Winning a Cloud Race in Ten Simple Steps
There are two ways to win a techno race - to run faster in the crowd or to step in a new direction.
And the last one did yield some magnificent wins, which have being coming from unusual places: a garage (Steve Jobs), a laptop in the plane (Bill Gates), a bicycle shop(Wright bros.
) Check the Discovery channel 'Inventions that Shook the World'.
It's a big one.
I've had my share in doing strange things in weird situations and here are my strange results: we can get much more agility and put more payload onto the Internet Cloud if we switch from the Internet Zeppelins to the personal Internet Jetpacks.
I'm no Steve or Bill, comfortable and blessed with the comprehension of that.
I'm not after building a great company at all.
But people, I'm flying in the Cloud in my Jetpack for a while now, would love to share it like the Big Z did: "Hey, guys, the swell is happening over on the north shore!" In more accurate terms the Cloud Tool I'm bragging about is many times more (not just more) efficient and easy to use then the frameworks of the Big Flyers like Microsoft and Zoho and Amazon and Google, whom I respect immensely.
And my humble "apple" does pretty much everything the "oranges" do.
Anyways, I took a few well known steps - just sideways (don't ask me why) - and cooked that CloudBurger(aka Datalator aka Jetpack).
And here we are, please try the recipe.
1.
Start with Declarative (vs.
Imperative) Programming.
Feel the Data supremacy over the Algorithms.
Replace those new(...
) and add(...
) directives with a table containing names, types and properties of UI controls when building an application's screens.
Add a piece of code to utilize those declarations and issue new() and add() statements in a short standard loop.
Add another piece of code - a visual designer to build those screens and save them to those tables.
2.
Use Views to build the Application Data Model.
In a data driven application a visual representation of a Database Table - views and forms - usually contains enough information to recreate that Table.
Add a piece of code generating a Database Table from already built application's screens (see above).
Add another piece of code glueing the Views and the Table together and one more, interpreting the user input.
Now you can build and serve a single Table on a standalone computer.
3.
Visually define Table relations.
The one-to-many relation is a simple one; it powers such data structures as menu-submenus and department-employees kinds.
Modify your visual designer to assign the property of a "kid-parent" type to a Database Table.
The "parent" property of the Employees Table should point to the Departments and the "kid" property of the Department Table will point back.
The same pair of properties works well to define relations in menu-submenu structures.
4.
Define mechanics to navigate along and between related Tables.
The stereotype perception of "related tables" might be reflected in an SQL statement of the following type: SELECT * FROM (kid-table) WHERE (kid-table-key-field = value-of-parent-table).
Replace that complex (to non-techie) statement with more "organic" approach: display the parent table and highlight the "current" parent row.
Define some interface gestures to move the " current" row inside the Table.
Define the "enter" gesture like: a click on the "current" row must take a user along the link by generating and executing that freaky SQL above - silently on the background.
When ready, the "kid" table must be visualized.
And only those "kid" records, which conform to the "WHERE...
" clause, must be brought along.
And that is cool - you have a simple pure visual system with no code to write and bugs to chase later.
By defining the Table "relation" property (a link), employing an idea of a "View row cursor", synchronized with the "Table row cursor", and defining a "follow the link" action - you can build powerful runtime to navigate inside and along hierarchical data structures.
Any hierarchical structures that is, all served by the standard interpreter and a few bytes of a "link" property.
5.
Add functionality to work with the content of related Tables.
If the runtime remembers a parent record UID which was used to navigate to the "kid" Table, it might easily generate something like INSERT INTO (kid-table)...
WHERE (kid-table-key-field = parent UID).
The same goes for "UPDATE...
" statements, when user(or API) uses a Table form.
A gesture (or API) might commence a record deletion process, which will generate the only "DELETE...
" statement if applied to the kid table, or many "deletes" - for all linked kid records and a parent record - if applied to the parent table.
It is not difficult to use the technique on multilevel hierarchies, thus providing full CRUD functionality for visually designed customary hierarchical data structures.
Click and connect - like Lego blocks.
6.
Add functionality to support many-to-many Tables relations.
< If you could convert user gestures into SQL statements with a simple WHERE clause - you can handle more complex ones where WHERE terms are connected with AND, OR and NOT predicates.
Believe me it is much harder to come up with straightforward interface representation and gesture language to control many "parent" Tables with selected rows at the same time.
And we need them all to create a complex query.
If I did it you can do it too.
And it makes the magic work: all the Tables and all the relations are visually built, the interface objects namespace is separated from the Database namespace, all the CRUD operations are enabled by default, all the content always ACID as the runtime controls potentially ambiguous actions and all the business logic beyond default CRUD could be expressed in terms "load the Table(name)", "find the record where...
", "get/set the record field(name)" and "save/delete the record".
Plus you get the zero-maintenance application Database if the generation of SQL statements is bug-free.
Plus you get the SaaS ready platform if your runtime collects stats about Tables usage.
My does.
And you can actually explain what you are doing to your 5th grader kid or your boss which is always impressive.
I have no boss.
7.
Make a stack of activated Tables visible.
When surfing along the links in the application database - say from Departments to Employees and further to Employee's Records - you must keep the stack of all "parent" Tables - because they define SQL statements parameters when fetching or altering the "kid" Table data.
Now make that stack visible(like I did) or show a three of active records of active Tables (like Project Navigators do).
Devote some screen space to help to navigate an application and The User will love it.
Remember that even a complex application is limited and well structured unlike the World Wide Web, thus allowing the application map be shown and used.
8.
Add support for different data types.
Keep data on the file system or in the Database, but allow your interface access all the different types.
I managed to serve only Strings, Numbers, Dates, Times, Currencies, Texts, HTMLs and Images.
It would be nice to be able to work with everything else as well - Flash, PDF, DOC, MP3 - you name it.
Takes some more time though.
9.
Separate the Interface and the Data Model.
Add a messaging layer between your Tables and their interface representation.
By now you have keyboard/mouse/touch/kinect gestures defined to control the front end already.
Detect such a gesture, code it and pass to the server.
RMI, AJAX, SMTP, HTTP, JMS - any messaging technique will do.
Mine works on plain TCP sockets.
Nothing fancy here - the request-respond protocols are relatively simple and well studied.
At this time it might be nice to add some real-time collaboration to the system.
You have to create feedback mechanics to push "server status changed" signals to every active client.
And you might not want to open listening port(s) on the client side.
Well, I used "long polling" with the client responsible for the lost connection healing.
With such a feedback line you can be sure that all the clients will be notified about the content or design changes.
And that is up to you now how to react.
I decided that relevant changes of the relevant Tables must be immediately reloaded asynchronously on the background.
That client-server architecture will work with or without messages encoding.
I scramble messages just enough to scare away an occasional sniffer.
But the system will not be consistent without transactions support and collisions marshalling.
You will not want to delegate content locking to the Database of your choice, because your choice of the Database might be changed over time.
If the system supposed to work with wide range of Databases - have your own locks: acquire the record lock to edit it, lock all the parent records to prevent them from deletion and lock the layer when you going to redesign it.
Release the locks when appropriate.
Now cache everything cacheable on the client side before querying the server and on the server side before querying the Database.
The Database takes care about caching queries results internally.
At last - and that's a big part - you have to think about improving the scalability.
I did not - my present server is plain multithreaded and does not support clasterization.
It will by the time I have applications with hundreds of simultaneous users.
10.
Provide convenient API.
You have to pass control to relevant custom event handlers before and after processing interface events on the client side.
You may want to call the custom code to process runtime originated events, like painting or not-enough-memory cries.
You may also want to custom process the server events, like "content has been changed".
In some cases you may want to change the way how the Table(s) visualized - think of a game programming.
You will want for sure to add some convenience calls.
And it all has to be easy to comprehend and employ.
Conclusion.
At this point you are already far ahead of all the competition.
You have a light tool which replaces a whole bunch of technologies.
You can build the application when talking to a customer and rebuild it without stopping the server.
Your apps are reliable and consistent, easy to deploy and easy to use.
You grow your library of styles and reusable Tables, you have your visually designed reports (just another View of the Table(s)) and you can import/export content in CSV format.
Your server hosts unlimited number of applications and serves unlimited groups of unlimited users, collects SaaS statistics, does not crash and does not requires attention.
Fast, reliable and cheap.
Could it be better? Indeed.
Scalability, security, speed, cleaner implementation, support for more data types, more dressed up components-Tables (chat, videoconference, mail client, etc.
), support for mobile platforms, API for different languages, IDEs integration (should Datalator be incorporated into IDE or IDE incorporated into Datalator?), Web Server integration (should Datalator be...
).
It could be truly RESTful if the business logic is kept on the server - along with the Table content and definition - and loaded on runtime.
Self-contained Table - a smart plug-and-play component, what could be better? Sky is the limit.
Prove me right.
The story is told and excitement is over, now comes the time of business development.
But speaking about of sometimes strange ways of looking at things - there are a few more perpendicular questions to be answered one day: Why the Text is the only feasible comprehensive representation of a computer program? Can we create a convenient 3D environment for immersive general programming by team, RPG style? Why Claude Shannon alphabet is immutable in his famous Source Coding Theorem? Can the theoretical compression limitations be overcome by extending the alphabet? Why Artificial Intelligence mainstream research studies the logic before recognition? What if we solve the mystery of the Tabula Rasa first? Well, another stories for another time.
I'll keep you posted.
And the last one did yield some magnificent wins, which have being coming from unusual places: a garage (Steve Jobs), a laptop in the plane (Bill Gates), a bicycle shop(Wright bros.
) Check the Discovery channel 'Inventions that Shook the World'.
It's a big one.
I've had my share in doing strange things in weird situations and here are my strange results: we can get much more agility and put more payload onto the Internet Cloud if we switch from the Internet Zeppelins to the personal Internet Jetpacks.
I'm no Steve or Bill, comfortable and blessed with the comprehension of that.
I'm not after building a great company at all.
But people, I'm flying in the Cloud in my Jetpack for a while now, would love to share it like the Big Z did: "Hey, guys, the swell is happening over on the north shore!" In more accurate terms the Cloud Tool I'm bragging about is many times more (not just more) efficient and easy to use then the frameworks of the Big Flyers like Microsoft and Zoho and Amazon and Google, whom I respect immensely.
And my humble "apple" does pretty much everything the "oranges" do.
Anyways, I took a few well known steps - just sideways (don't ask me why) - and cooked that CloudBurger(aka Datalator aka Jetpack).
And here we are, please try the recipe.
1.
Start with Declarative (vs.
Imperative) Programming.
Feel the Data supremacy over the Algorithms.
Replace those new(...
) and add(...
) directives with a table containing names, types and properties of UI controls when building an application's screens.
Add a piece of code to utilize those declarations and issue new() and add() statements in a short standard loop.
Add another piece of code - a visual designer to build those screens and save them to those tables.
2.
Use Views to build the Application Data Model.
In a data driven application a visual representation of a Database Table - views and forms - usually contains enough information to recreate that Table.
Add a piece of code generating a Database Table from already built application's screens (see above).
Add another piece of code glueing the Views and the Table together and one more, interpreting the user input.
Now you can build and serve a single Table on a standalone computer.
3.
Visually define Table relations.
The one-to-many relation is a simple one; it powers such data structures as menu-submenus and department-employees kinds.
Modify your visual designer to assign the property of a "kid-parent" type to a Database Table.
The "parent" property of the Employees Table should point to the Departments and the "kid" property of the Department Table will point back.
The same pair of properties works well to define relations in menu-submenu structures.
4.
Define mechanics to navigate along and between related Tables.
The stereotype perception of "related tables" might be reflected in an SQL statement of the following type: SELECT * FROM (kid-table) WHERE (kid-table-key-field = value-of-parent-table).
Replace that complex (to non-techie) statement with more "organic" approach: display the parent table and highlight the "current" parent row.
Define some interface gestures to move the " current" row inside the Table.
Define the "enter" gesture like: a click on the "current" row must take a user along the link by generating and executing that freaky SQL above - silently on the background.
When ready, the "kid" table must be visualized.
And only those "kid" records, which conform to the "WHERE...
" clause, must be brought along.
And that is cool - you have a simple pure visual system with no code to write and bugs to chase later.
By defining the Table "relation" property (a link), employing an idea of a "View row cursor", synchronized with the "Table row cursor", and defining a "follow the link" action - you can build powerful runtime to navigate inside and along hierarchical data structures.
Any hierarchical structures that is, all served by the standard interpreter and a few bytes of a "link" property.
5.
Add functionality to work with the content of related Tables.
If the runtime remembers a parent record UID which was used to navigate to the "kid" Table, it might easily generate something like INSERT INTO (kid-table)...
WHERE (kid-table-key-field = parent UID).
The same goes for "UPDATE...
" statements, when user(or API) uses a Table form.
A gesture (or API) might commence a record deletion process, which will generate the only "DELETE...
" statement if applied to the kid table, or many "deletes" - for all linked kid records and a parent record - if applied to the parent table.
It is not difficult to use the technique on multilevel hierarchies, thus providing full CRUD functionality for visually designed customary hierarchical data structures.
Click and connect - like Lego blocks.
6.
Add functionality to support many-to-many Tables relations.
< If you could convert user gestures into SQL statements with a simple WHERE clause - you can handle more complex ones where WHERE terms are connected with AND, OR and NOT predicates.
Believe me it is much harder to come up with straightforward interface representation and gesture language to control many "parent" Tables with selected rows at the same time.
And we need them all to create a complex query.
If I did it you can do it too.
And it makes the magic work: all the Tables and all the relations are visually built, the interface objects namespace is separated from the Database namespace, all the CRUD operations are enabled by default, all the content always ACID as the runtime controls potentially ambiguous actions and all the business logic beyond default CRUD could be expressed in terms "load the Table(name)", "find the record where...
", "get/set the record field(name)" and "save/delete the record".
Plus you get the zero-maintenance application Database if the generation of SQL statements is bug-free.
Plus you get the SaaS ready platform if your runtime collects stats about Tables usage.
My does.
And you can actually explain what you are doing to your 5th grader kid or your boss which is always impressive.
I have no boss.
7.
Make a stack of activated Tables visible.
When surfing along the links in the application database - say from Departments to Employees and further to Employee's Records - you must keep the stack of all "parent" Tables - because they define SQL statements parameters when fetching or altering the "kid" Table data.
Now make that stack visible(like I did) or show a three of active records of active Tables (like Project Navigators do).
Devote some screen space to help to navigate an application and The User will love it.
Remember that even a complex application is limited and well structured unlike the World Wide Web, thus allowing the application map be shown and used.
8.
Add support for different data types.
Keep data on the file system or in the Database, but allow your interface access all the different types.
I managed to serve only Strings, Numbers, Dates, Times, Currencies, Texts, HTMLs and Images.
It would be nice to be able to work with everything else as well - Flash, PDF, DOC, MP3 - you name it.
Takes some more time though.
9.
Separate the Interface and the Data Model.
Add a messaging layer between your Tables and their interface representation.
By now you have keyboard/mouse/touch/kinect gestures defined to control the front end already.
Detect such a gesture, code it and pass to the server.
RMI, AJAX, SMTP, HTTP, JMS - any messaging technique will do.
Mine works on plain TCP sockets.
Nothing fancy here - the request-respond protocols are relatively simple and well studied.
At this time it might be nice to add some real-time collaboration to the system.
You have to create feedback mechanics to push "server status changed" signals to every active client.
And you might not want to open listening port(s) on the client side.
Well, I used "long polling" with the client responsible for the lost connection healing.
With such a feedback line you can be sure that all the clients will be notified about the content or design changes.
And that is up to you now how to react.
I decided that relevant changes of the relevant Tables must be immediately reloaded asynchronously on the background.
That client-server architecture will work with or without messages encoding.
I scramble messages just enough to scare away an occasional sniffer.
But the system will not be consistent without transactions support and collisions marshalling.
You will not want to delegate content locking to the Database of your choice, because your choice of the Database might be changed over time.
If the system supposed to work with wide range of Databases - have your own locks: acquire the record lock to edit it, lock all the parent records to prevent them from deletion and lock the layer when you going to redesign it.
Release the locks when appropriate.
Now cache everything cacheable on the client side before querying the server and on the server side before querying the Database.
The Database takes care about caching queries results internally.
At last - and that's a big part - you have to think about improving the scalability.
I did not - my present server is plain multithreaded and does not support clasterization.
It will by the time I have applications with hundreds of simultaneous users.
10.
Provide convenient API.
You have to pass control to relevant custom event handlers before and after processing interface events on the client side.
You may want to call the custom code to process runtime originated events, like painting or not-enough-memory cries.
You may also want to custom process the server events, like "content has been changed".
In some cases you may want to change the way how the Table(s) visualized - think of a game programming.
You will want for sure to add some convenience calls.
And it all has to be easy to comprehend and employ.
Conclusion.
At this point you are already far ahead of all the competition.
You have a light tool which replaces a whole bunch of technologies.
You can build the application when talking to a customer and rebuild it without stopping the server.
Your apps are reliable and consistent, easy to deploy and easy to use.
You grow your library of styles and reusable Tables, you have your visually designed reports (just another View of the Table(s)) and you can import/export content in CSV format.
Your server hosts unlimited number of applications and serves unlimited groups of unlimited users, collects SaaS statistics, does not crash and does not requires attention.
Fast, reliable and cheap.
Could it be better? Indeed.
Scalability, security, speed, cleaner implementation, support for more data types, more dressed up components-Tables (chat, videoconference, mail client, etc.
), support for mobile platforms, API for different languages, IDEs integration (should Datalator be incorporated into IDE or IDE incorporated into Datalator?), Web Server integration (should Datalator be...
).
It could be truly RESTful if the business logic is kept on the server - along with the Table content and definition - and loaded on runtime.
Self-contained Table - a smart plug-and-play component, what could be better? Sky is the limit.
Prove me right.
The story is told and excitement is over, now comes the time of business development.
But speaking about of sometimes strange ways of looking at things - there are a few more perpendicular questions to be answered one day: Why the Text is the only feasible comprehensive representation of a computer program? Can we create a convenient 3D environment for immersive general programming by team, RPG style? Why Claude Shannon alphabet is immutable in his famous Source Coding Theorem? Can the theoretical compression limitations be overcome by extending the alphabet? Why Artificial Intelligence mainstream research studies the logic before recognition? What if we solve the mystery of the Tabula Rasa first? Well, another stories for another time.
I'll keep you posted.
