Friday, June 26, 2015

Enablers of change leading up to Platform as a Service (PaaS)

Recognizing that the computing “services” are the center of attention, PaaS is primarily about using a collaborative platform to increase service efficiency (scale, cost,…) while 
  • Developing services.
  • Deploying services.
Before getting deeper into PaaS, we should 1st understand the burning problem that PaaS is addressing.   The answer to this question, interestingly, lies in the evolution of change that led to the creation of Platform as a Service.
While the IT side of the house was keenly interested in scaling service deployments, the developer side of the house was also keenly interested in “scaling” (which means creating more efficiency) in the development of services.

When people loosely talk about a dynamic, flexible and scalable IT environment, they really mean it in reference to services.  What the customer “really” wanted from the cloud (scaling the service) and what the customer really got, from the cloud  (scaling the infrastructure), was an incomplete answer to their services creation and delivery problems.  Platform as a Service, as we know of it “today” enables a solution to address both sides of the customer problem (deployment and development).
  • Deployment of services:     The IT side of the house looked at Platform as a Service as the ability to automatically deploy and scale “services” on demand at "internet scale".  At the time, scaling of services was already being addressed through the use of network load balancers, within data centers. While a little more static, the data center model worked well when the capacity of the traffic was better understood. What was not really addressed here was the scaling to the ever changing demands of the “internet” part of the equation.  This required more automation in the data center to automatically use (or release) idle resources as being done in the cloud.  It eventually took the spike in demand from mobile based services to recognize the unquantifiable capacity needs of internet traffic, and to really address the economics of scaling services within a cloud like environment.
  • Development of services:    At about the same time another important factor influenced the definition of Platform as a Service as we know of today.  The developer side of the house looked at Platform as a Service quite differently from how the IT and deployment side of the house perceived its value. They (the developer), looked at Platform as a Service as a way to economically address scaling the development of services.  There were many attempts along the way to formalize such efforts (like the one from Sun Labs called Project Caroline, that I am familiar with).   
Moving to Platform as a Service:
 All these early efforts around PaaS suffered from a fundamental problem. The disconnect between the “enterprise” and the “internet” companies on the “constructs” of what constitutes as a service.

The way that services were constructed in the enterprise (using an ESB), was very different in the way that services got constructed by web and internet companies (using the RESTful style of web service construction).  Naturally it was difficult to define a common platform for developing and deploying services if there was no clear agreement on the constructs of a service.  Undeterred, the web and internet companies forged ahead with their definition of services, and created mechanisms leading to an efficient DevOps framework.


Eventually the tepid appreciation for the RESTful style of web services by the enterprise, opened up opportunities to a more commonly acceptable framework for services and the PaaS model emerged.   (For the curious reader, I briefly talk about the evolution of service construction in the footnote below - click the read more button).

Platform as a Service (PaaS):

The principles of DevOps / Continuous Integration, Continuous Delivery, enabled new modern, scalable, distributed, platforms into an integrated framework to develop, test, deploy, modify, and upgrade services in a very efficient, scalable, and agile like environment. 

Using the principles behind the DevOps model and layering a service delivery framework above the IaaS layer in the cloud led to the foundations of what we call today as Platform as a Service. There are many flavors in today’s PaaS offerings, some involving very specific capabilities (which may or may not use Web services, or some of the examples as mentioned above). Some or all of the methodologies mentioned above can be found in today's Platform as a Service frameworks.


In summary: 

As I mentioned in one of my previous pioneering-cloud-my-personal-journey the concepts leading to PaaS are “not conceptually new ideas”.  As an example, while at Sun Micosystems, we had 
  • Implemented almost all of the capability mentioned under “continuous development/ continuous deployment” to enable the efficient development/development of Sun Cluster.  While these home grown tools were efficient, they were also expensive to develop and maintain. Developing such home grown capabilities, was not unique to us, with many other enterprises also involved in such expensive endeavors.  What is different today is the efficient use of “standard” tools like Git, Gerrit, Jenkins, Puppet/Chef, Vagrant, Docker,…. which enable a DevOps framework, but were not readily available at the time.  This led us (and everyone else) to develop such home grown tools while also developing the product.  With the adoption of web services in enterprises, such capability quickly got adopted in the development of services, and terms like DevOps/CICD got deeply rooted.  This finally allowed the cost burden to shift into where it should be – in product development.
  • The service delivery framework for PaaS as in the case of Open shift and Cloud Foundry are  also remarkably similar in concepts to the HA Service framework that my group had created in multiple versions of Sun cluster.  (Perhaps more in the next blog).
  • Recognizing the importance of efficient service delivery, the N1 group at Sun Microsystems had at the time, a general service delivery product “N1-SPS (service provisioning system)" to enable the easy deployment of “services”.   While, this provided some exceptional capabilities as part of the N1 suite of products, it was also too early in the adoption cycle for enterprises.  It also lacked the integration of development—to-deployment as characterized in today’s Platform as a Service.
I am very excited about the future of cloud technology.  For a long time I have held a strong belief that Platform as a Service (PaaS) is really where differentiation in cloud computing will happen.  The dirty little secret in Silicon Valley is that very little is being invented that has not already been done before (in some way form or manner), but, I am hopeful that future technologies will be built bearing in mind any mistakes of the past.


---------------------------------------------------------------------------
Footnote on the evolution of computing services
----------------------------------------------------------------------------

Traditional construction of services in the enterprise:
The way that services were traditionally built in the Enterprise was through a Service Oriented Architecture (SOA).   Without getting into the complexities of SOA, lets consider a simplistic explanation of how services were enabled through the ESB (Enterprise Service Bus).  Here the desired service used the enterprise service bus to communicate with a multitude of “basic/composite” services (a basic service could be functions supported through a MySQL database).   One of the primary functions of the ESB was to perform data transformations. The messaging system/broker was the communication system between data services.



The enterprise would then scale the service through various mechanisms in the data center.  However, what is important is that scaling was crafted for a known capacity and something the enterprise architects had some level of knowledge about and control over.

Web model for building services
The traditional enterprise model of service construction was far too complex and expensive for the web & internet world.  Unlike the enterprise, Web companies chose not to use an ESB model, but instead chose to construct their services by “rolling their own stack” on a per service basis.  Here they would simply attach the business logic for their service using open source middleware components that they needed in order to create a service deployment solution.  An example could be the LAMP Stack + application/business logic all rolled as a "service" deployment.

Two key differences between enterprise service deployment and Web services deployment revolved around:

  • Scaling the service.  Web services would need to be scaled for an unknown demand (since these would be on the Internet).   Various mechanisms were used to enable capacity on demand.
  • Cost: Using open source middleware frameworks/ suites like those available from Apache foundation, RedHat/JBoss, WSO2, Spring Source, Mulesoft, Talentd, Fusesoft, etc, allowed quicker development and were less expensive.
Such a model had major advantages for web companies (by taking out some of the cost and complexity).  It also tended to restrict the “dynamic” expandability/change of “new” basic components & services into such an existing web stack. 


Internet/Web Services API’s
Many framework components (like MySQL) provide an easy standard way of reading/ writing/ changing the attributes of a service through the use of API’s.  Such API’s, traditionally, were not easily accessible through the web.    The desire to have such functionality to be web enabled, resulted in the creation of Internet based “Web services API’s”.   Classic examples of such API’s are Amazon AWS services:  (ex: Databases: Amazon RDS, DynamoDB,  Storage:  Amazon EBS,  Amazon elastic File System, Analytics:  Amazon AMR,  Amazon Kinesis, …. and many more).  

Though much can be said on the topic of “Internet Web services API’s”, it is beyond the scope of this particular blog.  What is important in the context of this blog, is that, such API’s reduced some of the complexities of service integration when used in the development of Internet services.  
 
DevOps / Continuous Integration Continuous Deployment (CICD)
With the tepid acceptance of the RESTful style of Web services in the enterprise (and without getting into the RPC vs RESTful style of Web services), some of what started out as a mechanism to automatically deploy and “scale services” through Platform as a Service, very quickly also morphed into the ability to provide such flexibility & scaling in the “development of services” through a DevOps/CICD (Continuous Integration/Continuous Deployment) framework.   
  • Continuous Integration:  Web services API’s when used in conjunction with other web enabled development tools such as:
    • Github:  Internet scale source code management repositories 
    • Gerrit: creating an automated environment for code reviews 
    • Jenkins CI:  Automating the workflows from build to test (also referred to as a Continuous Integration Server) 
enabled an extremely efficient and scalable environment for the creation of complex services, involving large number of developers working in distributed locations. 



  • Continuous Deployment:   Once these services were created they needed to be deployed very quickly within a very distributed, very complex and highly scalable deployment topology.  To enable the deployment of such services many tools such as:
supplied the extremely flexible & scalable environment needed to deploy such services. 









No comments: