Private cloud IaaS Evolution - Failures were the stepping stones to success

Before we examine cloud computing lets 1^st understand some of the factors that precipitated the need for cloud computing.

Cloud computing started with the Web, allowing  Enterprises to promote their information in a form that could be accessed from anywhere (geographically) and from anything (devices) without needing  any client side software (VPN,…).  Realizing the convenience of the Web,  Enterprise users also wanted access to their private Enterprise information through the Web as well (email, CRM & ERP data, HR data, financial data,…).  

All this external access caused a large headache for the Enterprise IT staff where the carefully articulated rules of the intranet simply did not apply to the internet (capacity planning/scaling, security being under a physical lock and key,…). The Enterprise IT staff had to scramble to meet the requirements of their customers through adhoc scripts and home grown solutions/processes designed to scale and secure their current environment to meet the requirements of the internet. 

The formalization of these processes is what I believe to be one of the fundamental requirements that led to today’s Cloud computing.

But that still leaves the question of what does it offer IT departments?  In its simplest form, the Cloud was attempting to solve 3 fundamental IT problems.  Managing Cost, Risk and Service levels.

 

Though there are many technologies that make up  Cloud Computing, there were 2 fundamental technology building blocks that became the foundation for modern clouds.

1.    Virtualization: In this context, virtualization is an abstract representation of a new resource from the underlying physical resources (compute, storage, network, services).   

Note: There is a common misconception that virtualization involves a single large resource (ex: a physical computer) to be partitioned into smaller pieces (host virtualization like Xen, or, as in the case of OS virtualization like LXC). 

Virtualization can also go the other way, where many smaller resources (physical computers) can be aggregated together to also form a larger single resource (ex: shared memory computing system).

2.   Clustering: Allows the above partitions to re-form into dynamic groups with an interdependency between them.   These interdependency's are basic clustering requirements like heartbeats, quorum, cluster membership, fencing, service management frameworks (another future blog of mine),…. of distributed computing.

Having these 2 disparate capabilities of resource optimization and distributed systems coming together on a single platform/system, enabled a flexible and powerful framework; thus setting the stage for cloud computing. 

It took a long time to adopt cloud computing because Enterprises were justifiably wary of the issues exposed by the public cloud (security, performance, failures,..). and so IT providers scrambled by providing private cloud’s (that’s where the money was).  Early efforts in Private cloud IaaS solutions were also not very successful.  There was a fundamental problem that I saw, essentially,...   

What problem were we really solving for the enterprise through this private cloud IaaS?

IaaS gave Enterprises a consistent way to dynamically and efficiently reuse computing resources at the infrastructure level.  However the Enterprises had already solved this problem though various home grown scripts of their own and were opposed to ripping and replacing their familiar well-orchestrated home grown scripts with another vendor’s solution.  Having no consistent standard between various IT providers, IaaS solutions simply added to the controversy.

A lot of this changed with the emergence of initial standardization efforts (Open Stack, Cloud Stack, AWS,..) to provide a consistent approach towards IaaS. 

 

At least 2 things worked in Open Stack’s favor:

1. Adoption:  Many IT vendors abandoned/aligned their private efforts towards Open Stack.
2. Open development: Unlike the politics of normal standards in the computing industry, Open Stack took a more pragmatic approach by providing Open Source development to everyone in the industry (and then took an aggressive stance in formalizing releases).

Of course, we can always debate the problems with Open Stack (vendors morphying the stack, Open Stack aimed at developer play rather than a deployment ready model, ...), nonetheless Open Stack finally provided the necessary ground swell for the Enterprise data centers to take cloud computing seriously.