07 January 2014 Posted by Paul Burns
Efficiency, along with adding value through technological innovation, has always been a defining goal of IT organizations. IT efficiency is driven by the desire to deliver better performance and value at lower costs. Cost optimization and price-to-performance optimization are continually taken further by the capabilities and economics of new technologies. These have included faster, less-expensive hardware, automation, x86-based virtualization and, in some cases, cloud services. History teaches us that each new efficiency limitation is simply a threshold for more efficient, more economical IT services.
Before describing what could be the next big step for IT efficiency, let’s take a look back at how IT organizations used to provision compute resources for their applications.
Dedicated Servers, Inefficiency and Sprawl
For many years, IT organizations used separate servers to deploy each of their applications, largely due to conflicting software dependencies. Each server was right-sized for the application it ran by configuring the appropriate amount of CPU, RAM, and disk. Of course resource consumption for most applications was (and generally still is) unpredictable, with peaks and valleys associated with different times of day or year. Therefore, IT staff members would “over provision” the hardware to support the application’s peak load. They would often go a step further, provisioning even more resources than expected for peak times to provide a performance and capacity buffer.
Far from efficient by today’s standards, this commonly resulted in average CPU utilization rates as lows as 10-15%, sometimes 20% if they were lucky. At the time, though, these rates were considered effective, even economic. That was until server “sprawl” became a real problem. As the number of managed servers grew, the costs of management, data center space, power and cooling became exorbitant.
Virtualization and Server Consolidation Drive Efficiency
The widespread availability of x86-based virtualization helped significantly improve utilization rates. Multiple applications could be deployed on a single server without fear of conflicting dependencies. This allowed IT organizations to further optimize their resources and they began reaching CPU utilization as high as 30-40%, sometimes 50%.
This worked so well that it created an important and popular trend called “server consolidation.” IT organizations began to buy larger servers and place more applications on them. Suddenly, it wasn’t about right-sizing servers for each application, it was about optimizing server utilization and improving IT efficiency, including the cost of server management, space, power, cooling, etc.
Cloud Computing Adds Efficiency – But It’s Not Perfect
While not forgetting that speed, flexibility and agility are often the most important benefits of cloud computing, it is important to acknowledge that cloud computing has taken virtualization and resource sharing to a new level of efficiency. Service providers and their customers both get benefits they wouldn’t normally receive with dedicated or virtualized servers. By serving multiple tenants from a single pool of shared resources, public cloud utilization rates can hover near 90% efficiency. Cloud customers also get efficiency benefits such as avoiding capital expenditures on servers that sit idle much of the time.
Of course cloud computing is not perfectly efficient and customers have discovered new challenges in their cloud environments. The vast majority of public cloud service providers host multiple tenants on each server by offering compute resources as virtual server instances. One downside of sharing a single physical server with other tenants is that your neighbors’ activities can actually impact the performance of your virtual server. This “noisy neighbor” problem is partially the result of service providers over subscribing their servers in order to achieve their own efficiency objectives.
Back to the Future with Over Provisioning
As IT organizations have gone from virtualizing their own infrastructure to purchasing virtual instances from service providers, their tendency to over provision resources hasn’t changed. When managing their own virtual machines on dedicated hardware, it’s at least possible to pair compatible applications without experiencing resource competition. Still, IT staff have a tendency to over provision each physical server as well as virtual machines to ensure resources are available at peak times. When purchasing cloud instances, IT staff members are back to playing guessing games about how other VMs and applications on their shared cloud server may impact them. As identified by Cloudyn and others, cloud users chronically over provision their instances.
Cloud service providers know this and anticipate that their customers won’t need all the capacity they’ve provisioned. They also realize that most applications on the same server will not demand peak resource consumption at the same time. To get the most out of potentially underutilized server resources, providers over subscribe their servers. While this results in better overall utilization rates for the provider, there are times when applications compete for resources and experience performance issues.
How Cloud Billing and Metering Really Works
According to the National Institute of Standards and Technology, one of the characteristics that defines cloud computing is measured service. Providers often advertise “pay-as-you-go” or “pay-per-use” services, but they rarely deliver on the promise of consumption-based pricing. Instead, customers typically pay for the time for which they’ve reserved the cloud resources or instance, not the actual amount of CPU cycles, RAM or local/ephemeral storage consumed by the instance.
If cloud customers don’t keep instances busy, they realize a real opportunity cost. With typical applications consuming just 10-15%, or perhaps as much as 20%, of a CPU on average, 80% or more of the reserved resources are essentially wasted. Of course, many applications have fluctuations in their actual resource consumption, so it’s almost impossible to achieve 100% utilization for a single application. Even individual instances of scale-out applications designed specifically to work in cloud environments can exhibit this behavior.
Cloud Server Efficiency
It is quickly becoming time to find and apply the next step in IT efficiency. Remember, greater efficiency doesn’t simply mean cost savings. It means making the best use of resources – people, technology, cash – to deliver the greatest value. As cloud resources are used more efficiently, more resources will be available to deliver extra value.
One of the most basic steps cloud users should take is to right size their instances. In other words, they should avoid the inefficiencies of over provisioning. Cloud users should also choose the most cost-effective pricing plans, whether that is on-demand, reserved or spot instances (or their equivalents, depending on the provider). Finally, cloud users should closely monitor cloud spending, ensuring that instances are released when not in use.
Of course, all these steps still result in cloud virtual machines that are underutilized due to the fact that application resource consumption varies over time. So, how can you address this?
Rewriting your application to scale in and out as resource demand fluctuates is one way. But this can be expensive and, depending on the application, you may still have individual instances whe actual resource consumption fluctuates.
Finding a cloud provider that meters based on actual usage is another way. ProfitBricks has done a lot here through dynamic instance scaling. As resource consumption grows, your individual instances can automatically scale up. Scaling down currently comes with a reboot, but this – along with their per minute billing, tenant isolation and other innovations – is very much in the right direction. A cloud 2.0 approach if you will.
The Next Step for IT Efficiency: Cloud Server Consolidation
The inefficiencies of cloud virtual machines bare a remarkable resemblance to those of dedicated servers running single applications without virtualization. An entire cloud VM is dedicated to each application, leaving it significantly underutilized on average. Shouldn’t we be able to apply virtualization and consolidate cloud servers just as we have physical servers?
The short answer is, yes. In fact, a couple interesting solutions for enabling cloud server consolidation already exist:
- Ravello Systems, has developed a “Cloud Application Hypervisor” that can gather and run multi-tiered applications on a single instance in the cloud. Normally, the VMs running each application component would need their own dedicated instance to work properly. Ravello’s Cloud Application Hypervisor gives customers the freedom to use their reserved instances the way they would typically use VMs in their own environment. Ravello has demonstrated that VM consolidation with their Cloud Application Hypervisor can increase the performance of multi-VM applications up to 2x in Amazon Web Services, a testament to the solution’ real world effectiveness.
- Docker, has taken a slightly different approach by offering an open-source engine that uses “containers” rather than a hypervisor to isolate applications running on the same operating system. These containers encapsulate entire applications and can be deployed in almost any environment, including public cloud environments such as Amazon Web Services. In essence, Docker creates a self-sufficient, technology-agnostic container for your entire application and then lets you deploy anywhere.
Both of these solutions offer other benefits in dev/test and DevOps scenarios. They improve instance utilization, moving cloud users closer to an actual pay-per-use pricing model. They also help prevent organizations from stepping back to older methods of right-sizing servers and instances.
While these solutions promise greater efficiencies for cloud service customers and move them closer to pay-per-use pricing models, service providers could face challenges if cloud server consolidation is done broadly. One of the things keeping costs low for service providers is the assumption that customers won’t come close to reaching 100% average utilization for their instances.
If many customers suddenly maintain higher utilization levels, providers will need to seriously reconsider their tendency to over subscribe resources. This means provisioning fewer VMs per server, which could significantly impact their profitability and return on investment.
It is also possible that service providers might eventually need to meter on actual consumption instead.
Have you performed cloud server consolidation? Will you try it now? We’d love to hear from you in the comment section below!
]]>
Recent blog posts
- Revisiting the Benefits of SaaS
- Compliance Drives Buying Decisions in the Cloud
- Changing Competitive Dynamics for IaaS Providers
- Good News for Cloud Job Seekers in 2015
- Colocation: The New Center of Hybrid IT
- The Problem with Forced Cloud Reboots
- OnApp Federation Grows Stronger with SolusVM Acquisition
- HP to Acquire Eucalyptus: Questions, Comments and Concerns
- Is It Really ‘Game Over’ for Most IaaS Providers?
- The Big Cloud Winners Are…Not Just the Hyperscale Players