Home > Blogs > VMware vCloud Blog > Monthly Archives: December 2010

Monthly Archives: December 2010

Cloud Architecture Patterns: Service VM

By Steve Jin, VMware R&D

Intent

Provide an easy way to provision new infrastructure and application services for a computing cloud

Category

Behavioral

Problem

To run a large-scale computing infrastructure, you will need many different types of services, including compute, storage, and networking, among others. After virtualization has successfully detached compute from the physical hardware, it’s very easy to provision and scale compute. But compute requires storage and networking which are lagging behind. To maximize the benefits of virtualization and cloud computing, it’s natural to push the storage and networking in the same direction.

Looking beyond the infrastructure to consider applications, we need various types of services such as database, directory, messaging, and more. I’ve covered the App VM pattern that allows using IaaS for PaaS in a previous blog. While you can pack some of these services into an application VM, the problem is that it scales well but does not follow the aspectual centralization pattern.

Solution

The solution is to have dedicated VMs for either the infrastructure level services or the application level services. In so doing, you can use the same provisioning mechanism with service VMs as with any other virtual machines.

One extra benefit with infrastructure level service VMs is that you don’t need dedicated or special hardware. This hardware freedom saves you on capital spending and operations expenses, and frees you to plan and allow resource sharing and management across the traditional boundaries among compute, storage, and networking.

With the application service VMs, you can simplify the app VMs by removing the service components already offered by the service VMs. These service VMs can be managed and maintained by different teams. That’s one less thing the application teams need to worry about!

Using service VMs does come with challenges. First, your service VM should have stable service interfaces that others can bind to. It’s preferably based on open standards. For the storage VM, you want to use NFS, iSCSI, and other open protocols.

Secondly, your service VMs should be highly available. Whoever builds on top of your service VM can’t be any more available than your service VMs. You have to raise the bar first.

Thirdly, you will need to have a stable service catalog so that people can pick and provision easily. You should not remove any service VM that is in current use, but you still need to patch and upgrade your service VMs.

Lastly, you will need to manage the service VMs effectively and efficiently. Some service VMs might have special requirements. For example, a storage service VM that serves others VMs on the same host should not be relocated to other hosts for good performance. You have to prevent arbitrary migration in this case.

Applicability

Consider a service VM pattern when you want to:

  • Virtualize and standardize all your IT infrastructure and application infrastructure for flexibility and cost saving;
  • Simplify the IT planning and provisioning management.

Consequence

The infrastructure service VMs may not perform to the same level of its physical equivalents on a per unit basis, but may outperform them while scaling out. On the management side, you will have many more VMs to manage with considerations on special requirements on service VMs. It may require more investment on management stacks which may offset the cost savings from your unified infrastructure.

While your application VMs can leverage services VMs, it affects portability. Your application VMs can no longer freely move to other clouds alone due to service accessibility or performance.

Known Uses

In today’s market, you can find many different service VMs such as VMware Shield Zones. It is deployed as a virtual appliance and integrated with VMware vCenter™ Server, and “helps make it easy to centrally manage and enforce compliance with security policies across large pools of servers and virtual machines. Built-in auditing capabilities make compliance straightforward and verifiable.” On the storage side, you can find products such as the HP Lefthand virtual storage appliance.

On the application related services you have more choices. In fact, you can build your own dedicated service VMs based on your own requirements. For example, a service VM preinstalled and configured with MySQL database.

Related Patterns

VM Factory: create new instances for service VMs.

App VM: can leverage the services provided by service VMs.

VM Pipeline: combine several service VMs in a row for serial processing.

Author: Steve Jin, top 50 blogger on cloud computing, author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. Subscribe to this blog via Email or RSS, follow on Twitter.

BlueLock Webinar Tomorrow: Public Cloud for the Pragmatic Enterprise

When is public cloud relevant to larger organizations? Join BlueLock and VMware tomorrow from 11:00-11:30am ET (8:00-8:30am PT) to learn how public cloud is most useful to the enterprise. BlueLock and VMware will discuss what the public cloud is NOT about as well as how your enterprise can begin to take steps toward a logical public cloud business model.

BlueLock was recently named one of VMware’s five global certified vCloud Datacenter providers, and jointly developed solutions with VMware that are specifically designed, based on real experience and research, to help the enterprise take advantage of the public cloud capabilities that mean the most to their business. Tomorrow, BlueLock will provide a demonstration of their innovative Virtual Datacenter solution based on vCloud Director technology from VMware.

Register for the webinar here.

Cloud Architecture Patterns: VM Pipeline

Steve Jin, VMware R&D

Intent

Provide a configurable structure for modularized information processing

Category

Structural

Problem

Complicated data processing involves many distinctive and repetitive steps. Each of these steps can be mapped to a software module that is independently developed and assembled for particular cases of data processing.

Given the elastic nature of cloud computing, it’s a perfect platform for data processing. We need a solution that is flexible in two ways:

1. Modularized components for data processing;

2. Configurable so that different modules can be re-used easily in various cases.

Solution

Define the key steps of processing and map each of them to a virtual machine. For better performance, one option is to have multiple VMs for one step but that may mean more work on coordination.

While deciding what steps are there, you need to think about not only the current project but also re-use of the corresponding virtual machines for future projects. It’s helpful to look beyond the immediate project and consider it from the perspective of the data processing and come up with re-usable virtual machines.

The topology of the pipeline is pretty straight forward. You have virtual machines in a line as shown below:

Vm1
A variation of this serial structure illustrated above is to have multiple VMs in one step as a cluster to improve performance. All the virtual machines in one cluster will be the same configuration to simplify management.

Vm2

As you can see, you need to flow information from one step to its next step. There are two types of flows: control flow and data flow. The control flow signals the completion of the current step so that its next step can get started. The data flow is the processed output data from the current step to its next step as input.

There are two different strategies for passing information. You can do it from a centralized repository or stream from one VM to the next. Here are the main things to consider:

1. Direct messaging. It includes RPC, Web Services, or socket-based communications. It requires the current step to know what the next step’s virtual machine is and how to create connections.

2. Indirect messaging. It leverages standard messaging queues and decouples the sender (current VM) and receivers (next VMs). An added benefit is that the messages can be stored and delivered later in case the receiver is busy or offline.

3. DB sharing. It uses standard databases as the common information store. All the VMs involved in the processing should have access to the database.

4. Distributed file system. It could be a Cluster file system, or just NFS-based shared file storage.

5. Cloud storage. With this approach, you open an account with a service provider and store your data there.

So which one should you choose for your project? From a business perspective, these options all come with varying costs for development, initial setup and on-going maintenance. From a technical perspective, the major technical considerations in your choice are:

1. Amount of data. When there are huge amounts of data, you don’t want to move data from one VM to another very often. In that case, having a centralized database makes a lot of sense. Whenever you have a centralized data accessed by multiple parties, synchronization becomes critical for smooth processing.

2. Timing requirement. If you have a demanding requirement on timing, you may want to consider direct messaging and sacrifice the benefits of decoupling your VMs.

As mentioned early, the control and data flows are two different flows. You can choose a different approach for either of them. For example, you choose messaging queue to pass on control messages and database for data flow.

If you don’t have complicated data processing, the simple and autonomous pipeline architecture may be good enough. If you have complicated data processing – for example, multiple pipelines interwoven together and data flows into different routes depending on current processing result. If that is the case, you may need a management application to manage and coordinate the processing.

Applicability

Consider a VM pipeline pattern when you want to:

  • Divide data processing into multiple steps that can map to virtual machines;
  • Freely assemble and configure specialized VMs as steps of information processing;
  • Easily isolate and encapsulate processing logics into VMs;
  • Better control the resource allocation and management for processing efficiency;
  • Design your system for better scalability.

Consequence

Mapping each step into a virtual machine instead of a process has extra overheads that range from more storage consumption and slower performance due to runtime VM switching. You will also need extra IP addresses and see more network traffic. To manage these virtual machines, you may need to acquire or upgrade your management system. These are costs you should consider when considering this approach for scalable data processing.

Known Uses

It’s a common practice to use a process pipeline pattern for information processing. I personally haven’t read about use of this approach at the virtual machine level. If you have any examples to share, please feel free to leave a comment!

Related Patterns

VM Factory: create new VM instances for each step.

Stateless VM: minimize the management of the VMs especially being used in one step.

This article was originally posted on www.doublecloud.org.Visit the site for more information on virtualization, cloud computing, and other enterprise technologies.

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to Email or RSS, and follow on Twitter.

Moving to the Cloud with Virtacore vCloud Express

Matthew D. Sarrel, Sarrel Group

I’m going to be moving my test platforms from Terremark to Virtacore where I will still be running the vCloud Express platform.  This is actually sort of cool that VMware takes the vCloud Express offering and helps multiple cloud providers bring it to market.  So while I’m sure that some aspects of the interface and management will be different, I’m also sure that much of what really matters will be the same.  I don’t think there will be an easy way to migrate my VM’s from Terremark to Virtacore, but they are just test machines anyway.

Virtacore has been around the online or cloud space since 2002 and has helped thousands of customers move services into the cloud.  They offer a 100% uptime guarantee, which they can do through partnerships with Equinix, the top tier data center provider.  According to their website, they already manage over 10,000 virtual environments.

Virtacore vCloud Express is fully VMware compatible and it looks to me like I will be able to migrate VM’s between internal servers and their cloud servers.  They have a unified management platform, vCore, that is a console that you can quickly and easily add, remove, change or monitor all the VM’s in my cloud.  The other thing that looks good is that file-level backups are available which gives me some piece of mind.

Virtacore vCloud Express is going to be available as beta within the next few weeks. I’m pretty psyched to try it out and write about it.

Matthew D. Sarrel (or Matt Sarrel) is executive director of Sarrel Group, a technology product testing, editorial services, and technical marketing consulting company.  He also holds editorial positions at pcmag.com, eweek, GigaOM, and Allbusiness.com, and blogs at TopTechDog.

SkyTap is Making Cloud an Affordable Reality

By David Davis

This week I interviewed Sundar Raghavan, CMO at SkyTap. At the end of his demonstration of the SkyTap public cloud, I was highly impressed in the ease of use, affordability, and practicality of their offering. There are really an unlimited number of use cases. In this article, I'll explain how the SkyTap cloud works, from an IT Admin/End User perspective and why I was so impressed with what SkyTap had to offer.

Who is SkyTap?

At the start of the interview with Sundar, I learned that he was previously a product manager at Google for two years where he evangelized their cloud offerings. Today, he leads product management and marketing for SkyTap in Seattle. SkyTap is a private company that is backed by well-known venture capital companies Madrona, Ignition, Washington Research, and Bezos Expeditions (yes, that is Jeff Bezos of Amazon's VC company). At this point I was thinking "Big names with Big Experience – Google and Amazon". Started in 2006, SkyTap has a partnership with CSC to use their datacenters in Seattle and Chicago – which is where your data would physically reside if you used the SkyTap cloud. The goal of SkyTap is to provide cloud automation solutions for businesses of all sizes. Common use cases include SMBs who need agility and simplified IT, developers who need test/demo environments, training companies that need on-demand infrastructure, and even enterprises that "move to the cloud" for all the common reasons that you would.

How much does SkyTap cost?

Much of my professional IT experience was with a family-owned medium-size enterprise that, as you can imagine, was always cost conscious (or sometimes we called it "cheap"). From that, a habit that I have gotten into is always asking how much some IT product or service costs, within the first few minutes of learning about it. Thus, I know how much of a factor price is to many companies.

With many of the potential customers for SkyTap being SMBs, independent development companies, or startups, I asked Sundar about the entry cost to use the SkyTap cloud. He directed me to the Google Marketplace where SkyTap is selling their Express Subscription for $250 per month with no contract. This includes their self-service portal, 5 users, 5 SVMs (equal to 1 CPU and 1GB of RAM), 500 SVM hours, and 200GB of storage. To me, this is offering about 5 virtual servers for $250 per month, or $50 per server – that's affordable. In fact, SkyTap says that you should be able to reduce your current infrastructure costs by as much at 70%.

With my most important question satisfied, let's move on.

Does SkyTap use VMware Technologies?

My second most crucial question for Sundar was whether or not they use VMware technologies in the SkyTap cloud. If the answer was no I was going to hang up the phone (just kidding). Actually, SkyTap runs the underlying cloud on VMware vSphere. You can import and export VMDK or OVF files from your current vSphere infrastructure (don't worry, there is no "Hotel California", Paul). They don't use vCloud Director or the vCloud API yet but they are working on how this will fit into their existing infrastructure.

With that second most crucial question satisfied, let's move on to how does this thing really work.

SkyTap's Five Best Features

SkyTap is really your virtual datacenter "in the cloud". You have virtual servers, networks, and storage that you can configure into whatever design you choose. Here are SkyTap's five best features, in my opinion:

1. Appealing Self-Service Design – SkyTap's feast of pre-built VM templates that you can deploy yourself makes getting up and running fast (even with more complex apps). By having these pre-built VM templates that have either OS or OS+App (like SQL Server) you are going to save a ton of time getting your VM (and app) up and running. SkyTap must work hard to give you many choices but also many them easy to configure and easy to understand. Even something like configuring multiple networks for your virtual infrastructure is straight forward to do. Here is what the typical SkyTap administration interface looks like:

Skytap1

2. Real Hybrid Cloud - SkyTap isn't only public cloud. They support the hybrid cloud model such that, using SkyTap, you can create a VPN between your internal LAN and your SkyTap LAN. You can then have the servers in the SkyTap cloud and the servers on your local LAN work together. I would guess that almost all of SkyTap's customers would be using Intenet-facing apps but Sundar said that about 70% of their customers are just using private/internal apps in the SkyTap cloud vs. about 30% that are Internet-facing – surprising news!

Skytap2
3. Network Automation - SkyTap's networking configuration is easy to do and their network automation features allow you to create advanced network designs and security policies with a few clicks (here is a link with info on their new network automation features).

Skytap3

4. Import and Export - you can import & export VMware VMDKs as well as OVF files from vSphere or VMware Workstation – all self-service. This way, you could quickly deploy your existing VMs/apps into the SkyTap cloud or bring VMs back in house whenever you see fit.

5. Real Customer Examples – If you choose to move your infrastructure to the cloud with SkyTap, it's not like you have to be the first to do it. There are many case studies of businesses that are already using the SkyTap cloud successfully  (even small businesses too). As an example, Sundar referred me to a small IT training company that uses SkyTap to roll out virtual classroom labs as needed and then destroy them when done. He also referenced a SMB that was tired of trying to keep up with hardware refresh, backup management, and IT skills so they moved their IT infrastructure to SkyTap.

Summary

For anyone getting started in cloud computing I think that the daily administration of Infrastructure as a Service (IaaS) is hard to envision, at first. The thought of having just one web interface to all your infrastructure located "in the cloud" could be very bad (if the UI is bad or has limited features) or very good (single interface could be nice if full featured and easy to use). Kudos to SkyTap because they are  one company that is really doing IaaS right and covers all the bases (affordable, easy to use, using VMware, import/export, and nice library of template VMs to deploy fast). I hope that SMBs, developers, trainers, and enterprises alike will take SkyTap for a test drive. I am requesting a demo account now so look forward to my future articles showing how to use SkyTap.

David Davis is a VMware Evangelist and vSphere Video Training Author for Train Signal. He has achieved CCIE, VCP,CISSP, and vExpert level status over his 15+ years in the IT industry. David has authored hundreds of articles on the Internet and nine different video training courses for TrainSignal.com including the popular vSphere video training package. Learn more about David at his blog or on Twitter and check out a sample of his VMware vSphere video training course from TrainSignal.com.