By Wesley Beary (aka
geemus), Engine Yard

Fog is the Ruby cloud
computing library. It helps Ruby
developers interact with cloud services. Here is our logo:

 So, the quick and dirty on fog, top to bottom:

* Models provide a simplified interface, making
clouds easier to work with and switch between;

* Requests allow power users to get the most out of
the features of each individual cloud;

* Mocks make testing and integrating a breeze.

As I was getting my head
wrapped around working with cloud resources and trying different services I
found myself wishing for more open, powerful tools. From this desire I started
working on basic wrappers to many services, which organically grew into a
number of new items for my toolbox. The end result combines common model
abstractions, low level calls and mocks to provide an easier and more uniform
cloud experience.

To interact with a
service you first need to initialize a connection:

    vcloud
= Fog::Terremark::Vcloud.new(

      :terremark_vcloud_username
=> INTENTIONALLY_LEFT_BLANK,

      :terremark_vcloud_password
=> INTENTIONALLY_LEFT_BLANK

    )

Ease of Use via Models

The common model
abstractions make it faster and easier to get started working with the basics
in several important ways. First by normalizing nouns across services, you can
just work with servers instead of needing to know that slices, instances, vApps
and servers have many shared operations. Next, they contain knowledge of what
information is required for different operations. This allows you to set up the
basic operations and depend on the models to let you know when you leave out an
important value. Finally, they provide helpers to make common operations
easier.

Some examples of the
model abstractions:

    #
list task models

    vcloud.tasks.all

    #
list task models (lazy_loaded)

    vcloud.tasks

    #
load task model by id

    task
= vcloud.tasks.get(task_id)

    #
use the wait_for and ready? helpers to wait for task completion

    task.wait_for
{ ready? }

Great Power and Responsibility via Low Level Calls

The low level calls also
provide important functionality. To start, they provide all the functionality
that the models provide a simpler interface to. They also provide an overflow
valve when the basic common functionality just won't cut it. In addition to
these important functions, they are the first part of a new cloud provider to
be implemented in fog, so they provide early access to functionality that the
models will later contain.

Some examples of the low
level calls:

    #
instantiate a vapp_template {12 => Ubuntu JeOS 9.10 (64-bit)}

    vcloud.instantiate_vapp_template('fog',
12)

    #
deploy a vapp_template

    vcloud.deploy_vapp(vapp_id)

Faster Development via Mocking

Last but not least, fog
provides mocks to make development faster and more pleasant. When fog is placed
in mock mode it accepts the same calls, but rather than sending out the actual
calls it just keeps track of the resulting state changes in memory. This allows
developers to try out provisioning code without having to wait for servers to
become available and without having to incur the cost of resources up front. These
are usually the last piece to arrive for a provider, so I have yet to add them
for vCloud, but this should provide a preview of goodness to come.

Mocks are enabled like
this:

    #
turn on mocking

    Fog.mock!

By Our Powers Combined

Putting these together
provides powerful tools for evaluating cloud providers and implementing
provisioning automation. vCloud is the newest provider in fog and currently has
many low level calls, a few models and no mocks. You can help implement more
support and follow its progress on github here or you can see a more
complete example of fog with vCloud here.

geemus (aka Wesley Beary) is an avid Rubyist and Open
Source enthusiast. When not spending his spare time working on enumerable
overly ambitious open source projects, he spends his days developing the Engine Yard Cloud.

By Steve Jin, VMware engineer

This entry was reposted from “DoubleCloud”
a blog for architects and developers on
virtualization and cloud computing.

SOA has been a buzzword in
enterprise computing for a long time now. Cloud computing is a relatively new
buzzword but it’s gaining momentum as SOA seems to be falling from favor.
Interestingly, with these two terms many people get confused: is cloud
computing just a new name
for SOA? And can they play well together?

Definitions

Before drawing a conclusion,
let’s take a look at the definitions from wikipedia.org:

Service
Oriented Architecture (SOA): is a flexible set of design principles used during
the phases of systems development and integration. A deployed SOA-based
architecture will provide a loosely-integrated suite of services that can be
used within multiple business domains.

Cloud
Computing: is Internet-based computing, whereby shared resources, software and
information are provided to computers and other devices on-demand, like a
public utility.

As we’ve seen in various
articles and books, SOA is mainly for enterprise while the cloud computing
definition suggests it’s for Internet-based services.

What’s Common?

In a previous blog, I mentioned
that cloud as architecture pattern (CAA), which can be leveraged in enterprises as well. If we compare SOA and CAA, they look somewhat similar and share some core principles.

First, both emphasize the
service concept. By definition, a service is performance of work by one for
another. Both CAA and SOA delegate work to other parts of the system, either by
the service provider or other business components. With that delegation, people
can use the services without worrying about the implementation details and
scalability. Most important, services in CAA and SOA can be shared by multiple
applications and users, thus 
optimizing resource utilization.

Second, both promote loose
coupling. Each architecture demands minimum dependencies among
different parts of the system. As a result, any single change on one part of
the system has limited impact on the overall system.

What’s Different?

SOA and CAA differ significantly:

* Horizontal versus vertical services.

The services in SOA mainly focus on business. Each service may represent
one aspect of the business. Combined together, these services consist of a
business application or solution. In this sense, the services are horizontal.

The services in CAA are mainly layered according to typical software
stacks. The lower services support the upper services to deliver applications.
Therefore, I call them vertical services.

* Application vs. infrastructure

SOA is for application
architecture. The dividing of different
components is based on their roles in the SOA applications. More often
than not, you start with a business problem and then abstract out the services.
These services can be re-used by other applications in the future.

Cloud Computing is for IT
delivery. The dividing of different services is based on their roles in a software
stack that is mostly well
defined. You don’t need a problem before defining the cloud services. The
services can be easily re-used by all applications.

Conclusion

SOA and cloud computing share many common principles, but also differ
significantly in their role in IT architecture. SOA is mainly an application
architecture with horizontal services; while cloud computing is an IT
architecture with vertical services.

Given the differences, SOA and cloud computing complement each other
very well. I will talk about SOA over
Cloud: A Perfect Match for Enterprise Computing in a future blog. Stay
tuned.

VMware engineer
Steve Jin is author of VMware VI & vSphere SDK (Prentice Hall), founder of open source VI Java API, and is the chief blogger at DoubleCloud.org

By Steve Jin, VMware engineer

Amazon released its SDK for Java last month. It’s a
complete set of APIs that cover almost all the services from EC2, S3, SQS, RDS
to SimpleDB. Overall Amazon has done a consistent job, but the SDK is NOT really
object oriented. And I think that is critical for higher development
productivity. 

Structure of the SDK

Before explaining why it’s not quite there, let’s take a
look at the structure of the API itself. The SDK includes many packages. For
each service, you find two or three packages: basic, model, and possibly util
package.

In each of the basic packages you can find the two
interfaces, and two implementation classes. Let’s pick EC2 as a sample here.
The AmazonEC2 and AmazonEC2Async are two interfaces and implemented by
AmazonEC2Client and AmazonEC2AsyncClient respectively. More methods are defined
in the synchronous than the asynchronous versions with the majority of them
overlapping with similar method names.

The first two versions wait and get you results upon return.
The third version doesn’t wait and gets the result later.

Overall Amazon has done a consistent job all over the
different types of services. The consistency is always a great attribute for
good API design.

What Should Have Been Done Differently?

If you have done Web Service programming before, you know
the SDK is about the same level as the stub generated by a Web Service toolkit
in terms of abstraction. With that, you have a fat “client” taking care of
everything, including all the possible operations. This is an easy and
straight-forward way for the SDK developers to come up with a SDK for the Web
Services. But it’s not so easy or straight-forward for the actual SDK users.

As most object-oriented (OO) programmers know, it’s a bad
idea to have everything in one fat class. If you have everything in one class,
then it’s not really OO programming. That is actually a typical mistake many
programmers make while transitioning from a procedural language like C to an
object-oriented language like C++.

So what about the classes in the model packages? These
classes are really data objects, whose methods are limited to getters and
setters. In other words, you can’t do anything meaningful with these data
objects. These data objects serve as the parameter and return types for the
methods in the two classes: one for synchronous and the other for asynchronous
calls. Some of them are actually over-used.

Let’s take a concrete example. The following is a method
from AmazonEC2Client class:

The DeleteSubnetRequest is a data object which holds really
one string as a subnet ID. It would be better to have the signature as:

This is just a minor issue anyway. The bigger issue is that
the SDK doesn’t have real Subnet class to which you can add methods like
create(), delete(). If yes, you will find the application code can be as
straight-forward and easy as this:

Lack of Clear Object Model

The root problem with Amazon’s approach is the lack of a
clear object model, which I believe is the core of every successful OO API
design.

Nowadays, Web Services and REST are pretty popular. By
nature, they are all procedural. Focusing on these can easily shift your
attention from the core design to the design of the requests/responses schema,
URLs etc. Then you can easily lose the vision of the forest for the individual
trees.

The lack of an object model is not a big deal when the API
scope is small. When it gets bigger, we will see clear usability issues for the
API users.

VMware vSphere API has a great object model as described in
its API reference, but gets lost in the Web Services SDK due to the inherent limitation
of Web Services. Because the vSphere API is a lot more complicated than Amazon’s
SDK, users find it difficult to use. If you’re curious to learn more about how
VMware sets out to solve these same problems as Amazon’s latest AWS SDK, first
check out the open source vSphere Java API.
It re-creates the object model and for first-timers it can be much easier to
use than the vSphere Web Services.

VMware engineer Steve Jin is author of VMware
VI & vSphere SDK (Prentice Hall), founder of open source VI
Java API, and is the chief blogger at DoubleCloud.org

You’re invited! Come join VMware vCloud and our
partner Terremark at the infamous Irish
Rogue pub and restaurant in New York City on April 20 from 6:30 – 8pm for
our first-ever vCloud Tweetup.

Register here
to reserve your seat by the fireplace!

Located at 356 W 44th Street in the Hells Kitchen
neighborhood, what better place for cloud heads to gather? Here you can
casually meet and greet some of the founding members of the vCloud ecosystem,
including:

*  William
Shelton, Director of Cloud Computing and Virtual Appliances at VMware

*  Jason
Lochhead, CTO of Hosting Services at Terremark Worldwide

*  Simon
West, Chief Marketing Officer at Terremark Worldwide

*  Adrian
Cole, founder of the open source jclouds framework and the cloudhackers
collective (see his video
talking about the benefits of using Cargo to automate app deployment to vCloud
Express)

Be sure to ask about how vCloud sets apart from the
rest, and what's new on the horizon…. and be sure to thank VMware and
Terremark for the snacks and drinks!

To keep
current on all vCloud Tweetups and other vCloud activities, follow us on
Twitter @vcloud or become a fan of our vCloud
Facebook page.

jclouds project founder Adrian Cole (see his video
talking about the benefits of using Cargo to automate app deployment to vCloud
Express) updates us on recent cloud discussions that we want to share with our
vCloud blog fans.

This week cloudhackers from
around the world met up in person and online for our first ever hack day. 
The epicenter was in San Francisco where the founding cloudhacker-SF meetup group
crashed CloudKick's headquarters for
the day.  cloudhackers-MTL
congregated at Station-C, and all our
distributed friends met up on freenode irc #cloudhackers.  

By 1pm, CloudKick's desks,
futons, and beanbags were full of hackers, attracted in no small part by an
extremely heavy order of 19 burritos, snacks and drinks, all compliments of VMware
vCloud Express.  At the same time, our irc channel was full of
commotion.  It was a combination of newcomers, and lots of cool stuff from
cloudseed and Adam Jacob, CTO from Opscode.  So much chatter, that I think
we overran geemus' irc client

Our goal was to add
content to clouddemo on github so that users could figure out what tools they
want without already knowing their names.  For example, some people know
they need to use chef and would prefer to use ruby.  Accordingly, we
decided to name cloudseed's
demo “cloudseed-bootstrap-chef-ruby.md.” Pretty easy, right?

The afternoon continued
with several of us hacking improvements to our tools or explaining our tools to
others. Wesley Beary completed Terremark vCloud support in his ruby library
called Fog, which was amazing
progress in just a couple hours.  The dotcloud guys teamed up with
libcloud for an interesting discussion on cloud images continued on the
libcloud mailing list later that night.

One of the coolest things
we were able to accomplish was to decide on a logo!  Emie Lee from design
shop byemie.com
donated a day of creative services to our group and created a few options for our
logo.  After some local to San Francisco deliberation and vetting via irc,
we decided on the logos below (listed also on the vCloud
Facebook page): one is for Twitter and the
second for general use.  We love it and I hope you like it too!

It was really fun to have
everyone online, and great that Hugo Duncan
organized the Montreal
cloudhackers to join us from Canada.  Feedback has been really
positive, so be sure we'll do it again.

Let
me know if you want to get involved with cloudhackers. Comment below or share
your thoughts with me on Twitter at @jclouds or at @vcloud. We’ll also be
talking about cloudhackers (with some photos) on the vCloud
Facebook page. Become a fan!

Cloudhacker Twitter logo

Cloudhacker general use logo

jclouds
project founder Adrian Cole (see his video talking about the benefits
of using Cargo to automate app deployment to vCloud Express) updates us on
recent cloud discussions that we want to share with our vCloud blog fans.

First Redis
Meetup

Engine Yard recently held a meetup on Redis,
the first in the world I hear. About 50 developers attended. It was a very very
very geeky crowd.  Very C, Ruby and Python centric, too, with some Clojure
fans as well. Java was not mentioned in discussions or asides.  

Developers are very happy with Redis, which
is essentially an atomic data structure server. They love the ease of creating
tools on its API, which was lamented as a problem in and of itself, since
everyone does it differently.  This “differently” problem surfaced in the
naivety of Redis, which doesn't handle scaling well.  For example, there
is only one master of data, so people implement sharding in the clients and
make up their own code (consistent hash) to pass requests to the right server.
 Not only is this difficult to get right, but since they are doing it
differently, you cannot use the perl Redis client on a Redis server being
accessed by the ruby one, since they don't use the same sharding approach. This
part feels very immature, but very consistent with the low clustering aptitude
present in other things like memcached.  

Redis is
Fast to Start

However limiting, it didn't seem
to be a big deal for a few reasons. One is that most developers cannot or do
not have the patience to setup traditional data grids.  Redis apparently
takes under two minutes to get running.  The other is that developers are
using Redis for extremely focused bits and only using one language.  For
example, Engine Yard is using Redis to deal with the polling problems of cloud
APIs like EC2.  Essentially, their pollers hit the Redis server, and update
only if a state change happened (like running to terminated).  Their other
code only pays attention to Redis, and therefore doesn't spam on multiple
pollers saying the same thing is terminated.  This is a very focused use
and all their code is ruby anyway, so it isn't a big deal. Moreover the data
size is measured in kilobytes so sharding is not useful.  Other uses were
to deal with deduping redundantly delivered messages.  This was due to
rabbit mq not having a good HA story outside erlang itself.  Ezra Zygmuntowicz, a co-founder at Engine Yard, basically sets up redundant exchanges and has some
wrapper code to use Redis as a distributed lock.  Craigslist uses Redis as
a queue for spam filtering. There were a couple mentions of removing rabbitmq
with Redis in the future, as provoked by the audience.  The audience
absolutely loved Redis founder Salvatore Sanfilippo for being very responsive
24 hours a day and keeping the API easy to use and install.

Let me know if you’ve played with Redis yet. What do you
think of it? Comment below or share your thoughts with me on Twitter at
@jclouds or vCloud at @vcloud. We’ll also be talking about Redis on the vCloud Facebook page. Become a
fan!