This is a guest blog post by anynines, a Cloud Foundry hosted cloud provider from Europe.
In the past few years, the perception of cloud hosting has been revolutionized by the growth of IaaS and PaaS environments. However, many of the major hosting companies are US-based. In order to assure that data subject to European Union data privacy regulations stays in the EU, it might be important for some users to choose an IaaS that is completely EU-based (unlike AWS, for example).
We are committed to solving this challenge. Say “hello” to anynines – a solution for European cloud enthusiasts (or any others!) looking for safe, 100% European PaaS hosting.
Matching IaaS to your PaaS
Firstly, we needed an IaaS that was EU-based and that worked with our PaaS.
We had two strong IaaS layers to choose from: OpenStack, and VMware vSphere. Both of these are popular and work with Cloud Foundry, BOSH and Fog. Anynines actually runs both vSphere and OpenStack. Our core components are powered by VMware, and our Swift storage service is a part of a separate OpenStack setup.
Nearly every standard web application requires some kind of file storage, and we felt that we needed to find an AWS S3 replacement for our PaaS users.
Why OpenStack Swift?
OpenStack Swift offers a REST API to store and retrieve files or any unstructured data at low cost. Highly scalable, a Swift setup can consist of anything from a few, up to thousands of machines, handling dozens of petabytes of storage… without a single point of failure!
Another important aspect is that Swift also offers scalable file access. Store massive data – access it massively. Simple!
Implementing a Cloud Foundry service
Starting to implement a Cloud Foundry OpenStack Swift service was a bit challenging, as Cloud Foundry itself is being improved every day. However, with the help of Pivotal and their open door policy, we received a lot of help during a two week stay in San Francisco. Thanks, folks!
Now that the story is over, it’s time to share our “recipe”: an implementation of OpenStack Swift as a service for Cloud Foundry applications.
Types of Cloud Foundry services
First, we needed to decide if the service would manage credentials only or also be responsible for provisioning service instances. Provisioning service instances means that the service implementation will be responsible for creating service processes, e.g. spinning up Redis instances. These kind of scenarios can become cumbersome, depending on whether you want isolation between service instances by using Warden containers, and high availability in case a service node breaks down.
The management of OpenStack Swift is out of the scope of Cloud Foundry in this case, so our service implementation fell into the first category: managing credentials only. Therefore, the service implementation required a running OpenStack instance somewhere in or near the datacenter.
How the Swift Service is made
What about the basic service structure? Current Cloud Foundry services rely on VCAP Services Base – a repository with base code for all current service implementations.
These base classes provide a separation between a service gateway and service nodes.
While the service gateway is responsible for the communication with the Cloud Controller, the service nodes represent service plans, and are responsible for the actual service work. The latter could be, for instance provisioning Redis instances, or in our case talking to OpenStack to create new tenants and users.
Implement from scratch, or inherit?
As Cloud Foundry moves forward, communication between the Cloud Controller and the service gateway may change. Your service implementation should always be tested to be compliant with the current Cloud Controller’s interface.
Cloud Foundry developers and operators who want to integrate Swift as a native Cloud Foundry service could start with the VCAP::Services::Base class. Eventually, they may wish to revisit the minimum viable implementation to remove code related to multi-tenant service provisioning with Warden, since Swift manages tenants outside Cloud Foundry service nodes. We thought about building a new, more lightweight class, but for a minimal viable Swift service and an initial service implementation, VCAP::Services::Base delivered a scaffold to start on, and enabled us to focus on the implementation logic of our new service.
What we needed to implement
The major implementation work in developing a service breaks down into building four key methods: [un]provision(), and [un]bind().
- During service provisioning, a new Cloud Foundry service instance is created. For MySQL this might be a database, and for Redis a new instance.
- On bind, user credentials are generated, granting access to the resource. For MySQL, a user is allowed to access the corresponding database, for example.
Along with these methods, a credential generating method is required. You might also want to store some service metadata in a service database, such as tenant names for maintenance and/or caching purposes.
OpenStack Swift service for Cloud Foundry
Now that we have looked at how to create a Cloud Foundry service in general, we can focus on how we built the OpenStack Swift service.
We created a basic implementation in order to make a running OpenStack Swift easy. Cloud Foundry users don’t have to struggle with OpenStack itself, but can just use the service right away. The Swift service assumes an OpenStack environment is up and running already.
The overall architecture and control flow for our Cloud Foundry Swift service looks like this:
OpenStack tenants, users and role assignments
As the name implies, in OpenStack a tenant, also referred to as a project, is a construct isolating resources between different tenants, such as different Cloud Foundry organisations or users.
In our case, when creating a Cloud Foundry service, individual service instances needed to be isolated from one another.
Therefore:
- on provision we create a new tenant.
- on binding an app to a service we create a user.
This allows us to bind multiple apps to a single Swift tenant, enabling us to share files between the apps. By creating multiple users, apps can have permissions to access a certain Swift account granted or revoked.
Fog and storage providers
A Ruby gem named Fog takes care of interactions between the service and OpenStack.
Tenant, user and role operations are done using the Identity class of the OpenStack provider, but operations that access the storage (Swift) have been implemented using the HP provider. This is because we found the HP storage provider to be more stable than the OpenStack provider as HP’s cloud actually uses a slightly modified OpenStack, underneath. However, it turned out that there’s a slight incompatibility between OpenStack Swift and HP’s flavor of it when it comes to generating temporary URLs.
Temporary URLs are required when you want a third party to access a file in a private directory. In this case, a temporary valid URL could be generated that allowed easy GET access to a file that would otherwise require a proper authorization header. Fortunately, the Fog and HP provider maintainers were more than helpful and gave good feedback while integrating a corresponding pull request. The patch allows OpenStack Swift and anynines Swift service users to pass in a so-called account meta temp url key, and thus use Fog to generate temporary URLs without this issue becoming a problem.
Using the Swift Service
For a detailed description on how to use the anynines Swift service within applications, have a look at our Swift howto.
The Cloud Foundry Swift Service is Open Source, and the code can be found on github. You can try it out with your applications today on anynines.com.
The Future
Next up, we’ll set our sights on plan and quota management. A very simple approach could be to have two plans. A very restricted basic plan with a limited storage quota and an unlimited plan that will be billed pay-as-you go.
There are more questions to consider: Swift integration using BOSH. Standalone Swift without OpenStack Keystone and Nova – could we wrap it up using BOSH, and make it a part of a Cloud Foundry release? This would ease the overall management of the PaaS, as BOSH remains the primary management tool. However, Swift is by nature a very I/O intense system. Would the I/O overhead caused by virtualization be acceptable? Or should it remain on bare metal? We look forward to working with the community to address these discussions!
