Home > Blogs > VMware Consulting Blog > Tag Archives: Michael Francis

Tag Archives: Michael Francis

Hybrid Cloud and Hybrid Cloud Manager

Michael_FrancisBy Michael Francis

Disclaimer: This blog is not a technical deep dive on Hybrid Cloud Manager; it talks to the components of the product and the design decisions around the product. It assumes the reader has knowledge of the product and its architecture.

Recently, I have been involved with the design and deployment of Hybrid Cloud Manager for some customers. It has been a very interesting exercise to work through the design and the broader implications.

Let’s start with a brief overview of Hybrid Cloud Manager. Hybrid Cloud Manager is actually comprised of a set of virtual appliances that reside both on-premises and in vCloud Air. The product is divided into a management plane, control plane, and data plane.

  • The management plane is instantiated by a plugin in the vSphere Web Client.
  • The control plane is instantiated by the Hybrid Cloud Manager virtual appliance.
  • The data plane is instantiated by a number of virtual appliances – Cloud Gateway, Layer 2 Concentrator, and the WAN Opto appliance.

The diagram below illustrates these components and their relationships to each other on-premises and the components in vCloud Air.

MFrancis_Logical Architecture Hybrid Cloud Manager

Figure 1 – Logical Architecture Hybrid Cloud Manager

The Hybrid Cloud Manager provides virtual machine migration capability, which is built on two functions: virtual machine replication[1] and Layer 2 network extension. The combination of these functions provides an organization with the ability to migrate workloads without the logistical and technical issues traditionally associated with migrations to a public cloud; specifically, the outage time to copy on-premises virtual machines to a public cloud, and virtual machine re-addressing.

During a recent engagement that involved the use of Hybrid Cloud Manager, it became very obvious that even though this functionality simplifies the migration, it does not diminish the importance of the planning and design effort prior to any migration exercises. Let me explain.

Importance of Plan and Design

When discussing a plan, I am really discussing the importance of a discovery that deeply analyses
on-premises virtual workloads. This is critical, as the Hybrid Cloud Manager creates such a seamless extension to the on-premises environment, we need to understand:

  • Which workloads will be migrated
  • Which networks the workloads reside on
  • What compute isolation requirements exist
  • How and where network access control is instantiated on-premises

Modification of a virtual network topology in Public Cloud can be a disruptive operation; just as it is in the data center. Introducing an ability to stretch layer 2 network segments into the Public Cloud and migrating out of a data center into Public Cloud increases the number of networks and the complexity of the topology of the networks in the Public Cloud. So the more planning that can be done early the less likely disruptions to services will need to occur later.

One of the constraints in the solution revolves around stretching layer 2 network segments. A Layer 2 network segment located on-premises can be ‘stretched’ to one virtual data center in vCloud Air. So we have some implications of which workloads exist on a network segment, and which vCloud Air virtual data centers will be used to host the workloads on the on-premises network segment. This obviously influences the creation of virtual data centers in vCloud Air, and the principals defined in the design, which influence when additional virtual data centers are stood up – compared with growing an existing virtual data center.

Ideally, an assessment of on-premises workloads would be performed prior to any hybrid cloud design effort. This assessment would be used to size subsequent vCloud Air virtual data centers; plus, it would discover information about the workload resource isolation that drives the need for workload separation into multiple virtual data centers. For instance, the requirement to separate test/development workloads from production workloads with a ‘hard’ partition would be one example of a requirement that would drive a virtual data center design.

During this discovery we would also identify which workloads reside on which networks, and which networks require ‘stretching’ into vCloud Air. This would surface any issues we may face due to the constraint that we can only stretch a Layer 2 segment into one virtual data center.[2] This assessment really forms the ‘planning’ effort in this discussion.

Design Effort

The design effort involves designs for vCloud Air and Hybrid Cloud Manager. I believe the network design of vCloud Air is a critical element. We need to determine whether to use:

  • Dynamic routing or static routing
  • Subnet design and its relationship to routing summarization
  • Routing paths to the Internet
  • Estimated throughputs required for any virtual routing devices
  • Other virtual network services
  • Egress optimization functionality from Hybrid Cloud Manager
  • And finally, we need to determine where security network access points are required

The other aspect is the design of the virtual compute containers, such as virtual data centers in vCloud Air. The design for vCloud Air should define the expected virtual data center design over the lifecycle of the solution. It would define the compute resource assignment to each virtual data center initially, and over the lifecycle as anticipated growth is factored in. During the growth of use, the requirements for throughput will increase on the networking components in vCloud Air, so the design should articulate guidance around when an increase in size of virtual routing devices will need to occur.

The vCloud Air platform is an extension of the on-premises infrastructure. It is a fundamental expectation that operations teams have visibility into the health of the infrastructure, and that capacity planning of infrastructure is taking place. Similarly, there is a requirement to ensure that the vCloud Air platform and associated services are healthy and capacity managed. We should be able to answer the question, “Are my virtual data center routing devices of the right size, and is their throughput sufficient for the needs of the workloads hosted in vCloud Air?” Ideally we should have a management platform that treats vCloud Air as an extension to our on-premises infrastructure.

This topic could go much deeper, and there are many other considerations as well, such as, “Should I place some management components in vCloud Air,” or, “Should I have a virtual data center in vCloud Air specifically assigned to host these management components?”

I believe today many people take an Agile approach to their deployment of public cloud services, such as networking and virtual compute containers. But I believe if you are implementing such a hybrid interface as offered by Hybrid Cloud Manager, there is real benefit to a longer term view to the design of vCloud Air services to minimise risk if we paint ourselves into a corner in the future.

Some Thoughts on Hybrid Cloud Manager Best Practices

Before wrapping up this blog, I wanted to provide some thoughts on some of the design decisions regarding Hybrid Cloud Manager.

In a recent engagement we considered best practices for placement of appliances, and we came up with the following design decisions.

MFrancis_Design Decision 1

MFrancis_Design Decision 2

MFrancis_Design Decision 3

Key Takeaways

The following are the key takeaways from this discussion:

  • As Hybrid Cloud Manager provides a much more seamless extension of the on-premises data center, deeper thought and consideration needs to be put into the design of the vCloud Air public cloud services.
  • To effectively design vCloud Air services for Hybrid Cloud requires a deep understanding of the on-premises workloads, and how they will leverage the hybrid cloud extension.
  • Network design and ongoing network access controlling operational changes need to be considered.
  • Management and monitoring of the vCloud Air services acts as an extension of the data center needs to be included in the scope of a Hybrid Cloud solution.

[1] Leverages the underlying functionality of vSphere Replication; but isn’t a full vSphere Replication architecture.

[2] This constraint could be overcome; however, the solution would require configurations that would make other elements of the design sub-optimal; for example, disabling the use of egress optimization.


Michael Francis is a Principal Systems Engineer at VMware, based in Brisbane.

New! Making Your Move to the Cloud – Part 2: Best Practices for Building a Migration Plan

In Part 2 of the “Making Your Move to the Cloud” series, Michael Francis and RadhaKrishna (RK) Dasari discuss best practices for building a Cloud migration plan for your organization.migration to cloud

“The primary technological goal of any migration project is to transfer an existing compute resource/application from one environment to another, as quickly, efficiently, and cost effectively as possible. This is especially critical when considering a migration to a public cloud; considerations must include security controls, latency and subsequent performance, operations practices for backup/recovery and others. Though VMware now provides technology in Hybrid Cloud Manager to minimize any downtime or eliminate downtime; these considerations persist. VMware Professional Services often assist customers with assessing their environment and focusing on generating an effective, actionable migration plan to achieve this goal. It adopts the philosophy of spending the time up front in the plan to reduce the time and increase likelihood of success of a subsequent migration project.”

Read the full blog for more information: http://vmw.re/1LhuAQ8 

New! Making Your Move to the Cloud – Part 1: Building a Structured Approach to the Cloud

profservices

 

“If you’re considering moving to the cloud, you no doubt recognize that cloud computing represents the opportunity to minimize overhead and maximize agility so IT can deliver significant value back to the organization. By leveraging a more agile and efficient IT environment that leverages pools of elastic, self-managed virtual infrastructure—consumed as a service at the lowest possible cost—you can innovate more rapidly while maintaining reliability, security and governance for all applications and services under IT control.”

In this new blog post, Michael Francis and RadhaKrishna (RK) Dasari share their approach to the cloud. This four-part weekly series will help continue the discussion about the benefits of the cloud, address challenges, and provide tips and best practices.

Read the full blog here.

DevOps and Performance Management

Michael_Francis

By Michael Francis

Continuing on from Ahmed’s recent blog on DevOps, I thought I would share an experience I had with a customer regarding performance management for development teams.

Background

I was working with an organization that is essentially an independent software vendor (ISV) in a specific vertical; their business is writing software in the gambling sector, and⎯in some cases⎯hosting that software to deliver services to their partners. It is a very large revenue stream for them, and their development expertise and software functionality is their differentiation.

Due to historical stability issues and lack of trust between the application development teams and the infrastructure team, the organization introduced into the organization a new VP of Infrastructure and an Infrastructure Chief Architect a number of years previous. They focused on changing the process and culture − and also aligning the people. They took our technology and implemented an architecture that aligned with our best practices with the primary aim of delivering a stable, predictable platform.

This transformation of people/process and technology provided a stable infrastructure platform that soon improved the trust and credibility of the infrastructure team with the applications development teams for their test and development requirements.

Challenges

The applications team in this organization, as you would expect, carries significant influence. Even though the applications team had come to trust virtual infrastructure for test and development, they still had reservations about a private cloud model for production. Their applications had significant demands on infrastructure and needed to guarantee transactions per second rates committed across multiple databases; any latency could cause significant processing issues, and therefore, loss of revenue. Visibility across the stack was a concern.

The applications team  responsible for this critical in-house developed application designed the application to instrument it’s performance by writing out flat files on each server with application-specific information about transaction commit times and other application specific performance information.

Irrelevant of complete stack visibility, the applications team responsible for this application was challenged with how to monitor the performance of this custom distributed application performance data from a central point. The applications team also desired some means of understanding normal performance data levels, as well as a way to gain insight into the stack to see where any abnormality originated.

Due to the trust that had developed with the infrastructure team, they engaged with them to determine whether the infrastructure team had any capability to support their performance monitoring needs.

Solution

The infrastructure team was just beginning to review their needs for performance and capacity management tools for their Private Cloud. The team had implemented a proof-of-concept of vCenter Operations Manager and found its visualizations useful; so they asked us to work with the applications team to determine whether we could digest this custom performance information.

We started by educating them on the concept of a dynamic learning monitoring system. It had to allow hard thresholds to be set, but also⎯more importantly⎯determine the spectrum of normal behavior based upon data pattern prediction algorithms for an application; both as a whole and each of its individual components.

We discussed the benefits of a data analytics system that could take a stream of data, and
irrespective of the data source, create a monitored object from it. The data analytics system had to be able to assign the data elements in the stream to metrics, start determining normality, provide a comparison to any hard thresholds, and provide the visualization.

The applications team was keen to investigate and so our proof-of-concept expanded to include the custom performance data from this in-house developed application.

The Outcome

The screenshot below shows VMware vCenter Operations Manager. It shows the Resource Type screen that allows us to define a customer Resource Type, which allows us to represent the application-specific metrics and the application itself.

MFrancis1

To get the data into vCenter Operation Manager we simply wrote a script that opened the flat file on each of the servers participating in the application; it read the file and then posted the information into vCenter Operations Manager using its HTTP POST adapter. This adapter provides the ability to post data from any endpoint that needs to be monitored; because of this vCenter Operations Manager is a very flexible tool.

In this instance we posted into vCenter Operation Manager a combination of application-specific counters and Windows Management Instrumentation (WMI) counters from the Windows operating system platform the apps run on. This is shown in the following screenshot.

MFrancis2

You can see the Resource Kind is something I called vbs_vcops_httpost, which is not a ‘standard’ monitored object in vCenter Operations Manager; the product has created this based on the data stream I was pumping into it. I just needed to tell vCenter Operations Manager what metrics it should monitor from the data stream – which you can see in the following screenshot.

 MFrancis3

For each attribute (metric) we can configure whether hard thresholds are used and whether vCenter Operations Manager should use that metric as an indicator of normality. We refer to the normality as dynamic thresholds.

Once we have identified which metrics we want to mark, we can create spectrums of normality for them and affect the health of the application, which allows us to create visualizations. The screenshot below shows an example of a simple visualization. It shows the applications team a round-trip time metric plotted over time, alongside a standard windows WMI performance counter for CPU.

MFrancis4

In introducing the capabilities to monitor custom in-house developed applications using combinations of application-specific custom metrics, a standard guest operating system and platform metrics, the DevOps team now has visibility into the health of the whole stack. This enables them to see the impact of code changes against different layers of the stack so they can compare the before and after from the perspective of the spectrum of normality for varying key metrics.

This capability from a cultural perspective brought the applications development team and infrastructure team onto the same page; both teams gain an appreciation of any performance issues through a common view.

In my team we have developed services that enable our customers to adopt and mature a performance and capacity management capability for the hybrid cloud, which⎯in my view―is one of the most challenging considerations for hybrid cloud adoption.

 


Michael Francis is a Principal Systems Engineer at VMware, based in Brisbane.

SDDC is the Future

Michael_Francis

 

 

By Michael Francis

 

VMware’s Transformative Growth

Over the last eight years at VMware I have observed so much change, and in my mind it has been transformative change. I think about my 20 years in IT and the changes I have seen, and feel the emergence of virtualization of x86 hardware will be looked upon as one of the most important catalysts for change in information technology history. It has modified the speed of service delivery, the cost of that delivery and subsequently has enabled innovative business models for computing – such as cloud computing.

I have been part of the transformation of our company in these eight years; we’ve grown from being a single-product infrastructure company to what we are today – an application platform company. Virtualization of compute is now mainstream. We have broadened virtualization to storage and networking, bringing the benefits realized for compute to these new areas. I don’t believe this is incremental value or evolutionary. I think this broader virtualization―coupled with intelligent, business policy-aware management systems―will be so disruptive to the industry that it will be considered a separate milestone potentially, on par with x86 virtualization.

Where We Are Now

Here is why I think the SDDC is significant:

  • The software-defined data center (SDDC) brings balance back to the ongoing discussion between the use of public and private computing.
  • It enables the attributes of agility, reduced operational and capital costs, lower security risk, and a new of stack management visibility.
  • SDDC not only modifies the operational and consumption model for computing infrastructure, but it also modifies the way computing infrastructure is designed and built.
  • Infrastructure is now a combination of software and configuration. It can be programmatically generated based on a specification; hyper-converged infrastructure is one example of this.

As a principal architect in VMware’s team responsible for the generation of tools and intellectual property that can assist our Professional Services and Partners to deliver VMware SDDC solutions, the last point is especially interesting and the one I want to spend some time on.

How We Started

As an infrastructure-focused project resource and lead over the past two decades, I have become very familiar developing design documents and ‘as-built’ documentation. I remember rolling out Microsoft Windows NT 4.0 in 1996 on CDs. There was a guide that showed me what to click and in what order to do certain steps. There was a lot of manual effort, opportunity for human error, inconsistencies between builds, and a lot of potential for the built item to vary significantly from the design specification.

Later, in 2000, I was a technical lead for a systems integrator; we had standard design document templates and ‘as-built’ document templates, and consistency and standardization had become very important. A few of us worked heavily with VBScript, and we started scripting the creation of Active Directory configurations such as Sites and Services definitions, OU structures and the like. We dreamed of the day when we could do a design diagram, click ‘build’, and have scripts build what was in the specification. But we couldn’t get there. The amount of work to develop the scripts, maintain them, and modify them as elements changed was too great. That was when we focused on the operating stack and a single vendor’s back office suite; imagine trying to automate a heterogeneous infrastructure platform.

It’s All About Automated Design

Today we have the ability to leverage the SDDC as an application programming interface (API) that abstracts not only the hardware elements below and can automate the application stack above― but can abstract the APIs of ecosystem partners.

This means I can write to one API to instantiate a system of elements from many vendors at all different layers of the stack, all based on a design specification.

Our dream in the year 2000 is something customers can achieve in their data centers with SDDC today. To be clear – I am not referring to just configuring the services offered by the SDDC to support an application, but also to standing up the SDDC itself. The reality is, we can now have a hyper-converged deployment experience where the playbook of the deployment is driven by a consultant-developed design specification.

For instance, our partners and our professional services organization has access to what we refer to as the SDDC Deployment Tool (an imaginative name, I know) (or SDT for short). This tool can automate the deployment and configuration of all the components that make up the software-defined data center. The following screenshot illustrates this:

MFrancis1

 

Today this tool deploys the SDDC elements in a single use case configuration.

In VMware’s Professional Services Engineering group we have created a design specification for an SDDC platform. It is modular and completely instantiated in software. Our Professional Services Consultants and Partners can use this intellectual property to design and build the SDDC.

What Comes Next?

I believe our next step is to architect our solution design artifacts so the SDDC itself can be described in a format that allows software―like SDT―to automatically provision and configure the hardware platform, the SDDC software fabric, and the services of the SDDC to the point where it is ready for consumption.

A consultant could design the specification of the SDDC infrastructure layer and have that design deployed in a similar way to hyper-converged infrastructure―but allowing the customer to choose the hardware platform.

As I mentioned at the beginning, the SDDC is not just about technology, consumption and operations: it provides the basis for a transformation in delivery. To me a good analogy right now is the 3D printer. The SDDC itself is like the plastic that can be molded into anything; the 3D printer is the SDDC deployment tool, and our service kits would represent the electronic blueprint the printer reads to then build up the layers of the SDDC solution for delivery.

This will create better and more predictable outcomes and also greater efficiency in delivering the SDDC solutions to our customers as we treat our design artifacts as part of the SDDC code.


Michael Francis is a Principal Systems Engineer at VMware, based in Brisbane.