Nothing speaks to the value of a solution, or helps the prospective users get a glimpse into their future experience like a real-life customer success story. This is why vendors never miss a chance (and work hard) to publish customer case studies when a good one comes around. However, these one-pagers or two-minute videos don’t often do justice to customers who put their time, enthusiasm, and yes – money – to bring their projects to success, and then agree to share their experience with the world. As we say, the devil is in the details. In our quest to provide effective content in the best tradition of Hi-Tech marketing, we sometimes lose the depth and the excitement that hides between the printed lines or video frames.
Just like with any other customer case study, we have a 2-minute version, and you are welcome to watch it.
We do believe, however, that a deep dive would be a lot more interesting. With that in mind, I’d like to bring to you the story of Vistra Energy – as told in their own words by the three key people who were instrumental to the project.
The Vistra Energy Customer Case Study
Vistra Energy (formerly Energy Future Holdings) is a consumer and commercial energy provider based in the state of Texas, and are the largest generator and retailer of electricity in this fast-growing and highly-competitive market. Their integrated portfolio of competitive businesses consists primarily of Luminant and TXU Energy. Luminant generates and sells electricity and related products from their diverse fleet of generation facilities totaling approximately 17,000 MW of generation in Texas, including 2,300 MW fueled by nuclear power, 8,000 MW fueled by coal and 6,000 MW fueled by natural gas, and it is a large purchaser of wind-generated electricity as well. TXU provides retail, consulting and cost saving services.
In 2014, Vistra Energy embarked on a major IT transformation project, and VMware was honored to be selected as the vendor of choice with full range of our SDDC solutions. Recently we met and spoke with three of Vistra Energy’s IT leaders who agreed to share their insights with us:
- Paul Reyes, Chief Information Security Officer, VP of Infrastructure and Operations
- Chris Cantu, Sr. Managing Architect
- David Blasingame, Sr. Managing Architect
This article presents their story, in their own words, as they tell us about the goals, the solution, the implementation, and the results of this project.
The Project Goals and the Business Case
The original goal for Vistra Energy’s two business units was to drive down costs. As Paul Reyes explains, “Energy is a very competitive industry now, with prices under $20 per megawatt of power. In this cut-throat market, the IT teams are challenged to support the LOB to lower their cost and make sure they provide their service with the highest margin possible, and do that with the highest levels of quality service.”
“In 2014 we were challenged to lower our costs in IT, so we looked at many different ways to provide cost savings initiatives to our BUs.” – Paul Reyes
One area was reducing the h/w and s/w maintenance and labor costs from an outsource prospective, and increase efficiency while providing a lower-cost service to customers. Vistra Energy was looking for massive savings, so they looked at the mission-critical environment, which is the largest environment handling the majority of the maintenance, license, and outsource costs.
Paul goes on to say:
“We took an initiative to automate and consolidate into a converged environment with dynamic provisioning services. That equated to $18-20M investment. In return, what we said to the business was: if we invest this amount into IT, we will produce $54M return on investment, in the form of reduction of maintenance costs, license costs, and [outsourced] labor. This is how we created a business case focused on cost reduction while providing similar or better services to the LOB.
While the original task was lowering our utilization of resources, both physical and labor, another technical challenge that we had to solve was: are we going to improve quality, or reduce quality? Are we going to improve time-to-market, or reduce time-to-market? And while doing all that, we had to consolidate 900+ various systems into one footprint. But doing that, in turn, puts all our eggs in one basket. So, we had to mitigate through virtual isolation and separation of our BUs, for application stacks, and license models.
Vistra Energy had to consolidate 900+ various systems, and automate provisioning across 11 different end points. The new architecture allowed us to dynamically provision our resources with VMware products, and enabled us to grow and shrink on demand with the business.”
More on that in a second…
The Major Transition
The VMware products that Vistra Energy selected include most of the VMware virtualization and automation stack. After this project, the virtualization levels reached 95-98%, with vRealize Suite Enterprise providing automation and orchestration of the key services’ provisioning throughout the entire system.
Chris Cantu, who is responsible for operations and keeping the critical systems up and running, recalls the situation before and after, “We were a typical IT silo’ed shop with network team, security team, and compute and virtualization teams.”
“With vRealize Automation and NSX, we really got a homogenized view of IT.” – Chris Cantu
“Roles that traditionally only focused on the network side, now can focus on security side, and vice versa. We also see that in the compute and storage areas: we don’t have a traditional storage team anymore, and we did before. It’s not that the storage team or administrators went away, but they are also now looking at the compute, and the compute team members that focused purely on the virtualization side, are now looking into vSAN administration, as well as NSX. We don’t have lines between these functions anymore, as they all are software-defined abstractions that are managed through similar skillsets and technology.
There is definitely a transition within the organization as far as the placement of a workload. The idea of VMware being just virtual machines within your data center is long past. We are certainly looking at public cloud computing, a hybrid model. In order to do that you need to have that agility and flexibility to move workloads across the cloud providers.
Now, as we support both our BUs with all their critical services, we support “four nines” availability applications for our users. At the end of the day, we hold all the applications that support our business. All applications run in a virtual form.” – Chris Cantu
Why VMware
“We don’t view VMware as a vendor, we view them as a partner. They are embedded in our team. When we call for critical support, we know they are on the phone even before my own team members are. So they are really an extended part of our team, proactively working with us. Other vendors do that too, but our view is that VMware is up there pretty much number one.” – Chris Cantu
Another important criteria that contributed into the final decision was the fact that VMware could be delivered as part of the Dell-EMC’s Enterprise Private Cloud pre-integrated solution. A long-time EMC customer, Vistra’s management team felt confident that the ROI of the overall project would be higher and realized much faster when most of the critical components come from the proven source.
VMware is not the only vendor out there, and customers have choice. The selection process that Vistra Energy has gone through brought them to VMware, but not immediately. They did try other products that implement similar functionality. As David Blasingame tells us, they did have problems with other solutions. Most of the issues they came across were around delivery, in particular inconsistent delivery of a VM or OS, items that were missed and had to be added manually, which defeats the purpose of automation.
“What I like about VMware is that it provides an integrated stack from the hypervisor to software-defined networking, to automation.” – David Blasingame
As Vistra Energy was solving a time-to-market problem, and also a consistent delivery, in the end they selected VMware vRealize Suite. Over the course of the project, David Blasingame and his team implemented a number of use cases around intelligent operations, IT automation, and DevOps.
“VMware has provided a standard method of delivery, and very consistent. There is no company that provides that, from what we looked at.” – David Blasingame
Listen to David Blasingame sharing more of his experience of working with VMware
The Implementation and the Results
The implementation was very deep. For example, with vRealize Automation, Vistra Energy integrated it with 11 different end-points, including security scans, anti-virus, tripwire, and several different things. The critical part of the automation project—development of end-point interfaces split equally between Vistra Energy and VMware. Vistra Energy actually had to rebuild their entire organizational structure around automation and hire an automation engineer who could implement “infrastructure as code” through vRealize Automation. They also had to change their entire policies around change management. Now that they have infrastructure on demand, they no longer need to have a two-week window to accommodate all the manual provisioning and approval steps. Now it’s 20 minutes.
“If you are not looking at automation today, you are already a year behind, or more…” – Chris Cantu
On the operations side, Vistra Energy was able to achieve the single pane of glass to manage their entire environment. They can now see as events are happening in real time, and be a lot more proactive.
Chris Cantu talks about vRealize Operations in even more personal terms, calling it a product that is “very near and dear to my heart”. As he explains,
“In a traditional sense, it is the applications vs. the infrastructure team, whose fault is it, who was the last person to touch it, while the teams were trying to restore services or address a performance issue. Prior to vRealize Operations, that was a pain because everybody was looking at a different pane of glass. So one of the things that the new operations management tools did within IT was to open up that view to both applications and infrastructure teams outside of compute. Now it is easy to narrow it down and pinpoint where the pain is, [and] who truly is the victim. And the management packs that are available out there on the third-party markets provide the operations team with even more visibility.”
Another interesting insight from Chris Cantu – about accountability and ownership
“One outcome of our project was financial – consolidate and reduce costs while providing similar or better services as before. What [Vistra Energy] IT leadership didn’t expect was their business units coming back to them and saying: ‘What just happened? We didn’t see any outage, very little disruption in our process, but now we have environments that are running at 50% better, and sometimes running at 300% better.’ What it translates into is the ability to turn invoices faster, run batch jobs in a shorter time. For example, one business process took 24 hours to run before, and now it takes 30 minutes! So I can come in two hours later than I needed to before. This was not in our business case, so this is on top of the financial benefit.” – Paul Reyes
LOB: “What did just happen? We didn’t see any outage, very little disruption in our process, but now our environments are running at 50% better, and sometimes running at 300% better.”
LOB: “One business process took 24 hours to run before, and now it takes 30 minutes!”
Another unexpected benefit was that the LOBs didn’t really expect that much of a quality improvement. They’re currently seeing about 75% improvement in availability, at around “four nines”.
“Additionally, now we have that great sense of staying current with technology – leaving that ‘technology debt’ behind. With automation, our work-live balance is starting to come back. Before, we had to do all our services provisioning at night, on the weekend, and now that we automated all these services we don’t need to sacrifice our personal time for that. No more calls in the middle of the night.” – Paul Reyes
Hear Paul Reyes talk about the outcomes of the project– in more detail
“And our customers – they benefit a lot too. When they need to be live in a month – that wouldn not have been thought of last year. Now we can be very agile and meet their needs dynamically, which in turn allows them to commit to their customers with a lot more confidence.”—Paul Reyes
We are in it together
Vistra Energy didn’t do it alone. VMware professional services had a large part in the overall enablement, especially on the vRealize Automation side. The partnership had started early in the process with the discussions of the scope and the deliverables. The long-term strategy with DevOps and fully automated model was built-in from the onset. Staying focused on this long-term strategy was a challenge, as sometimes it seemed easier to cut corners and make certain decisions that would take the course away from the main goal. But the VMware professional services and Vistra Energy implementation teams challenged each other, and in the end came up with the solutions that allowed the project to stay current while not losing site of the bigger goal.
Lessons Learned, and the Advice
In the words of Paul Reyes: “The advice I’d give to the folks who are on a similar journey, whether it is converged footprint or automation: when you try to sell that from the infrastructure or operations side, it’s very difficult to demonstrate business value. So you really need to look at what’s important to the business, and heavily partner with them so the application folks feel like they own part of it, and need to transform as much as the IT does.”
Watch Paul Reyes with more advice – need to know and understand your landscape
“Initially we started transforming from bottom up, and met a lot of resistance from the applications teams because it wasn’t their project. So we stopped and reset, and asked: who on the applications side will own it with us, so it’s part of your goals as well? Once we the business was involved and committed, it became a lot smoother. IT side of the house normally doesn’t drive the business outcome. But this was dramatic when we were able to take the total cost of ownership, and identify specific segments where IT was the enabler.” – Paul Reyes
The future
The future? Vistra Energy has gone through a vast amount of change – from storage, network, server, and virtualization prospective, to the entire way they operate their IT today. They have a roadmap to further drive DevOps methodology. They recognize that in this digital age infrastructure and operations are changing.
“Now, we are really taking on ‘software defined everything’. We started on our infrastructure, server, [and] storage. We are now looking at the network side of the house, and making it a true hybrid cloud realm. So we realize we are on this journey in the long terms, and we have only made the first steps.” –Paul Reyes
As Paul Reyes concludes, “My role as a traditional service provider of infrastructure and operations is going away simply because the dynamics of new technology. So we are now staying ahead of that, with all the automation and agility we bring, we are adapting and becoming a lot more relevant with our services to the business, allowing them to be a lot more current with their services and applications.”
