Learn how the telecommunications giant embraced software and how it delivers results fast.
If you think of Verizon as just a mobile phone company, you haven’t been paying attention. The company, like the telecommunications industry as a whole, is making aggressive moves into adjacent industries that live well above the network. Owning the pipes that transport mobile traffic is a good business. But an even better business combines it with original content creation and layers on digital services that drive real customer engagement and loyalty.
That’s the rationale behind Verizon’s recent acquisitions of Yahoo!, which includes digital properties like Tumblr, and Complex, a site for millennial men. Then there’s Hum, Verizon’s connected car service, and ThingSpace, a web-based Internet of Things platform, both of which Verizon launched in 2015. And don’t forget Verizon FiOS, Verizon’s home Internet and TV service.
What do all these disparate services have in common?
“Verizon is moving into all these new markets and all those new markets are software,” said Josh Stone, a Senior DevOps Platform Engineer at the company, speaking at Cloud Foundry Summit in Santa Clara last month. “Verizon is becoming a software company. And in order to be competitive in the marketplace as a software company, we have to deliver software faster to the market. And Cloud Foundry is one of the ways that we deliver that software faster.”
The Need for Speed
Sure, Verizon has been developing software for a while. But Verizon developers were largely restricted to building and running their software on traditional infrastructure that sacrificed speed for stability. In order to differentiate with custom software, a change in approach was clearly needed.
“We want developers to do what they do well and what we pay them to do — write code that ends up in production”
—Josh Stone, Verizon
In 2015, Verizon took the first step towards change by adopting Pivotal Cloud Foundry®. Pivotal Cloud Foundry is a platform for deploying and operating modern applications. It helps large teams automate many of the administrative tasks that slow down the process of moving code into production like managing dependencies and configuring network policies.
“We want developers to do what they do well and what we pay them to do — write code that ends up in production,” Stone said. “Anything else is a waste.” Pivotal Cloud Foundry removes much of the overhead developers traditionally have to deal. This allows them to focus on developing software, quickly getting it into the hands of real users, and then iterating based on feedback.
Verizon originally deployed Pivotal Cloud Foundry in its own private cloud running on OpenStack. By June of 2015, it had five apps running on the platform. It has since migrated the platform to VMware and grown the number of platform instances and apps running on the platform considerably. Today, Verizon is operating Pivotal Cloud Foundry in six different data centers supporting over 100 applications and 4,000 containers, Stone said.
Most importantly, developers are happy and software releases are now measured in hours and days instead of weeks and months.
Transparency and Automation
While there are a number of factors responsible for Verizon’s success in accelerating software delivery, two keys are transparency and automation, according to Stone.
Adopting Pivotal Cloud Foundry required developers to let go of some of their traditional responsibilities. This is a good thing, of course, as these responsibilities, like architecture considerations and dealing with middleware versions, are precisely the things that often impede developer productivity. But some developers at Verizon were reluctant to let go until they were confident the operations team and the platform itself was up to the task.
“That’s where you provide that transparency,” Stone said. “You take over ownership of operations, but you provide as much transparency to view into those operations, to help build up that level of trust between developers and operations.”
Verizon’s operations team built trust with developers by monitoring and sharing platform metrics like foundation health based on specific KPIs, capacity remaining on the platform, and underlying virtual machine health. Stone’s team also also tracks the performance of smoke tests, which run continuously against each of Verizon’s six data centers running Pivotal Cloud Foundry. It alerts the team to performance bottlenecks and other issues “so we know as operators if there’s an issue with the platform long before a developer opens a service ticket,” Stone said. And this is where automation comes in.
The team uses Concourse, a tool for setting up and running continuous integration pipelines in Pivotal Cloud Foundry, to respond to smoke test alerts. Once the operations team applies a fix an issue, Concourse will automatically apply the fix in the future, should the issue arise again. There’s no need for manual intervention.
“Automate everything,” Stone said. “If we’re performing an action against and environment it should be done through automation because chances are we’re going to have to do it to the [others].”
Automation not only speeds up incident response times but also provides a consistent experience for developers across each of the platform’s dozen foundations.
“Before Concourse, we had trouble keeping all of our environments consistent, and that led to a poor developer experience,” Stone said. “If an app pushed in one environment but failed to stage in another because the Java buildpack was different, that’s a poor developer experience and that’s something that would slip through the cracks before we brought in Concourse.”
Positioned for Growth
Verizon is already benefiting from its investment in faster software development. Its Internet of Things offerings generated over $240 million in revenue in the fourth quarter of 2016 alone and over 30,000 cars are connected to its telematics products, which include Hum.
With developer trust growing stronger by the day and automation ensuring a consistent development environment, Verizon is well positioned to continue making inroads into new markets. The company may have its roots in mobile phones and networking, but its future is firmly centered around software.
Change is the only constant, so individuals, institutions, and businesses must be Built to Adapt. At Pivotal, we believe change should be expected, embraced, and incorporated continuously through development and innovation, because good software is never finished.
How Verizon Uses Software to Break Into New Markets was originally published in Built to Adapt on Medium, where people are continuing the conversation by highlighting and responding to this story.
