During a plenary session this morning at the Strata Conference in New York City, Pivotal’s Josh Klahr, Vice President of Data Products, cut through the Big Data hype to illustrate what a data-driven enterprise looks like in action. Klahr presented a use case of how Hadoop-centric, data science-informed enterprises will derive value from this paradigm shift.
Klahr cited Mark Andreessen’s 2011 editorial “Why Software is Eating the World,” in which Andreessen stated, “Over the next 10 years, the battles between incumbents and software-powered insurgents will be epic.” Noting that this is consistent with his experiences with Fortune 500 enterprises, Klahr said that “In order to compete in the 21st century, the winning companies are going to need to be software companies. These companies need to learn how can to use data as a strategic asset to create the next generation of data-driven apps.”
As an example of this transition in action, Klahr pointed to Pivotal investor and customer, General Electric (GE). “While in the past you may not have thought about GE as a software company,” he said, “they are very quickly emerging as an innovator in the space of data driven applications.” Over the past year, Klahr said, GE has launched 24 data-driven apps across their business units, ranging from predictive maintenance applications for jet engine turbines to predictive outcome tools for hospital professionals. The company has driven over $400 million in new bookings from the first 10 apps to hit the market.
Hadoop is a primary catalyst and enabling technology for this transition to the data-driven enterprise, Klahr stated. “While general interest in the dominant relational databases has been steadily decreasing,” he said, “Hadoop interest has grown significantly in its short lifespan. This is the emergence of the next data fabric.”
Speaking from his professional experience working with Big Data at Yahoo prior to joining Pivotal, Klahr noted that while Hadoop is an important and powerful tool, it isn’t always the right tool for the job. “Success creating a data driven enterprise may rely on Hadoop as a core platform component,” he said, “but it takes an entire toolkit of supporting capabilities to ultimately be successful.”
To address this challenge, Klahr explained, Pivotal views “Hadoop as the platform for our Big Data architecture,” while building new capabilities on top of Hadoop to create a well-rounded toolkit for data-driven enterprises. One of these key technologies is HAWQ, Pivotal’s interactive and fast SQL query engine for Hadoop. Equally crucial are capabilities to develop realtime, user-facing applications. “It’s not just the internet giants who are building this type of application any more,” he said. “It’s companies like American Express, founded in 1850, which are building realtime, targeted offer platforms by analyzing the huge amounts of data available in Facebook’s social graph.”
Concluding his plenary session, Klahr summarized what all these companies share: “An application vision, powered by Hadoop, driven by data science, all within the traditional enterprise.” He invited attendees to not only embrace the transition, but also establish their own vision of what it means and looks like for an enterprise to be data-driven. “Now it’s your turn,” he said. “Create your own vision for being data driven, go out and build that first application — drive real value from your Big Data — and then do it again.”
