Big Data: Big Opportunity or Big Challenge?

By Barton Kaplan

We are undoubtedly living in the era of big data. For any doubters, McKinsey estimates that the amount of new data that enterprises around the world stored on disk drives in 2010 equaled 7 exabytes, growing at a compound annual rate of 40 percent.[1]

The opportunities for enterprises to leverage this data abound, at least in theory. In the retail sector alone, McKinsey estimates that big data could raise operating margins by a whopping 60 percent.[2] And in US healthcare, the potential benefits are as much as USD $300B annually.[3] Further, venture capital is flowing to a slew of new startups that are creating tools and algorithms that can parse data at scale.

But the reality inside enterprises paints a different picture. The biggest misconception with big data is that the sheer volume means there is more meaningful insight to be had. But as Nassim Taleb has convincingly shown, as the amount of data goes up, the signal to noise ratio goes down, by as much as 200 times.[4]

This phenomenon is reflected in corporate survey data. In a poll of over 8,000 employees by CEB, only a third indicated that the information they needed to do their jobs was available. And less than half said that corporate sources of information were usable.[5]

Meanwhile, at a time when over 80 percent of employees at the typical organization are now considered knowledge workers, fewer than 40 percent have the combination of analytical skills and judgment needed to use that data to drive better decisions,[6] without which no value can be gained. Data scientists alone can’t make up for a capability gap across the broader employee base.

So how do organizations tackle the myriad information management challenges that big data has exacerbated so they can realize some of the benefits? In my work with companies, the following four tactics have proved effective:

1. Elevate the importance of information: One large financial institution put information on par with people, process, and technology as a discrete enabler of business capabilities, leading to a fully funded, five-year information management strategy and roadmap.
2. Develop differently for data: At a major insurance company, IT leadership replaced their software development lifecycle with a data-centric development lifecycle for information-intensive projects.
3. Make data analysis user-friendly: IT can no longer keep up with business demand for reports and dashboards. Instead of being the bottleneck, work instead to expose corporate data in a secure yet user-friendly manner so employees can quickly and easily generate their own reports. At one organization I worked with, they set up an interface modeled on Amazon’s with the motto “shopping for data.” The result was a 50 percent reduction in support staff time to deliver analytic services.
4. Upskill your people: Rather than trying to land a few hard to find, expensive data scientists, focus instead on raising the skill sets of the 8 out of 10 employees in your organization who are considered knowledge workers. This starts with an understanding of the profile of an employee with high “insight IQ.” Try the following quiz to see if you fit the bill.

——–

Barton Kaplan is a business solution strategist with VMware Accelerate Advisory Services and is based in Maryland.