Big Data is no longer a new term, its a fact, and its one of the fastest growing area in IT. 90% of the world’s data has been created in just the last two years, and 80% of it is unstructured data such as video or photos.
Currently, the most popular Big Data modalities are centered around structured data, such as in stock pricing prediction in Financial Services, and semi-structured data including text data from services like Twitter and social graph data from the “Likes” of Facebook and LinkedIn. More recently, a significant new data modality is emerging. Unstructured data from video and images demand much more attention in the current Big Data market.
Digital devices that generate millions of pixels in a flash are in the pockets of billions of people worldwide. We have a handful of online video archiving systems, such as YouTube, where users upload 100 hours of video every minute! But its not just on YouTube—if you look around your office, you will find several surveillance cameras. Working at home? Your neighbors may have a sign on the wall announcing “Surveillance video recording.” Commute to work? Watch out for the little boxes on top of the traffic lights.
These ubiquitous video recording devices generate data that will require analysis. There have been researchers and engineers working on how the analysis of this unstructured video and image data will work. An eye-catching recent video analytics project is the Deep Learning “cat-face on YouTube” project led by Google and Stanford last year. The machine learning analytics results are impressive, but in terms of platform building, it is still limited. They explained the “dataset is constructed by sampling frames from 10 million YouTube videos”, on a home-constructed system. Most likely, they didn’t spend much effort decoding the entirety of each video file.
That project is just a modest teaser of what can be done with big unstructured video and image data. In this post, I will focus on how security analytics are related to video and image analysis. But first, how big is the data we’re talking about?
At most companies, those security cameras operate 24×7 and archive the most recent “hot data” for future investigation. A typical enterprise will generate about a terabyte of video every day from multiple sites around the office campus. Here is some quick facts I collected from my EMC field sales friends who have been providing storage solutions to Fortune 500 clients:
- Generate 1 terabyte+ per day
- 720p IP camera collects HD images with millions of pixels in milliseconds
- Hundreds of cameras
- Hundreds of terabytes to a few petabytes of data storage
- Store at least 30 days
As a result, the IT department of each enterprise owns a huge amount of video data, but much of it is NOT being analyzed. As a previous blog post suggests, IT departments in large enterprises are capable of uniting datasets they currently store in silos. It’s time we dig into those datasets for the gold of insight.
Motivation
The Pivotal Data Science Team has a vision that a unified analytics platform is key to Big Data success within the enterprise. In line with this belief, we are building a large-scale video analytics platform on Hadoop, aiming to bridge unstructured data to structured insights.
In 2006-2007, I enjoyed a great opportunity to work on designing and deploying large scale video analytics systems in major cities. As a research engineer in charge of implementing and deploying multiple key projects, I got a chance to take a deeper look into the existing video analytics market. In short, the options are:
- Proprietary systems that are built and maintained by different vendors.
- Software solutions more focused on real time analytics (motion detection, counting vehicles on highways, etc) rather than insight-oriented analytics or in-depth analytics.
These solutions typically process the incoming video stream in real-time. This is probably the only time the analytics algorithms will touch these data. The metadata produced will then be associated with triggering alarms, while the video data will be stored for a short time in an archiving file system.
Platform Features
Pivotal Data Science Team isn’t looking to reinvent the wheel. The video analytics platform we are building compliments real-time surveillance video analytics by performing the analytics current solutions cannot achieve.
A big data infrastructure, such as Pivotal’s Hadoop platform offers the ability to glean deeper insights from the video data. Data scientists will be able to run their favorite video analytics algorithms, as many times as they want, on the large video data sets. Since it’s Hadoop, data scientists will also enjoy the advantage of no longer coding for a proprietary, home-brewed system.
Challenges and Solutions
Performing sophisticated video analytics on Hadoop is no small task. There are multiple challenges to tackle:
1. Video Transcoder
The first challenge is deciding how to deal with compressed video data, which suffers from a number of legacy limitations.
Decades ago, the MPEG (Moving Picture Experts Group) standard was proposed for efficiently encoding/decoding sequence of image frames, with intra-frame and inter-frame coding to deliver high quality video streams bounded by transmission bandwidth. The major hurdle is that the MPEG committee couldn’t predict the Big Data revolution decades ago. As a result, the MPEG compression format is extremely unfriendly to mainstream distributed systems such as Hadoop or MPI.
The Pivotal Data Science team has recently developed a successful prototype of a distributed video transcoding system on Hadoop. It can transcode gigabytes of hour-long MPEG-2 videos to a Hadoop sequence file of image frames in only minutes, using a six-node Hadoop cluster running on VMware virtual machines.
The secret sauce here is the smartly-engineered MapReduce jobs that can seamlessly decode each video chunk on HDFS in a distributed fashion. In a future article we’ll go further into detail on how this works.
2. Video Analytics
Crunching the video data into image frames is the first step. The next step is more exciting—performing deep analytics on the Hadoop-friendly data.
Hadoop MapReduce is a powerful and industry-proven scalable technology. But while it can solve lots of problems, it can’t solve all of them.
In this new adventure into the uncharted world of video analytics, Pivotal performed a high-level review on several commonly-applied computer vision algorithms.
Video analytics is all about extracting structured insights out of unstructured data. By carefully dissecting the typical video analytics system, we identified several key components that could take advantage the Hadoop infrastructure.
We found that MapReduce fits naturally for several components in a video analytics algorithm, such as feature extraction and object detection/recognition in computer vision. MapReduce provides linearly scalable performance, requiring little effort to craft parallelism. Instead, data scientists can focus on developing the analytic algorithm itself, and let the platform handle the rest.
In a couple minutes, on a six node Hadoop VM cluster, we were able to perform restricted region anomaly detection in 5 gigabytes of MPEG-2 video files. We extracted all jail-breakers’ trajectories, using the UK government’s benchmark video footage dataset. Again, everything runs in parallel naturally, thanks to Hadoop.
3. Analytics by SQL
Nowadays, the most common post-event investigations on surveillance video are done by hardworking security officers who manually browse through hours of video footage.
With a good video analytics platform, we can further leverage the structured insights Hadoop provides by using an advanced query language like SQL.
A simple example of this would be the question: “When did you see this red car yesterday?” A more sophisticated question would be, “How fast did this red car drive in the parking lot yesterday between 9AM to 12PM?” These sorts of questions can be written in a simple SQL SELECT query. Thanks to Pivotal’s Hadoop distribution and HAWQ (HAdoop With Query), we no longer need to browse through days of video footage like most security officers currently do.
4. Engagement Models with our Data Science Team
Pivotal’s Data Science Team, previously the EMC Greenplum Data Science Team, has been one of the largest in the Bay Area for years, providing Big Data Analytics solutions to various Fortune 500 enterprises. Once we launch this large scale video analytics platform, multiple engagement models for video analytics will open up:
- Utilization of this platform for existing video data archives.
Such technology can be further applied to use cases like: video indexing to enable content based video search, traffic analysis based on trajectory analysis for optimizing city transportation, and more. - Scaling up the existing video analytics algorithms to Hadoop and MPP Database.
Some clients may already have a home brewed in-house architecture, and seek opportunities to scale up their analytics capabilities. Our veteran Data Scientist will be able to help them through migration and integration. - Correlating video data with other structured data sources.
Once the structured insights are extracted from the video dataset, joining with other data sources can generate deeper insights from siloed data sources. For example, we will be able to help correlate retail stores’ surveillance camera data with transaction logs.
The potential is tremendous and the tools are at hand. We have already demonstrated that Hadoop is a great way to bridge unstructured data to structured insights. Applying these capabilities to the petabytes of unstructured video and image data produced on a daily basis will have a major impact. Large-scale video analytics may be the sleeping leviathan in our industry for now, but it’s awakening, and will have a major impact on the future of Big Data.
