Home > Blogs > VMware vFabric Blog


4 Examples of Big Data Trends

Whether you are doing Algo Trading or Agriculture, your business is becoming more and more reliant on data. And what is it about that data that is common? It’s Big. It’s Fast. It’s Flexible.

These three trends (i.e. big, fast, and flexible) in new data and analysis are stretching the database in ways we hadn’t even conceived of in the past. To better articulate and think clearly about these trends, let’s analyze some examples so we may understand how these disruptors are impacting the data market. As well, we will look at an example of a fourth trend – how cloud delivery models impact data.

Agriculture – Example of Big Data

The first disruptive change we will talk about is Big Data.

Register for VMworld!
Click Here

Register for Session APP-CAP1250 – Fast Data Meets Big Data:
Click Here

Follow all vFabric updates at VMworld on Twitter:
Click Here

This term has gotten a lot of coverage in the press, but in simple terms, big data is about finding better ways of storing and analyzing massive amounts of data.  With the staggering quantities of data being produced today, the simple strategy of putting all of your data into a traditional data warehouse is prohibitively expensive and insufficient to the task.  How do we store and use data to meet business needs?  Big data solutions take large amounts of structured and unstructured data, run algorithms against datasets to refine data, and glean business-critical insight.

In agriculture the data patterns are changing radically. I bet you never thought about farming as a big data opportunity. Next generation farm equipment like combines and tillers are going to be able to take soil samples as they move along, perform analysis on those samples, and feed the results of the analysis back to the manufacturer for crunching on a macro scale. This will result in a better understanding of what is happening in that entire area and make it possible to adjust things like the amount or types of fertilizer and chemicals that should be applied. If the farm equipment manufacturers figure out how to harness all this information, this kind of big-picture analysis could change the commodity trading markets forever.

Algo Trading – Example of Fast Data

The second disruptor we are seeing is Fast Data. Fast data usage is changing. Whereas it used to be a notion of processing a stream of data serially as fast as you can, it is now about horizontally scaling that processing and delivering information from data in real-time. Traditional databases, and even specialty databases like tick databases have served us well in processing data quickly in the past. They will still continue to play a critical role in delivering data with low latency requirements.

Algo Trading is still about applying algorithmic models to analyze fast moving market data in near real time, but there are definite trends of moving to horizontally scalable, memory oriented, non-relational data technologies allowing users to optimize for speed and performance with which they can analyze the data that is coming at them. Technologies like GemFire that enable parallel real-time analysis of data as it becomes available in a horizontally scalable and fault tolerant way can provide dramatically faster understanding of market data and competitive advantage for the traders who use them.

Sentiment – Example of Flexible Data

The third disruptor we will talk about is Flexible Data. We use the term Flexible Data to describe how applications today are using new sources of data both structured and unstructured, and combining them to produce analytics that simply weren’t possible before. This unstructured data is forcing us to think about storing data in new ways too.  No longer is data simply stored in a relational table, but a host of new relational and non-relational data structures have emerged – many of which are open-source. Things like Object databases for semi-structured data, document oriented databases, and even graph databases to discover the relationships of things under analysis. When we think about how applications are being developed today, we see an increasing number of applications being built using these new technologies.

There are some trading firms who are using alternative data sets that were not used before to establish or enhance their trading strategies. For instance there is at least one fund that decides their strategy each day for what they believe will be the top ten winning stocks based on sentiment data they glean from Twitter. This is a case of unstructured data being applied to a business that relied almost entirely on structured data for years.

Big Data and an Example of the Cloud-Based Delivery Model

In addition to these three data-related changes, there is a fourth disruptor –the cloud delivery model itself. The whole notion of hybrid cloud or “cloud bursting” has major implications for the data management layer of cloud-based applications. You may be able to get 1000 nodes provisioned in a public cloud in minutes, but if you can’t get the data needed for the computation there for hours, you have a problem. So, one emerging pattern is to trickle feed the data to the public cloud on just a few nodes of an elastic memory oriented, cloud ready data store as it is being generated. This way, when you are ready to burst into the public cloud to perform the computation, most of the data is already there.

There are, in fact, some public cloud implementations that have sprung up simply because they have a lot of useful public data. The NYSE cloud, for instance, has instant access to all of the reference and market data for all of the instruments traded on that exchange. Users who want to analyze their performance against NYSE’s historical data need only push an abstract view of their trading decisions to the public cloud so that they can learn what decisions worked well, what didn’t, and what might have worked better based on an understanding of historical trends.

New Beginnings for Technology

It is not just the database that is being stretched though. Traditional batch analysis mechanisms are unable to deliver the value that more real-time analysis can. The notion of near-real-time, closed-loop analysis patterns is becoming a reality, and is delivering new business value sometimes even from old datasets. If you can analyze your big data to find the good patterns, and then codify rules to help you detect those patterns in real time, you can significantly change the value of your data.

One thing is clear. There is a lot more data out there than there used to be, and traditional methods of collection, storage and analysis are ill suited to handle it all. We live in a place in time where creativity and new science are once again able to make significant changes in what had become a pretty stodgy area of technology.

>> For more info on VMware’s data-related product offerings, see the following vFabric Product Pages:

Mike-stolz-headshot About the Author: Mike Stolz is a Global Field Architect for the vFabric division of VMware. He came to VMware in 2010 as part of the GemStone acquisition after serving 4 years at GemStone as VP of Architecture. In his role here, Stolz leverages his expertise in targeting, developing and delivering innovative technology solutions to expand VMware’s Emerging Products Division within the financial services, travel, health care and other communities. During the ten years prior to joining GemStone Stolz was Director and Chief Architect of Merrill Lynch’s Global Markets and Investment Banking division. In this role, Stolz was responsible for the design and development of trading systems and trading support systems for Fixed Income, Currencies, Commodities, Derivatives, Liquidity and Risk Management.

78 thoughts on “4 Examples of Big Data Trends

  1. Pingback: 5 Trends Driving Change in App Architectures | VMware vFabric Blog - VMware Blogs

  2. Ray Woods

    Great !

    please keep me in touch !
    Thanks !
    ray

    Reply
  3. Pingback: The Best VMware vFabric Stories of 2012 & What’s In Store for 2013 | VMware vFabric Blog - VMware Blogs

  4. Pingback: VMware vFabric Blog: The Best VMware vFabric Stories of 2012 & What’s In Store for 2013 | Virtualization

  5. Micah Norsworthy

    Looking to find or become a sisterwife? Check out sisterwives.

    Reply
  6. تور کیش

    Thanks a lot very good article
    well done

    Reply
  7. انجام پایان نامه مدیریت

    THANKS A LOT VERY GOOD ARTICLE.

    Reply
  8. خرید vpn

    good look

    Reply
  9. مد لباس

    Whether you are doing Algo Trad

    Reply
  10. موزیک

    ing or Agriculture, your business is

    Reply
  11. گرافیک و کاریکاتور

    becoming more and more reliant on data. And what is it about that data

    Reply
  12. اقتصاد

    that is common? It’s Big. It’s Fast. It’s Flexible.

    Reply
  13. دانستنی ها

    These three trends (i.e. big, fast, and flexible) in new data and analy

    Reply
  14. اخبار خودرو

    sis are stretching the database in ways we hadn’t even conceived of in the past. To better articulate and think clearly about these trends, let’s analyze some examples so we may under

    Reply
  15. تجارت

    stand how these disruptors are impacting the data market. As well, we will look at an example of a fourth trend – how cloud delivery models impact data.

    Reply
  16. فضای مجازی

    Agriculture – Example of Big Data
    The first disruptive change we will talk about is Big Data.

    Reply
  17. تبلیغات

    This term has gotten a lot of coverage in the

    Reply
  18. اخبار اجتماعی

    press, but in simple terms, big data is about

    Reply
  19. نرم افزار

    terms, big data is about finding better ways of storing and analyzing

    Reply
  20. اخبار حوادث

    massive amounts of data. With the staggering quantities of data

    Reply
  21. اخبار روز موبایل

    being produced today, the simple strategy of putting all of your data into a traditional data warehouse is prohibi

    Reply
  22. اخبار ورزشی

    tively expensive and insufficient to the task. How do we store and use data to meet business needs? Big data solutions take large amounts of structured and unstructured data, run algo

    Reply
  23. فناوری اطلاعات

    ing to be able to take soil samples as they move along, perform analysis on those samples, and feed the results of the analysis back to the manufacturer for crunching on a macro scale.

    Reply
  24. انتشارات و تحقیقات

    This will result in a better understanding of what is happening in that entire area and make it possible to adjust things like the amount or types of fertilizer and chemicals that should be

    Reply
  25. اخبار سیاسی

    Algo Trading – Example of Fast Data

    Reply
  26. بازی های رایانه ای

    The second disruptor we are seeing is Fast Data. Fast data usage is changing. Whereas it used to be a notion of processing a stream of data serially as fast as you can, it is now about

    Reply
  27. اخبار بین المللی

    horizontally scaling that processing and delivering information from data in real-time. Traditional databases, and even specialty databases like tick databases have served us well in pro

    Reply
  28. مجله خبری ویدئویی

    in near real time, but there are definite trends of moving to horizontally scalable, memory oriented, non-relational data technologies allowing users to optimize for speed and performance with which they can analyze the data that is coming at them. Technologies like

    Reply
  29. اموزش برنامه نویسی

    In addition to these three data-related changes, there is a fourth disruptor –the cloud delivery model itself. The whole notion of hybrid cloud or “cloud bursting” has major implications for the data management layer of cloud-based applications. You may be able to get 1000

    Reply
  30. گاد سرور

    t is not just the database that is being stretched though. Traditional batch analysis mechanisms are unable to deliver the value that more real-time analysis can. The notion of near-real-time, closed-loop analysis patterns is becoming a reality, and is delivering new

    Reply
  31. طراحی سایت

    THANKS A LOT FOR THIS POST. VERY GOOD

    Reply
  32. یو پی اس

    thank you for you …

    Reply
    1. خرید بک لینک

      http://upsonline.ir/ups-battery/

      Reply
  33. اخبار ایران و جهان

    thanks.

    Reply
  34. نقد فیلم

    thanks a lot

    Reply
  35. تفکر استراتژیک

    gooood

    Reply
  36. دستگاه حضور و غیاب در تبریز

    in near real time, but there are definite trends of moving to horizontally scalable, memory oriented, non-relational data technologies allowing users to optimize for speed and performance with which they can analyze the data that is coming at them. Technologies like

    Reply
  37. تورچ جوشکاری

    thanks

    Reply
  38. نوشیدنی آلوورا برمس

    thanks fo your post

    Reply
  39. یو پی اس apc

    http://upsco.ir/

    Reply
  40. VM certifications

    VMware Certified Advanced Professional 6 (Desktop and Mobility Deployment) – The industry-recognized VCAP6-DTM Deploy certification validates that you know how to deploy and optimize VMware Horizon 6 (with View) environments. It demonstrates that you have the understanding and abilities required to leverage best practices to provide a scalable and reliable Business Mobility platform for your business. Some of the subjects involve: Configuring and managing Horizon View components, configuring cloud pod archituecture, configuring Group Policy settings related to Horizon View, Configuring and optimizing desktop images for Horizon View & Mirage, Configuring and managing App Volumes AppStacks, Configuring desktop pools, Configuring and deploying ThinApp packaged applications, Configuring VMWare Identity Manager, etc.Szumigalski.com was designed by Sebastian to propagate his passion for PowerShell & VMWare. Sebastian is an IT expert employed in Singapore for more that fifteen years who’s on a regular basis looking for brand-new expertise to refine his technical competencies & know-how. Ever since, Sebastian has become a member of PowerShell User Group & VMWare VMug group, and has also been involved in every one of the get-togethers held in Singapore. This blog site will show you exactly how Sebastian manage to systemize some of his every day work making use of PowerShell. There’s study guides meant for the VCAP6-DTM exam, which are personally created by Sebastian. Sebastian is accredited with VCAP6-DTM, and is expert with virtualization & server maintenance from four years experience in automation. The need for VMWare knowledgable admins and technicians are ever-increasing in the current technology industry. Become familiar with more details on PowerShell & VMWare with Szumigalski.com!

    Reply
  41. hotgt

    thas good fro you

    Reply
  42. sianoor

    very mucht that alow

    Reply
  43. sianoor

    vry good to see you

    Reply
  44. sianoor

    سیانور
    جوان جی تی
    نیوز جی تی
    هات جی تی

    Reply
  45. طراحی سایت اصفهان

    web design in isfahan

    Reply
  46. سنگ مصنوعی

    cement plast

    Reply
  47. تولید کننده قطعات لاستیکی

    rubber in isfahan

    Reply
  48. سرویس پارچه ای آشپزخانه

    kitchen

    Reply
  49. زیورآلات

    zivar alat arzan

    Reply
  50. طراحی وب سایت در اصفهان

    طراحی وب سایت در اصفهان

    Reply
  51. طراحی سایت در اصفهان

    طراحی سایت در اصفهان

    Reply
  52. تولید سنگ مصنوعی

    تولید سنگ مصنوعی

    Reply
  53. لوله اتصالات

    HI GOOD SITE
    I was a good article thanks for nice site
    لوله اتصالات

    Reply
  54. طراحی سایت در تبریز

    Thanks for your site

    Reply
  55. کاندوم خاردار

    قطار مشهد

    Reply
  56. لوله و اتصالات پلی پروپیلن

    فروش لوله و اتصالات و تجهیزات شساختمانی.برای سفارش و خرید با ما تماس بگیرید.لوله های یو پی وی سی و سه لایه و بسیار مقاوم

    Reply
  57. لوله و اتصالات سفید آب

    فروش لوله و اتصالات

    Reply
  58. لوله و اتصالات یو پی وی سی

    thanks for the website.this site is very good

    Reply
  59. لوله و اتصالات upvc

    تولید و پخش لوله

    Reply
  60. لوله و اتصالات فاضلاب

    تولید کننده لوله و اتصالات فاضلاب

    Reply
  61. لوله و اتصالات آب آشامیدنی

    تولید ککنده لوله و اتصالات آب آشامیدنی

    Reply
  62. وایت برد

    تولید و پخش وایت برد و وایت برد شیشه ای

    Reply
  63. وایت برد شیشه ای

    بهترین و پرفروش ترین انواع تخته وایت برد

    Reply
  64. انجام پایان نامه

    خدمات انجام پایان نامه و انجام پایان نامه کارشناسی ارشد

    Reply
  65. طراحی و دکوراسیون منزل

    شرکت پر سیاه مجری خدمات دکوراسیون داخلی

    Reply
  66. pp لوله و اتصالات

    لوله و اتصالات ppو اتصالات داخلی ساختمان

    Reply
  67. لوله و اتصالات

    شرکت کاسپین پلیمر آبراهان تولید و پخش انواع لوله

    Reply
  68. لوله و اتصالات پلیکا

    لوله پلیکا و لوله های 3 لایه کاسپین پلیمر آبراهان

    Reply
  69. کابینت کرج

    طراحی و ساخت کلیه سفارشات ام دی اف

    Reply
  70. کابینت سازی کرج

    کلیه سفارشات کابینت در کرج و تهران

    Reply
  71. گیت تردد نفر رو میله ای

    گیت کنترل نفر – میله ای – شیشه ای تبریز

    Reply
  72. تزریق-بوتاکس

    تزریق-بوتاکس تزریق-بوتاکس تزریق-بوتاکس!!!

    Reply
  73. تزریق-ژل

    تزریق-ژل تزریق-ژل تزریق-ژل!!!

    Reply
  74. دانلود فول آلبوم خارجی

    دانلود فول آلبوم خارجیدانلود فول آلبوم خارجی

    Reply
  75. انتقال وجه

    thank youuu

    Reply
  76. انتقال وجه

    very very thank you

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

*