Home > Blogs > VMware vFabric Blog

5 Key Challenges with Financial Services Data

5-Key-Challenges-with-Financial-Services-Data---HEADLINE-GFX-Global-Trading-CentersDespite a troubled world economy, the data needed to run financial services keeps getting faster, bigger, more global, and more diverse.

As a result, there are new challenges to remaining nimble – quickly harnessing data in new ways to seize business opportunity, and to holistically manage risk.

In this blog, we’ll discuss the challenges that are spurring innovation and forcing change in Financial Services Application and Data Architectures. In a related session at VMworld, (US session APP-CAP3080 – Big Data, Fast Data in the Cloud Ready Front Office), you’ll hear more about our work to provide high performance, cost effective solutions to address 5 key challenges to managing financial services data.

Going to VMworld in Barcelona?


Register for VMworld!
Click Here

Register for Session (APP-CAP1250 – Fast Data Meets Big Data):
Click Here

Follow all vFabric updates at VMworld on Twitter:
Click Here

Challenge 1 – Economic Factors

Regardless of who we blame for the economic implosion of 2008, financial services firms continue to serve an indispensable function in the global economy.  New securities issuance and trading create access to capital for business growth, wealth creation, and preservation.   Still, financial services leaders face business model dynamics in this new economy.

  • Declining Profit Margins
    • How do I keep and grow my client base?
    • How do I protect and increase my profits?
  • Increasing Regulatory Scrutiny
    • How do I adapt to constantly changing regulations?
    • How do I automate to wring out the cost of compliance?
  • Increasing Regulatory Requirements:
    • Dodd Frank  requires SEF (swaps execution facilities) – these add more trade and quote data volume and increase reporting  requirements.
  • Brain Drain
    • How do I keep my best technologists from defecting to Silicon Alley?  CTO’s admit – FS is not “cool.” And, yes – New York State has some other employers besides financial service firms.

Challenge 2: Financial Services Data is getting Faster

A widely used indicator of financial services data growth is market data (also known as “tick data”). OPRA (Options Price Reporting Authority) periodically publishes projections for options – contracts that give the buyer the right, but not the obligation, to buy or sell a financial instrument at a particular price for a particular period of time. Options quotes and trades (collectively, the “ticks”) represent 60-70 percent of market quotes and trades for publicly traded securities.

Screen shot 2012-08-08 at 10.05.47 PM

What do we see?  The table above shows how OPRA has had to bump up projections for messages per second because tick volumes are growing faster than they had anticipated. Bandwidth also keeps keep drifting upward, with 2014 projections at almost 4 Gigabits per second sustained. Worse, according to OPRA, current burst volume can exceed 8 Gbps.

How soon is it before you need another 10 Gbps network pipe? The bottom line is that data volumes continue to grow, and competition escalates for existing network and computing infrastructure on the trading floor and in the data center. Another wrinkle is that tick data is stored for use in analytics. Huge data volumes make data storage more costly and harder to manage.

Challenge 3: Financial Services Data is getting Bigger

Retention projections are an indicator of bigger data. Here is what a client shared with us recently – and just for one product type – equities (i.e. trades and execution data). Generally, the regulatory requirement dictates saving data for 7 years.

Screen shot 2012-08-08 at 9.49.21 PM

What do we see? 2X growth year over year – with almost 8 petabytes of storage after 7 years (and this is with compressed data). Old thinking holds that only Tier 1 (data that is less than a week old) would need to be easily accessible, to fix trade breaks and answer inquiries. Tier 2 (less than 4 years old) and Tier 3 data would gather dust – accessed only infrequently – for example, in response to a regulatory audit.

New thinking has all retained tiers available not just for regulatory audits, but also for revenue generation and risk management uses:  predictive modeling, analytics, and research. The bottom line is that more data needs to be more accessible than it previously has been and at exponentially larger scale.

Challenge 4: Financial Services data is more Global

Attracting investment from the outside, local markets around the world are looking to modernize trading, clearance & settlement, and compliance processes to attract global investors from outside of their region. The converse is also true. In work with the NYSE, we see exchanges outside of the US and EMEA looking to enable investors in their own region or country to gain broader access to global markets.


The bottom line is that transactional and market data will grow everywhere. Global investors will need new ways to view their portfolios and to control their trading across many regions.

Challenge 5: Diversity – Integrating New Sources

More and more data sources are becoming available – from intelligent feeds that look for content and sentiment in news and other textual sources to social networking data. New business opportunities, and risks, are lurking in this data. How do you harness it?

The old thinking is when a financial services “quant” (i.e. quantitative analyst) or a trader gets a new idea, the idea requires new data. Then, the data is captured (usually in a spreadsheet or business intelligence tool) and is massaged to produce an observation.  Yet, the data is not usable for other purposes.

The new thinking is “when new sources become available, the data is discovered and mapped to internal meta-data and a data dictionary” (or noted as a new type). Information fabric services are extended to add the source, making it accessible for the quant, and for other users. This requires the same amount of time to onboard the data with more reusability. For data to be on-boarded quickly and ready for reuse, data management fabrics and services must be nimble and must scale.

The Call to Action

The days of free-wheeling financial services IT expansion are long gone. Financial services leaders must innovate to compete and survive. Containing costs while innovating takes new, global thinking, and a sober eye toward what was heretofore unthinkable:

  • Consolidated, shared services within a firm? Across firms?
  • Pushing “big beastly applications” to the cloud?
  • A global, fit-for-purpose financial services cloud for trading, trade processing, analytics, risk and compliance?
  • Upending conventional wisdom:
    • Reducing data movement through modern data management
    • Bringing analytics to the data, instead of consolidating data in a warehouse

See you at the session (US session APP-CAP3080 – Big Data, Fast Data in the Cloud Ready Front Office).

Screen shot 2012-08-09 at 11.00.29 AM About the Author: Nancy Turbé’s 22 years of experience in the Capital Markets industry include leadership roles in Investment Banking Technology, as well as work for key technology vendors. Nancy’s portfolio of work is the source of her deep knowledge of Capital Markets functions and technology from Front Office to Back office, and across asset classes. Prior to joining EMC, Nancy led Capital Market engagements for Accenture, focusing on Risk and Data Management Architecture and Technology. At Bear Stearns, Nancy managed Program Trading Technology, and then Prime Brokerage Risk Technology. Nancy also played a key role in introducing Illustra, the first commercial instance of Postges, to Investment Banks. Nancy currently leads the Capital Markets Practice for EMC Consulting.

8 thoughts on “5 Key Challenges with Financial Services Data

  1. Studio Paci - Dottori Commercialisti e Revisori Contabili in Milano

    I feel that is one of the so much vital information for me. And i’m happy reading your article. But want to statement on few basic things, The website taste is great, the articles is really nice : D. Good job, cheers

  2. suzuki violin

    I know this if off topic but I’m looking into starting my own blog
    and was curious what all is required to get set up? I’m assuming having a blog like yours would cost a pretty penny?
    I’m not very internet savvy so I’m not 100% positive.
    Any suggestions or advice would be greatly appreciated.

  3. jonuniverse.com

    Hello, i believe that i saw you visited my site thus i got here to “return the
    want”.I’m attempting to in finding things to improve my website!I
    assume its adequate to use some of your ideas!!

    Just wish to say your article is as astounding. The clarity in your post is just excellent and that i can assume you are an expert on this subject.
    Fine along with your permission let me to snatch your RSS
    feed to keep up to date with drawing close post. Thank you a million and please continue the enjoyable work.

    Its like you read my mind! You appear to grasp so much about this, such
    as you wrote the book in it or something. I believe that you just
    can do with a few % to pressure the message house a little bit,
    however other than that, that is wonderful blog. An excellent read.
    I’ll definitely be back.

    Thank you for the auspicious writeup. It in truth was a amusement account it.
    Glance complicated to more brought agreeable from you! By the way, how could
    we keep in touch?

    Very interesting information!Perfect just what I was looking for!

    I was reading through some of your content on this site and
    I believe this website is very informative ! Continue posting .

    Real superb info can be found on weblog .
    I like this web blog very much, Its a very nice office to read and find information.

    Very interesting points you have observed , regards for
    putting up. “Whatever one man is capable of conceiving, other men will be able to achieve.” by Jules Verne.

    I like this blog its a master peace ! Glad I noticed
    this on google. “Live…..Laugh…..GOLF” by
    Kathryn Schaefer Plaum.

    Wohh precisely what I was looking for, regards for putting up.
    “Be nice to everyone on your way to the top because you pass them all on the way down.”
    by Fred Hufnagel, Sr..

    I always was interested in this subject and stock still am, thankyou for putting up.

    Thankyou for all your efforts that you have put in this.
    very interesting info . “Nurture your mind with great thoughts.” by Benjamin Disraeli.

    I like this blog very much so much superb info .
    “Our national flower is the concrete cloverleaf.” by Lewis Mumford.

    Outstanding post, I conceive website owners should acquire a lot from this site its real user friendly .

    “Frisbeetarianism is the belief that when you die, your soul goes up on the roof and gets stuck.” by George Carlin.

    Rattling nice design and superb articles , nothing else we need : D.

    Keep up the great work , I read few articles on this
    internet site and I think that your web site
    is very interesting and has got lots of fantastic information.

    Its excellent as your other blog posts : D, thanks for posting .
    “For peace of mind, we need to resign as general manager of the universe.”
    by Larry Eisenberg.

    I consider something genuinely interesting about your
    weblog so I saved to bookmarks .

    Very interesting topic, thanks for posting.

    You have remarked very interesting details! ps decent site.

    Very interesting information!Perfect just what I was searching for!

    I was examining some of your articles on this internet site and I believe this internet site is rattling instructive!
    Retain posting.

    Really wonderful information can be found on web site.

  4. سئو

    I thank you for your excellent site

  5. دانلود آهنگ

    nice post

  6. کاندوم خاردار

    دانلود آهنگ

  7. بلیط هواپیما

    تیک بان یک سایت معتبر خرید بلیط هواپیما می باشد که می توانید به صورت آنلاین و اینترنتی و با گارانتی معتبر بلیط هواپیما خودتان را تهیه کنید. بلیط هواپیما [Airline tickets] می خواهید؟ تیکبان یکی از اولین های فروش و رزرو بلیط هواپیما می باشد و همیشه مشتری مداری را به عنوان شاخص فعالیت خود در نظر گرفته است و هر روز می کوشد که رضایت خاطر مشتریان را جلب کند.

  8. چاپ بک لایت چیست

    I am eternally grateful for everything you’ve taught me.


Leave a Reply

Your email address will not be published. Required fields are marked *