As a result, there are new challenges to remaining nimble – quickly harnessing data in new ways to seize business opportunity, and to holistically manage risk.
In this blog, we’ll discuss the challenges that are spurring innovation and forcing change in Financial Services Application and Data Architectures. In a related session at VMworld, (US session APP-CAP3080 – Big Data, Fast Data in the Cloud Ready Front Office), you’ll hear more about our work to provide high performance, cost effective solutions to address 5 key challenges to managing financial services data.
Challenge 1 – Economic Factors
Regardless of who we blame for the economic implosion of 2008, financial services firms continue to serve an indispensable function in the global economy. New securities issuance and trading create access to capital for business growth, wealth creation, and preservation. Still, financial services leaders face business model dynamics in this new economy.
- Declining Profit Margins
- How do I keep and grow my client base?
- How do I protect and increase my profits?
- Increasing Regulatory Scrutiny
- How do I adapt to constantly changing regulations?
- How do I automate to wring out the cost of compliance?
- Increasing Regulatory Requirements:
- Dodd Frank requires SEF (swaps execution facilities) – these add more trade and quote data volume and increase reporting requirements.
- Brain Drain
- How do I keep my best technologists from defecting to Silicon Alley? CTO’s admit – FS is not “cool.” And, yes – New York State has some other employers besides financial service firms.
Challenge 2: Financial Services Data is getting Faster
A widely used indicator of financial services data growth is market data (also known as “tick data”). OPRA (Options Price Reporting Authority) periodically publishes projections for options – contracts that give the buyer the right, but not the obligation, to buy or sell a financial instrument at a particular price for a particular period of time. Options quotes and trades (collectively, the “ticks”) represent 60-70 percent of market quotes and trades for publicly traded securities.
What do we see? The table above shows how OPRA has had to bump up projections for messages per second because tick volumes are growing faster than they had anticipated. Bandwidth also keeps keep drifting upward, with 2014 projections at almost 4 Gigabits per second sustained. Worse, according to OPRA, current burst volume can exceed 8 Gbps.
How soon is it before you need another 10 Gbps network pipe? The bottom line is that data volumes continue to grow, and competition escalates for existing network and computing infrastructure on the trading floor and in the data center. Another wrinkle is that tick data is stored for use in analytics. Huge data volumes make data storage more costly and harder to manage.
Challenge 3: Financial Services Data is getting Bigger
Retention projections are an indicator of bigger data. Here is what a client shared with us recently – and just for one product type – equities (i.e. trades and execution data). Generally, the regulatory requirement dictates saving data for 7 years.
What do we see? 2X growth year over year – with almost 8 petabytes of storage after 7 years (and this is with compressed data). Old thinking holds that only Tier 1 (data that is less than a week old) would need to be easily accessible, to fix trade breaks and answer inquiries. Tier 2 (less than 4 years old) and Tier 3 data would gather dust – accessed only infrequently – for example, in response to a regulatory audit.
New thinking has all retained tiers available not just for regulatory audits, but also for revenue generation and risk management uses: predictive modeling, analytics, and research. The bottom line is that more data needs to be more accessible than it previously has been and at exponentially larger scale.
Challenge 4: Financial Services data is more Global
Attracting investment from the outside, local markets around the world are looking to modernize trading, clearance & settlement, and compliance processes to attract global investors from outside of their region. The converse is also true. In work with the NYSE, we see exchanges outside of the US and EMEA looking to enable investors in their own region or country to gain broader access to global markets.
The bottom line is that transactional and market data will grow everywhere. Global investors will need new ways to view their portfolios and to control their trading across many regions.
Challenge 5: Diversity – Integrating New Sources
More and more data sources are becoming available – from intelligent feeds that look for content and sentiment in news and other textual sources to social networking data. New business opportunities, and risks, are lurking in this data. How do you harness it?
The old thinking is when a financial services “quant” (i.e. quantitative analyst) or a trader gets a new idea, the idea requires new data. Then, the data is captured (usually in a spreadsheet or business intelligence tool) and is massaged to produce an observation. Yet, the data is not usable for other purposes.
The new thinking is “when new sources become available, the data is discovered and mapped to internal meta-data and a data dictionary” (or noted as a new type). Information fabric services are extended to add the source, making it accessible for the quant, and for other users. This requires the same amount of time to onboard the data with more reusability. For data to be on-boarded quickly and ready for reuse, data management fabrics and services must be nimble and must scale.
The Call to Action
The days of free-wheeling financial services IT expansion are long gone. Financial services leaders must innovate to compete and survive. Containing costs while innovating takes new, global thinking, and a sober eye toward what was heretofore unthinkable:
- Consolidated, shared services within a firm? Across firms?
- Pushing “big beastly applications” to the cloud?
- A global, fit-for-purpose financial services cloud for trading, trade processing, analytics, risk and compliance?
- Upending conventional wisdom:
- Reducing data movement through modern data management
- Bringing analytics to the data, instead of consolidating data in a warehouse
See you at the session (US session APP-CAP3080 – Big Data, Fast Data in the Cloud Ready Front Office).