Another look at Data Warehouse management

The challenges around data propagation and data warehouse management are well documented in the investment management industry. Data underpins nearly every step in the value chain, yet its collection, management, storage, and analysis remain difficult. Cutting-edge technology provides part of the solution, but technology is only one component of a holistic approach. Breaking down silos and legacy systems is critical. But this requires a change in mindset if investment management firms want to remain nimble, innovative, and competitive.

Traditionally, data warehouse management glitches have been seen as the preserve of the technology team. Fixing data storage and interrogation problems was not easy. The advent of cloud-based solutions has enhanced agility, transparency, and data storage, but technology only goes so far in solving broader data challenges. A more effective long-term approach involves pivoting away from the IT team while increasing business user involvement to create a metadata-driven, user-friendly information management framework for your day-to-day operations.

Start with your business objectives in mind

The first step in developing an effective data warehouse strategy is gaining a better understanding of your challenges from an operational and developer perspective. For the former, the complexity around setting up data feeds, developing reports, and building dashboards are frequent stumbling blocks that we address in our client engagements. Equally as important is tackling the fact that data warehouses, which structure data in a standardized manner, can become rigid without proper long-term planning. If this happens, you may find it difficult to upgrade existing processing logic, much less introduce new asset types and address evolving regulatory and reporting requirements.

Portfolio managers and finance teams also face complications from the multiple accounting systems they need to handle various asset classes and investment vehicle types, as well as varied versions of net asset value (NAV). If these complexities are only unearthed after your data warehouse becomes operational, you will lose time and struggle to keep pace as your business needs change.

Developers, on the other hand, must be well versed in investment management domain knowledge, including how your business operates, the asset classes involved, trade and position management, and the web of third-party systems that offer point solutions to different steps in the fund lifecycle. If you look externally for support in developing your data warehouse, make sure your partner has relevant industry experience as well as technical know-how.

Focus on user experience in managing data warehouse operations

A deeper dive into these components enables firms to develop a software and data ecosystem that prioritizes ease-of-use over technical complexity. This will require technology changes that drive simplification and enhance traceability. Such technology changes will also need to encompass processes such as monitoring, governance, and support, as well as organizational training.

On the first point, abstracting technological complexity enables non-programmers to add or amend your data propagation, transformation, extraction, and visualization routines; this is similar to how Excel empowers analysts to convert text to data and add filters and charts. In addition, the metadata-driven nature of the data ecosystem allows the storage of linkages between components. This lets non-technical users debug issues without looking into the code.

From a process point of view, scheduling and exception-handling procedures, and a well-defined escalation matrix are critical. Provide support on how to escalate and resolve issues. Ensure strong governance that revolves around greater controls over reporting and changes to feed processing. If partnering with an external provider, choose one that possesses a deep knowledge of investment management operations, knows the right team members to involve and questions to ask, and can demonstrate the ability to deliver future-proof data warehouse and ecosystem solutions.

As for your people, they need to be well equipped and trained on the fundamentals of feed processing, reporting concepts and data modeling. Involvement throughout the solution development project is critical, not just to get critical input, but also to develop a new mindset around information management and build buy-in for new ways of working.

From data storage to information management

Investment firms can reap several benefits from this information management framework. Done successfully, you will reduce total cost of ownership (TCO) and increase efficiency, which in turn improves your return on investment (ROI). You’ll free up senior members of your development team to focus on strategic initiatives while queries are handled by experienced but non-technical users, whose job it is to look at end-to-end data processes. Last but certainly not least, reporting is simplified as day-to-day changes can easily be made to warehouse components.

About the author, Vishal Patil

Vishal Patil is the Senior Director, Advisory & Product Development at Linedata. He has 20+ years of experience in designing technical and outsourcing solutions for global and regional investment managers. He and his teams have helped dozens of firms achieve efficiencies and scale in software development and support services. Vishal leads Linedata’s advisory and software development services group in India