According to TabbFORUM, investment managers annually spend millions of dollars in avoidable direct costs due to poor data management technology and operations architecture. Within many large organizations, the management of data is left solely to the business verticals, and in some cases to individual teams. This breeds data duplication, and the quality of the final data can be questionable at best. To add insult to injury, data is obfuscated through poor usage of tools like Excel. BMA’s, (Business Managed Applications) when built in Excel are fraught with data linage issues, along with maintenance of the sheet, which in the best of cases is tedious. Through these current entrenched practices, companies are throwing millions of dollars away.
There is an alternative for those who are brave enough to go down this path, and that is the Data Lake and associated services designed to expose key business information from a single trusted source. Although this sounds great, it does not come cheap. You have to pay to play, and to see the costs of your data infrastructures drop, you need to invest in building those services. The impact of this change is far reaching and will affect most if not all of your business. But that’s the point, because this is a transformative change. Once you have walked that path, costs will drop, and confidence on the data will rise since you have a properly managed data platform.
This is probably why initiatives as outlined above in many cases will fail and fail hard. In general, it is not the technology stack that is at fault, but the people that are responsible for differing aspects of the data, leading to turf wars between departments. This leads to anemic progress and weakens the proposition to a point solution for a specific project.