It is well understood that data is the fuel of AI. Underappreciated is the role that AI can play in solving many long-standing challenges that enterprises have in managing their data. Reality is that the big data "problem" of constantly growing scale, many types of data, and the accelerating speed at which data is created is beyond human-powered intervention to resolve. AI is good at these sorts of problems, and as much as AI needs data, the data landscape of today and tomorrow needs AI to master it.
Every practitioner in the field of data and analytics has stories of data management projects gone awry, and the data landscape guilty of causing many of these problems is only getting more complicated. It is not just the data itself that presents a challenge; new data management technologies are creating sprawling, disparate data architectures that span on-premises, the public cloud, and all points between. From data maps that once created are instantly out-of-date, through attempts at manually maintaining master data repositories, managing data has become an enterprise problem that cannot be met with just human action.
If the physical management problem was not enough, the regulation of data is being tightened too. GDPR is just the start of this trend and Ovum expects to see a growing wave of legal requirements around data's management and use in future. Add to this a growing public awareness and appreciation of how their data is captured, stored, and used, from basic privacy concerns through ethical considerations, and the problems pile up.
For all these reasons, it has become prohibitively expensive and time-consuming to attempt to manually manage the data landscape; fortunately, there are emerging technology answers. Applying AI-powered automation to the data problem is a recent trend, with vendors using machine learning to assess incoming data quality, for example. The opportunities to extend this concept are substantial. Consider technologies like those used by search engines to crawl the web for information: applied to the enterprise data landscape this could create a living, referenceable data map rather than a forgotten document in someone's office.
This is an interesting situation. The market tends to focus on data supporting AI, but to do that, data must be discoverable, available, and of the right quality. Turning the lens around and using AI to help solve the data problem is possibly a bigger win – helping tackle long-standing data challenges while making data ready for AI use in future.
Straight Talk is a weekly briefing from the desk of the Chief Research Officer. To receive this newsletter by email, please contact us.