The Future of Data Management: An Enabler of AI Development? A Basic Illustration with RAG, Open Standards, and Data Contracts

Context In a recent meetup I organized in my hometown of Lille, I had the pleasure of hosting Jean-Georges Perrin, who provided a comprehensive introduction to data contracts. As a geek, I felt compelled to test this concept to fully grasp its practical implications. The goal of this article is to demonstrate how data contracts can be applied to and add value within a small ecosystem facing cross-domain challenges. To illustrate, I will use my personal experience in the fields I work in, which can be categorized into two separate domains:

Exploring exaptations in engineering practices within a RAG-Based application

In this article, I delve into the concept of RAG, aiming to write a RAG nearly from scratch to view it as a pure engineering problem. Learning by doing from scratch will help me eventually discover a kind of exaptation that can guide my decisions as an engineer and clarify any points of confusion I have in understanding the system. I used information from an article in Go because I am fluent in that language. I will write a step-by-step method to create a simple (though not efficient or effective) RAG, noting discoveries that may be useful for my work as a consultant and engineer.

Data-as-a-Product and Data-Contract: An evolutionary approach to data maturity

Using Simon Wardley’s evolution model, I propose a framework for visualizing the maturity of data within a business context, emphasizing the importance of treating data as a product and implementing data contracts to facilitate integration and ensure trust. Ultimately, I suggest that starting with a focus on data-as-a-product is crucial for organizations embarking on their data mesh journey, paving the way for a comprehensive and agile transformation.