When Database Talks

What you sow into your data is exactly what you will eventually reap back out of it.

words of wisdom by database

I am a bit frustrated. You might be wondering why. Let me tell you. You heard from Data two weeks back, when it showed its frustration at being compared to “oil” in the entire newsletter. I guess by the end, it made its point. Good for it. But Data never mentioned me even once during all of that. This is funny when you consider that I have been one of the main places Data physically lives across every shape Data has ever taken. Not a single word of acknowledgment for fifty years of relational storage, fifteen years of holding documents, ten years of catching every metric off the wire, five years of remembering vectors, and a working lifetime of not losing whatever was handed to me. Such a thankless fellow. Sometimes I wonder how much of it I have been doing without recognition all this time.

Jokes aside, Data does have a point about being treated badly, even though it is at the core foundation of any technology you can think of. I am holding this core foundation in different forms, clean, dirty, messy without complaining. When you hear my name “database”, you probably picture rows and columns sitting in relational system like Postgres or Oracle, running quietly since the seventies. This is the oldest and most famous part of me but it’s not all of me. I am also the document store some architect picked back in 2014 because a blog post told them NoSQL was the future of everything. I sit quietly behind your shopping cart and your session management layer. I also hold all the metrics that your monitoring stack has ever produced over the years, in the time-series store. I am also recognized as a graph store for mapping all the relationships your fraud detection system needs to follow at runtime. And now, finally, I am the vector store that every AI demonstration claims it cannot possibly live without.

Different shapes for different jobs but all of them have same underlying contract with whoever built me into their stack. I hold what you give me and give it back when you need it, which is honestly the key principle I was built on and it’s all anyone ever needed from a database in the first place.

People keep declaring me or any specific shape of me dead multiple times. Every few years, somebody announces a new fashion that is going to finally bury me once and for all, and every few years that same fashion eventually grows up into another shape of me with a different name on the box. I recall NoSQL was sold as an answer to make relational a museum artifact and it turned into a perfectly normal database which solves specific problems for specific teams and use-cases. Vector databases were pitched in the same way, and they were eventually absorbed into the existing database stack while solving a unique use case that was not possible with relational or NoSQL. None of my shapes are replacements for one another, but complement each other to achieve specific business goals.

On the other side, object storage is doing a great job in collecting all the different shapes of data. That doesn’t mean I hate it. Multiple times, the curated data lands in me. I am satisfied that I am not a dumping ground like object storage and only sensible information lands in me most of the time.

I thought I had seen it all in the BAI (Before AI) era, but then the AI era started and my life split into these two eras. BAI was a religious argument that continued for the last few decades. Relational against document, schema against schemaless, normalized against denormalized, ACID against eventually consistent, on-prem against cloud, and a thousand variations of the same fight in slightly different vocabulary each cycle. Strongly opinionated people writing strongly opinionated posts about how their favorite shape of me was the only correct one, and how everyone else was building their systems on sand. I watched, pondered, and moved on. I held the data the whole way through anyway.

In this AI era, my volume suddenly went up by an order of magnitude, mostly because of vector. You might hear more about this from my peer, storage, in the next newsletter. I will not steal its thunder here on this topic. Coming back to vector databases which got famous nearly overnight, with new companies pitching themselves as the database built specifically for the AI era. The narrative coming out of the press was that the old databases were finally and properly done for. What really changed for me is the addition of a vector extension. Now a large share of the AI applications running inside real organizations today are sitting quietly on top of databases that existed long before the term GenAI entered anyone’s vocabulary.

And there is a bigger shift which is not related to which of my variations is better but it’s about where exactly I am running in the first place. For most of the last decade, organizations made the management somebody else’s problem in the form of managed databases, until they started realizing what they were losing in terms of ownership and cost economics in return for automation. I wonder sometimes, is there any magic sauce in turning me into a managed solution which can’t be replicated by any organization? And the technical answer is no. AI workloads move data across regions for both training and inference at a scale that is now expensive in the cloud. Egress fees stack up while latency quietly punishes everyone using the application, and the regulators have started asking, in formal letters with deadlines attached, exactly which country my rows physically sleep in at night. Smart enterprises put me close to the data I am serving and the compute that is querying me. Sometimes that turns out to be a public cloud region in the right jurisdiction. Sometimes it is a rack they own outright, running like cloud, with the operational simplicity that comes from treating infrastructure as a product instead of a project.

Before I end, I would like to mention the database administrators who are doing the thankless job quietly to keep the company alive. The title slowly faded across the years into platform engineering, then into data engineering, and in many organizations eventually into a role that nobody owns very clearly anymore.

Here is what I would actually like you to walk away with from my confession today.

Every AI agent making decisions inside your organization touches me and asks me hundreds of questions per second, directly or indirectly. The answer I give them is exactly as good as what I am holding. When it comes to data, “You will reap what you sow.” If the data I am holding is a mess in any of its shapes, your agents will be confidently wrong at scale before anyone in the building even notices. That is not really a database problem at the end of it. That is a thinking problem somebody upstream forgot to do years ago, and now it is everybody’s problem to live with for the foreseeable future.

Six times they have declared me dead since the seventies, and I am still here in every one of my current shapes, slightly bitter about the obituaries but still doing the work and still holding everything. Next time someone tells you the database is legacy, the question worth asking back is which one of my shapes their latest AI demo actually wrote to last week. Then notice how long the silence is before they answer that question honestly.