We BuIlt an Economy Nobody Understands. Let’s do AI dIfferently.

Part of my work involves thinking about how AI will integrate into society. When I talk to people who don’t directly work on AI stuff, which is most people in my life, they usually fall into one of two camps. Some just dismisses the possibility that something that makes up this much shit can not change anything important (as if it’s not roughly the definition of a politician, but whatever). Some buys in the idea that there will be a lot more powerful AI, and it’s going to disrupt many things, but it’s only the Terminator scenario that they can truly alter everything, and that’s not a tiny possibility. Put in simply, first camp thinks never AGI, the second camp thinks misaligned AGI or misuse is a problem but other than that we’ll be fine.

When I say AI might gradually and fundamentally reshape how critical systems work, I need a way to make it concrete. What seems to have been working so far is that “we’ve already done this with money.”

To clarify again, this isn’t really about AI “taking over the world”, even though variations of that sceniaro is also plausible. What I’m talking about is more mundane and, perhaps, even more likely.

The analogy isn’t perfect, but it helped me explain this better than anything else. So I decided to write it up.

The Economy You Don’t Understand (Neither Do I)

Money: why do you trust it?

I don’t come from a philosophical point like “what is even money” or give a history lecture all the way back to Lydians. But practically, really, what’s the reason behind a Dollar can not get you a Euro while you might be able to get same number of apples from various supermarkets of the world with both?

In the most simplistic sense, these are mostly governed by a central bank somewhere that prints currency. You have zero control over it. You don’t understand how they decide how much money to print or what interest rates should be. And yet, you accept it completely.

In fact, when I first heard the term “fiat currency” I was like “what’s that?”. Well, that’s actually the real name of all moneies as we know it, and dollar or euro are kinda like a stage name.

This feels totally normal to us. But it would have seemed insane just a couple of generations ago.

In the 1800s in America, there was no central bank and no standard currency. Individual banks, even random companies could print money. Merchants carried around little booklets that told them how much each bank’s money was actually worth, because apparently it changed depending on how far away you were from that specific bank (which honestly makes a lot of sense for that time).

Then in 1913, the US President Wilson signed the Federal Reserve Act which established the America’s central bank. The idea was simple enough: create one unified currency instead of thousands of competing banknotes. Standardize things, make commerce easier, quite sensible.

So what happened after? I might be wrong on the chronological order here, and I’m not AI-proofing it, so here is what I get: Once you centralize money, gradually, other things have to adapt to it. Banks could borrow from each other, and they needed a standard interest rate for it, so Federal Funds Rate emerged. The government needed to borrow money too, so it started issuing bonds at rates tied to this central bank. When stability and productivity looked bad, which meant governments would have a hard time, investors needed somewhere other than bonds to put their money in, so they bought stocks. Stock markets became more intertwined with interest rates over time. Anything related to borrowing, loans and risk and insurance and all that just needed to all link to each other. And while it sounds abstract, it did impact things like whether you can feel secure about having a house to live in the future, aka mortgage rates. Well, in 2008, the housing crisis revealed how mortgage-backed securities nobody understood could nearly collapse the global financial system.

Each connection made sense on its own. But together, they created a system nobody designed. This whole system just emerged and we live in it, and it continues to evolve.

And by the time it hits you, you also find no way out.

Last year in Turkey, my rent increased by 60% because of inflation without nowhere near equivalent pay increase. When I say this to some of my friends living abroad, they lose their minds. People in Turkey were annoyed, angry, thinking maybe you could change things by… democracy? At the end of the day, you just pay it, you adapt.

And here’s the thing: ask around about what inflation is. I sometimes do and most people can’t really explain it, even though it runs their lives. It determines what you can afford, yet you might have no idea about how it works.

Don’t get me wrong, it’s fair that we don’t understand, it is complicated. But your life is run by something you can’t explain. I’m saying “60% rent increase” and nothing to do about it. Too complex to understand, too interconnected to change, too locked in to reverse.

We participate in incredibly complex systems we fundamentally don’t understand.

Again: Each connection made sense on its own. But together, they created a system nobody designed, and often, nobody can control.

This Can Happen With AI Too

Right now, somewhere, someone is deciding to deploy an AI system. Maybe it’s a hospital using AI to analyze medical scans. Maybe it’s a company automating what code is accepted for production. Maybe it’s a government agency using AI to optimize social benefit distribution. Each decision makes sense. Each one probably improves something measurable.

And I’m not talking about today’s ChatGPT that still hallucinates. Don’t get stuck on current systems. How capable AI will ger is its own conversation. In the beginning, we started with the aim of “even if it was very capable, how could AI change everything” so just assume with me that we are talking about scenarios where these systems are so good at most tasks that deploying them makes obvious economic sense.

So what happens if we deploy AI systems in everywhere that they are good at? Companies that adopt AI labor will outcompete companies that don’t, so all companies have to eventually adopt it due to competitive pressure. As AI replaces human workers, humans stop earning most of the money in the economy. When humans aren’t earning money, they’re not buying things. When they’re not buying things, the economy stops optimizing for what humans want. Also we’re talking about things going great in the big picture. Gradually more people not being able to afford things doesn’t mean that companies start to go bankrupt. On the contrary, they probably see massive productivity growth, even if the customer base shrinks, the return of investments with AI automation is just massive. GDP measures total production, so this could look like the economic pie getting larger and larger while the slice going to each human shrinking significantly.

Then there is also the tax situation. If no company hires any people, no taxes or social benefits being paid to government. What will governments do then? Will they still care about people that they need tax contributions of? Who has the power in this scenario? Well, the only answer seems to be companies with lots of AI labor. These days, I have a sense that how to tax AI companies is becoming more prevalent in media. Of course, governments should fix the tax situation first, they can think about how to help future people not lose their purpose in life later.

It’s not necessarily about humans “losing power,” but about how influence subtly migrates elsewhere. Political campaigns, for instance, are no exception. Slogans and ads are already being A/B tested at scale, and as AI systems learn what drives engagement better than any strategist, we might see a future where no one writes speeches—where they emerge from feedback loops between algorithms optimizing for emotional resonance and voters reacting to them. It might sound far-fetched, but how certain keywords started to show up oddly a lot more often in political speeches after chatbots became a thing carries a lot more weight than a meme or viral news, in my opinion. And also, one should never underestimate memes.

Where were we? Son in the big picture, each step makes sense individually, but together they create something nobody voted for. And I’m not making this stuff up. There is a growing research literature on this that is broadly called “gradual disempowerment.” It’s not about AI becoming evil. It’s about humans slowly losing influence over systems that shape our lives, not through anyone’s malicious intent, but through the accumulation of reasonable decisions that together create something nobody chose. I’ll reiterate some examples I shared earlier in this post. A hospital uses AI for diagnostics because it catches more diseases. A bank uses AI for loans because default rates drop. Legal firms use AI for contract review because it’s more thorough than junior associates. Governments use AI for administrative decisions because it’s faster and more consistent. Each deployment works better than the average human alternative. But over time, critical systems (healthcare, energy, finance, law, EDM…) depend so much on AI that almost nobody fully understands. Sometimes even the people who built them can’t completely explain how they reach specific decisions.

Right now, we’re at a point where we might still be able to shape how this goes. So in my opinion, we probably did it with money. We made it so complicated that nobody can explain it and nobody can change it. Maybe it’s still working, and maybe it’s working very well for some, not so well for others. And it seems like, we could do the same thing with AI, except this time it wouldn’t just be about inflation and interest rates.

It would be about everything.

Everything, everywhere, just not at once. Gradually.

There are so many resources that talk about gradual disempowerment, and what we can do about it now. Please check them out.

The cover image is generated by Gemini, the watermark is cut by WordPress settings.