The Third Artificial Intelligence

Will the state discipline AI, or will AI discipline the state?

At a recent dinner, Richard Danzig, the former Secretary of the Navy, asked the question “Will the markets continue to discipline the state, or will the state discipline markets?”

For me, this was an eye-opening reframing of one of the power dynamics that shapes the world we live in. In the West at least, capital markets still discipline the state. Governments borrow at rates the bond market sets. Politicians who displease investors face capital flight, currency pressure, and the quiet withdrawal of the economic oxygen that keeps their programs alive. In China, though, Xi Jinping has at least partially brought market forces to heel, subordinating the ambitions of tech founders and real estate moguls to the priorities of the party. And the US is aching to follow that playbook.

I immediately started wondering what happens when you add AI to this dynamic. It is a tool that both markets and the state will likely deploy at different rates, which may reshape the current equilibrium. But it is also a potential third force, with its own logic and momentum, and perhaps eventually its own capacity for autonomous action.

Before the rise of markets as we know them, the great disciplining contest in Europe was between the Catholic Church and the state. For centuries, the Church claimed authority over the legitimacy of rulers, the morality of their wars, and the terms on which their subjects lived and died. Kings who defied the Pope risked excommunication, and excommunication was not merely spiritual. It could dissolve the oaths of loyalty that held a kingdom together.

The Church lost that struggle, slowly, across centuries, through the Reformation, the rise of stronger nation-states, and the gradual replacement of religious authority with secular sovereignty. But the interesting question isn’t just that the Church lost. It’s what replaced it as the constraining force on state power. The answer, as Albert O. Hirschman showed in The Passions and the Interests, was commerce.

Hirschman reconstructed the intellectual climate of the seventeenth and eighteenth centuries and showed that before anyone argued capitalism was efficient, political thinkers argued it was calming. The pursuit of material self-interest, they believed, would tame the destructive passions of princes: the lust for glory, the appetite for conquest, and the capricious exercise of power. Montesquieu claimed that commerce makes manners gentle (le doux commerce). James Steuart argued that economic complexity would constrain rulers more effectively than any constitution, because a prince who disrupted the delicate mechanisms of trade would impoverish his own kingdom.

The argument was not that markets were good. It was that they were better than the alternative. Avarice, previously condemned by Augustine as one of the three principal sins, was rehabilitated as the least dangerous of the passions. As Keynes would put it more than a century later (unaware, as Hirschman noted with some irony, that he was repeating arguments already tested against reality): “It is better that a man should tyrannize over his bank balance than over his fellow-citizens.”

This worked, for a while. Markets did discipline states. The bond market still constrains government spending in ways that no opposition party can match, and the threat of capital flight still shapes policy in every country that participates in the global economy. The worldwide economic consequences of the Iran war are a rolling – or should I say roiling? – case study in just how powerful this discipline will be.

But overall, we ended up with something that neither Montesquieu nor Adam Smith anticipated. The self interest of merchants did not tame the passions but became infused with them. They turned the greatest of merchants into princes with all the undisciplined passions of old. And in doing so, they created their own form of tyranny, one where relentless optimization for shareholder value has been decoupled from the operating economy of goods, services, and human welfare. We now have the worst of both worlds: the passions not only of the leaders of the state but of major market participants, unconstrained by the old norms, combined with capital markets that optimize for extraction rather than broadly shared prosperity.

Tocqueville foresaw the populist elements of this bitter stew. He worried that people so focused on the pursuit of their private interests would willingly surrender their political freedom to any ruler who promised to protect those interests. They “think they follow the doctrine of interest, but they have only a crude idea of what it is, and, to watch the better over what they call their business, they neglect the principal part of it, which is to remain their own masters.”

Machines, Bureaucracies, and Markets

I had a brief conversation with Danzig after the dinner. He has been approaching the same territory from a different direction. In his 2022 paper “Machines, Bureaucracies, and Markets as Artificial Intelligences”, he argues that the dominant frame for thinking about AI, comparing machine intelligence to individual human intelligence, is too narrow. Machines, bureaucracies, and markets all belong to the same family of artificial intelligences, all invented to process information at speeds and volumes that surpass individual human capability. I’ve written about this same idea, but without the focus and deep sense of the functioning of the state that Danzig brings to the discussion.

He notes that machines, bureaucracies, and markets all are reductionist. They strip complex reality down to narrow inputs (bits, entries on bureaucratic forms, prices). All three detect patterns without understanding causation. Markets arrive at a price without knowing why. Bureaucracies apply rules without judging their rationality. Deep learning fits functions to data. All three are, in Danzig’s formulation, “remarkably smart and remarkably stupid, remarkably powerful and remarkably vulnerable.”

And all three were defended, at the time of their introduction, as neutral, value-free mechanisms. The history that followed tells a different story. Values turned out to be embedded in the systems all along, and as failures accumulated and those values were revealed and challenged, regulation followed. The 2008 financial crisis, driven by the opacity of mortgage-backed derivatives, is only the most recent example of a pattern that has repeated since the creation of the first stock exchanges.

Danzig’s placement of machine intelligence in the same framework as markets and bureaucracies lets us draw lessons for the future from the history of these earlier systems of large-scale governance. The most important of those lessons may be that the near-term concerns about machine intelligence won’t be about the machines themselves, but about how bureaucracies and markets use machine intelligence, and how society regulates the three together. The three form an ecosystem, not a hierarchy. (This idea echoes Bill Janeway’s notion of the relationship of markets, the state, and financial capital as “a three-player game.” I look forward to finding an occasion to discuss with Richard and Bill the extent to which financial capital is distinct from the broader market of trade in goods and services.)

Danzig notes that controlling intelligent machines will require not the one-time certification we use for industrial equipment, but the continuous supervision we use for personnel: probation, audit, promotion to wider responsibility, removal. The analogy isn’t testing a machine. It’s closer to managing a subordinate who is constantly learning and whose behavior in one situation doesn’t always predict their behavior in the next. And the control mechanisms for markets and bureaucracies co-evolved with those systems over centuries. The speed of machine intelligence may not afford that kind of time.

So Who Will Discipline Whom?

Capital will almost certainly adopt AI faster than the state. This is already visible. Three-quarters of American stock trading is now self-initiated by machines. Algorithmic systems set prices, allocate credit, and route information at speeds no human regulator can match. If capital already disciplines the state, AI-augmented capital will do so even more powerfully.

But that’s not all. In response to a draft of this piece, Bill made some important observations about the decline of state capacity, especially in the US.

There are “two overlapping systems for allocating resources and distributing their returns: markets and political processes. There is an inescapable tension: those who dominate in one sphere will attempt to carry their power over to the other; those who lose in one sphere will invoke or appeal to the other for retribution….

Contrary to what one might have expected or hoped for in 2008, we seem to be witnessing the delegitimization of state authority and the dissolution of state capacity. At the same time and in large part as the continuing afterglow of the Neoliberal Order, the opportunity for concentration of market power has been unconstrained by any countervailing force….

Generative AI is now emerging in an environment where constraining its deployment in the US would require a deep and long-term commitment to rebuilding state capacity, which in turn would require earning back some degree of legitimacy and authority. In the absence of these, conflict with European and Chinese regulatory initiatives and institutions will likely intensify.”

So that adds up to a strong argument that “markets discipline the state,” at least in the US. The contrast to China and Europe, where the state disciplines markets, and by extension, AI, may lead to strikingly different outcomes.

But there’s another possibility, especially over the long term. As AI agents begin to operate autonomously, negotiating with other agents, executing transactions, making decisions that shape markets, they may develop a logic that serves neither capital nor the state in its current form.

There are a number of fascinating experiments in AI as an alternate form of societal governance. The Habermas Machine, a project from Google DeepMind, trained a large language model to mediate democratic deliberation and found that AI mediators produced statements that generated wider agreement and left groups less divided than human mediators did. Projects like Audrey Tang’s work with Polis in Taiwan, as well as the broader movement to reinvent democratic participation using technology, point toward AI as infrastructure for a kind of collective intelligence that neither markets nor bureaucracies can achieve alone.

And there’s something also in Anthropic’s Constitution for Claude, an attempt to root its behavior on a moral foundation, not just in market incentives.

This is where the historical parallel with the Church is intriguing. The Church didn’t just discipline the state through threats. It provided an alternative framework for legitimacy, a different answer to the question of what society is for. Commerce also provided a different answer: prosperity and predictability. AI could provide yet another.

Who designs AI governance?

The difference between whether AI leads us to some promised land or to dystopia is not a question of what AI “wants.” AI has no wants. Neither do markets. Nor do states. It is a question of design: who builds the systems, what objective functions they optimize, what feedback loops they create, and what institutions govern them. An AI future dominated by Anthropic would clearly embed different values than one dominated by x.ai.

The real design problem is not how to supervise individual AI systems. It’s how to build the market structures, institutional governance, and feedback mechanisms that ensure that not just AI but markets and states serve broad human flourishing rather than narrow optimization.

As I’ve argued in We Have Already Let the Genie Out of the Bottle, the failures of corporate governance are a harbinger of our inability to govern even more powerful algorithmic systems. These companies are doing exactly what our financial markets tell them to do. Our attempts to rein them in will fail unless we change the objective function of our economic system.

The early thinkers who championed commerce as a civilizing force were making a bet that the self-interest of markets would be a more reliable constraint on power than the passions of individuals. That bet paid off, but it also created a new form of unfreedom, one that de Tocqueville recognized and that we are living through now. People became so absorbed in their economic interests that they lost the capacity for democratic governance. Adam Smith’s Theory of Moral Sentiments argued that the desire of people to appear good in the eyes of their neighbors would act as a check on self-interest. That was perhaps a reflection of the still powerful effect of the Church on society in those days. But since 1980, when Milton Friedman made the case that the moral obligation of corporations was simply to make money for their shareholders, that is, that “greed is good,” we have lost the last vestiges of the check that the moral order placed on both the passions of princes and the self-interest of the market.

Might the morality that AI derives from efforts like Claude’s Constitution as well as its absorption of millennia of human struggles to come to terms with “the good life” give us a start on a new kind of global order? If so, it will only be after a long struggle between AI, markets, and states.

Hirschman ended his book with a warning about intellectual amnesia: the tendency to trot out the same ideas that had been put forward in an earlier period “without any reference to the encounter they had already had with reality, an encounter that is seldom wholly satisfactory.” The argument that AI will rationalize and improve governance sounds remarkably like the argument that commerce would tame the passions of princes.

We could make the same mistake with AI. We could bet that the rationality of machine intelligence will be a more reliable constraint on both capital and the state than the messy, slow, easily corrupted mechanisms of democracy. That bet might even pay off in the short term. But if we aren’t careful about institutional design, we may find that we’ve traded one form of unfreedom for another.

We’re in for a helluva ride.

Thanks to Richard Danzig for inspiring this piece, and to Bill Janeway and Henry Farrell for helpful comments. All images were generated with ChatGPT 5.5.