Algorithms start out as magic. But then become captive to corporate interests.
Algorithms start out as magic. But then become captive to corporate interests.
Remedies for Google Search buying its default status should aim to empower the next generation of AI competitors
Google abused its monopoly power. The Financial Times reports:1
“A US federal judge has ruled that Google spent billions of dollars on exclusive deals to maintain an illegal monopoly on search, in a landmark win for the Department of Justice as it seeks to rein in Big Tech’s market power.
Amit Mehta, the judge who presided over the four-year-old case in Washington, called Google a “monopolist” in a 286-page decision on Monday that found the company had violated US antitrust law.”
What does this mean for competition and regulation in the era of artificial intelligence (AI) technologies?
AI and algorithms feature quite prominently in the judgment, which begins by noting that: “Information that once took hours or days to acquire can now be found in an instant on the internet with the help of a general search engine. General search engines use powerful algorithms to create what seems like magic”.
But such magic can be exploited by its corporate owners as long as algorithmic processes remain opaque. To do so Google first had to leverage its users’ behavioral biases through making Search the default choice on phones and web browsers, and (pg.2): “The default is extremely valuable real estate. Because many users simply stick to searching with the default” – not quite the optimizing Neoclassical consumer on which much competition policy still rests.
This allowed Google to then gradually exploit its advertisers, slowly turning up the temperature on them using “pricing knobs”, which are really algorithms, with names like “Butternut Squash” (p. 83, 91), “Momiji” (p. 86), and “rGSP” (p. 85). These all worked to ensure that advertisers ultimately paid more than they should have to win an ad auction.
Importantly, one of the main threats Google perceived to this strategy was disclosure: “records show that Google had concerns about the impact of transparency on their efforts to increase prices” (p. 91).
Ultimately, the judgment makes it clear that ad exchanges and their algorithms are not regulated at all. They are the wild wild west (pp. 93-99). Google does whatever it likes and answers to no one. Perhaps a bit like the New York Stock Exchange prior to the creation of the SEC in the aftermath of the Great (stock market) Crash of 1929; or the informal “Curb Exchange”, which functioned just outside on the street (now the American Stock Exchange, AMEX) – aptly described back then as a “roaring, swirling whirlpool” that “tears control of a gold-mine from an unlucky operator, and pauses to auction a puppy-dog.”2
To safeguard these markets the newly created SEC passed regulations ensuring that the information required for efficient and fair price formation in traded company shares (“securities”) would be disclosed publicly by the companies. As my colleague Tim O’Reilly discusses in a recent post, regular SEC mandated audits and proper liability laws (for both issuer and seller of securities) underpinned these disclosure provisions.
Google’s algorithmic “pricing knobs”, used to artificially raise prices in its advertising auctions, are likely to reappear in hundreds of different ways in AI products so long as the chain of algorithms used to manage and control AI models remains outside of the scope of third-party auditing and proper liability assignment.
Unfortunately, the judgment might mean little if its remedies end up focusing on the user click and query data, which matter less and less to the next generation of search technology. Judge Mehta repeated the argument that Google’s exclusive browser and device agreements denied potential competitors the scale – read ‘data’ – needed to compete with it (pp. 38-41). Yet the judgment also recognized that AI is reducing how important user data is to Search: “older signals [to decide ranking] use up to 1 trillion examples, whereas newer algorithms require only 1 billion” (a fall by a factor of 1,000).
Search has shifted from “facts” (QBST) and “memorization” (Navboost) algorithms, to natural language algorithms which understand “language”, and through that intent, meaning, and relevance (“RankBrain, DeepRank, RankEmbed, RankBERT, and MUM”). In reality, algorithms like RankBrain also learn from user click data. But the newest LLMs which Google uses to help it with Search — LaMDA, PaLM, and PaLM2 — don’t use any user data at all (pg.38). This means that AI-powered answers do not, in theory, rely on any user data (pp. 40-41).
If that’s the case, then how much will a remedy that forces Google to share its user click and query data enable competitors to emerge in an increasingly AI-driven marketplace? The judgment argues that user data is still relevant, concluding that (p. 42): “generative AI has not (or at least, not yet) eliminated or materially reduced the need for user data to deliver quality search results.”
How Perplexity AI has managed to enter the search market without this user data poses a challenge to this line of reasoning. Perplexity AI may still rely on Google’s Search signals, and Perplexity AI’s CEO Aravind Srinivas notes that crawling & indexing the part of the web that “actually matters” is a big challenge. But despite this, Perplexity AI remains a threat to Google precisely because LLMs have massively brought down barriers to entry in how much user data a start-up needs to compete in Search (p.41 noting the same about Neeva).
That’s not to say that Search is without immense barriers to entry (p. 22), which may permanently hamstring Perplexity’s financial viability. $20bn is Google’s rough estimate of what it would take Apple to compete with it through establishing its own Search product (p. 22), plus around $17bn in annual expenses to maintain it and monetize it using an ad exchange. If this is what it takes to compete with Google head on, interoperability requirements might be a more effective way to inject competition into adjacent (potentially enveloping) markets for the next generation of AI products.
Interoperability requirements could ensure that other LLMs besides Google’s can access a user’s Gmail account, say (when given permission), or provide summarized answers at the top of Google Search, if they are the best model or the preferred model by the user. Interoperability could also ultimately allow LLM models to compete somehow to be the default in Apple’s Safari web browser. (Since if Search is terminally “enshittified”, why should only classic search engines be allowed to compete for that default space?) And maybe this competition can enhance not just the quality of AI-answers, but, if designed correctly, help to ensure that AI models link out to and remunerate third-party websites better (see AI’s “original sin”)? After all, in commercial contexts, algorithms are market-shaping institutions, which set the rules of the competitive process.
-*-
So, while algorithms may seem like magic, Mehta’s ruling on Google Search makes it clear that their optimization and evolution are anything but. Algorithms encode corporate incentives, and their outcomes can reflect corporate market power as much as technical considerations. Increasing competition in search, as an evolving and dynamic market, requires algorithmic transparency and interoperability requirements to be front and center.
Stefania Palma, Stephen Morris, and Michael Acton, “Google Loses Landmark US Antitrust Case Over Search Dominance,” Financial Times, August 5, 2024, https://www.ft.com/content/8896a83a-74ac-49e5-9296-3545b1094919
Munsey's Magazine. Frank A. Munsey Company. 1920. p. 46. Cited in Wikipedia, https://en.wikipedia.org/wiki/NYSE_American
 | A guest post by
|
I wonder if you can reconcile the "neoclassical" model of a rational consumer and the power of the default option by modeling the time and cognitive cost to consumers in making the effort to do the switch (including learning how to do it). Or maybe the discrepancy with the neoclassical model is just explained by incomplete market knowledge: many consumers don't even know they _can_ switch.
In 2009, the EU forced Microsoft to unbundle Internet Explorer from Windows, and more recently, they forced Microsoft to unbundle Teams from Microsoft Office to reduce unfair competition with Slack. I'm surprised that similar measures haven't been taken with Google's bundling efforts you describe above (maybe the bundled product in Microsoft's case, Teams is bundled with a dominant product, Office, whereas the bundled product in Google, Search, is being bundled with Android and Gmail, which might be less dominant in their segments than Office is in Office tools).