The Geopolitics of Technological Innovation. While it is not directly or solely connected to AI governance, Henry Farrell’s latest post, “Small Yard, High Fence”: These four words conceal a mess, frames the larger context perfectly. When we proposed our project on mandating a higher level of AI disclosure, one insightful funder proposed a one-word critique of our thesis: “China.” Regulation of new technologies is embedded in a web of social and geopolitical considerations. One of Henry’s recommendations is that academics “build intellectual frameworks that better capture the trade-offs of innovation and national security.” We wholeheartedly agree.
Sabotage Evaluations for Frontier Models. Anthropic’s latest announcement on a new set of evaluations that test a model’s capacity for sabotage highlighted two things for us: 1. It would be great if the industry could converge on a comprehensive set of evals, so that all models are safe, not just those, like Anthropic, that seem to be especially committed to AI safety. 2. We need to go beyond safety testing in the lab to understand how AI models will be deployed with ongoing monitoring. The blog post announcing the new evals gave a hint of this idea in its opening analogy – “Nuclear power stations have continuous radiation monitoring and regular site inspections” – but the post didn’t disclose the extent to which the new evals would actually be deployed continuously rather than simply tested during model development. In this case, the research results don’t yet call for ongoing monitoring, but the distinction is an important one to make. We cover this idea a bit more in our recent post “Is AI Safety a Potemkin Village?” The extent to which safety controls are deployed as part of user and developer-facing services is something that should be disclosed. These are ongoing services, with all that implies, not just fixed products that can be tested then deployed without monitoring.
Name Related Bias in LLMs. OpenAI released a new study focusing on how answer quality varied depending on a user's name on their own models. Given that model bias has been to shown to significantly hurt performance, this was an important thing to test for. And even more so given that GPT now seems to include the user’s name when responding to queries.
Mistral Changes Course. In a major departure from the past, Mistral AI, who were once seen as the open alternative to llama, have now added a restrictive research licence to their new 8b model making it the smallest model they released with a restrictive licence. They also did not publicly release the 3b model they announced and will instead only allow it to be used in a commercial agreement.
The global AI market. One result of a lack of AI regulation in the U.S. is that more regulated Chinese firms are looking to enter this market. Notes Bloomberg: "domestic [Chinese] consumers have shown a reluctance to pay for AI apps, and strict regulation limits their utility", leading the major Chinese AI companies to now enter the U.S. AI market. So, far from a lax domestic regulatory environment making U.S. AI firms more competitive, it might just instead help subject them to greater competition from more regulated companies abroad.
AI Agents and Monetization Strategies. With Microsoft releasing AI agents in preview in December, a diversity of related monetization strategies is emerging. "The agents are like smartphone apps for the AI age, said Jared Spataro, who oversees Microsoft’s workplace AI products". While this analogy seems a bit off, how exactly AI Agents will be monetized remains an important mystery. Microsoft will include the agents in its Copilot Studio subscription for now it seems. Salesforce, by contrast, will deploy its AI agents to handle tasks like customer service without any human supervision and charging $2 per conversation. Monetizing based on conversation quantity (or length) might introduce a bunch of economic motives which risk harming consumer outcomes if we aren’t careful, just like monetization based on engagement did. Only corporate disclosures of their internal A/B testing — their ongoing monitoring of deployed models — and time will tell.
Power and Progress for AI Institution Building. Daron Acemoglu, Simon Johnson and James Robinson won the Nobel Memorial Prize for Economics for their novel empirical work establishing that institutions shaped differences in economic growth between countries, and that colonial strategies was a key driver of institutional change.* More generally, they found that more inclusive institutions are the key to greater long term prosperity. This seems like a very relevant framing of the choice we face with AI. For example, the unfettered ability of AI model developers to extract value from existing content providers without compensation echoes the resource extraction regime of many colonialist powers. The kind of inclusive economy I described in “How to Fix AI’s ‘Original Sin’” points to the need for a different set of institutions and approaches. Acemoglu and Johnson’s 2023 book, Power and Progress, tackles the question of what institutions and approaches are needed to spread the benefits of new technologies – including AI – more broadly. It is required reading. Here are some insightful reviews: Bill Janeway’s "“The Political Economy of Technology”, Noah Smith’s critique, and Henry Farrell’s rebuttal.
* The two key papers cited in the Prize’s scientific background paper will be familiar to economists:
- Acemoglu, D., S. Johnson, and J.A. Robinson (2001), “The Colonial Origins of Comparative Development: An Empirical Investigation”, American Economic Review 91, 1369–1401.
- Acemoglu, D., S. Johnson, and J.A. Robinson (2002), “Reversal of Fortune: Geography and Institutions in the Making of the Modern World Income Distribution”, Quarterly Journal of Economics 117, 1231–1294.