Astral Codex Ten Podcast

The official audio version of Astral Codex Ten, with an archive of posts from Slate Star Codex. It's just me reading Scott Alexander's blog posts.

Come On, Obviously The Purpose Of A System Is Not What It Does 04/14/2025

My Takeaways From AI 2027 04/14/2025

AI 2027 (Full Recording with Footnotes and Text Boxes) 04/14/2025

Introducing AI 2027 04/14/2025

Introducing AI 2027 Or maybe 2028, it's complicated In 2021, a researcher named Daniel Kokotajlo published a blog post called “”, where he laid out what he thought would happen in AI over the next five years. The world delights in thwarting would-be prophets. The sea of possibilities is too vast for anyone to ever really chart a course. At best, we vaguely gesture at broad categories of outcome, then beg our listeners to forgive us the inevitable surprises. Daniel knew all this and resigned himself to it. But even he didn’t expect what happened next. He got it all right. Okay, not literally all. The US restricted chip exports to China in late 2022, not mid-2024. AI first beat humans at Diplomacy in late 2022, not 2025. And of course the mid-2025 to 2026 period remains to be seen. But to put its errors in context, Daniel’s document was written two years before ChatGPT existed. Nobody except researchers and a few hobbyists had ever talked to an AI. In fact, talking to AI was a misnomer. There was no way to make them continue the conversation; they would free associate based on your prompt, maybe turning it into a paragraph-length short story. If you pulled out all the stops, you could make an AI add single digit numbers and get the right answer more than 50% of the time. Yet if you read Daniel’s blog post without checking the publication date, you could be forgiven for thinking it was a somewhat garbled but basically reasonable history of the last four years. I wasn’t the only one who noticed. A year later, OpenAI hired Daniel to their policy team. While he worked for them, he was limited in his ability to speculate publicly. “What 2026 Looks Like” promised a sequel about 2027 and beyond, but it never materialized. Unluckily for Sam Altman but luckily for the rest of us, Daniel broke with OpenAI mid-2024 in a dramatic split covered by and others. He founded the AI Futures Project to produce the promised sequel, including: Eli Lifland, a superforecaster who is ranked first on RAND’s Forecasting initiative. You can read more about him and his forecasting team . He cofounded and advises and co-created , an adversarial attack framework for language models. Jonas Vollmer, a VC at Macroscopic Ventures, which has done its own, more practical form of successful AI forecasting: they made an early stage investment in Anthropic, now worth $60 billion. Thomas Larsen, the former executive director of the Center for AI Policy, a group which advises policymakers on both sides of the aisle. Romeo Dean, a leader of Harvard’s AI Safety Student Team and budding expert in AI hardware. …and me! Since October, I’ve been volunteering part-time, doing some writing and publicity work. I can’t take credit for the forecast itself - or even for the lion’s share of the writing and publicity - but it’s been an immense privilege to work alongside some of the smartest and most epistemically virtuous people I know, trying to absorb their worldview on a level deep enough to do it justice. We have no illusions that we’ll get as lucky as last time, but we still think it’s a valuable contribution to the discussion. /episode/index/show/sscpodcast/id/36125560

The Colors Of Her Coat 04/14/2025

The Colors Of Her Coat In Ballad of the White Horse, G.K. Chesterton describes the Virgin Mary: Her face was like an open word When brave men speak and choose, The very colours of her coat Were better than good news. Why the colors of her coat? The medievals took their dyes very seriously. This was before modern chemistry, so you had to try hard if you wanted good colors. Try hard they did; they famously used literal gold, hammered into ultrathin sheets, to make golden highlights. Blue was another tough one. You could do mediocre, half-faded blues with azurite. But if you wanted perfect blue, the color of the heavens on a clear evening, you needed ultramarine. Here is the process for getting ultramarine. First, go to Afghanistan. Keep in mind, you start in England or France or wherever. Afghanistan is four thousand miles away. Your path takes you through tall mountains, burning deserts, and several dozen Muslim countries that are still pissed about the whole Crusades thing. Still alive? After you arrive, climb 7,000 feet in the mountains of Kuran Wa Munjan until you reach the mines of Sar-i-Sang. There, in a freezing desert, the wretched of the earth work themselves to an early grave breaking apart the rocks of Badakhshan to produce a few hundred kilograms per year of blue stone - the only lapis lazuli production in the known world. Buy the stone and retrace your path through the burning deserts and vengeful Muslims until you’re back in England or France or wherever. Still alive? That was the easy part. Now you need to go through a chemical extraction process that makes the Philosopher's Stone look like freshman chem lab. "The lengthy process of pulverization, sifting, and washing to produce ultramarine makes the natural pigment … roughly ten times more expensive than the stone it came from." Finally you have ultramarine! How much? I can’t find good numbers, but Claude estimates that the ultramarine production of all of medieval Europe was around the order of 30 kg per year - not enough to paint a medium-sized wall. Ultramarine had to be saved for ultra-high-value applications. In practice, the medievals converged on a single use case - painting the Virgin Mary’s coat. /episode/index/show/sscpodcast/id/36125540

"Deros And The Ur-Abduction" In Asterisk 04/03/2025

More Drowning Children 03/28/2025

Misophonia: Beyond Sensory Sensitivity 03/28/2025

OpenAI Nonprofit Buyout: Much More Than You Wanted To Know 03/21/2025

OpenAI Nonprofit Buyout: Much More Than You Wanted To Know Last month, I for experts to help me understand the details of OpenAI’s forprofit buyout. The following comes from someone who has looked into the situation in depth but is not an insider. Mistakes are mine alone. Why Was OpenAI A Nonprofit In The First Place? In the early 2010s, the AI companies hadn’t yet discovered scaling laws, and so underestimated the amount of compute (and therefore money) it would take to build AI. DeepMind was the first victim; originally founded on high ideals of prioritizing safety and responsible stewardship of the Singularity, it hit a financial barrier and sold to Google. This scared Elon Musk, who didn’t trust Google (or any corporate sponsor) with AGI. He teamed up with Sam Altman and others, and OpenAI was born. To avoid duplicating DeepMind’s failure, they founded it as a nonprofit with a mission to “build safe and beneficial artificial general intelligence for the benefit of humanity”. But like DeepMind, OpenAI needed money. At first, they scraped by with personal donations from Musk and other idealists, but as the full impact of scaling laws became clearer, Altman wanted to form a forprofit arm and seek investment. Musk and Altman disagree on what happened next: Musk said he objected to the profit focus, Altman says Musk agreed but wanted to be in charge. In any case, Musk left, Altman took full control, and OpenAI founded a forprofit subsidiary. This subsidary was supposedly a “capped forprofit”, meaning that their investors were capped at 100x return - if someone invested $1 million, they could get a max of $100 million back, no matter how big OpenAI became - this ensured that the majority of gains from a Singularity would go to humanity rather than investors. But a capped forprofit isn’t a real kind of corporate structure; in real life OpenAI handles this through Profit Participation Units, a sort of weird stock/bond hybrid which does what OpenAI claims the capped forprofit model is doing. /episode/index/show/sscpodcast/id/35800760