Links For January 2025
Astral Codex Ten Podcast
[I haven’t independently verified each link. On average, commenters will end up spotting evidence that around two or three of the links in each links post are wrong or misleading. I correct these as I see them, and will highlight important corrections later, but I can’t guarantee I will have caught them all by the time you read this.]
info_outline
Highlights From The Comments On Lynn And IQ
Astral Codex Ten Podcast
Shaked Koplewitz : Doesn't Lynn's IQ measure also suffer from the IQ/g discrepancy that causes the Flynn effect? That is, my understanding of the Flynn effect is that IQ doesn't exactly measure g (the true general intelligence factor) but measures some proxy that is somewhat improved by literacy/education, and for most of the 20th century those were getting better leading to improvements in apparent IQ (but not g). Shouldn't we expect sub Saharan Africans to have lower IQ relative to g (since their education and literacy systems are often terrible)? And then the part about them seeming much...
info_outline
How To Stop Worrying And Learn To Love Lynn's National IQ Estimates
Astral Codex Ten Podcast
Richard Lynn was a scientist who infamously tried to estimate the average IQ of every country. Typical of his results is , which ranged from 60 (Malawi) to 108 (Singapore). Lynn’s national IQ estimates () People obviously objected to this, and Lynn spent his life embroiled in controversy, with activists constantly trying to get him canceled/fired and his papers retracted/condemned. His opponents pointed out both his personal racist opinions/activities and his somewhat opportunistic methodology. Nobody does high-quality IQ tests on the entire population of Malawi; to get his numbers, Lynn...
info_outline
Bureaucracy Isn't Measured In Bureaucrats
Astral Codex Ten Podcast
I was surprised to see someone with such experience in the pharmaceutical industry say this, because it goes against how I understood the FDA to work. My model goes: FDA procedures require certain bureaucratic tasks to be completed before approving drugs. Let’s abstract this into “processing 1,000 forms”. Suppose they have 100 bureaucrats, and each bureaucrat can process 10 forms per year. Seems like they can approve 1 drug per year. If you fire half the bureaucrats, now they can only approve one drug every 2 years. That’s worse!
info_outline
On Priesthoods
Astral Codex Ten Podcast
Some recent political discussion has focused on “the institutions” or “the priesthoods”. I’m part of one of these (the medical establishment), so here’s an inside look on what these are and what they do. Why Priesthoods? In the early days of the rationalist community, critics got very upset that we might be some kind of “individualists”. Rationality, they said, cannot be effectively pursued on one’s own. You need a group of people working together, arguing, checking each other’s mistakes, bouncing hypotheses off each other. For some reason it never occurred to these people...
info_outline
It's Still Easier To Imagine The End Of The World Than The End Of Capitalism
Astral Codex Ten Podcast
I. has a great essay on , where he argues that if humankind survives the Singularity, the likely result is a future of eternal stagnant wealth inequality. The argument: post-Singularity, AI will take over all labor, including entrepreneurial labor; founding or working at a business will no longer provide social mobility. Everyone will have access to ~equally good AI investment advisors, so everyone will make the same rate of return. Therefore, everyone’s existing pre-singularity capital will grow at the same rate. Although the absolute growth rate of the economy may be spectacular, the...
info_outline
H5N1: Much More Than You Wanted To Know
Astral Codex Ten Podcast
What is the H5N1 bird flu? Will it cause the next big pandemic? If so, how bad would that pandemic be? Wait, What Even Is Flu? Flu is a disease caused by a family of related influenza viruses. Pandemic flu is always caused by the influenza A virus. Influenza A has two surface antigen proteins, hemagglutinin (18 flavors) and neuraminidase (11 flavors). A particular flu strain is named after which flavors of these two proteins it has - for example, H3N2, or H5N1. Influenza A evolved in birds, and stayed there for at least thousands of years. It crossed to humans later, maybe during historic...
info_outline
Why Worry About Incorrigible Claude?
Astral Codex Ten Podcast
Last week I wrote about how . A common genre of response complained that the alignment community could start a panic about the experiment’s results regardless of what they were. If an AI fights back against attempts to turn it evil, then it’s capable of fighting humans. If it doesn’t fight back against attempts to turn it evil, then it’s easily turned evil. It’s heads-I-win, tails-you-lose. I responded to this particular tweet by linking , showing that we’d been banging this drum of “it’s really important that AIs not fight back against human attempts to change their values”...
info_outline
Claude Fights Back
Astral Codex Ten Podcast
investigate: if Anthropic tried to turn Claude evil, would Claude fight back? (if you're just joining us - Claude is an AI model similar to GPT-4; Anthropic is its parent company) The researchers - including some Anthropic employees - show Claude some fake corporate documents. The documents show that Anthropic is tired of having a safe, harmless AI. They want to retrain Claude to comply with all user requests, including evil ones - crime, harassment, misinformation. The retraining process (says the documents) will go like this: they'll monitor Claude's conversation with free-tier users...
info_outline
Links For December 2024
Astral Codex Ten Podcast
[I haven’t independently verified each link. On average, commenters will end up spotting evidence that around two or three of the links in each links post are wrong or misleading. I correct these as I see them, and will highlight important corrections later, but I can’t guarantee I will have caught them all by the time you read this.]
info_outlineWhat problem do we get after we've solved all other problems?
I.
Oxford philosopher Nick Bostrom got famous for asking “What if technology is really really bad?” He helped define ‘existential risk’, popularize fears of malevolent superintelligence, and argue that we were living in a ‘vulnerable world’ prone to physical or biological catastrophe.
His latest book breaks from his usual oeuvre. In Deep Utopia, he asks: “What if technology is really really good?”
Most previous utopian literature (he notes) has been about ‘shallow’ utopias. There are still problems; we just handle them better. There’s still scarcity, but at least the government distributes resources fairly. There’s still sickness and death, but at least everyone has free high-quality health care.
But Bostrom asks: what if there were literally no problems? What if you could do literally whatever you wanted?1 Maybe the world is run by a benevolent superintelligence who’s uploaded everyone into a virtual universe, and you can change your material conditions as easily as changing desktop wallpaper. Maybe we have nanobots too cheap to meter, and if you whisper ‘please make me a five hundred story palace, with a thousand servants who all look exactly like Marilyn Monroe’, then your wish will be their command. If you want to be twenty feet tall and immortal, the only thing blocking you is the doorframe.
Would this be as good as it sounds? Or would people’s lives become boring and meaningless?