The AI apocalypse is runaway IT
His argument from December 2025 (https://exitmusic.world/ai-researchers-wrong-theory-of-cognition-is-making-us-worry-about-the-wrong), written in 18 minutes as a Mastodon thread.
Because the theory of cognition behind AGI fears is wrong (thread-mind-and-machine), the superintelligence scenario is a distraction. The real risk is "more proximate and more mundane", the same one that has played out for centuries: we deploy technologies whose socioeconomic impact we can't forecast, to make a profit. The AI apocalypse looks like accelerated runaway IT: replacing components of complex socioeconomic infrastructure with AI causes damage through unpredictable failure modes rippling through complexity. Two aggravators:
- software substituted for people in decisions where there's an incentive to delegate accountability to a system that can't be questioned;
- a proliferation of software problems impossible to diagnose and impossible to fix.
"We are only fighting the dumbest, greediest instincts our human society produces. And that is something we know how to do."
Connections: the framing note on convivial-computation (the non-proliferation frame justifies concentration); where judgement belongs (thread-judgement-in-machines); one of the lenses of the ai-roundup.
Provenance: James's published notes at exitmusic.world, read in full 2026-10-26 (raw/notes/exitmusic-world-2026-10-26.md). His statement of his own position; not a sourced claim about the world. Disposition: stated.
<!-- dip-exitmusic-links --> Place in the departure-in-place network (author's reading): section 2, the argument behind the aside that AI is "marginal to these drivers, or possibly an accelerant" (pmc-dependency-chains).
Related: thread-mind-and-machine, thread-judgement-in-machines, convivial-computation, ai-roundup