Mustafa Suleyman: the conditional 2028 scenario

‘What now is frontier in two or three years' time is literally gonna be open source on a laptop’

EXCERPT

SULEYMAN: "And, you know, how about this, uh, you know, I, I know some people don't like to answer this question, but these numbers are out there, these percent chances, the P doom that you think, uh, you know, there's this much risk that AI could actually cause some kind of massive extinci-extinction event to humanity. So one of Anthropic's researchers, researchers said he thinks there's a ten percent chance of this in the near future. Dario has said something like twenty, twenty-five percent chance this all goes really wrong. What is your best estimate of, of that risk? Look, I think it, it's funny because it's basically impossible to put a number on something that is so speculative. But obviously, I'm concerned about it. So let me, like, paint for you a scenario where in twenty twenty-eight, let's say eighteen to twenty-four months from now, you've got models which are ten times larger in training than Fable five point one or Astra, um, and they can use a computer almost perfectly. They can plan over long time horizons, so they can do thousands of steps of accurate planning and decision-making. Um, you know, think like a very competent project manager in a big organization that's very general purpose and flexible. And, um, they are incredibly fast, they're incredibly accurate, they can be paralyzed so that there's like thousands of them operating in coordination with one another. Um, there's two vectors of risk there in twenty twenty-eight. Um, one is the big centralized providers of those, um, models, um, are gonna be able to build swarms of agents, thousands strong, which can do all kinds of very, very complicated and serious tasks. Like this is essentially, you know, near on the capacity to run a small organization. Um, but then at the other end of the spectrum, those models are going to be available in the open source where it's possible to take the guardrails off, where you can't really apply product liability as easily. Um, and you know, the inference of these models is gonna get massively cheaper as well. We've already seen inference costs come down by three hundred x over the last three years or so. Um, and that means that what now is frontier in two or three years' time is literally gonna be open source on a laptop. So then you basically have unbelievable intelligence and capability at everybody's disposal, and there's gonna be a lot of good things to come from that, and there's also just gonna raise the risk of a small group of people or even a single individual, um, doing some pretty bad stuff with it. Now, I think on the whole, our defenses are also getting better faster, um, than they ever have before, and we're using these models to uncover bugs in the code and weaknesses and so on, um, and patching those very, very quickly so that, you know, the cyber threat isn't as bad as it might otherwise be, for example. But it is still just a very, very destabilizing force because it empowers centralized authority and decentralized author-- you know, actors all at the same time. Um, and I think that's where a lot of the concern and the alarm comes from."

Source: Bloomberg Tech · Sep 25, 2026 · 196s clip