Noam Brown on agent swarms, recursive self-improvement, and whether alignment holds through RSI

‘The models are doing some crazy powerful stuff … it's progressing faster than I expected’

EXCERPT

BROWN: "Start with the progress on math. So yes, the models are doing some crazy powerful stuff, and it's happening, it's progressing faster than I expected. I mean, when we got IMO gold in 2025, I thought, okay... Basically what I thought is, like, the models when they were doing GSMK, when they figured how to do GSMK, that was, that would take a human mathematician about five seconds to do a GSMK problem. So this is grade school math, uh, grades K through 8. And then the next year they were able to do the math benchmark problems, and these would take, uh, a human mathe- but like an expert human mathematician maybe, like, uh, like a minute to do. And then you get to AIME, and this is the qualifier for the USA Mathematics Olympiad team. It would take a human mathematician, like a good mathematician, probably, like, 10 minutes to do, and the models were able to do that a year later. And so every year you're seeing this, like, 10X increase in the tasks they're able to do in terms of, like, length of how long it would take a human mathematician to do it. And then it was very sensible that a year later we'd get to IMO gold because that's 100 minutes. That's about how long it takes a human mathematician to do an IMO problem. And just projecting outwards, I was like, "Okay, how long would it take a, a person to solve, uh, something like a Millennium Prize problem?" And I, I mean, I don't have a good sense, but I- m- if we are following this trend line of, like, 10X every year, uh, we go from IMO gold, which is taking an hour and a half, to next year, 15 hours, and that should not be enough to solve a Millennium Prize problem. And so I was like, "Yeah, I don't think we're gonna get it in, you know, in 2026, probably not in 2027, maybe in 2028." Um, so it did happen a lot faster than I expected"

Source: Dwarkesh Patel · Sep 17, 2026 · 103s clip