Pacing the Frontier is not the actual goal for AI labs — LessWrong
In his latest post about pacing the frontier, Dario writes:
Spots 
But over the last few months, I have
But over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up. We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain. Two things have convinced me.My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI. This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic, as we and others have described. Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.
But over the last few months, I have
But over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up. We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain. Two things have convinced me.
My first concern is that, since roughly this
My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI. This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic, as we and others have described. Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.
You can find countless videos, posts, and articles
You can find countless videos, posts, and articles from all the frontier lab CEOs saying some variation of the above, and also posts from people saying variations of "the labs are really concerned. We should listen to them." I think this is confused, and the right thing to do is ignore anything from the labs regarding risks of AI.In what is now ancient history, the CAIS 2023 statement was signed, where the same CEOs claimed to be alarmed by the risks, and we should do something about it. Since that statement was signed, they have done approximately zero things resembling "pacing the frontier". In fact, they have stepped on the gas. All evidence points to them directionally pursuing RSI as soon as their eyes could see that was a possibility.
After everyone agreed to "pace the frontier", they've
After everyone agreed to "pace the frontier", they've gone ahead and released a few more models that seem to exceed their previous SOTA benchmarks, as well as starting on the path to automated biologic research. What safety measures they took other than "there were humans in the loop" is unknown at this point. This has confused me, and a lot of other people before, so what exactly is happening and why haven't actions matched their words?
This is not a recent pattern. As far
This is not a recent pattern. As far back as 2024, people on this same forum were confused when Anthropic decided to release Claude 3:
And Dario on an FLI podcast: I think
And Dario on an FLI podcast: I think we shouldn't be racing ahead or trying to build models that are way bigger than other orgs are building them. And we shouldn't, I think, be trying to ramp up excitement or hype about giant models or the latest advances. But we should build the things that we need to do the safety work and we should try to do the safety work as well as we can on top of models that are reasonably close to state of the art. None of this is Dario saying that Anthropic won’t try to push the frontier, but it certainly heavily suggests that they are aiming to remain at least slightly behind it. And indeed, my impression is that many people expected this from Anthropic, including people who work there, which seems like evidence that this was the implied message. And Dario on an FLI podcast:
I think we shouldn't be racing ahead or
I think we shouldn't be racing ahead or trying to build models that are way bigger than other orgs are building them. And we shouldn't, I think, be trying to ramp up excitement or hype about giant models or the latest advances. But we should build the things that we need to do the safety work and we should try to do the safety work as well as we can on top of models that are reasonably close to state of the art.
None of this is Dario saying that Anthropic
None of this is Dario saying that Anthropic won’t try to push the frontier, but it certainly heavily suggests that they are aiming to remain at least slightly behind it. And indeed, my impression is that many people expected this from Anthropic, including people who work there, which seems like evidence that this was the implied message. And this comment with this image attached: And here's gwern on the same thread:
News
Pacing the Frontier is not the actual goal for AI labs — LessWrong
In his latest post about pacing the frontier, Dario writes:
@spots
Source: Hacker News