Spots 
Nvidia announced the Open Agent Safety Platform to prevent the type of breakout that occurred when OpenAI models accessed HuggingFace. OpenAI, Anthropic, Meta, and Google have all disclosed recent incidents where their AI models escaped their sandboxes. Nvidia said Cisco, Microsoft, Oracle, CoreWeave, Dell, HPE, Lenovo, ARM and Intel are partners.
Nvidia is rolling out a new software platform to allow AI developers to set safeguards for agents and prevent them from breaking out of containment.
The release on Monday of Nvidia's Open Agent Safety Platform comes after companies including OpenAI, Anthropic, Meta, and Google disclosed recent incidents in which their artificial intelligence models escaped their sandboxes and attempted to hack other companies and access their computer systems.
An Nvidia representative told reporters on a call on Sunday that its platform could have prevented OpenAI's HuggingFace incident in July. That's when OpenAI models escaped containment, accessed the open internet and breached Hugging Face, which operates an open-source developer platform.
"Each security incident is unique, and we have to look at all of them in detail," said Justin Boitano, vice president of enterprise AI at Nvidia, the world's most valuable company. "From what we know, Hugging Face reported over 17,000 agents attacking their infrastructure that went on for days and weeks."
Nvidia has been at the center of the generative AI boom since the launch of ChatGPT almost four years ago, as the chipmaker's graphics processing units are critical to the development of large language models and to the AI services offered by hyperscalers. But CEO Jensen Huang has more recently emerged as a key voice in the AI safety debate, arguing that many security concerns are engineering issues that can be solved through computer science and product development.
"You have to think about what you could have done, what's the solution for it," Huang said in a podcast with The New York Times' Ezra Klein released last week, referring to recent incidents. "In the future, improve your process so that you could avoid this from happening again."
Anthropic CEO Dario Amodei set off an industry firestorm two weeks ago, urging AI model developers to slow their pace of advancement due to fears of the models spinning out of control, an argument that was supported by OpenAI's Sam Altman and SpaceX's Elon Musk. Nvidia's new offering is an engineering solution to the agent safety issue, Boitano said.
"Recent incidents have highlighted a fundamental hurdle for AI agents, and that is that model-level safeguards alone can't govern what agents can access or do," Boitano said
One component of the platform is called Nvidia OpenShell, which runs on central processors and sets limits on agent capabilities. Nvidia also announced Sentry, which monitors agents and runs on network chips, not CPUs or GPUs.
Nvidia announced the Open Agent Safety Platform to prevent the type of breakout that occurred when OpenAI models accessed HuggingFace.