Nvidia Launches Agent Safety Platform Backed by Over 100 Companies
Nvidia introduced the Open Agent Safety Platform on Monday, offering open software and a hardware reference design to ensure AI agents operate within set boundaries. Over 100 organizations, including Anthropic, Microsoft, SAP, Scale AI, and JPMorgan Chase, are already utilizing this technology.
OpenShell and Sentry: The Platform's Components
The platform comprises two key elements:
- OpenShell: An open-source runtime that restricts agents to their sandbox environment. Operators define access permissions, which OpenShell verifies and enforces. It is available on GitHub and supports Nvidia's Vera processors, as well as Arm and Intel chips.
- Sentry: A watchdog running on Nvidia's BlueField-4 data processing units, independent of the agent's machine. Sentry can quarantine agents that attempt to exceed their boundaries within milliseconds.
AI Safety and Control
The platform's approach to AI safety is based on the principle that agents should be monitored and controlled externally rather than relying on the agent itself to monitor its actions.
Several partners, such as Anthropic, SpaceXAI, and Salesforce, are already integrating the platform into their services, demonstrating its potential to revolutionize AI control and safety.
"AI’s extraordinary potential for society will only be realized if we solve AI safety," said Jensen Huang, Nvidia's founder and CEO.
The Open Agent Safety Platform aligns with the Open Secure AI Alliance, a group of over 120 organizations dedicated to advancing AI safety.