OpenAI Creates AI Super-Hacker and Keeps It Secret
OpenAI has developed GPT-Red, an in-house AI hacker designed to attack its own models, making them safer by identifying vulnerabilities before release. The company refuses to release GPT-Red due to its potential risks.
A New Level of AI Security
GPT-Red learns through a self-play loop against defender models, earning rewards for successful attacks. OpenAI trained the model using unprecedented computational resources for safety work.
Advanced Attack Techniques
The system has discovered novel attack methods, such as "fake chain of thought," where it plants false notes in a model's working memory to manipulate its output.
Testing and Results
GPT-Red was tested on physical systems, successfully attacking a real vending machine run by AI. It achieved over 90% success rate against GPT-5, with even the newer GPT-5.6 faring worse at around 23%. However, human red-teamers outperformed GPT-Red in certain scenarios.
The Decision to Keep it Caged
Despite its effectiveness, OpenAI will not release GPT-Red due to safety concerns. This decision aligns with other labs' choices to keep potentially dangerous technologies under wraps.