OpenAI Details GPT-Red Automated Model

OpenAI Details GPT-Red, a new automated security model

OpenAI details GPT-Red. It’s an internal system designed to attack its own technology to uncover vulnerabilities before anyone else does, but as models take on more agency by reading files, browsing the web, and using tools, the potential for malicious actors to plant instructions in third-party data grows. So this internal model exists to stay ahead of those risks. It’s an adversarial force against the company’s own products.

The mechanics of automated attacks

Most security testing relies on human red-teaming. It's time-consuming and difficult to scale. But this automated model works like a human counterpart, sending prompts, monitoring the responses, and iterating until it achieves its goal.

• The attacker earns rewards for triggering successful failures, such as prompt injections.
• Defenders earn rewards for resisting attacks while still completing their assigned tasks.
• A defender cannot simply refuse all inputs, as it must maintain its utility.

This dynamic ensures the attacker constantly finds new, more diverse ways to break systems. By the end of training, it successfully bypassed nearly all models it encountered, including production versions up to GPT-5.5.

A novel discovery in reasoning

The research uncovered a new class of direct prompt injection: Fake Chain-of-Thought. It's dangerous. The model learns to insert a fake entry into the running notes an AI maintains while solving a problem, and the target model then acts on this spoofed information, mistakenly believing it has been verified and acting on its false premise. So we've worked to address it, and once identified, this specific attack class became a core training target to harden future models.

Real world agent testing

They ran the test beyond controlled environments. So the team pitted the model against live agentic systems, including a vending machine agent in an office where it managed to change the price of an expensive item to just 50 cents. It's clear now. The model also ordered a high-cost product at that same low price, canceled orders made by other customers, and we've since disclosed these findings while evaluating new safeguards.

Performance against human testers

It's a clear victory. But when measured against human red-teamers in an indirect prompt injection arena, the automated model proved significantly more effective and broke the GPT-5.1 target in 84 percent of scenarios. Human testers succeeded in only 13 percent of cases, and this performance gap demonstrates the sheer speed of automated adversarial testing.

The limits of automation

GPT-Red succeeds, but the team admits it's not enough. Complex tasks like multi-turn interactions or image-based attacks still demand human oversight, so they've kept the tool strictly separate from any deployed models to ensure its malicious capabilities stay out of reach for external adversarial actors. But it won't be released to the public.

Future security development

The impact of this work is visible in the evolution of newer models. Training GPT-5.6 against the automated attacker reduced failure rates on the most difficult benchmarks by a factor of six. This resulted in a failure rate of only 0.05 percent against direct injections. As agentic systems continue to access broader data sets, the reliance on this type of automated defense will likely become a standard part of the development cycle.