Zhipu GLM-5.3 Cybersecurity Claims Scrutinized

Zhipu GLM-5.3 became an overnight sensation in the AI world when headlines proclaimed a Chinese model had beaten America's best at finding software vulnerabilities. The Beijing-based company, which also operates as Z.ai, launched its coding-focused model on August 14 and published a technical release note detailing its performance against rivals. One sentence buried in that document tells a more careful story than the viral coverage suggested.

"Capability is growing fastest exactly where we are furthest behind," Zhipu wrote about its own cybersecurity results. That admission cuts against the triumphant framing that followed the release.

The Number That Traveled

The benchmark that grabbed attention was CyberGym, a test where GLM-5.3 scored 84.5% against 83.8% for Anthropic's Mythos 5 and 83.6% for OpenAI's GPT-5.6 Sol. A seven-tenths-of-a-percentage-point lead over two American frontier models made for a compelling story about the balance of power shifting in AI security research.

Vulnerability discovery carries real weight in this debate. A model that can read source code and identify exploitable flaws serves both defenders auditing their own software and attackers probing someone else's. Anthropic keeps equivalent work behind restricted access for precisely that reason. Zhipu plans to publish GLM-5.3's weights openly.

Three Benchmarks, Three Different Pictures

Of the three cybersecurity results Zhipu published, CyberGym is the narrowest. The other two tell a different story. But the company doesn't hide that fact, and it's upfront about the contrast between them, so readers can weigh the findings without spin. That honesty is rare.

ExploitBench requires a model to reason about a real vulnerability and how it would be exploited. GLM-5.3 scores 54.4% there, which more than doubles its predecessor's 24.4%. But Mythos 5 hits 78.0% and GPT-5.6 Sol reaches 76.5%. The gap is enormous.

ExploitGym measures how many exploitation tasks a model completes within a fixed time budget. GLM-5.3 finishes 105 tasks in two hours and 130 in six. Mythos 5 completes 181 and 247 respectively.

The coverage was thin. But those two results describe the actual distance between the models, and it's a gap that speaks volumes about where each system truly stands. Finding a flaw and building a working exploit from it are fundamentally different jobs, and Zhipu's own reading acknowledges that the further along that chain a test sits, the further behind its model falls. That's the takeaway.

Which Anthropic Model, Exactly?

Part of the confusion in the coverage stems from Zhipu comparing three different Anthropic models in three different places. The main benchmark table sets GLM-5.3 against Opus 4.8. The performance charts use Fable 5. The cybersecurity section uses Mythos 5. A quick reader comes away with a single comparison that does not actually exist.

On coding tasks, the picture is mixed rather than dominant. GLM-5.3 leads Opus 4.8 on some tests and trails it on others. Zhipu states plainly that its model remains behind Claude Fable 5 on the company's own internal coding benchmark.

How The Tests Were Run

The methodology footnotes contain something the summaries skipped. Zhipu evaluated GLM-5.3 on CyberGym, ExploitGym, ExploitBench, Terminal Bench and several other tasks inside Claude Code 2.1.207, Anthropic's coding agent. Using a common setup across models ensures a fair comparison, and Zhipu documents its settings. But a Chinese open-weights model's frontier claims are being measured through American agent software. That says something about where the tooling layer sits in this competition that the scores themselves do not.

Hold that thought. Two further details deserve attention before we treat the CyberGym result as settled. The score is a single run, reported as pass@1 across 1,507 tasks, with no variance figures given, so we can't gauge how stable that number really is. But a gap of seven tenths of a point between two single runs isn't a gap anyone should lean on. It's just too thin.

The ExploitGym time budgets were also normalised using throughput rates from Artificial Analysis. Rescaling factors are listed for GLM-5.3, Kimi K3 and Qwen3.8 Max, but not for Mythos 5.

Finding a flaw and building a working exploit from it are different jobs. Zhipu's reading is that the further along that chain a test sits, the further behind its model is.

The Vulnerability Count And The Number That Is Missing

Beyond the benchmarks, Zhipu says it worked with security teams in China to run its models against real codebases. The company reports identifying 2,436 vulnerabilities across 269 open-source projects. The severity split breaks down as 107 critical, 990 high, 1,286 medium and 53 low. The oldest flaw dates back to 1981, and the average vulnerability had been sitting in code for 26.6 years before discovery.

One discrepancy is worth carrying carefully. Zhipu's summary panel labels 1,097 findings as critical and high, which matches the severity table. The body text of the same release describes those 1,097 as medium-to-high. Several outlets have reproduced the second version.

The count also arrives after what Zhipu describes as expert review, screening and deduplication. The raw model output is not what is being reported. Of the 2,436 findings, 53 have been publicly disclosed, and 2,383 remain under embargo. The release does not say how many were previously unknown, and it does not say how many were independently reproduced. Those two figures would turn a volume claim into a capability claim.

What Matters More Than The Benchmark Table

Two things in the release carry longer consequences than the CyberGym margin. The first is efficiency. Zhipu reports GLM-5.3 reaching 31.4% on its internal coding benchmark at around 50,000 output tokens per task. Opus 4.8 manages 29.5% using 120,000 tokens. Slightly better work for less than half the tokens is a cost argument, and cost determines whether security teams outside the largest budgets can run these tools at all.

The second is distribution. Zhipu says the weights will be published once safety evaluation and hardening are finished. That has not happened yet. Until it does, the open-weights claim is a commitment rather than a fact.

If the release holds, a model with documented vulnerability-discovery capability becomes something any team can download and run locally. That includes markets that will never have access to an export-controlled American model. The weights are due at the end of August.

Zhipu's own release acknowledges the gap between finding vulnerabilities and exploiting them. The CyberGym headline made the company look like a leader. The ExploitBench and ExploitGym numbers describe a follower that is improving quickly, but is not there yet.

Zhipu GLM-5.3 is a serious piece of work with impressive efficiency gains and a genuine security finding capability. The company deserves credit for publishing results that do not flatter it. The coverage that turned a narrow single-run benchmark into a narrative of Chinese AI supremacy deserves considerably less.