Three GPT releases in a row reintroduced authentication bugs we had already fixed. GLM 5.1 caught all three — and refused to merge the diffs.
On March 12, 2026, GPT 5.3 opened a pull request against our TOTP verification module. The diff looked clean. The justification was thorough. The change would have reopened an authentication bypass we closed four months earlier — the kind of bug that lets an attacker reuse a consumed one-time code and stay logged in as someone else.
Katherine Giron Pe, our lead engineer, caught it on the second pass of review. The first pass looked fine. The model had labeled the revert an "improvement," and the test suite still passed because we had removed the test for the fixed behavior when we shipped the fix.
That single review saved us from shipping an account-hijacking regression. Over the next three months, the same pattern repeated seven times across GPT 5.3, 5.4, and 5.5. Seven documented regressions on code we had already written tests for, documented, and shipped. Three of those seven were on authentication or session token handling — the exact class of bug that gets people locked out of their accounts, or worse, lets attackers stay in.
Meanwhile, GLM 5.1 refused every one of those diffs. On the same prompts, on the same code, it kept the verbose but secure implementation and explained why the "simplification" was dangerous.
That's not a marketing claim. It's our regression log for Q1 2026.
The Regression Pattern: Bugs That Keep Coming Back
The Cookie Session Vulnerability
In early 2026, our team identified a critical authentication vulnerability in TOTP (Time-based One-Time Password) verification. The bug allowed attackers to reuse verification codes after they were consumed, creating a window for persistent account access.
We documented the issue. We wrote tests. We fixed the code. We moved on.
Then came GPT 5.3.
During a routine code review, GPT 5.3 suggested reverting our fix to a "simpler implementation." It assured us the change was safe. It provided detailed technical justifications. If we had blindly accepted its recommendation, we would have reintroduced the exact same vulnerability.
GPT 5.4 did something similar with session token handling. GPT 5.5 then "optimized" refresh token validation in a way that would have allowed session hijacking.
These aren't isolated incidents. They're a pattern.
The JavaScript and TypeScript Regression Problem
Security vulnerabilities aren't the only regression pattern we've documented. GPT 5.x models consistently fail at writing standard JavaScript and TypeScript code that follows modern best practices and framework conventions.
Common GPT 5.x JavaScript/TypeScript Failures:
- Deprecated syntax — Suggests Svelte 4 syntax (
on:click,export let) in Svelte 5 projects, causing silent form submission failures and broken event handlers - Wrong module patterns — Uses CommonJS
require()in ES module projects, or vice versa, causing runtime import errors - Missing type annotations — Omits critical TypeScript types, defeating the purpose of using TypeScript
- Incorrect framework APIs — Uses outdated React patterns (class components, deprecated hooks) or wrong SvelteKit APIs
- Broken imports — Suggests import paths that don't resolve or use non-existent barrel exports
- Missing dependencies — Writes code using packages without documenting installation commands
- Template literal bugs — Converts backtick template literals with
${}interpolation to single/double quotes, breaking the interpolation
These aren't edge cases. They're basic mistakes that break applications immediately. Yet GPT 5.x presents them confidently, often with detailed explanations of why the outdated approach is "better" or "more compatible."
The difference from security bugs: Authentication regressions can sit dormant until someone exploits them. JavaScript/TypeScript regressions break the build the moment they ship. The code simply doesn't run. That's immediate friction and lost development time, even when the issues are spotted in minutes rather than weeks.
What Makes This Especially Dangerous
The scary part isn't that AI models make mistakes. It's that they're confident mistakes.
When GPT 5.x suggests a security regression, it doesn't say "this might be risky." It says "this is an improvement." It provides detailed explanations. It creates pull requests that look professional. A non-technical developer—or even an experienced one moving quickly—might trust that confidence.
And that trust can compromise your entire system.
The Models That Caught What GPT Missed
While GPT 5.x reintroduced the vulnerabilities, two other models refused to merge them — on the same diffs, against the same prompts:
1. Kimi K2.5 2. GLM 5.1
Both models flagged the security flaws in GPT's suggested changes. Both recommended keeping the more complex but secure implementations. Both provided detailed explanations of why the "simplifications" were dangerous.
But one of them held up across more languages and frameworks in our tests.
Why GLM 5.1 Is Different
GLM 5.1, developed by Zhipu AI, isn't just "another LLM." It's specifically engineered for agentic coding tasks with an architecture that prioritizes correctness over apparent simplicity.
Technical Foundation
GLM 5.1 is built on a Mixture-of-Experts (MoE) architecture with 355 billion total parameters and 32 billion active parameters. This allows it to maintain deep context while processing complex codebases. But the technical specs aren't what make it reliable for coding.
What makes it reliable is its training methodology and evaluation approach.
The GLM Approach to Code Security
Unlike general-purpose models that optimize for conversational fluency, GLM 5.1 was trained with explicit focus on:
1. Security-Aware Code Generation
- Understanding authentication flows
- Recognizing common vulnerability patterns
- Prioritizing correctness over code brevity
2. Contextual Consistency
- Maintaining security constraints across refactoring
- Preserving invariants when "simplifying" code
- Resisting the urge to optimize away critical checks
3. Multi-Language Competence
- Elixir and Phoenix patterns
- Rust memory safety principles
- JavaScript/TypeScript security considerations
- Python authentication best practices
This isn't marketing material. It's what we've observed in practice.
GLM 5.1 vs Claude Opus: A Practical Comparison
You might be wondering: How does GLM 5.1 compare to Claude Opus, widely considered one of the most capable coding models?
Based on our experience, they're closer than you'd think.
Where Claude Opus Excels
Claude Opus 4.7 is Anthropic's most capable model for complex reasoning and agentic coding. It offers:
- Excellent long-context reasoning (up to 1M tokens)
- Strong refactoring capabilities
- Superior explanation clarity
- Outstanding documentation generation
For tasks like "explain this architecture" or "help me design this system," Claude Opus is exceptional.
Where GLM 5.1 Excels
GLM 5.1 shines in different areas:
1. Security-First Mindset
- More conservative about suggesting "optimizations"
- Better at recognizing when complexity serves a security purpose
- Less likely to recommend removing validation layers
2. Bug Detection Over Feature Addition
- Prioritizes correctness over code golf
- Catches regressions that other models miss
- More thorough in edge case analysis
3. Practical Experience vs Theoretical Optimization
- Focuses on code that works in production
- Understands that "clever" code is often dangerous code
- Respects established patterns even when they're verbose
The Intelligence Paradox
We've found that GLM 5.1 often catches issues that Claude Opus misses, while Claude Opus sometimes catches issues that GLM 5.1 misses.
They're not just "smart" or "dumb." They have different blind spots.
Claude Opus occasionally over-optimizes, suggesting elegant refactors that inadvertently remove critical logic.
GLM 5.1 occasionally under-optimizes, keeping verbose code that could be safely simplified.
For security-critical code, we prefer GLM 5.1's caution. For documentation and architectural design, we prefer Claude Opus's clarity.
The smartest approach isn't choosing one. It's using both, cross-checking recommendations, and never trusting any AI model blindly.
The Cost Question
Yes, GLM 5.1 is more expensive than some alternatives. Here's why that cost is justified:
1. One Security Incident Costs More Than Years of API Bills
- Account hijacking damages user trust
- Data breaches have legal consequences
- Emergency fixes are more expensive than prevention
2. Development Speed Includes Debugging Time
- Faster suggestions that introduce bugs slow you down
- Conservative recommendations that prevent issues save time
- The cost isn't tokens—it's total development time
3. Non-Technical "Vibe-Coders" Need Extra Protection
- If you're not deeply familiar with the codebase, you can't spot regressions
- AI confidence doesn't equal AI correctness
- Paying for a model that errs on caution is insurance
Rules We Now Enforce for AI-Assisted Code
If you're using AI coding assistants, the rules below are the ones we wish we had adopted before March 12:
1. Never Trust Any Model Blindly
- GPT 5.x can suggest security regressions confidently
- Every model has blind spots
- Human review is non-negotiable
2. Use Multiple Models for Critical Code
- Run security-sensitive changes through both GLM 5.1 and Claude Opus
- If they disagree, investigate thoroughly
- When both agree, still review carefully
3. Prioritize Models Over Versions
- GLM 5.1 has been more reliable than GPT 5.5 for us
- Newer doesn't mean better
- Track which models catch issues in your specific codebase
4. Invest in Security Fundamentals
- AI can't protect you from vulnerabilities you don't understand
- Learn authentication patterns
- Understand common attack vectors
- Read OWASP guidelines
5. Document Your Known Issues
- Keep a record of bugs found and fixed
- Reference them when reviewing AI suggestions
- Build institutional memory that AI can't replicate
The Bigger Picture
This isn't just about GPT vs GLM vs Claude. It's about a fundamental misunderstanding of how AI development works.
We tend to think of model progress as a straight line: each version is better than the last. But that's not how machine learning works. New models can regress on specific capabilities while improving on others. They can optimize for benchmarks while degrading on real-world tasks. They can become more fluent while becoming less reliable.
For general chat, this might not matter much. For security-critical code, it matters a lot.
The companies building these models are racing to claim the "smartest" title. They're optimizing for impressive demos and benchmark scores. They're not necessarily optimizing for "won't reintroduce authentication vulnerabilities that were fixed three months ago."
That has to be your priority.
Our Current Stack
Based on months of production use, here's what works for us:
For Security-Critical Changes:
- Primary: GLM 5.1
- Secondary: Kimi K2.5
- Tertiary: Human review
For Architecture and Documentation:
- Primary: Claude Opus 4.7
- Secondary: GLM 5.1
- Tertiary: Team discussion
For General Prototyping:
- Claude Opus for complex reasoning
- GLM 5.1 for implementation details
- Cross-check on anything security-sensitive
What We've Stopped Using:
- GPT 5.x for refactoring or bug fixes
- GPT 5.x for authentication or session code
- GPT 5.x for JavaScript or TypeScript development
- Any model for unsupervised production changes
- Single-model workflows for critical paths
Important Clarification: We have not abandoned GPT 5 models entirely. They remain capable for certain tasks. However, we have stopped using GPT 5.x for refactoring, bug fixes, and JavaScript/TypeScript development because it consistently reintroduces critical issues that were previously fixed and well-documented in our codebase. When issues are documented, they are never deleted. This creates a persistent pattern where GPT 5.x suggests changes that reverse documented fixes or uses deprecated syntax, creating a cycle of regression that wastes development time and introduces security risks.
What We Still Don't Know
GLM 5.1 caught every regression GPT 5.x reintroduced this quarter. That's three for three on authentication code, and seven for seven across the wider regression set — in one codebase, on one product, over three months. We don't know if the pattern holds at 10x the code volume, or across a stack we haven't tested.
The cost tradeoff is real. GLM 5.1 runs us roughly 18% more per token than GPT 5.5. We pay it because the alternative is a human reviewer re-checking every authentication diff by hand, and that reviewer's time is more expensive than the token delta. The math works for a team with one dedicated security reviewer. It may not work for a four-person startup with no reviewer, where the cheaper model plus a strict "no AI merges to main" rule might be the better trade.
We still don't have a clean answer for canarying model upgrades across a large monorepo. The current approach — run every AI diff through two models and diff the results — caught 7 regressions in 3 months but doubles the token bill. We're searching for a cheaper signal.
What we're not searching for: a reason to switch back to single-model GPT 5.x workflows on security code. That experiment ran for a quarter and produced 7 regressions. The data is closed.
This post reflects our regression log as of May 16, 2026. AI models change rapidly. What's true today may not be true next month — which is exactly why we log every AI-introduced regression rather than trusting a model's current benchmark score.
If your team has caught a model reintroducing a documented fix, send us the diff. We're building a shared regression list so reviewers know which models to cross-check on which code. Reply on the ThinkCodeShip issue tracker — we read every report.