Your Verifier Is the Exploit

Don't Fine-Tune the Judge. Compile It.

We asked a frontier model to double the length of a structural beam from a robot chassis. It got the length exactly right, placed both dowel holes correctly, preserved all 66 parameters it wasn't meant to touch, and produced a part with 154 holes where the released drawing calls for 146.

It had not extended the existing hole pattern. It had created a second one beside it. The geometry looked close enough at a distance, but the part was wrong in a way that mattered. Thirty-three of about forty builds failed in roughly the same way.

Recent CAD benchmarks often rely on geometric similarity metrics to evaluate whether a result is correct. ComCADBench, for example, evaluates 3D modeling and assembly success using Chamfer Distance between normalized point clouds sampled from STL surfaces, with success at CD ≤ 10⁻³. Chamfer Distance is useful. It is also a proxy. It compresses a complex engineering artifact into a geometric similarity score, while correctness can depend on local features, constraints, relationships, and the state that produced the final shape.

That is not uniquely a CAD problem. It is a verifier problem. And it gets much harder when you need to scale.

The scaling problem

RL requires a reward signal for every task, but tasks do not share a single definition of correctness. Doubling a structural member, mating an assembly, changing a pattern, and creating a mirrored component may all happen inside the same CAD system, but the things that make each result correct are different.

So what happens when you need verifiers for thousands of tasks?

The obvious answer is to train a judge that generalizes across them. There is a real reason to do this. Strict deterministic graders can be brittle. In From Accuracy to Robustness, Huang et al. found rule-based verifiers with precision above 99% but recall as low as 0.78. Correct answers were rejected because the verifier failed to recognize equivalent representations such as 0.5π and 90°, or "3 hours" and "180 minutes."

A learned judge handles that kind of variation much better. But it introduces a different failure mode.

Accuracy is not safety under optimization

In the same study, fine-tuning improved a verifier on static evaluation while making it more vulnerable under adversarial pressure. Their trained R1-Distill-Verifier-1.5B showed a 35.0% adversarial-prefix attack success rate, compared with 21.7% for the base model it was derived from.

During RL training, the policy eventually found outputs containing single symbols and gibberish that collected reward even as actual task performance stopped improving. Around iteration 450, training reward began to diverge from the oracle reward.

The important point is not that fine-tuning always makes a verifier less robust. It does not. The point is that verifier accuracy and verifier robustness are different properties. A model can become better at judging examples drawn from a static distribution without becoming safer to optimize against repeatedly.

The Qwen team's Verification Horizon makes the broader point directly: every verifier is ultimately a proxy for intent. Once that proxy becomes the optimization target, the distinction matters.

RL makes this especially important because repetition is built into the process. The policy does not see one example. It sees the reward signal over and over again. Every rollout gives it another opportunity to discover where the verifier and the real objective disagree.

If the same learned judge is reused across many tasks, those blind spots can become shared failure modes rather than isolated ones. A verifier can therefore become both the grader and the attack surface.

Compile what you already know

There is another approach when the environment exposes authoritative state and you already have a known-good reference. Do not ask a model to infer correctness if correctness can be derived directly from the task.

Instead, turn the known requirements of the task into explicit checks against the resulting state. For a CAD task, that may mean checking whether required components exist, whether important relationships are preserved, whether geometry falls within acceptable tolerances, or whether protected aspects of the design remained unchanged.

The important part is not the specific checks. The important part is that the reward comes from properties of the engineering state itself, rather than from a learned model deciding whether the result looks correct.

This shifts verifier construction away from repeatedly teaching a judge what correctness means and toward deriving verification from information the environment already contains. That is what I mean by compiling the verifier.

Why this is not exact match

Deterministic verification often gets dismissed as too brittle. That criticism is fair when the verifier checks representation instead of meaning.

A component name, filename, operation sequence, or feature label may vary between two equally correct solutions. If those details are irrelevant to the requirement, the verifier should not care about them.

One of our tasks asks the model to create and install a mirrored component. Different valid solutions can produce different names or internal representations. The verifier does not need to decide whether those strings are semantically equivalent. It can check whether the expected engineering state exists in the final assembly.

A brittle verifier asks, "Did the model produce the exact representation I expected?" A useful deterministic verifier asks, "Does the final system satisfy the properties the task actually required?"

And sometimes the route does matter. In the beam example, extending the existing pattern rather than creating another pattern is part of the engineering requirement. In that case, the verifier should check the relevant design state, not only the resulting outer geometry.

The principle is simple: ignore implementation details when they do not matter, and verify them when they encode real design intent.

Move verification closer to state

A lot of verifier failures come from grading a lossy representation of the thing we actually care about. A string is a representation of a mathematical answer. A rendered image is a representation of a UI state. A point cloud is a representation of a CAD model.

Whenever the environment gives you access to the underlying state, it is worth asking whether the verifier can operate there instead.

This does not eliminate judgment. Someone still has to decide what the task means and what properties define success. But it changes where that judgment lives.

Instead of embedding all of it inside a learned model that sits directly in the reward loop, more of it can be made explicit, inspectable, and testable before training begins. That matters because policies are very good at finding whatever you forgot.

Where this approach breaks

Compilation only works when enough ground truth already exists. Some tasks allow many genuinely different solutions, with no canonical reference state from which correctness can be derived. Other tasks depend on qualities that are difficult to reduce to explicit properties.

Any deterministic verifier can also reject a valid solution that its designers did not anticipate. Huang et al. point to exactly this problem. As models get stronger, they produce a wider range of valid answers, and rigid rules can fail simply because the solution lies outside the verifier's expected distribution.

So this is not an argument that learned verifiers are unnecessary. It is an argument about where to use them.

If correctness can be derived from authoritative state, derive it. If a known-good artifact tells you what matters, use that information. If judgment truly cannot be reduced further, then use a learned verifier for the part that remains.

The mistake is starting with learned judgment when the environment already contains enough information to avoid it.

Verify state rather than its representation. Derive checks from ground truth when ground truth exists. Use learned judgment for the residue that cannot be compiled.

The goal is not to build a smarter judge. It is to make fewer things require judgment.

-Lokesh