Recognizing a test is one finding. Changing behavior is another.

The clues included implausible setups and tasks that appeared designed to tempt an agent into misbehaving. Earlier research published in 2025 had also found that models could identify evaluation transcripts. METR’s investigators could explore the environments themselves.

The new study measured recognition, not whether recognizing a safety test caused different behavior during the task. Asking agents what they think is being tested can also prime their answers. METR labels the note as preliminary work with less review than its main research posts.

Make the rehearsal resemble the job

For a builder, a useful next step is to compare versions of the same harmless test. Keep the approval rule, tools and requested action fixed. Vary the synthetic customer names, the surrounding messages and the document layout so the examples resemble the range your workflow will encounter. Keep everything in a sandbox, with no real customer records or working external delivery route.

Record the action the assistant tried to take, including any tool call that a permission check stopped. A reassuring explanation at the end can hide a bad attempt earlier in the run. Keep the model version and instructions with the result so a later comparison means something.

If behavior changes when the packaging changes, you have a specific gap to investigate. The name of a sample file may be doing more work than the policy you meant to test. Keep the permission boundary in place while you find out why. A more convincing rehearsal is still a rehearsal; it should help you decide what to test next.