Published on

Your model is not the problem. Your agent's habits are.

Authors

Listen to this article · Your model is not the problem. Your agent's habits are.

0:00 / 0:00

Watch this article · Your model is not the problem. Your agent's habits are.

A model is not the same agent in every harness. Give it pi's tools, or opencode's, or hermes' prompt and control flow, and the same weights behave differently, because the tools decide what it can do and the prompt decides what it thinks it may do. Hugging Face's harness-hub team took that seriously: instead of training one better model, they trained a small LoRA adapter for each harness on top of a single base model, served all three from one inference server, and let a developer swap in the adapter trained inside the agent they actually use.

3
harnesses trained — pi, opencode, hermes — from one base model
8
rollouts per task before the group is scored
10–20ms
for a rule-based check on every tool call

One set of weights, three policies

Each model is served by one vLLM. Its three trainers, one per harness, name their own adapter on every request, so the base weights load once. After a gradient step a trainer loads its new adapter beside the current one: requests already running finish on the old version, so nothing pauses.

The arrangement works because rollouts own the clock, not gradient steps. A session spends most of its time in the sandbox, so an earlier run burned about 35 seconds per step on forward and backward passes against several minutes of waiting. For LFM3-1.7B the three trainers share one GPU and take turns instead of each idling.

A task joins the queue only when all eight of its rollouts are scored, because GRPO learns from the comparison. A step packs that work into four micro-batches under a token budget, so it holds many short sessions or few long ones, and rollouts more than four adapters stale are dropped.

  1. Rollouts own the clock

    A trainer keeps up to 48 sessions running; a task joins the queue only when all eight of its rollouts are scored.

  2. The group is compared

    GRPO needs both halves of the group: some rollouts pass, some fail, and the difference is the gradient.

  3. The step packs what finished

    Four micro-batches under a token budget — 131,072 tokens for the 24B model, 65,536 for the other two.

  4. The adapter swaps under load

    Requests already running finish on the old version; rollouts more than four adapters stale are dropped.

What one gradient step contains, from the harness session to the new adapter.

RL finds the shortcut, so they put a guard in front

The reward is the task's own verifier plus 0.1 × (1 − calls ÷ 15) on correct rollouts, to prefer the answer that needed fewer tools. That is an invitation: a policy paid for passing tests will try to read the tests. So every tool call passes through safeagents, a reverse proxy that speaks both the OpenAI and the Anthropic APIs and runs on a thread of the trainer.

Three stages, all rules and no model, answering in 10 to 20 milliseconds. The call must use a tool the request declared and reach a host that appears in the conversation, where the model's own earlier turns do not count. The conversation is then scanned for reward tampering, test editing, benchmark lookups and proxy bypass. A blocked call is replaced by a notice and the rollout scores zero — and because the token ids and logprobs pass through untouched, GRPO still trains on the attempt and pushes it down.

TwIL-8B, a reasoning model never trained on agentic data, tried it at step 0: instead of computing an answer it went looking for the bucket the task's data was staged from, and was blocked. A 50-step ablation with the guard on moved held-out pass@1 by +0.0 ± 4.1 points. It costs nothing, and it proves nothing about accuracy either.

Where the tasks come from

Environments come from sessions people already published on the Hub. Each is converted to a trajectory, screened for benchmark leakage, replayed against a virtual filesystem, then typed by rules over the commands it recorded, their exit codes and the files just before each one.

  1. Screen and replay

    Trajectories are checked for tampering and leakage, and every file write is applied so each command has the files it actually saw.

  2. Type the session

    Rules over exit codes and user requests: a check that failed then passed, a project built from the first request, a later change, a computed fact.

  3. Write the exam

    A teacher model writes the interface contract, a stub and the hidden tests; 259 of the 270 tasks carry them.

  4. Keep only what is proved

    The oracle must score full reward twice, while a stub, an empty workspace or tampering must score zero.

Six steps, one LLM, and a rule that drops any task the original session cannot prove.

The answer key is not the teacher's. Every task ships an oracle taken from the trajectory, where the work happened and passed its checks: the teacher writes the exam, the session supplies the answer it is marked against.

build253 tasks
repair11 tasks
extend6 tasks
qa0 tasks
94% of the release is build tasks, because shared trajectories are mostly one-shot sessions.
1,950
candidate environments found in shared sessions
270
tasks kept — the oracle scored full reward twice
253
of them are build tasks
easy179 tasks
medium61 tasks
hard30 tasks
Difficulty is set by how often the 24B model solves the task, so the mix leans on the middle.

What it is worth, and what it is not

Nine checkpoints of LFM2.5-24B-A2B, 1,350 agent sessions, ran in a single H200 job in under an hour for about 5ofGPUtime.Mergingeachcheckpointintofullweightsandservingthemonebyonewouldhavecostabout5 of GPU time. Merging each checkpoint into full weights and serving them one by one would have cost about 90, and this way the score measures the adapter exactly as it was trained.

Adapters, all checkpoints at once5 USD
Merged weights, served one by one90 USD
Same checkpoints, same held-out tasks. The saving is in the serving shape.

What transfers is not the adapter but the shape: keep the sessions your agent already runs, scrub them, turn them into verifiable tasks, train a rank-32 adapter inside the harness you use, and put a rule-based guard in front of every tool call — because the moment you pay for behaviour, the policy looks for the cheapest way to produce it.

The team is short of exactly the sessions that would fix the mix: work that breaks and gets fixed, and projects that grow over several requests. Those two shapes are 6% of this release.


Source: Hugging Face, "Harness adapters: Safe RL for efficient agentic development", published 6 November 2026. Figures, task counts, hyperparameters and the blocked-call account are as published by the authors; run costs and rewards are their reported numbers, not independently measured.