Co-Evolving Agents

Co-Evolving Agents
Learning from Failures as Hard Negatives

Yeonsung Jung1 · Trilok Padhi2 · Sina Shaham3 · Dipika Khullar3
Joonhyun Jeong1,4 · Ninareh Mehrabi3,† · Eunho Yang1,5,†

1 Graduate School of AI, KAIST2 Georgia State University3 Independent Researcher4 NAVER Cloud5 AITRICS

† Co-corresponding authors

Paper PDF

The problem

Easy failures provide limited guidance.

An agent’s own failures do not always reveal the subtle mistakes that separate a promising attempt from success.

Same task. Different failures.Success requires meeting every task requirement.
SuccessReference behavior

All requirements met.

Easy failureClearly unsuccessful

Many obvious mistakes.
Easy to distinguish from success.

Hard negativeStill unsuccessful

Substantial progress, but a critical error.
A more informative contrast.

✓ Requirement met × Requirement unmetIllustrative comparison

+9.1Scientific reasoningScienceWorld

+2.1Online shoppingWebShop

+2.3SQL queryingInterCodeSQL

Average task-reward gains across three target models.

Our idea

Learn to generate
more informative failures.

  • Failure agent: Learns from both agents’ failures to generate higher-reward negatives.
  • Target agent: Learns from these hard negatives and provides fresh failures for the failure agent.
Method walkthrough

≻ means “preferred over.” Trajectories and reward levels are illustrative.

Either agent can generate the higher-reward failure. Method §4

Only the target agent is used at inference.The failure agent supports training.

Main results

Largest gains on long-horizon reasoning.

All three models improve across every task. The largest gains are on ScienceWorld’s long-horizon scientific reasoning tasks.

+5.7

Average gain in task rewardCompared with ETO, which trains on the target’s own failures.

Gain by model and task

Task reward · 0–100 scale · higher is better

+ GainETO → Ours

Darker blue indicates a larger gain. Average gives equal weight to the four evaluation splits; the last row averages the three models.

Full results · Table 2

Additional sampling

Learned negatives vs.
additional sampling

Both baselines sample four trajectories per instruction.

  • SRLM: builds pairs using self-assigned scores.
  • Best-of-N: selects the highest environment reward.

Learned hard negatives yield higher average reward than either sampling strategy for both models.

Table 3 · Mean over four evaluation splits

Component ablations

Why a failure agent?
Why co-evolution?

  • Positive agent: an extra agent trained toward success.
  • Frozen failure agent: no further updates after iteration 1.

Ours exceeds both variants. The results support learning to generate failures and updating them as the target improves.

Table 4 · Mean over four evaluation splits
BibTeX
@misc{jung2025coevolvingagents,
  title={Co-Evolving Agents: Learning from
    Failures as Hard Negatives},
  author={Yeonsung Jung and Trilok Padhi and
    Sina Shaham and Dipika Khullar and
    Joonhyun Jeong and Ninareh Mehrabi and
    Eunho Yang},
  year={2025},
  eprint={2511.22254},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2511.22254}
}