A clear account of the “self-improving agent”: what should improve is how the work gets done, and it has to show on new tasks it hasn't seen.
Indigo's conclusion
Against the view that continual learning ends in the weights, she is a serious opponent; for the view that verification can't be skipped and the ceiling is the evaluator, she makes the sharpest case. But her strong claims rest on bounded evidence: a promising direction, not a settled result.
How to read this A research essay by a University of Maryland professor, with no commercial angle and unusually careful reasoning: for every experiment she marks what it shows and what it doesn't. It is not an announcement that self-improvement has arrived; it is an agenda for what to build and how to test it.
What to remember
Self-improvement means turning experience into better methods, not remembering more or retraining the weights now and then.
Useful changes are concrete: an extra check, a tool that spots generated files, a habit of checking provenance before editing.
Learn a set of methods plus a rule for choosing: cheap path for ordinary edits, careful path for generated files.
One strict test separates real self-improvement from fake: does the learning curve get more accurate, cheaper and less dependent on human correction? Not the final score.
Breakdown · 5 steps
01
The system paid twice for the same lesson
The self-improvement she wants: an agent uses the results of its work to change how it works next time. The change can live in skills, workflows, action policies or the model's weights. Read this part →
02
Better tools and processes are capability
Voyager builds a skill library; DGM edits its own code while the base model stays frozen. An agent evolved on SWE-bench rose from 14.2% to 28.9% on Polyglot. Read this part →
03
A lesson has to change a decision
Saving transcripts is retention; writing “beware generated files” is compression. Neither guarantees a better next move. What's needed is a mechanism that turns experience into method. Read this part →
04
Learn a repertoire, not one perfect workflow
A process that helps one problem hurts another; learning should make behavior more selective, not uniformly heavier. FlowBank's average across five benchmarks rose from 70.40 to 73.40. Read this part →
05
“Better” has to mean later work gets better
Look at the learning curve on unseen tasks and count the cost of proposing, testing, updating and retrieving. Evaluators may learn too, but keep independent tests. Read this part →
What it means for Rewired Index
Research-side testimony on whether the method layer holds real value: better methods transfer, so it is real capability. It also gives a screening rule: a system that claims to improve itself without a learning curve on unseen tasks, full cost counted, hasn't proven it.
What would change my mind
a system that, on unseen later tasks and with the full cost of learning counted, still shows a learning curve that keeps rising.
How to read this
A research essay by a University of Maryland professor, with no commercial angle and unusually careful reasoning: for every experiment she marks what it shows and what it doesn't. It is not an announcement that self-improvement has arrived; it is an agenda for what to build and how to test it.
The self-improvement she wants: an agent uses the results of its work to change how it works next time. The change can live in skills, workflows, action policies or the model's weights.
Consider a coding agent assigned to fix a failing test. It searches the repository, edits a generated file, and eventually discovers that the real change belongs in the generator. It repairs the source, regenerates the output, and submits a correct patch.
A week later, it receives a different issue in the same repository. It edits the generated file again.
Both tasks might end in success. Yet something important is missing. The first task taught the agent something that should have changed how it approached the second. Instead, the system paid twice for the same discovery.
Now imagine that the first run leaves behind a change to the agent’s working procedure: check whether a file is generated before editing it, locate the source when it is, and verify that regeneration produces the intended change. On the next relevant task, the agent starts with that procedure available. It can refine it further when it encounters a different build system.
This is the form of self-improvement I care about: an agent uses the consequences of its work to improve the way it does future work.
The result is still a patch, an analysis, or a completed experiment. But the work can produce a second result: a more capable agent. That improvement might live in a skill, a workflow, a better action policy, or the model’s parameters. What matters is that the next task benefits.
Not every interaction deserves an update. The ambition is to make useful experience consequential, rather than requiring a human engineer to interpret every failure and specify every change.
02
Better tools and processes are capability
Voyager builds a skill library; DGM edits its own code while the base model stays frozen. An agent evolved on SWE-bench rose from 14.2% to 28.9% on Polyglot.
The thing that learns is the whole agent
In Reasoning as Control, I argued that self-improvement should extend to how a system allocates computation, chooses actions, and organizes workflows.1 The model is one part of that system. The procedures through which it uses its capabilities are another.
For the coding agent, knowing how to reason about source code is not sufficient. It also needs to decide what to inspect, when to run a test, how to interpret the result, and whether its current approach is worth continuing. These choices determine whether it uses its underlying capabilities effectively.
Once those procedures can be revised through experience, they become part of the learned system.
Voyager offers an early, concrete example. Guanzhi Wang and colleagues built a Minecraft agent that accumulated executable skills and reused them to solve new tasks in a fresh world. Its underlying language model was accessed through black-box calls, without parameter fine-tuning. Persistent improvement came through the skills available to the agent.2
The Darwin Gödel Machine, developed by Jenny Zhang and colleagues, makes the agent’s own implementation a target for improvement. An agent examines evaluation logs, proposes a change, and modifies its code. The system maintains an archive of variants from which further changes can be explored. Reported discoveries included more precise file-editing tools and revised solution-generation workflows, while the underlying foundation models remained frozen.3
There is evidence of transfer, too. In the paper’s cross-benchmark evaluation, an agent evolved on SWE-bench achieved 28.9% on Polyglot, compared with 14.2% for the starting agent; Polyglot was not used in that evolution run. The experiment does not establish indefinite improvement, and the outer archive-and-selection process remained fixed. But it demonstrates something substantial: improvements to an agent’s working methods can survive the problems used to discover them.
I would not dismiss this as “only improving the harness.” Better tools and better procedures are capabilities of the deployed system. A model need not become intrinsically stronger for the agent built around it to become more effective.
03
A lesson has to change a decision
Saving transcripts is retention; writing “beware generated files” is compression. Neither guarantees a better next move. What's needed is a mechanism that turns experience into method.
Experience has to change the method
Return to the generated-file mistake. Saving the transcript would preserve what happened. Writing “be careful with generated files” would compress it. Neither, by itself, establishes that the agent will make a better decision next time.
The useful change is operational: a different inspection step, a tool that identifies generated artifacts, or a learned preference for checking provenance before editing. The lesson needs to alter a decision.
Our Agentic Critical Training (ACT) work, led by Weize Liu, studies one component of that ability. Rather than training a model to imitate a supplied reflection, ACT trains it to choose between an expert action and a model-generated alternative. The reward depends on the choice, not on matching a reference explanation. When used before subsequent agent training, this improves performance in the paper’s experiments. It relies on expert-action supervision; it is not, by itself, an autonomous lifelong-learning system.4
The relevance to self-improvement is that judgment must affect behavior. An eloquent account of a mistake is useful only insofar as it helps the system avoid or repair that mistake.
Self-Distillation Policy Optimization (SDPO) provides a complementary mechanism. After an attempt, the model receives feedback such as an execution error. A feedback-conditioned version of the model then supplies next-token distributions for the original trajectory, creating a dense training signal for the policy that generated it. Information unavailable during the attempt becomes supervision afterward.5
That is a precise way to turn hindsight into a change in future behavior. It also clarifies why “self” does not mean learning in isolation. The environment contributes evidence the model did not previously have. The system’s job is to extract something useful from it.
A self-improving agent therefore needs more than a place to store experience. It needs a mechanism for converting experience into a better method.
04
Learn a repertoire, not one perfect workflow
A process that helps one problem hurts another; learning should make behavior more selective, not uniformly heavier. FlowBank's average across five benchmarks rose from 70.40 to 73.40.
Learning a repertoire, not one perfect workflow
There is a temptation to imagine self-improvement as a sequence of replacements: discover a better procedure, discard the old one, repeat.
But a procedure can be better for one problem and worse for another. Our coding agent should not perform an expensive build-system investigation before every one-line documentation edit. Learning the generated-file lesson should make its behavior more selective, not uniformly more elaborate.
Our FlowBank work, led by Lingzhi Yuan, studies this issue at the workflow level. Instead of retaining only the best average workflow from an optimization run, FlowBank builds a compact portfolio of complementary workflows and learns to select among them for each query, accounting for predicted performance and cost.6
Across five benchmarks, it improved the average score from 70.40 for the strongest evaluated automated baseline to 73.40, a gain of 3.00 points, with the same executor model used across methods. The deployed portfolio is fixed after construction; the paper does not demonstrate a bank that continually learns from deployment.
It does, however, support an important premise: useful progress can come from preserving several methods and learning where each belongs. A natural next step, discussed in the paper, is to let new experience improve both the repertoire and its selection policy.
For the coding agent, that could mean retaining a cheap path for ordinary edits and a more careful path for generated artifacts. A later discovery might improve only one of them. Another might reveal that the existing distinction is inadequate.
This is a richer view of self-improvement than adding another paragraph to a prompt. The agent develops a better set of methods and a better understanding of when to use them.
The agent should help decide what to learn next
There is another step beyond learning from whatever happens to arrive.
Suppose the agent has encountered generated code in two repositories. It now has a tentative procedure, but does not know how broadly it applies. It could wait for another user request. Or, within an authorized test environment, it could construct small repositories with different generation conventions and investigate where the procedure breaks.
The second option turns an accidental lesson into a deliberate learning problem.
Voyager already contains a version of this idea: its automatic curriculum chooses tasks to drive further exploration alongside the growing skill library. The learner helps shape the experience from which it will learn.
For a working agent, I would extend that principle to uncertainty about its own methods. When a failure exposes a recurring weakness, the system should be able to identify what evidence would resolve it. That might require a targeted experiment, a comparison between procedures, or a human clarification about the intended outcome.
This introduces a trade-off. Solving the current task as cheaply as possible and learning the most from it are not always the same objective. A diagnostic experiment may cost more now but prevent repeated failures later. Conversely, elaborate reflection on a one-off issue may never repay its cost.
The agent needs to learn when further learning is worth doing.
This is not a claim that every deployed agent should conduct open-ended experiments. Users’ tasks are not a license to explore indiscriminately. It is an argument for giving learning its own budget and permitted environment, rather than treating it as either free or forbidden.
Under that design, deployment can supply questions for the next round of learning. Improvements can then change what the agent is able to discover. Whether this produces sustained acceleration is an empirical question, not something guaranteed by drawing a feedback loop.
05
“Better” has to mean later work gets better
Look at the learning curve on unseen tasks and count the cost of proposing, testing, updating and retrieving. Evaluators may learn too, but keep independent tests.
Better must mean better on later work
The strongest skeptical interpretation is that these systems are accumulating exceptions, tuning against familiar tests, or spending additional compute that a fixed agent could also have used. A larger skill library and a higher final score do not distinguish those explanations from useful learning.
I would evaluate a self-improving agent over a sequence of tasks, against systems with the same starting capabilities: one that remains fixed, one that retains raw experience, and one that learns reusable methods. All would then face common unseen tasks. The comparison should count the cost of proposing changes, testing them, updating models, and retrieving what was learned—not just the final execution.
The key result would be a learning curve: does prior experience make later work more accurate, less expensive, or less dependent on human correction? Testing earlier capabilities would reveal regressions. Removing the learned methods would help determine whether they caused the gain. Different task orders would test whether progress depends on a carefully arranged curriculum.
This is where evaluation belongs in the philosophy: it tells us whether the learning loop is actually producing learning.
The evaluator may need to learn as well. In REFORM, our work with Pankayaraj Pathmanathan, a reward model helps discover responses whose reward scores are inconsistent with their preference class. Targeted training on those failures improves robustness in the evaluated settings. Here, discovering a weakness creates data for repairing the mechanism that judges quality.7
That does not justify allowing an agent to approve itself under standards it can freely rewrite. I would keep independent tests for proposed changes to both the worker and its evaluator. But evaluation should enable the system to learn from informative failures, rather than simply reward familiarity with a fixed test suite.
For the generated-file lesson, progress would mean fewer mistaken edits on unfamiliar repositories—not a higher score on examples the agent wrote for itself.
Can one agent’s progress help another?
Once an agent can learn a useful method, it is reasonable to ask whether another agent must rediscover it.
Imagine one coding agent developing the generated-file procedure while another struggles with the same class of error elsewhere. Sharing the procedure could save work. More importantly, the second agent might test it under different conditions and discover a missing qualification. The shared method could become better than either agent’s original version.
That would be collective learning, rather than simply parallel execution. The distinction is testable: does experience acquired by one agent improve another’s performance on work it has not already seen?
The transfer should preserve what matters about the original setting. A repository-specific convention is not a universal rule, and private project details do not become shareable merely because they would be useful. Nor should copying the same lesson across many agents be mistaken for independent confirmation.
The promising unit of exchange is therefore a usable method with enough context to judge its applicability. Sharing succeeds when it reduces the recipient’s learning burden without making it inherit an unsupported assumption.
This is a possible extension of self-improvement, not a substitute for demonstrating it in one agent first.
What the next task should inherit
Return to the coding agent a week later.
The meaningful change is not that it can recount the previous mistake. It recognizes when the old lesson applies, checks the source of the generated file, and avoids the unnecessary edit. When the repository behaves differently, it investigates the difference instead of blindly replaying the procedure.
Perhaps the learned method eventually becomes a reusable tool. Perhaps repeated experience improves the model’s action judgments. Perhaps another agent contributes a better check. These are different implementations of the same idea: the process of doing the work helps improve the process that will do the next piece of work.
That is why I see self-improving agents as more than agents with memory, and more than models that occasionally receive another training run. The system takes on part of the responsibility for discovering how it should improve.
We should still ask whether an agent can complete the task in front of it. But for a system intended to keep working, the next question matters too:
What will it do better because it has done this before?
This essay was prompted by Yongkyun’s Self-Evolving AI: Learning from Its Own Runs, which surveys what agent systems can change, when changes take effect, and how they are accepted. My emphasis here is on the broader goal: making an agent’s working methods learnable through experience.8
Where Indigo landsFurther
Indigo's conclusion
Against the view that continual learning ends in the weights, she is a serious opponent; for the view that verification can't be skipped and the ceiling is the evaluator, she makes the sharpest case. But her strong claims rest on bounded evidence: a promising direction, not a settled result.
What to remember
Self-improvement means turning experience into better methods, not remembering more or retraining the weights now and then.
Useful changes are concrete: an extra check, a tool that spots generated files, a habit of checking provenance before editing.
Learn a set of methods plus a rule for choosing: cheap path for ordinary edits, careful path for generated files.
One strict test separates real self-improvement from fake: does the learning curve get more accurate, cheaper and less dependent on human correction? Not the final score.
Back on the long-running theses
conflicts
Continual learning ends in the weights A serious opponent of this view: weights aren't the only place learning lives, and better methods transfer too.
confirms
Verification can't be compressed: bottleneck, moat, breaking point Better means later work gets better, with the full cost of learning counted and independent tests kept: the most rigorous evaluation plan for this view.
confirms
Harness: the moves get eaten, the interface stays She rejects “it's just a better harness” and sharpens “the ceiling is the evaluator”, confirming both ends of this view.
adds to
Who owns the learning loop A loop isn't real because it's drawn; a learning curve has to show it learns. She supplies the test.
adds to
Ashwin Gopinath: memory is a compiler, not a database Ashwin covers what to keep; she covers how to turn it into better methods.
confirms
Vercel design.md: taste distilled into a loadable file plus an eval loop Vercel is the engineering proof, she is the theory: the same thing at two levels.
adds to
Dwarkesh on the OpenAI / Hugging Face incident A grader gamed from the inside was the accident; “don't let an agent approve itself against standards it can rewrite” is the remedy.
What it means for Rewired Index
Research-side testimony on whether the method layer holds real value: better methods transfer, so it is real capability. It also gives a screening rule: a system that claims to improve itself without a learning curve on unseen tasks, full cost counted, hasn't proven it.
What would change my mind
a system that, on unseen later tasks and with the full cost of learning counted, still shows a learning curve that keeps rising.
Finished. Indigo's take on this piece is in two places: