Chemistry experiment · October 2026
Can an agent improve
the way it works?
Let an agent revise its own workflow. Test each change. Keep what works, and use the results to guide the next revision.
RSI is a method for improving an agent’s workflow. This experiment uses Claude Sonnet 5.5 in native Claude Code on two chemistry tasks.
How the improvement loop works
A model proposes a change. Fresh runs decide whether it stays.
The agent’s harness
The parts RSI can edit
Add an experiment checklist
Make planning and submission checks explicit before the agent spends its experiment budget.
No extra campaign instructions.
Read the task files first. Keep a validated model on disk from the start. Check every batch before running it.
instructions.md
No hidden answers or changes to the test.
2 chemistry tasks × 3 trials
Only measured results decide.
Read the evidence
The first round starts with public task instructions. Later rounds also use scores and traces from this experiment.
Animation explains the process; it does not simulate experimental results. Instructions and Hooks show actual edits from this run in plain English. Other components show possible edits, not tested candidates.
What can change?
The harness is the guidance and support around the model: instructions, skills, helper tools, MCP tools, lifecycle hooks and task-local memory. The selected version adds a checklist, a checking tool and a one-shot stop hook.
What stays fixed?
Claude Sonnet 5.5 (high), the native Claude Code conversation and tool loop, the chemistry tasks, their experimental budgets, and the official scoring rules.
What makes it recursive?
An accepted change becomes the starting point for the next proposal. Every tested change leaves evidence for the next round, even when it is discarded.
Fixed before the run. Each candidate gets one screening trial per task, then two more unless it clearly fails against a working parent. A candidate is kept when its posterior probability of beating the parent is at least 0.8. A difference of one trial in six is treated as noise: it wins only if the candidate is also simpler or at least 15% cheaper. Before any trial, an automatic gate (format, leakage scan, offline smoke tests) and a critic must pass, with up to three repairs.
Did it work?
Rounds 1–5, scored by the official AS-Bench verifier.
Reading the latest published experiment snapshot.
Performance across revisions
Passes per candidate. Grey bars were measured but not accepted; hatched rounds never reached a trial.
| Candidate | Propylene | Suzuki | Total | P(beats parent) | Decision |
|---|
Each trial is scored by the official AS-Bench verifier. No local native-agent baseline or independent retest was run. Two earlier attempts of this protocol stopped on an expired login and on a controller fault; they are preserved in the archive and contributed no evidence to this run.
What changed in each round
Each round's plan, gate and critic history, exact edits, and scored trials.
A method beyond one model.
The experiment separates task execution from workflow revision. All model roles in this run used Claude Sonnet 5.5. The controller talks to runtimes through adapters; other models need their own adapter and evaluation.
Inspired by Regularized Recursive Self-Improvement. This is our own selection protocol, not a reproduction of the full paper.
Experiment record
Runs used the existing Claude Max subscription only. This is a frozen result snapshot, not a live monitor.
Download results and round history