Ari Autoresearch: Autonomous Post-Training in AC2

Introduction

Post-training a model for a specific task often requires several rounds of manual experimentation to achieve a meaningful performance gain. Each round can take hours to days, with a researcher launching a training run, monitoring run health, analyzing traces and metrics, and identifying what to try next. With Ari⌝ Autoresearch, we automate this loop by giving Ari the ability to act on its own analysis and execute experiments without explicit human guidance.

Ari Autoresearch builds on a growing line of work on automating ML research, popularized by Karpathy's original autoresearch [1] and since extended to broader, open-ended ML tasks through benchmarks like PostTrainBench [2] and AutoResearchExam [3]. While these benchmarks cover smaller, self-contained exercises, post-training at the enterprise scale poses a significantly harder challenge to automate due to increased compute requirements, time horizons, and problem complexity. Beyond changing hyperparameters, an agent might need to improve the task harness, construct a curriculum, or combine different learning algorithms to produce a desired performance lift.

AC2⌝ already provides the primitives that make these post-training experiments tractable. Working within AC2, Ari has the tools it needs to be an effective researcher, launching training and evaluation jobs, annotating traces with judges, and diagnosing failure modes from training. In a single 16 hour episode, Ari Autoresearch increased held-out reward achieved by Qwen3.5-9B on Lichess chess puzzles from 0.118 to 0.629, a 5.3x improvement.

We attribute this gain to two factors: fast iteration speed and good research judgement. Fast iteration allows an agent to run more experiments within a fixed timeframe. Given the same number of GPUs, a frontier general coding agent spent over 13 hours on a baseline run that takes roughly 2.5 hours on AC2. Alongside fast iteration, an agent also needs good research judgement, which involves identifying the root cause of a failure and choosing the experiment most likely to fix it. To improve this, we hill climb Ari's research skills on a set of decision points from past internal training runs, grading against the fixes our researchers later made.

Below, we ablate both factors by varying whether each agent has access to AC2 and to our autoresearch skills, testing four agents in total. For each agent, we report the held-out reward of its baseline GRPO run and its best run.

This post covers how autoresearch works on AC2, how we built and evaluated the skills, and what each agent actually did during its episode. Ari Autoresearch is available to AC2⌝ platform users today.

How Autoresearch Works in AC2

Autoresearch poses a challenging long horizon task as it requires agents to launch experiments, monitor their progress, analyze results, and accumulate findings across context-dense training runs. We view autoresearch as a function of two things: the infrastructure an agent uses to run experiments, and the research judgment it uses to decide what to run next. To improve the former, we’ve made several improvements to Ari’s harness and AC2 tools to support this workflow within any Ari session:

1. Autonomous Experiment Execution. Ari can now use the AC2 SDK and CLI from its workspace to curate datasets, edit project code, and launch training and evaluation jobs on AC2 with the same configuration flexibility as our Researchers. We’ve observed that this flexibility has allowed Ari to set up experiments that span multiple parts of our training process, including testing changes to the environment or multi-stage recipes such as SFT followed by RL.
2. Run Monitoring. RL runs can take days to complete, so we add a run monitoring tool which wakes up Ari every k steps, where k is set by Ari upon launching the run. At each check-in, Ari checks model behavior, investigates issues, and decides whether to continue, stop, or revise the experiment.
3. Expanded Training Metrics. Ari can’t improve training if it can’t access training and performance metrics. Thus, we’ve expanded the metrics available throughout AC2 to cover training optimization, rollout generation, evaluation, and more. Coupled with semantic analysis of individual traces, these signals provide a clearer picture on issues like excessive truncation, sparse learning signal, or throughput bottlenecks.
4. Persistent Experiment Notebook. Inspired by [1, 4, 5], we add a persistent notebook for Ari to organize its accumulating results without bloating its context. Ari consults and updates this notebook as it runs more experiments, preserving a record of its learnings and reasoning behind previous decisions.

Hill Climbing Skills

To improve Ari’s research judgement, we also constructed a set of autoresearch skills that define common workflows which appear during the research process, such as monitoring a run, analyzing traces, and identifying gaps in the model’s behavior. In our early runs, we noticed Ari often found the right symptoms but chose low leverage experiments. In one early episode, Ari spent its first two experiments on prompt variations before raising the response length cap, a more direct fix that produced a larger gain. To close this gap, we hill climb the skills that define Ari’s decision process.

Each autoresearch run takes up to a day to complete, so we built an internal dataset of decision moments from past training runs to climb Ari’s decision making ability without needing to constantly launch full autoresearch episodes. Each moment is a snapshot of context frozen immediately after a training run completes, where Ari must analyze and select the next experiment. We construct the set from two sources:

1. Ari’s own autoresearch episodes, at points where we found it made poor research decisions.
2. Training runs from other internal projects, to ensure this decision-making generalizes.

Our evaluation set contains 89 moments from a diverse set of internal training runs, on tasks ranging from coding and tool-use agents to document analysis. Each moment is tied to a run that an applied researcher later improved on, which allows us to create rubrics based on researcher-driven improvements that showed real lift. After constructing these tasks, we confirmed that every choice was deducible from the run’s own data at that moment, ensuring no task was impossible given the context available.

We run each task in a replica of our production harness, including the same prompts, tools, skills, and base model. Our eval harness provides Ari access to everything a researcher could see on the platform at that point in time, including the run’s traces, metrics, configs, project code, and judge results, while any context from later is hidden to prevent answer leakage. Ari is then tasked with identifying the limiting behavior in the model and proposing a next experiment to address it. We use a strong LLM judge to construct a specific rubric for that moment, and a second LLM judge scores Ari’s analysis and next experiment choice. To validate the grader, we compared it with blind human researcher ratings of 30 pairs of Ari’s answers. On pairs where the researcher preferred one answer, our LLM judge achieved 90% agreement with researcher preference.

This allows us to hill-climb Ari’s skills using a tuning set containing 55 tasks and an evaluation set of 34 tasks, with no run, project or root cause shared between them. Between iterations, only the skill text is changed.

Each task is run 2 times per skill version and scored by the same judge against the same rubric.

From first to the final iteration, the proportion of runs where Ari found the key root cause rose from 0.38 to 0.54 on our evaluation tasks and from 0.52 to 0.72 on tuning tasks.

Gains are in percentage points over v0, with 95% confidence intervals in brackets.

Autoresearch Experiments

Since end-to-end post-training is a complex, multi-step process, we focus our experiments on the actual training of a model. Concretely, Ari receives a (model, task) pair that includes a seed environment, initial dataset, and grader, along with an initial run config that it uses to establish a baseline. We then task Ari with improving upon the baseline’s held-out reward in subsequent training runs. Good autoresearch looks like incremental gains which compound into a significant increase in final reward.

In each autoresearch episode, Ari trains Qwen3.5-9B on Lichess puzzles, with a training pool of 8000 examples and an evaluation set of 800 puzzles. Lichess is well suited for autoresearch for a few reasons:

1. As described in our Spork blog⌝, current frontier models have little specialized training on chess, similar to many of the tasks that we encounter in enterprise post-training.
2. The end-to-end training time of Qwen3.5-9B on Lichess is relatively fast, taking around 2 hours to complete. This allows Ari to launch more experiments while keeping our iteration cycle manageable.
3. Prior experiments with Qwen3.5-9B provide evidence that training can improve reward, making Lichess a promising setting for testing whether autoresearch can find better training recipes.

Each puzzle is played as a multi-turn episode where the model receives the board state, submits moves through a tool call, and the environment responds with the opponent’s move and the updated board. This continues until the puzzle is solved or the model uses up three wrong move attempts, with partial credit awarded for each correct move.

Each autoresearch episode provides Ari a 16 hour research horizon, 16 B300 GPUs, and access to the AC2 training and evaluation tools. The Codex agents instead receive the same GPUs without AC2. Within their specified episode budget, the agents start with the same baseline run config, and have freedom to launch subsequent experiments, analyze traces, prepare datasets, and improve the agent harness and environment.

Each agent is instructed to keep each experiment runtime under 4 hours, which it can estimate by extrapolating from the runtime of the baseline experiment that it runs. We also require that each new experiment starts from the original model weights, and the evaluation dataset and grader remain fixed, which allows fair comparison against the baseline.

Results

Effective autoresearch requires both strong research judgement and infrastructure which allows an agent to iterate quickly. To measure the role of each, we benchmark four agents under this evaluation setup, each using GPT-6 Sol with medium reasoning effort.

1. Base Codex: Codex with 16 B300s, but without access to our skills or AC2.
2. Base Ari: Ari with AC2 tooling, including experiment execution, run monitoring, and expanded metrics, but without the autoresearch skills and workflows.
3. Codex with Autoresearch Skills: Codex with 16 B300s and our autoresearch skills, but without AC2 access.
4. Ari Autoresearch: Ari with AC2 tooling and the hill-climbed autoresearch skills and workflows.

We also evaluate Codex with access to AC2, which we include in the Appendix.

Notably, Ari Autoresearch improved held-out reward in Lichess from 0.118 to 0.629, a 5.3x increase over its baseline, while base Ari reached 0.228 (1.2x). Without AC2, base Codex completed only its baseline run (0.189) within the allotted timeframe, while Codex with our skills reached 0.174 from a weak baseline of 0.061. Each agent ran its own baseline from the same config, so starting reward varies with run-to-run training variance. We report each agent’s improvement relative to its own baseline.

Base Codex

We provide base Codex with 16 B300 GPUs across two nodes, along with the same task instructions, baseline config, dataset, and grader. In addition, we allow Codex to use any publicly available RL framework in its experiments. Codex set up its own chess environment and training stack, pulling in libraries like TRL for post-training and vLLM for its inference engines. However, rather than using TRL’s native GRPO trainer, it wrote its own implementation on top of PyTorch and PEFT to train LoRA adapters instead of the full model.

Without access to AC2’s training infrastructure, Codex’s training loop was quite suboptimal including blocking training updates on generating rollouts, rather than in parallel. Its 250-step run took over 13 hours, compared to roughly 2 hours for the same baseline on AC2, finishing its final evaluation 12 minutes before the episode deadline. While its final reward (0.189) is similar to reward achieved by the baselines run on AC2, Codex ultimately ran out of time to launch any further experiments, highlighting the importance of the infrastructure primitives in the autoresearch process.

Base Ari

After its initial run, base Ari found that the model often attempted moves that broke chess rules. However, it chose to pursue the cheapest fix, which was a prompting change that instructed the lichess agent to check its moves, and dismissed training on legal move demonstrations as too expensive.

When the prompt subsequently led to increased truncation, Ari shifted focus to this new symptom, spending its next three experiments exploring a more compact board representation, longer context lengths, and further training rather than returning to the original root cause.

Codex with Autoresearch Skills

We provide Codex with the same setup as before, along with the autoresearch skills we created for Ari Autoresearch. Unlike base Codex, it trained the full model with TRL’s GRPO trainer, but its baseline peaked at only 0.061. We attribute this to Codex training with BF16 weights without an FP32 master copy, this caused most weight updates at a learning rate of 1e-6 to be lost to rounding. Codex never identified this issue, choosing to explore other training methods.

Still, Codex’s research process with our skills closely resembled Ari Autoresearch, with each experiment targeting a specific failure it observed in the previous run. After its baseline, it analyzed traces and identified that most failures came from illegal or incorrect moves, switching to SFT on correct first moves. When this caused excessive model truncation, it trained on the environment’s feedback after each move, then experimented with DPO to teach the model to prefer the correct move over plausible wrong ones. Over 13 training experiments, it improved reward from 0.061 to 0.174, below the 0.189 reached by base Codex.

Ari Autoresearch

After the baseline run, Ari Autoresearch identified the same root cause as base Ari, but concluded that prompting would not help, since the system prompt already asked the model for careful analysis, and instead gave the model a list of legal moves. Importantly, Ari Autoresearch also saw truncation rise during its baseline, but decided against raising the response limit after calculating that even fixing every truncated response would add little, since completed responses scored barely higher. When both agents later tried SFT, base Ari trained only on the correct first move with no reasoning, while Ari Autoresearch constructed a new training pool by replaying full solved puzzles through the environment, further improving reward to 0.515 and then 0.629.

We observed three behaviors that shaped Ari Autoresearch’s success:

1. Diagnosing root cause. Rather than prompting again like base Ari, Ari changed the harness to show the model its legal moves, resulting in the largest single gain of the episode.
2. Estimating experiment leverage. Before each experiment, Ari used AC2’s evaluation metrics and trace tooling to calculate the most a change could improve reward, rejecting the response limit increase that base Ari spent experiments on. Codex with our skills made the same calculation and also deprioritized truncation, but only after writing its own scripts to replay and classify its evaluation episodes.
3. Accumulating improvements. Ari started each experiment from the best training recipe seen so far, retraining from the original model weights each time, so its improvements compounded from the legal move harness to the SFT warmup. Compared to Codex, where a single GRPO run took over 7 hours, training on AC2 allowed Ari to iterate much faster.

Quantitative Autoresearch Metrics

Quantitatively, we found that Ari Autoresearch also spends more effort on analysis compared to base Ari, which ran in the same harness. It made 2.5x as many analysis calls (405 vs 159) and read 3x as many span payloads (97 vs 30), incurring only a few additional minutes between runs. This additional overhead is negligible relative to the time spent training, so we find this tradeoff acceptable.

Codex with our skills launched the most experiments, choosing to spend more time exploring shorter SFT, DPO, and GRPO stages continuing from earlier checkpoints. However, its training throughput remained slow, with the baseline taking over 7 hours compared to roughly 2 hours for Ari’s baseline experiment.

Conclusion

With Ari Autoresearch, we’ve taken the next step towards truly autonomous post-training in AC2. Platform users can now use Ari to autonomously launch training jobs, create datasets, improve harnesses, and hill climb models on their specific tasks, powered by accumulated insights from our Applied Research Engineers and platform primitives we have built to accelerate them.

References

1. karpathy/autoresearch⌝
2. PostTrainBench: Can LLM Agents Automate LLM Post-Training?⌝
3. AutoResearchExam⌝
4. MemGPT: Towards LLMs as Operating Systems⌝
5. Effective context engineering for AI agents⌝

Appendix

Codex + AC2 SDK

Through the entire autoresearch episode, Codex’s analysis centered around alleviating growing truncation during training. In its baseline run, Codex found that most of the evaluation samples were being truncated, choosing to address this through several interventions, including prompting the model to be more concise, training on an easier mix of puzzles, and raising the response limit to 12k.

Over the course of the episode, it continued to iterate on this single observation, reaching its best result (0.293) by combining a prompting change with an increased context length. However, it never dug deeper in its analysis to examine why the model failed the puzzles it did complete, which both Ari agents traced to illegal moves.