Hill climbing Ari’s skills
Share of runs where Ari finds the key root cause, by skill version
Tuning (55 tasks)
Evaluation (34 held-out tasks)
Hill climbing Ari’s skills
30%
40%
50%
60%
70%
80%
Runs finding the root cause
0
1
2
3
4
5
6
7
8
Skill version (v0 to v8)
v0 tuning
v0 evaluation
Tuning 72%
Evaluation 54%