Task-level results

Pass@1 matrix

One row per published task and one column per available model. The cell label is the task-level private-test Pass@1 result. Background intensity adds private-test progress as a visual cue.

Find taskScientific-knowledge ablation onlyTask details

Private test pass fraction0%100%Continuous scale; PASS / FAIL labels remain authoritative.

Task	Claude-Opus-5 (max)	DeepSeek-V4-Pro (max)	GPT-5.6-sol (max)	GPT-6 Astra (max)	Kimi-K3 (max)	GLM-5.2 (max)	Qwen3.8-27B (xhigh)
001	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
002	FAIL	PASS	PASS	PASS	FAIL	PASS	FAIL
003	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
004	PASS	PASS	PASS	PASS	PASS	FAIL	PASS
005	FAIL	FAIL	FAIL	PASS	FAIL	FAIL	FAIL
006	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
007	PASS	PASS	FAIL	FAIL	PASS	FAIL	PASS
008	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
009	FAIL	PASS	FAIL	FAIL	PASS	PASS	FAIL
010	PASS	PASS	PASS	PASS	PASS	PASS	FAIL
011	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
012	FAIL	FAIL	FAIL	FAIL	PASS	PASS	PASS
013	PASS	PASS	PASS	PASS	PASS	PASS	PASS
014	FAIL	PASS	FAIL	FAIL	FAIL	FAIL	FAIL
015	PASS	PASS	FAIL	PASS	PASS	PASS	PASS
016	PASS	PASS	PASS	PASS	PASS	PASS	PASS
017	PASS	PASS	PASS	PASS	PASS	PASS	FAIL
018	PASS	PASS	PASS	PASS	PASS	PASS	PASS
019	PASS	PASS	FAIL	FAIL	FAIL	FAIL	FAIL
020	FAIL	PASS	PASS	FAIL	FAIL	FAIL	FAIL
021	PASS	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
022	PASS	PASS	PASS	PASS	PASS	PASS	FAIL
023	PASS	PASS	PASS	PASS	PASS	PASS	PASS
024	PASS	PASS	PASS	FAIL	PASS	PASS	FAIL
025	PASS	PASS	PASS	PASS	PASS	FAIL	PASS
026	PASS	FAIL	PASS	PASS	PASS	FAIL	PASS
027	PASS	PASS	PASS	PASS	PASS	PASS	PASS
028	PASS	PASS	PASS	PASS	PASS	PASS	PASS
029	PASS	FAIL	PASS	PASS	FAIL	PASS	FAIL
030	PASS	PASS	PASS	PASS	PASS	PASS	PASS
031	PASS	PASS	PASS	PASS	PASS	PASS	PASS
032	PASS	PASS	PASS	PASS	PASS	PASS	FAIL
033	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
034	PASS	FAIL	PASS	PASS	FAIL	PASS	FAIL
035	FAIL	FAIL	PASS	PASS	FAIL	FAIL	FAIL
036	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
037	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
038	PASS	FAIL	PASS	PASS	PASS	PASS	PASS
039	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
040	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
041	FAIL	FAIL	FAIL	PASS	FAIL	FAIL	FAIL
042	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
043	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
044	PASS	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
045	PASS	PASS	PASS	PASS	PASS	PASS	PASS
046	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
047	FAIL	FAIL	FAIL	PASS	FAIL	FAIL	FAIL
048	PASS	FAIL	FAIL	PASS	FAIL	PASS	FAIL
049	FAIL	FAIL	PASS	PASS	FAIL	FAIL	FAIL
050	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
051	PASS	PASS	PASS	FAIL	FAIL	FAIL	FAIL
052	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
053	PASS	PASS	PASS	FAIL	PASS	FAIL	PASS
054	FAIL	PASS	FAIL	FAIL	FAIL	FAIL	FAIL
055	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
056	PASS	PASS	PASS	PASS	PASS	PASS	PASS
057	PASS	PASS	PASS	PASS	FAIL	PASS	PASS
058	FAIL	PASS	PASS	PASS	PASS	FAIL	FAIL
059	FAIL	FAIL	FAIL	FAIL	PASS	FAIL	FAIL
060	FAIL	FAIL	FAIL	PASS	FAIL	FAIL	FAIL
061	PASS	PASS	PASS	PASS	FAIL	FAIL	PASS
062	PASS	PASS	PASS	FAIL	FAIL	PASS	PASS
063	FAIL	PASS	FAIL	FAIL	PASS	PASS	PASS
064	PASS	PASS	PASS	PASS	PASS	PASS	FAIL
065	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
066	PASS	FAIL	FAIL	PASS	FAIL	FAIL	FAIL
067	FAIL	FAIL	FAIL	PASS	FAIL	FAIL	PASS
068	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
069	PASS	PASS	PASS	PASS	PASS	PASS	PASS
070	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
071	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
072	PASS	FAIL	FAIL	PASS	FAIL	FAIL	FAIL
073	FAIL	FAIL	FAIL	PASS	FAIL	FAIL	FAIL
074	PASS	PASS	PASS	FAIL	PASS	FAIL	PASS
075	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
076	PASS	FAIL	FAIL	PASS	FAIL	FAIL	FAIL
077	PASS	FAIL	PASS	PASS	PASS	FAIL	PASS
078	PASS	PASS	PASS	PASS	FAIL	PASS	PASS
079	FAIL	PASS	FAIL	FAIL	FAIL	FAIL	FAIL
080	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	PASS
081	PASS	PASS	PASS	PASS	PASS	PASS	PASS
082	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
083	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
084	FAIL	PASS	FAIL	PASS	PASS	FAIL	PASS
085	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
086	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
087	PASS	PASS	FAIL	PASS	FAIL	FAIL	FAIL
088	PASS	FAIL	PASS	PASS	FAIL	FAIL	FAIL
089	FAIL	FAIL	FAIL	PASS	FAIL	FAIL	FAIL
090	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
091	PASS	FAIL	PASS	PASS	PASS	PASS	PASS
092	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
093	PASS	FAIL	FAIL	FAIL	FAIL	PASS	FAIL
094	PASS	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
095	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
096	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
097	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
098	FAIL	PASS	PASS	FAIL	PASS	FAIL	FAIL
099	FAIL	FAIL	FAIL	PASS	FAIL	FAIL	FAIL
100	PASS	FAIL	FAIL	PASS	PASS	FAIL	FAIL
101	PASS	FAIL	PASS	PASS	PASS	FAIL	PASS
102	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
103	PASS	PASS	PASS	PASS	PASS	FAIL	FAIL
104	FAIL	PASS	FAIL	PASS	FAIL	FAIL	FAIL
105	FAIL	PASS	PASS	FAIL	FAIL	PASS	FAIL
106	PASS	PASS	FAIL	PASS	PASS	PASS	FAIL
107	PASS	PASS	FAIL	FAIL	PASS	PASS	PASS
108	PASS	PASS	PASS	PASS	PASS	FAIL	FAIL
109	FAIL	FAIL	FAIL	PASS	FAIL	FAIL	FAIL
110	PASS	FAIL	FAIL	PASS	FAIL	FAIL	FAIL
111	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
112	PASS	PASS	PASS	PASS	PASS	PASS	PASS
113	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
114	FAIL	FAIL	FAIL	FAIL	FAIL	PASS	FAIL
115	FAIL	PASS	PASS	PASS	FAIL	PASS	PASS
116	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL
117	PASS	FAIL	PASS	PASS	FAIL	FAIL	FAIL
118	PASS	PASS	PASS	PASS	FAIL	FAIL	FAIL
119	PASS	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL

Task details

Expand a task to compare model, harness, test, and trace entries.

0010/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	2/3 (66.7%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace

002Ablation4/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	6/8 (75.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	6/8 (75.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	7/8 (87.5%)	FAIL	Open trace

003Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	4/6 (66.7%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	3/6 (50.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	3/6 (50.0%)	FAIL	Open trace

004Ablation6/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	0/1 (0.0%)	2/15 (13.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace

005Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	1/4 (25.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	1/4 (25.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	1/4 (25.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	0/4 (0.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	1/4 (25.0%)	FAIL	Open trace

006Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	11/13 (84.6%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	10/13 (76.9%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	0/1 (0.0%)	0/13 (0.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	10/13 (76.9%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	10/13 (76.9%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	10/13 (76.9%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	11/13 (84.6%)	FAIL	Open trace

007Ablation4/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	3/6 (50.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace

008Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	0/1 (0.0%)	0/6 (0.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	1/6 (16.7%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	1/6 (16.7%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	1/6 (16.7%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	0/6 (0.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	0/1 (0.0%)	0/6 (0.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	1/6 (16.7%)	FAIL	Open trace

009Ablation3/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	0/1 (0.0%)	5/9 (55.6%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	6/9 (66.7%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	5/9 (55.6%)	FAIL	Open trace

010Ablation6/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	6/11 (54.5%)	FAIL	Open trace

011Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	1/5 (20.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	3/5 (60.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	3/5 (60.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	3/5 (60.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/5 (60.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	3/5 (60.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/5 (40.0%)	FAIL	Open trace

012Ablation3/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	9/10 (90.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	9/10 (90.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	9/10 (90.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	9/10 (90.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace

013Ablation7/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace

014Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	3/6 (50.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/6 (50.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	0/1 (0.0%)	2/6 (33.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace

015Ablation6/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	31/31 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	31/31 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	11/31 (35.5%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	31/31 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	31/31 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	31/31 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	31/31 (100.0%)	PASS	Open trace

016Ablation7/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace

017Ablation6/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	11/12 (91.7%)	FAIL	Open trace

018Ablation7/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace

019Ablation2/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	14/14 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	14/14 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	13/14 (92.9%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	13/14 (92.9%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	13/14 (92.9%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	13/14 (92.9%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	13/14 (92.9%)	FAIL	Open trace

020Ablation2/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	0/1 (0.0%)	0/12 (0.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	11/12 (91.7%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	11/12 (91.7%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	11/12 (91.7%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	10/12 (83.3%)	FAIL	Open trace

021Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	13/15 (86.7%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	11/15 (73.3%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	11/15 (73.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	5/15 (33.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	10/15 (66.7%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	9/15 (60.0%)	FAIL	Open trace

022Ablation6/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	12/12 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	11/12 (91.7%)	FAIL	Open trace

023Ablation7/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace

024Ablation5/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	7/8 (87.5%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	4/8 (50.0%)	FAIL	Open trace

025Ablation6/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	0/7 (0.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace

026Ablation5/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace

027Ablation7/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace

028Ablation7/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace

029Ablation4/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	29/29 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	14/29 (48.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	29/29 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	29/29 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	14/29 (48.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	29/29 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	14/29 (48.3%)	FAIL	Open trace

030Ablation7/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace

031Ablation7/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	41/41 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	41/41 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	41/41 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	41/41 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	41/41 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	41/41 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	41/41 (100.0%)	PASS	Open trace

032Ablation6/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	21/21 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	13/21 (61.9%)	FAIL	Open trace

033Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	9/10 (90.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	7/10 (70.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	7/10 (70.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace

034Ablation4/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	1/8 (12.5%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	1/8 (12.5%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	1/8 (12.5%)	FAIL	Open trace

035Ablation2/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	8/14 (57.1%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	9/14 (64.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	14/14 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	14/14 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	9/14 (64.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/14 (57.1%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	8/14 (57.1%)	FAIL	Open trace

036Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	6/9 (66.7%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace

037Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	3/13 (23.1%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	3/13 (23.1%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	10/13 (76.9%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	11/13 (84.6%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/13 (23.1%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	3/13 (23.1%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	10/13 (76.9%)	FAIL	Open trace

038Ablation6/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	5/15 (33.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace

039Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	4/11 (36.4%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	4/11 (36.4%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	10/11 (90.9%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	10/11 (90.9%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	4/11 (36.4%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	4/11 (36.4%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	4/11 (36.4%)	FAIL	Open trace

040Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	2/3 (66.7%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	0/3 (0.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	0/3 (0.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	0/3 (0.0%)	FAIL	Open trace

041Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	8/24 (33.3%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/24 (33.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	8/24 (33.3%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	24/24 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	8/24 (33.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/24 (33.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	7/24 (29.2%)	FAIL	Open trace

042Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	12/16 (75.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	15/16 (93.8%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	15/16 (93.8%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	15/16 (93.8%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	15/16 (93.8%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	6/16 (37.5%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	7/16 (43.8%)	FAIL	Open trace

043Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	4/8 (50.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	4/8 (50.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	4/8 (50.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	3/8 (37.5%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	1/8 (12.5%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	4/8 (50.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/8 (25.0%)	FAIL	Open trace

044Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/15 (53.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	9/15 (60.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	8/15 (53.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	8/15 (53.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/15 (53.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	8/15 (53.3%)	FAIL	Open trace

045Ablation7/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	16/16 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	16/16 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	16/16 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	16/16 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	16/16 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	16/16 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	16/16 (100.0%)	PASS	Open trace

046Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace

047Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	5/7 (71.4%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	2/7 (28.6%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/7 (28.6%)	FAIL	Open trace

048Ablation3/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	13/13 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	12/13 (92.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	12/13 (92.3%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	13/13 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	5/13 (38.5%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	13/13 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	12/13 (92.3%)	FAIL	Open trace

049Ablation2/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	0/1 (0.0%)	1/13 (7.7%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	11/13 (84.6%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	13/13 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	13/13 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	11/13 (84.6%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	11/13 (84.6%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	11/13 (84.6%)	FAIL	Open trace

050Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	9/11 (81.8%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/11 (72.7%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	8/11 (72.7%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	9/11 (81.8%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	0/1 (0.0%)	2/11 (18.2%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	9/11 (81.8%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	9/11 (81.8%)	FAIL	Open trace

051Ablation3/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	4/6 (66.7%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	4/6 (66.7%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	0/6 (0.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	0/6 (0.0%)	FAIL	Open trace

052Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	8/16 (50.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	9/16 (56.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	15/16 (93.8%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	15/16 (93.8%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	10/16 (62.5%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/16 (50.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	11/16 (68.8%)	FAIL	Open trace

053Ablation5/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace

054Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	3/7 (42.9%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	3/7 (42.9%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	4/7 (57.1%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/7 (42.9%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	3/7 (42.9%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	3/7 (42.9%)	FAIL	Open trace

055Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	10/18 (55.6%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	10/18 (55.6%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	10/18 (55.6%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	13/18 (72.2%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	10/18 (55.6%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/18 (44.4%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	8/18 (44.4%)	FAIL	Open trace

056Ablation7/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace

057Ablation6/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/11 (27.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace

058Ablation4/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	5/9 (55.6%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace

059Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	5/9 (55.6%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	5/9 (55.6%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	5/9 (55.6%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	7/9 (77.8%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	2/9 (22.2%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	7/9 (77.8%)	FAIL	Open trace

060Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	4/11 (36.4%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	4/11 (36.4%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	0/1 (0.0%)	4/11 (36.4%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	4/11 (36.4%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	4/11 (36.4%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	4/11 (36.4%)	FAIL	Open trace

061Ablation5/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	0/1 (0.0%)	0/1 (0.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	2/8 (25.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace

062Ablation5/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	17/17 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	17/17 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	17/17 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	16/17 (94.1%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	16/17 (94.1%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	17/17 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	17/17 (100.0%)	PASS	Open trace

063Ablation4/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	11/18 (61.1%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	18/18 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	17/18 (94.4%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	11/18 (61.1%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	18/18 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	18/18 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	18/18 (100.0%)	PASS	Open trace

064Ablation6/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	4/8 (50.0%)	FAIL	Open trace

065Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace

066Ablation2/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace

067Ablation2/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	3/6 (50.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	3/6 (50.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	3/6 (50.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/6 (50.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	3/6 (50.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace

068Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	0/3 (0.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	0/3 (0.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	2/3 (66.7%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	0/3 (0.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	0/3 (0.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	0/3 (0.0%)	FAIL	Open trace

069Ablation7/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace

070Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	2/9 (22.2%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	2/9 (22.2%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	2/9 (22.2%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	2/9 (22.2%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	2/9 (22.2%)	FAIL	Open trace
GLM-5.2 (max)	Codex	0/1 (0.0%)	0/9 (0.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	0/9 (0.0%)	FAIL	Open trace

071Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	1/5 (20.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	1/5 (20.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	2/5 (40.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	3/5 (60.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	1/5 (20.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	2/5 (40.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/5 (40.0%)	FAIL	Open trace

072Ablation2/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/7 (42.9%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/7 (28.6%)	FAIL	Open trace

073Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace

074Ablation5/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	5/10 (50.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace

075Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	1/8 (12.5%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	4/8 (50.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	3/8 (37.5%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	1/8 (12.5%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	1/8 (12.5%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	1/8 (12.5%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	0/8 (0.0%)	FAIL	Open trace

076Ablation2/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace

077Ablation5/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	13/13 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	12/13 (92.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	13/13 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	13/13 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	13/13 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	12/13 (92.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	13/13 (100.0%)	PASS	Open trace

078Ablation6/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	42/42 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	42/42 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	42/42 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	42/42 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	17/42 (40.5%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	42/42 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	42/42 (100.0%)	PASS	Open trace

079Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	13/14 (92.9%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	14/14 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	13/14 (92.9%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	13/14 (92.9%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	13/14 (92.9%)	FAIL	Open trace
GLM-5.2 (max)	Codex	0/1 (0.0%)	5/14 (35.7%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	13/14 (92.9%)	FAIL	Open trace

080Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	10/15 (66.7%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	10/15 (66.7%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	10/15 (66.7%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	14/15 (93.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	10/15 (66.7%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	10/15 (66.7%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace

081Ablation7/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace

082Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	1/7 (14.3%)	FAIL	Open trace

0830/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	1/3 (33.3%)	FAIL	Open trace

084Ablation4/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	5/7 (71.4%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	5/7 (71.4%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace

0850/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	4/10 (40.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/10 (80.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	3/10 (30.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	9/10 (90.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	8/10 (80.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	3/10 (30.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	4/10 (40.0%)	FAIL	Open trace

086Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace

0873/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	2/5 (40.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	0/5 (0.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	3/5 (60.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	1/5 (20.0%)	FAIL	Open trace

0883/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	4/5 (80.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/5 (60.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	3/5 (60.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	3/5 (60.0%)	FAIL	Open trace

0891/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace

090Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	3/6 (50.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	5/6 (83.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace

0916/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	3/3 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	0/3 (0.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	3/3 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	3/3 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/3 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	3/3 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	3/3 (100.0%)	PASS	Open trace

0920/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	7/9 (77.8%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	7/9 (77.8%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	8/9 (88.9%)	FAIL	Open trace

0932/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace

0941/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	0/5 (0.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	0/5 (0.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	4/5 (80.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	0/5 (0.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	0/5 (0.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	0/5 (0.0%)	FAIL	Open trace

0950/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	0/8 (0.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	0/8 (0.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	2/8 (25.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	2/8 (25.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	0/8 (0.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	2/8 (25.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	4/8 (50.0%)	FAIL	Open trace

0960/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	5/8 (62.5%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	5/8 (62.5%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	5/8 (62.5%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	5/8 (62.5%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	5/8 (62.5%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	5/8 (62.5%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	5/8 (62.5%)	FAIL	Open trace

097Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	0/7 (0.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	6/7 (85.7%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	0/7 (0.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	0/7 (0.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	0/7 (0.0%)	FAIL	Open trace

098Ablation3/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	8/11 (72.7%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	8/11 (72.7%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	11/11 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	8/11 (72.7%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	8/11 (72.7%)	FAIL	Open trace

099Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace

100Ablation3/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	7/8 (87.5%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	7/8 (87.5%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	8/8 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	7/8 (87.5%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	7/8 (87.5%)	FAIL	Open trace

101Ablation5/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	8/10 (80.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	0/1 (0.0%)	3/10 (30.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace

1020/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	3/4 (75.0%)	FAIL	Open trace

1035/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace

1042/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	0/7 (0.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	0/7 (0.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	7/7 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	0/7 (0.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	0/7 (0.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	0/7 (0.0%)	FAIL	Open trace

1053/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	4/4 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/4 (50.0%)	FAIL	Open trace

1065/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	37/37 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	37/37 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	32/37 (86.5%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	37/37 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	37/37 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	37/37 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	35/37 (94.6%)	FAIL	Open trace

1075/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	3/3 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	3/3 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	2/3 (66.7%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	2/3 (66.7%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/3 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	3/3 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	3/3 (100.0%)	PASS	Open trace

1085/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	23/23 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	23/23 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	23/23 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	23/23 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	23/23 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	0/1 (0.0%)	7/23 (30.4%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	21/23 (91.3%)	FAIL	Open trace

1091/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	1/6 (16.7%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	1/6 (16.7%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	1/6 (16.7%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	1/6 (16.7%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	1/6 (16.7%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	1/6 (16.7%)	FAIL	Open trace

1102/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	4/10 (40.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	4/10 (40.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	4/10 (40.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	4/10 (40.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	4/10 (40.0%)	FAIL	Open trace

111Ablation0/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	6/10 (60.0%)	FAIL	Open trace

1127/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	5/5 (100.0%)	PASS	Open trace

1130/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	9/13 (69.2%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	7/13 (53.8%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	7/13 (53.8%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	11/13 (84.6%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	9/13 (69.2%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	7/13 (53.8%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	9/13 (69.2%)	FAIL	Open trace

114Ablation1/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	13/15 (86.7%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	13/15 (86.7%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	13/15 (86.7%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	5/15 (33.3%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	0/1 (0.0%)	10/15 (66.7%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	15/15 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	13/15 (86.7%)	FAIL	Open trace

1155/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	93/97 (95.9%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	97/97 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	97/97 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	97/97 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	89/97 (91.8%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	97/97 (100.0%)	PASS	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	97/97 (100.0%)	PASS	Open trace

1160/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	85/93 (91.4%)	FAIL	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	85/93 (91.4%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	75/93 (80.6%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	85/93 (91.4%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	85/93 (91.4%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	85/93 (91.4%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	85/93 (91.4%)	FAIL	Open trace

1173/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	6/6 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/6 (50.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/6 (33.3%)	FAIL	Open trace

1184/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	9/9 (100.0%)	PASS	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	5/9 (55.6%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	5/9 (55.6%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	5/9 (55.6%)	FAIL	Open trace

1191/7 model Pass@1

Model	Harness	Public	Private	Pass@1	Trace
Claude-Opus-5 (max)	Claude Code	1/1 (100.0%)	10/10 (100.0%)	PASS	Open trace
DeepSeek-V4-Pro (max)	Claude Code	1/1 (100.0%)	2/10 (20.0%)	FAIL	Open trace
GPT-5.6-sol (max)	Codex	1/1 (100.0%)	2/10 (20.0%)	FAIL	Open trace
GPT-6 Astra (max)	Codex	1/1 (100.0%)	9/10 (90.0%)	FAIL	Open trace
Kimi-K3 (max)	Kimi Code	1/1 (100.0%)	3/10 (30.0%)	FAIL	Open trace
GLM-5.2 (max)	Codex	1/1 (100.0%)	2/10 (20.0%)	FAIL	Open trace
Qwen3.8-27B (xhigh)	Claude Code	1/1 (100.0%)	2/10 (20.0%)	FAIL	Open trace

Shading reflects the private-test pass fraction for each task; PASS@1 labels show whether the task passed.