Multi-turn Wordle

Evaluating executive function and information gain across multi-turn gameplay.

Data Source: evaluate_wordle_multi_turn.
Win Rate vs. Info Gain: As games progress to the 6th turn, the task becomes easier and win rates naturally spike from 7-8% to 70-80%, reducing the relative importance of the Information Gain metric compared to single-turn evaluations.
Frontier Dominance in State Tracking: The top five models remain largely consistent but reorder: GPT-5.5 takes second place and Claude Opus overtakes Gemma-4-31b. This shift occurs because multi-turn gameplay demands superior working memory and dynamic state tracking, giving larger frontier models an edge.
Inhibitory Control: Rule violations scale proportionally with overall scores, demonstrating that high-performing models possess superior working memory management and inhibitory control.
Mid-Weight Efficiency: Gemma-4 maintains strong performance across both modes, ranking second in cost, speed, and info density despite being the slowest model tested.
Non-Reasoning Shortfalls: Grok non-reasoning falls far behind the pack, failing on both performance scores and cost efficiency.

Rank	Model	Overall Score	Win Rate	Avg Violations	API Error Rate	Penalty
1	gemini-3.1-pro-preview	1.1266	73.5%	0.041	5.8%	-
2	gpt-5.5	1.0919	69.2%	0.000	0.0%	-
3	claude-sonnet-4.6	1.0455	73.1%	0.173	0.0%	-
4	grok-4.20-reasoning	1.0289	65.4%	0.154	0.0%	-
5	claude-opus-4.6	1.0136	63.5%	0.077	0.0%	-
6	gemma-4-31b	1.0099	66.0%	0.149	9.6%	-
7	glm-5	0.8876	63.3%	0.490	5.8%	-
8	gpt-5.4	0.8377	59.6%	0.365	0.0%	-
9	deepseek-r1	0.7964	57.1%	0.381	19.2%	-
10	deepseek-v3.2	0.6449	44.7%	0.872	9.6%	-
11	qwen-3-next-80b-instruct	0.5165	30.0%	0.260	3.8%	-
12	qwen-3-235b-A22b-instruct	0.4257	34.0%	0.540	3.8%	-
13	gpt-5.4-mini	0.2185	26.9%	1.731	0.0%	-
14	grok-4.20-non-reasoning	0.1927	48.1%	2.923	0.0%	-
15	gpt-5.4-nano	0.0446	15.4%	1.481	0.0%	-
16	gemini-3-flash-preview	0.0000	64.5%	0.258	40.4%	Yes
17	qwen-3-next-80b-thinking	0.0000	50.0%	0.000	42.3%	Yes

Evaluating operational footprint. Sorted by a Combined Efficiency Score (33% Cost, 34% Speed, 33% Density).

Rank	Model	Combined Score	Cost ($)	Bang-for-Buck	Time (Mins)	Velocity	Info Density
1	gpt-5.4-2026-03-05	71.6 / 100	$1.3501	0.622	12.3m	0.0683	0.218
2	gemma-4-31b-it	49.1 / 100	$0.2276	4.438	278.1m	0.0036	0.094
3	gpt-5.4-mini-2026-03-17	28.9 / 100	$0.4999	0.440	6.0m	0.0368	0.049
4	qwen3-next-80b-a3b-instruct	27.8 / 100	$0.4145	1.255	16.7m	0.0311	0.020
5	gemini-3.1-pro-preview	27.2 / 100	$8.8613	0.128	108.9m	0.0104	0.139
6	gpt-5.5-2026-04-23	26.9 / 100	$5.8408	0.187	46.1m	0.0236	0.090
7	grok-4.20-0309-reasoning	15.9 / 100	$2.5295	0.407	48.1m	0.0214	0.014
8	claude-opus-4-6-default	15.5 / 100	$9.6204	0.105	66.6m	0.0152	0.048
9	deepseek-v3.2	13.8 / 100	$0.7213	0.887	97.5m	0.0066	0.026
10	grok-4.20-0309-non-reasoning	12.8 / 100	$0.9321	0.204	10.6m	0.0179	0.016
11	glm-5	9.9 / 100	$2.1075	0.422	105.2m	0.0085	0.017
12	claude-sonnet-4-6-default	9.4 / 100	$9.5718	0.110	107.4m	0.0098	0.024
13	deepseek-r1-0528	7.4 / 100	$4.7953	0.167	88.2m	0.0091	0.011
14	gpt-5.4-nano-2026-03-17	5.6 / 100	$0.1449	0.276	7.9m	0.0051	0.007
15	gemini-3-flash-preview	0.0 / 100	$2.9042	0.000	72.2m	0.0000	0.000
16	qwen3-next-80b-a3b-thinking	0.0 / 100	$1.2035	0.000	115.8m	0.0000	0.000