Project Page
We evaluate 35 language models with four prompt strategies, with and without reasoning. Success means reaching the goal within the step budget. The error decomposition explains behavior that task success alone cannot capture.
Takeaway 1
Across 35 models, success has strong negative linear relationships with log exploration error (R² = 0.893) and log exploitation error (R² = 0.874). Agents need to discover useful states and use that knowledge to complete the task.
These errors summarize trajectory behavior and complement success rate; they are not standalone rankings of overall agent quality.
Takeaway 2
On the same map and prompt, Hy3 and GPT-OSS-120B both reach the goal. Yet Hy3 incurs only exploration errors, while GPT-OSS-120B incurs only exploitation errors. The metric reveals which behavior needs improvement in each trajectory.
| Model | Steps / budget | Exploration error | Exploitation error |
|---|---|---|---|
| Hy3 | 148 / 210 | 8.1% | 0.0% |
| GPT-OSS-120B | 188 / 210 | 0.0% | 47.3% |
This is a comparison of two individual trajectories. Each error rate is normalized over the steps where that behavior is relevant.
Takeaway 3
Exploration-focused prompts reduce exploration error, while exploitation-focused prompts reduce exploitation error. For GPT-4.1, the exploration prompt raises success from 63% to 80% and lowers exploration error from 0.123 to 0.099.
Figure 4. The prompts share an environment description and action format; only one strategy sentence changes.
Takeaway 4
A compact summary of previously observed information helps agents act without generating reasoning traces. Compared with a baseline that retains its own reasoning traces, the harness achieves higher success, lower exploration error, and lower total step-budget usage for both evaluated models. The summary organizes existing observations and adds no new environment information.
| Model | Method | Success ↑ | Exploration error ↓ | Exploitation error ↓ | Avg. budget used ↓ |
|---|---|---|---|---|---|
| Gemini 3.1 Flash Lite | Reasoning | 50.0% | 0.190 | 0.193 | 77.1% |
| Structured memory | 76.9% | 0.144 | 0.083 | 64.9% | |
| GPT-4.1 | Reasoning | 66.7% | 0.159 | 0.227 | 72.6% |
| Structured memory | 72.2% | 0.121 | 0.293 | 69.2% |
GPT-4.1’s exploitation error increases from 0.227 to 0.293 even as success improves. The separate errors make this tradeoff visible.
Takeaway 5
Replacing arbitrary symbols with meaningful cooking-related names substantially reduces exploitation error, while exploration error remains similar. Semantic knowledge can help infer task dependencies, but it does not reveal where the task nodes are located.
| Model | Task names | Success ↑ | Exploration error ↓ | Exploitation error ↓ |
|---|---|---|---|---|
| Gemini 3.1 Flash Lite | Symbolic | 0.0% | 0.305 | 0.405 |
| Semantic | 25.0% | 0.291 | 0.061 | |
| GPT-4.1 | Symbolic | 25.0% | 0.275 | 0.456 |
| Semantic | 27.5% | 0.284 | 0.219 |
Takeaway 6
Exploration error rises with exploration demand, and exploitation error rises with exploitation demand. Increasing task DAG size also increases both errors. These controlled changes show that the metrics respond systematically to the requirements of the task.
Takeaway 7 / Minecraft Clone
In Minecraft Clone, Gemini 3.1 Flash Lite has zero success despite zero measured exploitation error without screenshot memory. Its high exploration error points to a discovery bottleneck. Including up to two prior screenshots improves success and lowers exploration error for both evaluated models.
| Model | Screenshot memory | Success ↑ | Exploration error ↓ | Exploitation error ↓ |
|---|---|---|---|---|
| Gemini 3.1 Flash Lite | Without | 0.0% | 0.948 | 0.000 |
| With | 88.9% | 0.807 | 0.000 | |
| GPT-4.1 | Without | 11.1% | 0.877 | N/A |
| With | 77.8% | 0.740 | 0.556 |
GPT-4.1 still incurs exploitation errors with screenshot memory. Improved success does not imply that every failure mode has been resolved.
Depending on the current state, an agent should discover unseen cells, visit known tasks whose prerequisites are satisfied, or return to the ready goal. When discovery and actionable tasks are both available, either strategy is allowed unless the goal is ready; then the agent must return to it.
Entering a previously unvisited cell or activating a ready task counts as progress and is error-free. Otherwise, the metric checks whether a move gets closer to any productive destination. With multiple destinations, it also detects loops and excessive revisits during periods without progress, while permitting necessary backtracking.
Errors count toward exploration, exploitation, or both according to the current state. Each error rate is normalized over the steps where that behavior is relevant, giving a behavioral profile alongside task success.
Explore all 13 sample animations. Each uses exploration=high, task_dag=hard, and seed=2. Select an animation to view it at full size.
Measure exploration and exploitation errors from action trajectories without inspecting hidden policies or prescribing a single reference strategy, with an extension to Minecraft Clone.
Partially observed grid maps and unknown symbolic task DAGs isolate reasoning from semantic priors and allow systematic control of exploration demand, exploitation demand, and task complexity.
Connect both errors to task success, distinguish failure modes hidden by successful outcomes, and quantify how prompts, structured memory, and semantic information change agent behavior.
@article{park2026exploration,
title={Exploration and Exploitation Errors Are Measurable for Language Model Agents},
author={Jaden Park and Jungtaek Kim and Jongwon Jeong and Robert D. Nowak and Kangwook Lee and Yong Jae Lee},
journal={arXiv preprint arXiv:2604.13151},
year={2026}
}