Adaptive Reasoning Has Measurable Headroom in Small Open-Weight Models

A 200-Problem Feasibility Study of Instance-Level Test-Time Compute Allocation

Aloysius Faustin Manohar — Draft preprint, September 2026

---

Abstract

Reasoning language models are usually run at a fixed inference budget, or one picked by hand, even though extra reasoning is worth far more on some problems than on others. We ask a narrow version of the question that follows from this: does a small open-weight reasoning model vary enough from instance to instance that an adaptive allocator could improve the accuracy–compute frontier at all?

Our test bed is DeepSeek-R1-Distill-Qwen-1.5B on 200 unique mathematical problems — 100 from GSM8K and 100 from MATH-500 — evaluated under fixed budgets, three-sample self-consistency, disagreement-triggered resolvers, and an untrained STOP/RETRY controller. None of the hand-designed adaptive methods beat the strongest baseline.

What they did expose is a clean split: two independent short solutions agreed on exactly half the set, and where they agreed final accuracy was 81%, against 41% with a 1024-token resolver and 48% with a 2048-token resolver where they did not (Fisher exact p = 8.94×10⁻⁹ and 1.66×10⁻⁶).

A post-hoc cheapest-correct oracle over the same action set reaches 76.0% accuracy at 747.7 parameter-weighted total tokens, where three-sample self-consistency reaches 68.5% at 2188.7: 7.5 percentage points more accuracy for 65.8% less normalized compute.

None of this amounts to a deployable controller. It does show that the allocation headroom is large, that a cheap difficulty signal exists, and that learning a policy over the marginal value of further reasoning is worth attempting.

Keywords: adaptive reasoning, test-time compute, reasoning language models, compute allocation, self-consistency, uncertainty, open-weight models

---

1. Introduction

Large reasoning models earn their gains by generating intermediate reasoning before committing to an answer, but not all of that reasoning pays for itself. Easy questions get over-processed; hard ones stay unsolved under the same budget. The practical question is one of allocation: with a finite inference budget, which prompts deserve more reasoning, and when should the model stop?

Learned budget prediction, hybrid think/no-think policies, difficulty-aware reinforcement learning, and constrained allocation have all been proposed [2–8], so adaptive reasoning is an active line of work rather than an unexplored one. Our question is narrower and mostly empirical. Does a small open-weight reasoning model expose cheap signals that predict whether extra inference will help, and is the gap between uniform allocation and ideal per-instance allocation wide enough to justify training a controller?

We test this with a 1.5B DeepSeek-R1 distilled reasoner and a 0.5B Qwen controller, and we report the controllers that failed alongside the ones that worked. A feasibility claim can mean three quite different things: that heterogeneous compute needs exist, that some measurable signal tracks them, or that a learned policy can act on that signal. We find evidence for the first two. We make no claim about the third.

Three findings drive the paper. Agreement between two short samples marks out a substantially easier subset of problems. The untrained STOP/RETRY controller collapses to RETRY and buys nothing. And a post-hoc cheapest-correct oracle sits well above every hand-designed policy we tried. The implication is that the next move is a trained allocator, not more prompt tuning.

Contributions. What follows is a transparent feasibility evaluation, not a new state-of-the-art method:

• A same-backbone comparison, across 200 problems, of fixed budgets, self-consistency, disagreement-triggered resolution, and a lightweight controller.
• A parameter-weighted total-token metric that charges prompt as well as generated tokens and scales controller cost by parameter count.
• Evidence that agreement between two short samples is a strong difficulty signal on both GSM8K and MATH-500.
• A post-hoc allocation oracle that puts a number on the headroom left for a learned controller.
• A negative result: a prompt-only 0.5B controller is not enough here, which narrows the next step to training rather than further hand-designed routing.

---

2. Related Work

Chain-of-thought prompting and self-consistency showed that spending more inference — whether on intermediate steps or on multiple sampled paths — improves reasoning [10, 11]. DeepSeek-R1 later showed that this behavior distills into much smaller open-weight models, including the 1.5B Qwen-based model we use [1].

Work on adaptive reasoning has since concentrated on avoiding uniform compute. SelfBudgeter predicts a query-specific reasoning budget before generation [2]. AutoThink and hybrid-reasoning methods learn whether explicit thinking is needed at all [3, 4]. Plan-and-Budget schedules compute across decomposed subproblems in a model-agnostic way [5]. CODA casts allocation as marginal accuracy gain against incremental token cost [6]. Constrained-policy work formalizes allocation under a global budget and trains lightweight classifiers to imitate oracle actions [7]. Learning When to Think trains the model to pick among NoThink, Short, and Long as its first response action [8].

We are not offering a competitor to these methods. The point here is to measure whether a small local, open-weight stack shows enough action-dependent variation to make a learned allocator worth the effort, and to find a cheap signal that could supervise one.

---

3. Experimental Design

3.1 Research Questions

• RQ1: Is there measurable headroom for per-instance test-time compute allocation over uniform allocation on a small open-weight reasoning model?
• RQ2: Does agreement between two short independent solutions work as a cheap indicator of problem difficulty or answer reliability?
• RQ3: Can a prompt-only lightweight controller exploit that headroom without training?

3.2 Models and Benchmarks

The reasoner was deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B [1] and the controller Qwen/Qwen2.5-0.5B-Instruct, both run in FP16 through Hugging Face Transformers. We sampled 100 unique GSM8K problems [9] and 100 unique MATH-500 problems drawn from MATH [10].

The final exploratory validation used a single inference seed (11); independent samples within a method came from deterministic seed offsets. The evaluation set is therefore 200 distinct questions, not repeated seed replicates.

The reasoner generated at temperature 0.6 with top-p 0.95. The controller ran at temperature 0 for STOP/RETRY classification. Correctness came from symbolic and numeric grading in the benchmark harness.

3.3 Compared Inference Policies

3.4 Compute Metric

Raw token counts hide controller overhead, so we report parameter-weighted total tokens as a normalized proxy for inference work. With R the sum of the reasoner's prompt and generated tokens, C the same sum for the controller, Pᵣ = 1.5B the reasoner parameter count and Pᶜ = 0.5B the controller parameter count:

WeightedTotal = R + C × (Pᶜ / Pᵣ) = R + C / 3

This is an approximation, not a FLOP or energy measurement. It stops controller work from being counted as free, but any hardware-specific conclusion should also report wall-clock latency, memory, and energy.

3.5 Statistical Analysis

The main comparison is accuracy against mean parameter-weighted total tokens. To ask whether pair agreement stratifies difficulty we run two-sided Fisher exact tests on the 2×2 tables of signal (AGREE/DISAGREE) against final correctness, for both pair-resolver policies. We treat the post-hoc oracle strictly as an upper bound and do not test it, since it picks actions using outcome information not available at deployment time.

---

4. Results

4.1 Overall Accuracy–Compute Trade-off

The best deployable baseline was SC3, at 68.5% accuracy for 2188.7 weighted total tokens per problem. Fixed-1024 got 63.5% at 720.0 tokens. PairResolve-2048 edged past Fixed-1024 on accuracy (64.5% against 63.5%) but spent nearly 2.74× the normalized compute to do it. The prompt-only STOP/RETRY controller managed only 61.0% at 1683.7 tokens. No hand-designed adaptive method improved the Pareto frontier.

The oracle looks quite different. It reaches 76.0% accuracy at 747.7 weighted total tokens: 7.5 percentage points above SC3 with 65.8% less normalized compute, and 12.5 points above Fixed-1024 for 3.8% more compute. Since it selects actions with hindsight, this is not a performance claim — it is a measurement of how large the allocation opportunity is.

Figure 1. Accuracy against parameter-weighted total tokens. The star marks a post-hoc upper bound, not a deployable policy.

4.2 Agreement as a Difficulty Signal

The two 512-budget initial solutions agreed on exactly 100 of the 200 problems and disagreed on the other 100. That split predicts downstream success sharply.

Under PairResolve-1024:

• AGREE subset: 81% correct
• DISAGREE subset: 41% correct
• Odds ratio: 6.13 (Fisher exact p = 8.94×10⁻⁹)

Giving the disagreement cases a 2048-token resolver lifted them only to 48%, for an odds ratio of 4.62 (p = 1.66×10⁻⁶).

The pattern holds across both benchmarks, though not equally. On GSM8K, 61 of 100 problems produced initial agreement and that subset reached 93.4%, while the 39 disagreement cases reached 61.5% with the 2048 resolver. On MATH-500 only 39 agreed, reaching 61.5%, and the 61 disagreement cases reached 39.3% after the 2048 resolver.

Agreement is no guarantee of correctness, but disagreement reliably picks out a harder set that stays hard even with more compute.

Figure 2. Accuracy split by the initial pair-agreement signal. Disagreement marks a much harder subset, even with extra resolver compute.

4.3 Dataset-Level Results

The signal is stronger on GSM8K than on MATH-500, and absolute accuracy is much lower on MATH-500. One reading is that a binary retry decision is too blunt for harder mathematics. A learned policy may need several actions (STOP, short continue, long continue, verify) rather than a single binary branch.

4.4 Failure of the Prompt-Only Controller

The 0.5B STOP/RETRY controller did not learn a policy. It emitted or fell back to RETRY on all 200 evaluation questions. Parse compliance was 67% overall (62% on GSM8K, 72% on MATH-500), so in practice the policy amounted to unconditional extra computation.

The failure is informative: the allocation signal needs to be learned from labeled state/action examples rather than coaxed out of a generic instruction prompt.

---

5. Discussion

5.1 What the Data Establish

Within limits, the experiment supports feasibility. Uniform allocation is clearly suboptimal on this sample, since different methods solve different questions and the resulting oracle gap is wide. A cheap observable — whether two short answers agree — partitions the set by difficulty. And the heuristic controllers fail, which says that bolting on a router does not by itself capture the headroom.

The signal and the oracle matter mostly in combination. An oracle gap with no predictive signal behind it would be an opportunity nobody could reach; a signal with no gap would not be worth acting on. Both are present here, which is what makes training a policy a reasonable next experiment rather than a guess.

5.2 What the Data Do Not Establish

The study does not show that a trained controller would reach the oracle frontier, that the signal survives outside mathematics, or that our token proxy maps onto energy or dollars. Nor does it claim adaptive reasoning as a novel idea — plenty of prior work already trains or schedules adaptive budgets [2–8].

The claim we can defend is narrow: on this 1.5B open-weight reasoner and these 200 problems, inference needs are heterogeneous and the post-hoc allocation headroom is large.

5.3 Implications for a Learned Controller

The obvious next step is turning inference trajectories into supervised state/action examples. Given a state containing the problem, a compact candidate answer, and perhaps a short uncertainty summary, one can score the realized utility of stopping against continuing. Labels should come from measured outcomes under alternative actions, not from another model's opinion about difficulty.

A first controller can train on binary STOP/RETRY targets; later versions can add compute levels and optimize quality minus λ times normalized inference cost.

The more useful target is the marginal value of compute: will another block of reasoning improve the answer enough to pay for itself? That shifts the question from "how hard is this prompt?" to "is more compute worth it from where we are now?", which permits escalation or early stopping during inference rather than a single routing decision made up front.

---

6. Limitations and Threats to Validity

• Only 200 unique problems, split evenly between GSM8K and MATH-500. Enough for a feasibility study, not for general claims.
• One reasoner architecture and size, and one controller model, in the final validation.
• One inference seed in the final validation. Publication-quality variance estimates need more.
• The cheapest-correct oracle is post hoc and optimistic by construction. It measures headroom, not achievable deployment performance.
• Parameter-weighted total tokens are a proxy. FLOPs, energy, GPU utilization, KV-cache memory, and monetary cost were not measured.
• The agreement analysis uses final correctness within each routing policy. A follow-up should label the correctness and calibration of each initial candidate directly, before resolution.
• Controller parse failures were treated as RETRY. Conservative, but inflates the controller's compute and may understate its efficiency.
• Every task here is mathematical. Coding, planning, tool use, retrieval, and general knowledge would all need covering before calling anything a general-purpose mechanism.

---

7. Conclusion

We set out to ask whether a small open-weight reasoning model varies enough across instances to make adaptive test-time compute allocation worth learning. In the limited sense tested here, it does.

Agreement after short reasoning separates easier problems from harder ones, and a post-hoc allocation oracle reaches 76.0% accuracy at 747.7 weighted total tokens against 68.5% at 2188.7 for three-sample self-consistency. The hand-designed STOP/RETRY controller captures none of it.

That points to a specific next step: learn the allocation policy from outcome-derived state/action labels, then check whether the trained controller moves toward the oracle frontier at matched compute.

The hypothesis worth testing next is that reasoning effort is a dynamic resource-allocation problem. Start cheap, watch for signs of uncertainty or disagreement, and keep going only while the expected marginal value of more reasoning exceeds what it costs.

---

8. Reproducibility and Data Availability

All compared methods ran through the same Stage 3 benchmark harness. The final raw file holds 1,200 rows (200 questions × 6 deployable methods) with exact prompts and predictions, method metadata, token counts, latency, backend identifiers, and correctness.

Question selection used seed 20260905 for GSM8K and 20260906 for MATH-500; inference seed was 11. Parameter normalization was 1.5B/0.5B. Benchmark code, the exact package lockfile, run logs, raw_results.csv, and the derived analysis scripts will be posted to a permanent repository before public release.

---

References

1. DeepSeek-AI. "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning." arXiv:2501.12948, 2025.
2. Z. Li et al. "SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoning." arXiv:2505.11274, 2025.
3. S. Tu et al. "Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL (AutoThink)." arXiv:2505.10832, 2025.
4. L. Jiang et al. "Think Only When You Need with Large Hybrid-Reasoning Models." arXiv:2505.14631, 2025.
5. J. Lin et al. "Plan and Budget: Effective and Efficient Test-Time Scaling on Large Language Model Reasoning." arXiv:2505.16122, 2025.
6. S. Wu et al. "CODA: Difficulty-Aware Compute Allocation for Adaptive Reasoning." arXiv:2603.08659, 2026.
7. Z. Zhai et al. "Adaptive Test-Time Compute Allocation for Reasoning LLMs via Constrained Policy Optimization." arXiv:2604.14853, 2026.
8. G. Kassenaar, Z. Yang, and V. François-Lavet. "Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation." arXiv:2608.20256, 2026.
9. K. Cobbe et al. "Training Verifiers to Solve Math Word Problems." arXiv:2110.14168, 2021.
10. D. Hendrycks et al. "Measuring Mathematical Problem Solving With the MATH Dataset." NeurIPS, 2021. arXiv:2103.03874.
11. X. Wang et al. "Self-Consistency Improves Chain of Thought Reasoning in Language Models." ICLR, 2023. arXiv:2203.11171.

---

Appendix A. Exact Numerical Results

Post-hoc oracle: accuracy = 0.760; mean weighted total tokens = 747.670; mean weighted generated tokens = 626.170.