Boule or Baguette?
November 16, 2026Reasoning traces are central to how modern AI models solve hard problems, but it remains unclear when they actually help. On a new dataset of over 23 million logic statements, we find that reasoning traces help when a task contains many varied problems that each take only a few steps, but hurt when a task has little variety and each problem requires a long chain of steps. On these long-chain tasks, contrary to common intuition, models that answer directly outperform reasoning models by as much as 50 percentage points. Our theory suggests why: reasoning traces help models generalize across diverse problems, but long traces dilute the information needed to reach the right answer.
AI has quickly become a ubiquitous technology, not only changing the way we search for information, but also automating many of these things we used to think through or reason about on our own. AI-driven reasoning is typically powered by reasoning traces: extra tokens that the model spends to improve its answer, analogous to how a human might “think” for longer on a difficult problem. In recent work, we study the benefits of reasoning traces, and whether they always help AI models perform better. Surprisingly, we discover a category of tasks in which additionally “thinking” actually hurts. Due to technical limitations of AI models, thinking for a long time can sometimes interfere with their ability to solve a problem. We study these cases in detail on a novel propositional logic benchmark that we construct for this purpose. We also develop a theory of the phenomenon in a parallel task based on syllogistic reasoning. Altogether, our results present a nuanced perspective on when extra “thinking” helps and when it hurts for AI.
1. Let’s think step-by-step
The last several years have witnessed a paradigm shift in how we prompt AI models. In the past, users would simply pose a question to a model, and the model would compose a response directly. Here is an example from an early study of reasoning prompts [1]:
Q: A performer holds sixteen balls. Golf balls make up half; of those, half are blue. Count the blue ones.
A: 8. (Incorrect.)
However, researchers noticed that better performance could be induced simply by asking the model to “think step-by-step.” Rather than produce an answer instantly, the model would instead output a reasoning trace (also called a chain-of-thought) that can lead it to the correct answer.
Q: A performer holds sixteen balls. Golf balls make up half; of those, half are blue. Count the blue ones. Let’s think step by step.
A: Golf balls: 16 / 2 = 8.
Blue ones: 8 / 2 = 4.
Answer: 4. (Correct.)
Interestingly, a simple change to the input prompt could substantially improve downstream performance. By allowing the model to “think” [2] using intermediate tokens, the model tends to find the correct answer more frequently. This discovery has led to a broad and rapidly growing collection of methods aimed at getting models to generate more and better reasoning traces.
2. More thinking = more success?
At the same time, it’s not always clear how — or even if — reasoning traces always help. Recent findings have shown that models sometimes largely ignore their stated reasoning when producing an answer, even when mistakes are inserted into the trace [3], and moreover models trained against a reasoning-trace monitor can learn to hide evidence of reward hacking while continuing to exploit the task [4]. Reasoning traces in general aren’t always interpretable. Further studies have found that prolonged reasoning traces can actually hurt model performance [5]. While more thinking would intuitively help the model perform better, prolonged intermediate token generation sometimes does just the opposite. Hence, we need a better framework for thinking about when exactly reasoning traces help, and when they may hurt.
3. Thinking logically with PITA
In our recent work, we present a new task to study more deeply the strengths and weaknesses of reasoning traces: the Propositional Inference TAsk (PITA). Our task is based on propositional logic, the formal process of reasoning about truth values. In this way, PITA captures a very pure and formal sense of reasoning, removing the messy confounds of natural language.
Each example in PITA is a statement in propositional logic, written as a series of symbols that represent truth (⊤), falsity (⊥), and propositional variables (p, q, r, …) that can take one of these two values. Symbols are further modified by logical connectives like “and” (∧), “or” (∨), “not” (¬), and “implies” (→). A full statement might look like p → (q ∨ ¬r). Some statements in propositional logic are tautologically true, which means the statement is true regardless of the truth assignments given to the underlying propositional variables [6]. Tautologies capture forms of inference that preserve truth, making propositional logic a foundation for mathematical proof.
The goal of an AI model in PITA is to predict whether a statement in propositional logic is tautologically true. (Note, a “false” label means that a statement is not a tautology. It need not be false under every assignment.) We compare models that are permitted to generate intermediate reasoning traces (“RT models”) against models that are not permitted intermediate generation, and must output a direct prediction (“DP models”). We fine-tune pretrained models on a subset of propositional logic statements, then evaluate length generalization: training uses statements whose proofs are at or below the median proof length in each subset, while testing uses statements with longer proofs. We measure the accuracy of the final classification, rather than requiring the
generated proof to be valid.
To construct the training traces for RT models, we record a proof-search procedure in Lean, a computer language for expressing and verifying math. A trace alternates between proof states and tactics, the commands used to transform them, including backtracking when needed. Figure 1 illustrates an example trace.

Reasoning trace models are trained to produce the intermediate proof before responding with a decision about whether the statement is (tautologically) true or false. Direct prediction models must output their decision immediately. Figure 2 compares the two input and completion formats.

We then compare the overall performance of RT against DP models on different subsets of propositional statements.
4. A boule or a baguette?
Our key finding is that reasoning trace models do outperform direct prediction models on longer-proof test examples, but the advantage depends on the subset of propositional statements. On some subsets, reasoning traces actually hurt.
The subsets of propositional logic that we study are defined by families of statements that share some characteristic. The specific subsets are:
- Full: statements with no restriction on the logical connectives
- Imply: statements that use only the implication operator (→)
- Or: statements of the form p → (q1 ∨ · · · ∨ qn), asking whether p appears among the variables on the right
- PHP: statements that are variations of the Pigeonhole Principle, expressed in propositional logic
For more details on why we picked these exact subsets, and on their more precise definitions, please see our paper.
For each subset, we can measure its depth and its breadth. We define depth as the average number of steps it takes to prove a statement in the subset: deeper subsets require more steps to prove. Figure 3 compares how the depth of each split grows with statement size.

We define breadth as the number of statements in each subset, while fixing the number of symbols. Broader subsets grow faster as we increase the size of the statement, compared with narrower subsets. Figure 4 compares how the breadth of each split grows with statement size.

Note that both Full and Imply are relatively broad but shallow. We call these “boule”-shaped subsets, because they are broad and shallow like a boule bread loaf. On the other hand, Or and PHP are narrower but deeper. We call these “baguette”-shaped subsets, because they’re narrow and long like a baguette loaf.
Figure 5 shows how this quality of being boule-like (broad and shallow) or baguette-like (narrow and deep) predicts the generalization advantage in our experiments: RT models tend to do better on boule-like tasks, while DP models tend to do better on baguette-like tasks.

One possible explanation for this outcome is the following. Deep tasks require longer reasoning traces. As we mentioned above, longer reasoning traces have been observed to hurt model performance, so reasoning models may struggle on these examples. On the other hand, direct prediction models tend to struggle on broader tasks, where the greater diversity of examples can make it harder to learn an immediate answer without an intermediate reasoning trace. Narrow and deep baguette-shaped subsets can therefore favor DP models, while broad and shallow boule-shaped subsets can favor RT models.
5. Final thoughts
In this work, we’ve explored situations in which reasoning traces can hurt or help. We study propositional logic in particular, connecting the benefit or harm of reasoning traces to the shapes of the underlying task subsets: reasoning traces help on boule-shaped subsets that are broad and shallow; reasoning traces hurt on baguette-shaped subsets that are narrow and deep.
There are many more details in our full paper, including a theory developed in a parallel setting based on syllogisms, that explores another case in which the same tradeoffs with reasoning traces are evident. We hope our discussion adds more nuance when considering the benefit of reasoning traces, and look forward to future work further exploring their strengths and weaknesses.
To learn more, find the code and paper links below.
- Adapted from Figure 1(c–d) of Kojima et al. (2022) [↩]
- It’s tempting to draw a direct analogy between human thinking and model reasoning traces. Many commercial model providers even call reasoning traces “thinking tokens.” However, a trace need not faithfully describe how a model arrives at its answer: for instance Turpin et al. (2023) show that models can rationalize answers influenced by biases they do not mention. A more precise way of thinking about reasoning traces is simply as intermediate generation that the model is permitted to use when producing its final answer.[↩]
- See Lanham et al. (2023), Measuring Faithfulness in Chain-of-Thought Reasoning. Relatedly, Chen et al. (2025) find that reasoning models often use hints without acknowledging them in their traces.[↩]
- Baker et al. (2025), Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation demonstrate this behavior under strong optimization against a monitor.[↩]
- See Wu et al. (2025), When More is Less: Understanding Chain-of-Thought Length in LLMs, and Hassid et al. (2025), Don’t Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoning[↩]
- We won’t be going into any technical detail on propositional logic in this blog post. For interested readers, we recommend the propositional logic chapter of Logic and Proof, a free introduction by Jeremy Avigad and Robert Y. Lewis with Floris van Doorn.[↩]