By moving reasoning from discrete token space into continuous latent space, tiny recursive models can outperform trillion-parameter transformers on problems like the ARC prize.
The Limits of Feedforward Reasoning
Large language models like GPT-2 achieved remarkable results by simply predicting the next token. But as the field demands genuine reasoning, the feedforward architecture reveals a fundamental limitation: a transformer can only perform as many computational steps as it has layers. For an incompressible problem like sorting a list of 31 items, a 30-layer model literally cannot complete all necessary comparisons in one forward pass. Chain-of-thought and tool use are workarounds that happen in discrete token space—but they are bounded by human knowledge. A model trained only on bubble sort traces will never discover merge sort on its own.
This is because transformers, for all their parallelism and training efficiency, have no latent reasoning mechanism. Every decode forces the model to retain the entire input history, and the only way to add more computation is to increase the number of layers or let the model generate more tokens. Neither approach discovers novel algorithms; they merely interpolate within the training distribution.
The Return of Recursion: HRM and TRM
Recurrent neural networks (RNNs) were designed to compress temporal information into a hidden state, allowing iterative reasoning without unbounded memory. But they suffered from vanishing or exploding gradients during training, which limited their depth. Two 2025 papers revive the recursive paradigm—this time with a clever training trick that sidesteps the old problems.
The Hierarchical Reasoning Model (HRM) uses three nested loops: a low-level recursion that updates a local hidden state, a high-level recursion that conditions on the low-level state, and an outer refinement loop that repeats the entire process multiple times. Crucially, during training the model does not backpropagate through all recursive steps. Instead, it uses truncated backpropagation (stop-gradient after one step) and treats each outer refinement as a new sample in a mini-batch over latent states. This pseudo-fixed-point iteration avoids gradient noise and makes training feasible.
With only 27 million parameters and trained on just 1,000 ARC Prize puzzles, HRM achieved state-of-the-art results—beating models like o3 that had zero correct solutions. The key insight was not the hierarchical structure itself, but the outer refinement loop, which scales the effective number of reasoning steps.
TRM: Simpler, Smaller, Stronger
Where HRM is complex, the Tiny Recursive Model (TRM) paper asks what can be stripped away. The answer: almost everything. TRM collapses the two separate low-level and high-level networks into a single weight-shared transformer layer. It keeps the outer refinement loop and the truncated backprop, but backpropagates through one complete recursion step instead of just one call. The result is a 7-million-parameter model that raises ARC Prize accuracy from 70% to 87%.
Perhaps most surprising is that on some tasks, like Sudoku, a simple feedforward MLP (without attention) matched the performance of a transformer within the recursive framework. On mazes, attention was still necessary. This suggests that recursion can compensate for architectural simplicity, and that the real power comes from the iterative, expectation-maximization style update of the latent state.
The Magic of Outer Refinement and Latent Space Reasoning
Both HRM and TRM treat reasoning as an optimization process. The model iteratively updates two latent states—call them ZL (local) and ZH (high-level)—conditioned on the input and each other. This is reminiscent of the expectation-maximization algorithm: fix one, update the other, then repeat. The final ZH is one linear transformation away from the answer.
For a problem like Sudoku, where each cell depends on others and the puzzle cannot be solved in one pass, this iterative refinement is natural. The model learns to hold a partial solution in ZL, propose a completion to ZH, then refine the local state with the new information. Because the updates stay in continuous latent space, they are far more expressive than token-by-token reasoning. The outer refinement loop at training time (repeating the process 16 times, each time with a different latent state but the same input) acts like a mini-batch over possible memories, providing diverse gradients without needing new data.
What This Means for the Future of AI
The combination of recursion and large-scale pre-training is the next frontier. Current LLMs excel at learning rich, semantically separated representations—the embedding space. But they do little actual reasoning inside that space. A tiny recursive model could inhabit that embedding space and perform iterative, latent reasoning on top of the pre-trained representations. The model would not need to generate tokens to think; it would refine a hidden state that directly interacts with the embedding.
This is not just an efficiency gain. It offers a path to discovering novel algorithms beyond human knowledge—something chain-of-thought cannot do. The recursive framework allows a model to learn iterative strategies from scratch, without being hand-holded through every step. As the field moves beyond scaling parameters, recursion may become the next necessary ingredient for general intelligence.