Two machine learning papers posted to arXiv on Nov 6, 2026 propose new model architectures: one focuses on the parameter-reuse mechanism of looped Transformers, the other on long-horizon value expansion for pixel-based offline reinforcement learning. Both sit at the frontier of architecture research — the first targets model efficiency, the second serves embodied-AI training.

Loopback Transformer: separating input strength from update weights

The paper "Loopback Transformer: Towards a Unified View of Residual Looped Transformers" notes that looped Transformers cut parameter count by repeatedly applying shared blocks, but parameter sharing alone does not determine how the results of repeated computation are reused. Taking the depth-memory view of residual connections, the authors analyze which representation is stored after each block application and how later reads weight it.

The work argues that a full block output carries both its input and a new update, so weighting that output also reweights the history already carried by the input. Reading a single update instead, and keeping the shared stack's input on a separate path, allows separate control of input strength, total update weight and the allocation of weight across updates. The Loopback Transformer implements these controls: geometric decay sets the total update weight and supplies a recency prior, while an attention mechanism learns how to allocate that weight.

On ClimbMix, Loopback achieves lower validation loss with 40-75% of the parameters of a standard Transformer baseline (excluding token embeddings and output head), at reference scales from 191M to 3B parameters, with training tokens and execution depth matched.

The paper also reports that at 1.4B and 3B parameters, Loopback has higher average accuracy across 13 downstream tasks; at 191M, 374M and 778M parameters it posts the lowest validation loss, with the highest 13-task average among the standard Transformer and seven looped-residual designs.

MAC-v2: reconstruction-free latent world model plus action chunking

The other paper, "MAC-v2: Long-horizon model-based value expansion from pixels," studies whether long-horizon model-based value expansion can serve as an effective framework for pixel-based offline RL. It notes that long-horizon value expansion from pixels is far harder than in state-based environments, because the agent must learn both the representation and its dynamics, and errors from either can compound recursively along imagined trajectories.

MAC-v2 therefore trains a reconstruction-free latent world model and combines it with action chunking to enable long-horizon model-based expansion from pixels. Because it learns representations from prediction and control objectives, MAC-v2 need not encode control-irrelevant visual detail that could hinder dynamics prediction. Action chunking further eases prediction error by reducing the number of recursive model predictions required for value expansion.

Across several vision benchmarks including OGBench, MMBench2, Robomimic and LIBERO-10, MAC-v2 delivers strong performance against model-free and model-based offline RL baselines. The analysis shows that long-horizon model-based value expansion is critical to MAC-v2's performance, and that action chunking and the latent representation learning objective are both essential for reliable long-horizon latent imagination.

What to watch

Both studies were released as preprints, with no mention of peer review or conference acceptance. Worth tracking: the availability of Loopback Transformer's code and models, MAC-v2's generalization to real robot tasks, and the reproducibility of both across different scales and task distributions.