TorchTitan Update: New Frontier Models, Faster MoE Training and TitanRL for Reinforcement Learning

PyTorch's TorchTitan adds Qwen3.5, Kimi K3 and DeepSeek-V4, an RL stack whose trainer and generator can produce bitwise-identical log-probabilities, and FlexShard, which cut Muon optimizer time by up to 79%.

TorchTitan Update: New Frontier Models, Faster MoE Training and TitanRL for Reinforcement Learning

TorchTitan, the PyTorch team’s native platform for large-scale pre- and post-training, has had a big year. The latest update adds support for a batch of frontier open-weight models, a reinforcement learning stack called TitanRL, and a set of performance features aimed at mixture-of-experts training at scale [1]. For teams choosing a training framework, the headline is that TorchTitan now covers the path from pretraining to RL in one PyTorch-native codebase.

Why it matters

Many RL stacks keep two implementations of the model: one for training and one for inference. Architecture changes have to stay aligned across both, and when a run goes wrong it is hard to tell whether the cause is the reward, the algorithm or the two copies quietly disagreeing [1:1]. TitanRL keeps the same model from pretraining through post-training and inside vLLM [1:2]. Combined with a batch-invariant mode that makes trainer and generator agree exactly, it removes one source of noise from RL experiments.

What happened

New models. TorchTitan added PyTorch-native support for Muse Spark Open, Muse Glimmer, Qwen3-VL, Qwen3.5, Kimi K2.7, Kimi K3 and DeepSeek-V4, as full training-stack integrations across its 5D parallelism stack. A new configuration system built on typed Python recipes lets users change parallelism, model components or hardware-specific kernels without forking the model [1:3].

TitanRL. Rollouts run asynchronously, with training and generation each scaling independently. The team reports stable training for hundreds of steps on MoE and dense models, including Qwen3.5-35B-A3B on SWE-Rebench with 64K context on 40 GB300 GPUs [1:4]. In supported on-policy configurations, batch-invariant mode makes the trainer and generator produce bitwise-identical log-probabilities, so the gap between them is exactly zero at every step [1:5].

Ecosystem. TorchTitan is now a training backend in veRL and Miles, the default backend for Meshy, and the inspiration for the training engine in AReaL [1:6].

How it works: the performance features

  • local_compile applies torch.compile to selected regions rather than the whole graph or block, keeping numerics intact without custom kernels [1:7].
  • torch_remat is a new activation-checkpointing API with fine-grained control over what is recomputed [1:8].
  • Dist-MoE, a newly open-sourced library, moves tokens between experts over symmetric memory and fuses that communication with expert compute on NVIDIA SM100+ GPUs, while staying CUDA-graphable [1:9].
  • FlexShard is a new data-parallel sharding library with placements FSDP2 cannot express. Its Owned and BlockShard placements let the distributed Muon optimizer skip all-gathers, cutting optimizer time by up to 79% on DeepSeek-V3 at 256 H100s. BlockShard also aligns shards with FP8 quantization blocks, so quantized parameters can be gathered without extra collectives, improving step time by up to 20% on Llama 3 405B at 256 H100s [1:10].
  • GraphTrainer offers a graph-mode training path whose numerical results match eager training bitwise [1:11].
  • torch_checkpointing removes per-save synchronization. On DeepSeek-V3 671B at 512 H100s, average trainer blocking per checkpoint fell from 841.2 ms to 34.1 ms [1:12].

At scale, TorchTitan matched Megatron’s loss convergence on the DeepSeek-V3 MLPerf training benchmark, run in MXFP8 on an internal GB300 cluster [1:13]. PyTorch 2.14 also ships a new opt-in NCCL backend, ported from torchcomms and validated on TorchTitan, with in-place process-group reconfiguration for fault tolerance [1:14].

What data and AI engineers should do

  • Use batch-invariant mode when changing RL algorithms or kernels. If rewards differ from your baseline, you can rule out trainer/generator mismatch.
  • Try FlexShard if you use Muon or blockwise FP8. Those are the cases where FSDP2’s sharding boundaries cost extra communication.
  • Adopt torch_checkpointing for frequent checkpoints at scale, where per-save synchronization used to stall training.
  • Start from the repository. Clone pytorch/torchtitan and follow its instructions; the recipes are plain Python and easy to review [1:15].

Limitations and open questions

Most numbers come from Meta’s own clusters, often on GB300 or H100 at hundreds of GPUs, and several features target NVIDIA Blackwell-class hardware, so results on other setups may differ. Bitwise-identical log-probabilities hold only in supported on-policy configurations [1:16]. The post shows RL curves but does not compare TitanRL’s throughput or cost with other RL frameworks.

Key takeaways

  • TorchTitan now supports Qwen3.5, Kimi K3, DeepSeek-V4 and more across its 5D parallelism stack. [1:17]
  • TitanRL keeps one model from pretraining to vLLM, and batch-invariant mode gives bitwise-identical log-probabilities in supported setups. [1:18]
  • FlexShard cut Muon optimizer time by up to 79% on DeepSeek-V3 and step time by up to 20% on Llama 3 405B. [1:19]
  • torch_checkpointing cut average trainer blocking per checkpoint from 841.2 ms to 34.1 ms at 512 GPUs. [1:20]

Sources


  1. PyTorch Blog, “TorchTitan Update: New SOTA Models, Performance at Scale, TitanRL”, 2026-10-22. https://pytorch.org/blog/torchtitan-update-new-sota-models-performance-at-scale-titanrl/ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎

Discussion

Join the discussion: create a free account or sign in to comment.

No comments yet.

Keep reading

Helion and NVIDIA CompileIQ: Autotuning GPU Kernels Down to the Compiler
AI Engineering

Helion and NVIDIA CompileIQ: Autotuning GPU Kernels Down to the Compiler

Kernel autotuners stop at tile sizes and warp counts. Helion can now search ptxas compiler controls in the same loop, using CompileIQ's Advanced Control Files, for 2 to 7% more speed on kernels already tuned.

Nov 19, 2026 · 4 min read
TorchTPU: PyTorch Runs Natively on Google TPUs, and Hugging Face Models Come Along
AI Engineering

TorchTPU: PyTorch Runs Natively on Google TPUs, and Hugging Face Models Come Along

Google and Meta built TorchTPU, a native PyTorch backend for TPUs with eager execution and torch.compile. Hugging Face Transformers, Diffusers and Kernels now treat "tpu" as just another device.

Nov 15, 2026 · 5 min read
Private AI Workflows on One NVIDIA DGX Spark: n8n, vLLM and LiteLLM With No Cloud API
AI Engineering

Private AI Workflows on One NVIDIA DGX Spark: n8n, vLLM and LiteLLM With No Cloud API

n8n served seven models from one 128GB DGX Spark behind a single OpenAI-compatible endpoint. Qwen3.6-35B-A3B ran at 120 tokens per second for one user and 359 in total for eight, but image generation cut chat speed to a third.

Nov 14, 2026 · 5 min read