Helion and NVIDIA CompileIQ: Autotuning GPU Kernels Down to the Compiler

Kernel autotuners stop at tile sizes and warp counts. Helion can now search ptxas compiler controls in the same loop, using CompileIQ's Advanced Control Files, for 2 to 7% more speed on kernels already tuned.

Helion and NVIDIA CompileIQ: Autotuning GPU Kernels Down to the Compiler

Every GPU kernel autotuner, whether Helion’s, Triton’s or the one inside torch.compile, searches the same kinds of knobs: tile shapes, warp counts, pipeline stages and memory access strategy. Below that sits a second set of decisions that nobody searches: how ptxas, NVIDIA’s PTX assembler, allocates registers and schedules instructions. Those are left at their defaults. A joint post from NVIDIA and Meta shows Helion now searching that compiler layer too, using NVIDIA’s CompileIQ, and finding extra speed on kernels that were already tuned [1].

Why it matters

Gains on top of an already-optimized kernel are the hard kind, and the post is careful about its baseline. The test kernels come from Helion’s pretuned_kernels/, about 27 kernels covering FP8 and NVFP4 quantization, RMSNorm, RoPE, grouped GEMM, Gated DeltaNet decode and a fused Qwen3 decode layer. Each ships with ahead-of-time tuned configurations per GPU architecture [1:1]. Squeezing a few percent more out of those, automatically and with a correctness check, is meaningful for kernels that run continuously in inference fleets.

What happened

Helion can now take compiler settings packaged as an Advanced Control File (ACF), a binary artifact applied at compile time that you can version and ship next to a kernel. ptxas exposes these controls starting with CUDA 13.3 [1:2]. There are two ways to use them [1:3]:

  • Pin one ACF as part of a configuration you have already chosen.
  • Search a list of ACFs with autotune_search_acf, as one more dimension alongside tile sizes and warp counts. Helion always adds “no ACF” to the list, so the compiler defaults stay in the race and an ACF is only picked if it measurably wins.

Most users will not generate ACFs themselves. CompileIQ publishes pre-tuned catalogs; the current Helion booster pack holds 24 ACFs tuned for B200 with ptxas from CUDA 13.4 [1:4].

How it works

Why search jointly. Kernel configuration and compiler controls are coupled through the hardware. Block size and pipeline stages set register pressure, and register allocation decides whether that pressure causes spills. Tuning one and then the other finds a local optimum; searching both in one loop is the only order that avoids it [1:5].

How CompileIQ finds ACFs. It treats compiler-control selection as hyperparameter optimization, running an evolutionary search: propose candidate control sets, compile each, check correctness against a reference, time the survivors and breed the winners [1:6].

The results. On an NVFP4 decode GEMV kernel on GB200, with GPU clocks locked and every result checked with a Welch’s t-test and Cohen’s d [1:7]:

  • Optimizing for runtime (8 generations, pool of 15) cut execution time from 0.0971 ms to 0.0928 ms, a 4.7% speedup.
  • Optimizing for achieved occupancy, the kernel’s profiled bottleneck, moved occupancy only from 21.60% to 21.76% but still gave a 1.75 to 2.0% runtime gain.
  • Multi-objective tuning found one ACF that was 6.9% faster and raised L1TEX cache throughput from 81.51% to 85.96%.
  • A single ACF tuned across two input shapes beat the defaults on both, by 1.05x and 1.01x.

The authors set expectations plainly: on highly optimized kernels, compiler-control tuning has shown 2 to 7% gains; less optimized kernels have seen double-digit improvements [1:8].

What data and AI engineers should do

  • Run it offline, then pin. Each ACF multiplies autotuning trials, so search once on kernels that matter and pin the winner with advanced_controls_file [1:9].
  • Start with the booster pack. Download it, point autotune_search_acf at it and autotune; no search space or objective function needed [1:10].
  • Match ACFs to hardware and toolkit. An ACF tuned for one architecture is not valid on another and can produce wrong results, not just slower code. Keep the correctness check in the loop [1:11].
  • Profile first. For end-to-end gains, find the kernels that dominate your model’s time, tune those and commit their ACFs with the code [1:12].

Limitations and open questions

The detailed results come from one kernel family on one GPU, measured by the teams that built the tools. The occupancy experiment shows that a proxy metric can move very little while runtime moves more, so choosing the objective takes judgment. CompileIQ itself ships under a proprietary NVIDIA license, though the ACF files it produces can be freely distributed [1:13]. And the approach is NVIDIA-specific today, even if the authors argue any compiler with tunable controls could be searched the same way.

Key takeaways

  • Helion can now autotune ptxas compiler controls together with kernel configuration, using Advanced Control Files. [1:14]
  • The compiler defaults always stay in the search, so the worst case is spent tuning time, not a slower kernel. [1:15]
  • On an already-tuned NVFP4 GEMV kernel, ACFs delivered 4.7% (runtime objective) and 6.9% (multi-objective) speedups. [1:16]
  • ACFs are tied to a GPU architecture and CUDA version; tune offline and pin the result. [1:17]

Sources


  1. PyTorch Blog, “One Search, Two Layers: Compiler Controls Inside PyTorch Helion using NVIDIA CompileIQ”, 2026-10-19. https://pytorch.org/blog/one-search-two-layers-compiler-controls-inside-pytorch-helion-using-nvidia-compileiq/ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎

Discussion

Join the discussion: create a free account or sign in to comment.

No comments yet.

Keep reading

TorchTitan Update: New Frontier Models, Faster MoE Training and TitanRL for Reinforcement Learning
AI Engineering

TorchTitan Update: New Frontier Models, Faster MoE Training and TitanRL for Reinforcement Learning

PyTorch's TorchTitan adds Qwen3.5, Kimi K3 and DeepSeek-V4, an RL stack whose trainer and generator can produce bitwise-identical log-probabilities, and FlexShard, which cut Muon optimizer time by up to 79%.

Nov 18, 2026 · 4 min read
TorchTPU: PyTorch Runs Natively on Google TPUs, and Hugging Face Models Come Along
AI Engineering

TorchTPU: PyTorch Runs Natively on Google TPUs, and Hugging Face Models Come Along

Google and Meta built TorchTPU, a native PyTorch backend for TPUs with eager execution and torch.compile. Hugging Face Transformers, Diffusers and Kernels now treat "tpu" as just another device.

Nov 15, 2026 · 5 min read
Private AI Workflows on One NVIDIA DGX Spark: n8n, vLLM and LiteLLM With No Cloud API
AI Engineering

Private AI Workflows on One NVIDIA DGX Spark: n8n, vLLM and LiteLLM With No Cloud API

n8n served seven models from one 128GB DGX Spark behind a single OpenAI-compatible endpoint. Qwen3.6-35B-A3B ran at 120 tokens per second for one user and 359 in total for eight, but image generation cut chat speed to a third.

Nov 14, 2026 · 5 min read