Metal Performance Shaders are supported in PyTorch 2.15

Meta description: PyTorch 2.15 adds FP8 on Apple silicon Macs, faster LLM decoding, lower video memory, and MPS improvements for local model builders on Macs.

Tags: PyTorch 2.15, Apple silicon, MPS, FP8, LLM decoding

Featured image: abstract friendly illustration style with no real people, no logos, and no text.

PyTorch 2.15 is aimed at people who run models on Apple silicon Macs with transformers, diffusers, or tools such as ComfyUI. It targets three practical problems: memory use, token generation speed, and operations that previously did not work reliably on the Mac graphics path. The PyTorch team at Hugging Face says that at the start of 2026, even small models on the MPS backend could require workarounds because of unsupported operations, sharp slowdowns, and memory use that grew unexpectedly. Their work tests real models from the Hugging Face Hub across language, image, and video tasks, then sends fixes upstream to PyTorch, often moving work from MPSGraph to purpose-built Metal kernels.

What changes first for Mac users

The release requires macOS 15 or later and Python 3.11 or later. If the operating system is too old or no Metal GPU is available, the MPS availability check now reports the reason. This matters for local workflows that previously fell back to CPU execution or failed when an operation was not supported on Mac.

LLM decoding gets faster and smaller

FP8 support arrives on MPS through torch.float8_e4m3fn tensors. Each value uses one byte, half the storage of FP16 or BF16. Multiplication uses F.scaled_mm, which supports tensorwise and rowwise scaling. This gives Mac users a way to keep large weights in memory while still using GPU matrix multiplication, and it makes more operations behave like their CUDA and CPU counterparts.

In token generation, the decode step processes one new token at a time, so matrix-heavy layers behave like matrix-vector products. Although matrix-vector kernels for torch.mm were added in 2.14, F.linear still used the MPSGraph path. In 2.15, F.linear can use those kernels for the common [1, 1, hidden_size] decoding input, and eligible bias additions are fused. Across 96 BF16 layer shapes from real models, the measured geometric-mean speedup is 1.23x. The gain is largest for small projections such as mixture-of-experts gates; large layers and language-model heads were already close to bandwidth-bound and stay near 1x.

Sampling also gets faster. softmax and log_softmax use dedicated Metal kernels. An FP32 softmax over a [1, 128256] row of logits drops from 57 microseconds to 7 microseconds, about 8x faster than the MPSGraph version. The release notes similar gains across other common vocabulary sizes.

Mixture-of-experts models route different token counts to different experts. Grouped matrix multiplication handles those uneven groups in one operation, and 2.15 gives MPS a native implementation. In 2.14, grouped matmul copied group offsets to the CPU and looped over groups. New Metal kernels read offsets directly on the GPU, using Metal Performance Primitives where available and a separate Metal path elsewhere; small groups, common during decoding, get their own kernel. The release attributes the gain to native grouped matmul, fewer GPU-CPU syncs, and faster top-k routing.

The release summary says MoE models decode 5-6.5x faster than in 2.14. On an M3 Ultra with torch.compile, Qwen3.6-35B-A3B goes from 9.4 to 61.3 tokens/s, and LFM2.5-8B-A1B goes from 23.6 to 123.6 tokens/s. A separate MoE throughput note reports about 1.3x more tokens per second for the tested M3 Ultra torch.compile path.

Linear algebra and training operations move to Metal

Complex64 tensors can now use LU factorization, LU solve, and triangular solve on MPS. That also unlocks dependent operations and gradients, including complex determinants. The release includes a small system solved directly on the GPU.

Small FP32 matrix batches up to 4x4 are inverted in one Metal kernel that combines factorization and solving, useful for camera geometry and image transformations. QR decomposition uses blocked updates, and triangular solves turn much of their work into efficient matrix multiplication. Gated DeltaNet workloads get a dedicated fast path for batches of 16x16, 32x32, and 64x64 FP32 triangular systems with at least eight right-hand sides.

SVD is more robust at numerical extremes. It handles very small singular values, very large or very small input magnitudes, and basis completion with full_matrices=True, making pseudoinverse and least-squares results more reliable. Audio models, video decoders, and training loops also get new Metal paths for convolutions, padding, dropout, interpolation backward, and reductions.

Video memory drops, image editing becomes correct

One of the clearest practical changes is memory. Wan-AI/Wan2.2-TI2V-5B-Diffusers makes 8,764 padding calls during a full 28-tile decode. With padding on Metal, the highest reported driver memory use falls from about 270 GiB to 8 GiB.

The same padding change fixes a correctness issue. On 2.14, the Qwen-Image-2.1 VAE encoder silently corrupted images on MPS, breaking image-edit workflows reported in ComfyUI. After the fix, the same encode-decode test now matches the CPU result at 29.95 dB PSNR, compared with 10.06 dB before.

More control and broader operation coverage

PyTorch 2.15 exposes public MPS streams and more reliable event timing. torch.mps.Stream and the torch.mps.stream context manager let Python code choose the stream, and Tensor.record_stream lets the allocator track a tensor used on another stream.

More operations now work on MPS: torch.frexp, torch.logspace, torch.take and Tensor.put_, torch.heaviside, torch.special.laguerre_polynomial_l, and unsigned-integer comparisons. torch.logspace does not yet support integer or boolean outputs with more than one element.

The MPS math implementation of scaled dot-product attention now handles batch and head broadcasting correctly.

What PyTorch 2.16 may add

Work on 2.16 is underway. Some changes are on main and in nightlies; others are still in review, so details may change before release.

For compiled BF16 decoding at a 2048-token context, 2.16 plans to commit the command buffer every 16 kernels. Once a decode path no longer contains MPSGraph operations, this lets the GPU start work while the CPU is still encoding later kernels.

Scaled dot-product attention is expected to move entirely to Metal. The planned path uses dedicated vector kernels for decoding and Metal Performance Primitives for half-precision prefill on newer macOS versions. Shapes that previously fell back to MPSGraph should benefit most. On an M5 Pro, grouped-query attention decoding with 32 query heads, 8 KV heads, head dimension 80, and a 4K context drops from 465 microseconds to 65 microseconds. A 4Kx4K prefill with head dimension 40 goes from 42 milliseconds to 6 milliseconds.

Batch norm forward and backward are in progress. MSE loss is reported up to 12x faster on small inputs, std_mean and var_mean up to 4x faster, and bincount has landed. In one bincount case, when most values fall into a single bin, counting 4 million integers used to take 6.8 seconds and now takes half a millisecond.

Bundled shaders will build for Metal 3.1 by default, with Metal 4 versions compiled only for kernels that need them. That shrinks libtorch_cpu.dylib by about 25%, from 384 MiB to 287 MiB, and prepares for Metal 4.1 shaders with faster matrix multiplication and native FP8 matrix multiplication.

Who should use it and who should skip it

Use PyTorch 2.15 if you have an Apple silicon Mac running macOS 15 or later, Python 3.11 or later, and you want lower-memory FP8 tensors, faster dense or mixture-of-experts decoding, corrected Qwen-Image-2.1 editing, or much lower peak memory for Wan2.2 video decoding. It can be tried in a separate environment as a release candidate. Skip it for now if your Mac cannot meet the operating system or Python requirements, or if no Metal GPU is available.

The team says the next focus is keeping attention and more operations on Metal, reducing graph breaks in torch.compile, and accepting feedback on GitHub for slow, missing, or broken behavior. It credits Nikita Shulga, Kurt Mohler, and Aaron Gokaslan for reviewing much of the work, along with everyone who contributed to MPS in this release.