Meta introduces Muse Spark Open, an open-weight frontier multimodal mixture-of-experts model.
Meta releases Muse Spark Open, an open-weight multimodal model with expert routing for coding, tools, one million context, and local inference.
Key Points Summary
- Meta introduced Muse Spark Open, an 816B-parameter open-weight multimodal mixture-of-experts model optimized for agentic coding.
- The model supports images, audio, video, tool use, and a 1M-token context, allowing it to process screenshots, voice notes, and screen recordings while working on code.
- Its mixture-of-experts design uses 256 experts with 8 active per token, delivering faster inference at roughly 42B-parameter compute while requiring memory for the full 816B model.
- Meta provides open-weight checkpoints, guidance for running and fine-tuning the model, and deployment recipes for llama.cpp, vLLM, SGLang, MLX, and NVIDIA Blackwell GPUs.
Meta description: Meta releases Muse Spark Open, an 816B open-weights expert-routed model for coding with images, audio, video, tools, 1M context, and llama.cpp support.
Tags: Muse Spark Open, Meta, agentic coding, multimodal inference, open weights
Featured image: Abstract friendly illustration of an open-weights model routing tokens through expert networks and multimodal inputs.
Muse Spark Open is for developers and teams running open-weights models locally or on large inference stacks. Meta presents it as the most capable US open-weights model to date, trained for agentic coding: reading a codebase, writing and debugging code, and calling tools. It also takes images, audio, and video as input, so the same model can read a screenshot, listen to a voice note, or watch a screen recording while it works on code. Meta says the release includes guidance for running and fine-tuning the model, plus examples of what it built with it.
Architecture: Sparse Experts with Dense Memory
Muse Spark Open follows Muse Glimmer, a small dense model from the same Muse series that Meta released in August and describes as ideal for running locally. Muse Spark Open is the full-size model in that series and introduces a Mixture-of-Experts architecture at a much larger scale.
The checkpoint repositories are Muse-Spark-Open-816B-A42B and Muse-Spark-Open-816B-A42B-NVFP4. Meta describes the model as coding-optimized, trained on real developer workflows such as multi-file refactors and long debugging sessions. It is agentic-focused: it plans, acts, observes, and self-corrects over long tasks, including browsing the web with images.
Both Muse Glimmer and Muse Spark Open use the same tokenizer and vocabulary. The chat template follows the Muse Glimmer template, with additions for the new modalities. In Muse Spark Open, the input embedding and the output layer are separate matrices of about 1.66B parameters each.
The language model has 62 layers and a hidden dimension of 8192. Each layer has 128 query heads and 16 key-value heads under grouped-query attention, an attention setup that shares key-value heads across query heads. Each head has 64 dimensions. The attention block has an output gate and an output normalization layer. The model also applies query-key normalization. Each block combines its input and output through a learned residual gate, not through a plain sum.
Muse Glimmer uses one dense feed-forward network in each layer. Muse Spark Open replaces that dense network with a Mixture-of-Experts block. Each block has 256 experts, and a router selects 8 experts for each token. Each expert is a feed-forward network that projects from 8192 to 4096 dimensions and back to 8192, then applies its own normalization to its output. The experts hold about 97% of the total model parameters.
Only 8 experts run per token, so the model computes each token with about 42B parameters. All 256 experts stay in memory, so deployment needs the memory of an 816B-parameter model, but runs at the much faster speed of a 42B model. The BF16 checkpoint is about 1.5 TB in safetensors format. The NVFP4 checkpoint stores parameters in 4-bit precision with negligible loss in accuracy, specifically for NVIDIA Blackwell GPUs.
In transformers, the BF16 checkpoint runs across 4 nodes of 8 H200 GPUs with tensor parallelism (TP=32), and the NVFP4 checkpoint runs on a single B200 node. Meta directs deployment readers to llama.cpp, vLLM, SGLang, and MLX recipes.
Multimodal Inputs and 1M Token Context
A perception encoder converts images and video frames into tokens for the language model, just like Muse Glimmer. The encoder has 1.8B parameters, 50 layers, a hidden dimension of 1536, and 16 attention heads. It splits each image into 14×14 patches and produces features with 4096 dimensions. Position information comes from learned absolute embeddings and 2D rotary position embeddings. The feed-forward network in each layer uses layer normalization and a Gaussian Error Linear Unit activation. Every fourth layer and the last layer use full attention, and the other layers use sparse attention.
Muse Glimmer 30B uses an encoder with the same architecture. The design follows the Perception Encoder work from Meta. Video input goes through the same encoder as images.
Muse Spark Open also accepts audio through an audio encoder. An audio tokenizer converts audio into tokens, and the chat template marks the audio position with <|audio|> tokens. The encoder emits one audio token every 80 ms, which yields 12.5 tokens per second. A 10-second audio clip therefore produces 125 input audio tokens.
The audio encoder itself is a convolutional pyramid. Each stage consists of a patch embedder with a different patch size and two convolutional blocks of one-dimensional kernels of size 7 with stride 1. These stages are repeated for patch sizes of [16, 6, 5, 4]. At the end of the final stage, temporal frames are fed to a final block of RMS normalization, a linear projection, and RMS normalization.
The context window is 1M tokens at inference. The configuration sets max_position_embeddings to 1,048,576. This includes image, video, and audio tokens in the same window as text.
Grouped-query attention keeps the key-value cache smaller than a fully separate key-value head design would allow. Each token adds 2 × 16 heads × 64 dimensions × 62 layers = 126,976 values to the cache. In BF16, this is about 250 KB per token and about 266 GB for a full 1M-token context.
Running the Model Across Inference Runtimes
The transformers implementation provides MuseSparkForConditionalGeneration for text generation from multimodal inputs and MuseSparkProcessor for preparing conversations. The processor combines the tokenizer, image and video processors, audio feature extractor, and chat template.
A DistributedConfig can be used to configure parallelism settings. The tp_size setting controls the number of devices across which dense layers are sharded, and ep_size controls the number of devices across which experts are distributed.
To use the processor, prepare a conversation, then pass messages to processor.apply_chat_template with add_generation_prompt=True, tokenize=True, return_dict=True, and return_tensors="pt". The processor output is passed to model.generate, and the newly generated tokens are decoded while excluding the input prefix.
The processor exposes a reasoning effort lever. Symbolic levels low, medium, high, and xhigh map to the integers 16, 64, 256, and 512 that the model accepts. Turning off reasoning is not possible with Muse Spark Open, so users who want fewer tokens for an easier task can keep reasoning at low. A numeric effort can also be set by including a line like Reasoning strength: 1 in the system prompt.
For images, the processor expands each image marker into the number of patch tokens required by the processed image, surrounded by image boundary tokens. This expansion keeps text positions aligned with the visual features passed to the model. The processor also handles multiple images within a single prompt.
Video prompts include frame timing as well as visual content. The processor handles videos with audio, which is an upgrade from Muse Glimmer thanks to the audio encoder. It downsamples video frames and audio to align frames together. By default, videos are sampled to 2 FPS, capped at 96 frames. Users can change this by setting the fps of the video processor.
The tradeoff is accuracy versus cost. More downsampling can lose frames between specific events, so a higher fps can help when more accurate generations are needed. However, higher fps requires more memory and increases latency for prefill. The frames are interleaved with text input that states timestamps, such as “Time: 3.5s”. To pass videos with audio, users can set load_audio_from_video=True when calling apply_chat_template. Preserving the video’s metadata with return_metadata=True lets the processor assign the correct framerate.
For audio inference, the audio tower consumes a raw waveform. The processor expects mono audio at 24 kHz and pads the waveform to a multiple of 1,920 samples, corresponding to one audio embedding per 80 milliseconds. Audio supplied at another sample rate must first be resampled. The load_audio utility in transformers can do this, and it uses torchcodec, librosa, or torchaudio backends, so one of those should be installed locally before running inference. If users work through apply_chat_template with messages, this handling is done under the hood. The processor inserts the matching number of audio tokens between audio boundary markers and returns a mask identifying real samples, allowing the model to exclude padded audio embeddings.
Muse Spark Open can do text-only and multimodal tool calling. It uses a channel-based chat template called Harmony, first introduced by gpt-oss. Users do not have to know the internals, but checking the channels the model routes to can help when outputs are low quality. In the message format, <|start|>assistant is intentionally left open; the model chooses a recipient with to=. Recipients include self for reasoning, commentary for comments before and between tool calls that are visible to the end user, and user for the final response. Tool-call routing is parsed when tools are passed to apply_chat_template.
Llama.cpp integrates Muse Spark Open and ships it in prebuilt binaries. A server can be started with a quant suffix attached to the model ID, and the built-in WebUI is available at localhost:9931. From the WebUI, users can add audio and images for inference and adjust the reasoning lever. Because reasoning cannot be turned off, keeping it at low is the option for using fewer tokens on easier tasks.
Meta tested quants on a Mac M3 Ultra with 512 GB. An imatrix-calibrated 2-bit quant runs at 18 tokens per second, with plenty of room left for the KV cache; the model uses 304 GB. The llama.cpp team ships different quantized versions, and Q2_K, Q4_K_S, and Q_8 were tested on Apple Silicon chips. Each quant can be run by adding the quant ID as a suffix, for example: llama serve -hf ggml-org/Muse-Spark-Open-816B-A42B:Q2_K.
The SGLang integration supports text, image, audio, and video inputs. Its NVFP4 reference configuration runs across four GB300 GPUs. From the SGLang integration environment, users download the checkpoint from the Hugging Face Hub and start the server. The checkpoint includes the chat template, which is passed explicitly, and the launcher configures Muse reasoning and tool-call parsers. Requests can be sent through an OpenAI-compatible API. The response separates the reasoning trace in reasoning_content from the final answer in content.
The vLLM reference integration supports text and image inputs with the NVFP4 checkpoint, which requires Blackwell series GPUs. After installing the reference plugin and its dependencies, users download the checkpoint from the Hugging Face Hub and launch the server. This configuration sets a 128K-token serving context. The plugin supplies the model implementation, checkpoint loader, reasoning parser, and tool-call parser. Requests use the same Hugging Face Hub model ID.
Muse Spark Open is also available in oMLX and mlx-lm. oQ quants optimized for oMLX are available in 3-bit and 4-bit and are ready to use. Meta tested the oQ3e quant on an M3 Ultra with 512 GB. The model takes about 344 GB, leaving room for the KV cache, and runs at about 225 tokens per second for prefill and 21 tokens per second for generation with a 1024-token prompt and 128 generated tokens. After downloading the model through the oMLX interface, users can chat with it in the built-in UI, add images, and adjust reasoning effort on the fly.
What This Means for Open-Weights Coding Work
For readers evaluating Muse Spark Open, the main story is a large sparse model with a heavy memory footprint. Even though each token uses only 8 experts and computes with about 42B parameters, all 256 experts remain in memory, so the deployment needs the storage of an 816B model. Quantized paths make local and smaller-node use possible, but they still require very large machines: the 2-bit llama.cpp test used 304 GB, and the oQ3e MLX test used about 344 GB on a 512 GB Mac M3 Ultra.
The multimodal design matters for coding workflows. Instead of moving between separate tools for screenshots, voice notes, screen recordings, and code, users can feed those inputs to one model and let it reason over a 1M-token context that includes text, image, video, and audio tokens. The model also supports tool calling, including multimodal tool calling, and exposes reasoning traces through APIs such as the OpenAI-compatible SGLang endpoint.
The practical takeaway is that Muse Spark Open is built for teams that can afford dense expert memory but want sparse per-token compute. For local experimentation, quantized versions provide a path, while BF16 and NVFP4 deployments target large GPU nodes. Meta positions the model as the full-size, coding-focused open-weights entry in the Muse series, and the release notes point to model repositories, inference recipes, fine-tuning guidance, and examples of what Meta built with it.

Comments (0)