Meta has released Muse Spark Open, an open-weight model with roughly 816 billion parameters (about 42 billion active per token) and up to a 1 million token context window. Hugging Face announced it on October 15, 2026 and calls it the most capable US open-weights model to date. It is trained for agentic coding and takes text, images, audio and video as input, but only outputs text.
What changes
It is a mixture-of-experts model: 62 layers, 256 experts per layer and 8 selected per token. The experts hold about 97% of the parameters. Since every expert has to sit in memory, you need the memory of an 816B model, but each token costs about as much as a ~42B one.
There are two checkpoints: BF16 (about 1.5 TB in safetensors) and NVFP4, a 4-bit version aimed at NVIDIA Blackwell GPUs that Hugging Face says loses negligible accuracy. In transformers, BF16 runs on 4 nodes of 8 H200 GPUs with tensor parallelism (TP=32), and NVFP4 fits on a single B200 node.
The context window is 1,048,576 tokens and includes image, audio and video tokens. With grouped-query attention (16 KV heads) the cache costs about 250 KB per token, around 266 GB for a full BF16 context. That number matters more than the parameter count when you plan capacity.
One operational detail: reasoning cannot be turned off. There are four levels (low, medium, high, xhigh, default xhigh), and for easier tasks the advice is to keep it at low to save tokens.
Why it matters in production
Until now, a model at this level for coding agents meant depending on someone else's API. Here you can host it yourself, with what that means for data residency, latency and predictable cost. The price is infrastructure: Hugging Face documents a validated Inference Endpoints setup with NVFP4 on 8 × RTX Pro 6000 (131,072-token context) or on 8 × H200 if you want the full million.
Day-one support covers llama.cpp, vLLM, SGLang, MLX and transformers, and the server exposes an OpenAI-compatible API. That lowers integration risk: your current client may work by changing the base URL. Watch for one difference: reasoning comes back separately in reasoning_content, so decide how your traces log or drop it.
It can also be fine-tuned. The post shows TRL SFTTrainer on the full checkpoint across 8 nodes of 8 H100s, training LoRA adapters on the attention layers with the experts frozen.