NVIDIA has released Nemotron 3.5 Super VL, an open multimodal MoE model with 120B parameters, 12B active per token. One model covers text, image, document and video input. The checkpoint uses NVFP4: 4-bit weights and activations on Blackwell, 4-bit weights with 16-bit activations on other GPUs.
All 120B weights must fit in memory even though only 12B run per token. NVIDIA says LiLiCorr speculative decoding speeds generation up to 2.3x on DGX Spark.
vLLM has day-0 support for Meta's Muse Spark Open, an open-weight MoE with 816B total and 42B active parameters and a 1M-token context. Weights come in BF16 and NVFP4, plus MXFP4 from AMD.
46 of its 62 layers use a 2048-token sliding window, so KV cache at 1M tokens is just over a quarter of all-full attention. The example serves NVFP4 at TP8 with expert parallelism and a DSpark draft model for speculative decoding.
NVIDIA released Dory1.0-4B, a 4B looped language model: a shared 20-layer block repeats between 8 input and 8 output layers, with interleaved sliding-window and full attention and up to 256K context. Loop count is set at inference; SWE-bench Verified goes from 50.3 at 1 loop to 61.8 at 4.
Per the model card, last-loop KV caching means extra loops add compute but no KV cache over a non-looped model. LLMScale estimates 11.12 GiB in BF16 and 6.60 GiB in INT4 for one user at 8K context.
| Precision | Memory | Fits on |
|---|---|---|
| BF16 | 10.59 GiB | RTX 3060 |
| FP8 | 6.89 GiB | RTX 3060 |
| INT4 | 5.04 GiB | RTX 3060 |
FP16 KV cache and vLLM's overhead on each device at its defaults. “Fits on”: the smallest device that holds it alone, or how many of the largest it takes. Change the context, users or GPU in the calculator
NVIDIA released Nemotron-Labs-Dory1.0-4B, an open 4B looped model. A shared 20-layer block sits between 8-layer input and output blocks and runs repeatedly. It mixes sliding-window and full attention and supports up to 256K context.
Per-loop caching grows the KV cache with each loop; last-loop mode keeps it at one-loop size. LLMScale estimates 11.12 GiB in BF16 and 8.05 GiB in FP8 for one user at 8K context. vLLM support is still an open PR.
| Precision | Memory | Fits on |
|---|---|---|
| BF16 | 10.59 GiB | RTX 3060 |
| FP8 | 6.89 GiB | RTX 3060 |
| INT4 | 5.04 GiB | RTX 3060 |
FP16 KV cache and vLLM's overhead on each device at its defaults. “Fits on”: the smallest device that holds it alone, or how many of the largest it takes. Change the context, users or GPU in the calculator
Unsloth has opened a Hugging Face repo for GGUF builds of DeepSeek-V4.1-Flash-1017, an MIT-licensed minor update to DeepSeek-V4.1-Flash with reported gains in reasoning, coding and agentic tasks. The page says the quants are not posted yet.
LLMScale estimates 748.49B parameters, about 173.27B active per token (MoE). For one user at 8,192 tokens of context: 701.1 GiB at FP8 (8× B200) and 374.9 GiB at INT4, enough for a 512GB M3 Ultra. Recommended context is 1M tokens.
| Precision | Memory | Fits on |
|---|---|---|
| BF16 | 1395.9 GiB | 10× B200 180GB |
| FP8 | 698.8 GiB | 5× B200 180GB |
| INT4 | 350.3 GiB | Apple M3 Ultra (512GB) |
FP16 KV cache and vLLM's overhead on each device at its defaults. “Fits on”: the smallest device that holds it alone, or how many of the largest it takes. Change the context, users or GPU in the calculator
Amazon has released ALoDLM-8B, a diffusion language model initialized from Qwen3-8B. It reuses shared transformer layers in a loop and gives unresolved tokens extra passes. A Reddit post claims about 3x Qwen3-8B's speed. The model card measures throughput single-stream on one B200, in a parallel-commit mode separate from its quality runs. The license is CC BY-NC 4.0.
LLMScale estimates for one user at 8,192 tokens of context: 18.37 GiB in BF16, 11.90 GiB in FP8 and 8.87 GiB in INT4.
| Precision | Memory | Fits on |
|---|---|---|
| BF16 | 17.34 GiB | RTX 3090 / Ti |
| FP8 | 9.71 GiB | RTX 3060 |
| INT4 | 5.90 GiB | RTX 3060 |
FP16 KV cache and vLLM's overhead on each device at its defaults. “Fits on”: the smallest device that holds it alone, or how many of the largest it takes. Change the context, users or GPU in the calculator
LLMScale