Unsloth opens GGUF repo for DeepSeek-V4.1-Flash-1017; quants not yet posted
Unsloth has opened a Hugging Face repo for GGUF builds of DeepSeek-V4.1-Flash-1017, an MIT-licensed minor update to DeepSeek-V4.1-Flash with reported gains in reasoning, coding and agentic tasks. The page says the quants are not posted yet.
LLMScale estimates 748.49B parameters, about 173.27B active per token (MoE). For one user at 8,192 tokens of context: 701.1 GiB at FP8 (8× B200) and 374.9 GiB at INT4, enough for a 512GB M3 Ultra. Recommended context is 1M tokens.
#deepseek #gguf #moe #quantization
DeepSeek-V4.1-Flash-1017-GGUF · 748.49B parameters · 8k context, one user
FP16 KV cache and vLLM's overhead on each device at its defaults. “Fits on”: the smallest device that holds it alone, or how many of the largest it takes. Change the context, users or GPU in the calculator
Amazon has released ALoDLM-8B, a diffusion language model initialized from Qwen3-8B. It reuses shared transformer layers in a loop and gives unresolved tokens extra passes. A Reddit post claims about 3x Qwen3-8B's speed. The model card measures throughput single-stream on one B200, in a parallel-commit mode separate from its quality runs. The license is CC BY-NC 4.0.
LLMScale estimates for one user at 8,192 tokens of context: 18.37 GiB in BF16, 11.90 GiB in FP8 and 8.87 GiB in INT4.
FP16 KV cache and vLLM's overhead on each device at its defaults. “Fits on”: the smallest device that holds it alone, or how many of the largest it takes. Change the context, users or GPU in the calculator
vLLM reports that an eight-GPU AMD Instinct MI355X system serving Kimi K3 reaches about 7,567 total tokens/s per GPU on the AgentX coding-agent benchmark. That is 24.4% more than NVIDIA HGX B300 (6,083) at 50 tokens/s/user P90. Both GPUs have 288 GB HBM3E.
The biggest gains came from cache handling. DCP8 splits the MLA KV history across GPUs, cutting a 142K-token conversation from about 1.96 GB to 245 MB per GPU. LLMScale estimates 1458.6 GiB at INT4 for one user at 8K context.
FP16 KV cache and vLLM's overhead on each device at its defaults. “Fits on”: the smallest device that holds it alone, or how many of the largest it takes. Change the context, users or GPU in the calculator
A r/LocalLLaMA recap lists four open-weight releases in 72 hours: Meta's Muse Spark Open (816B total, 42B active), Reflection's Beam (501B, 23B active), DeepSeek V4.1 Flash 1017 (510GB repo) and NVIDIA's Nemotron 3.5 Bolt (74B, 9B active).
Only Bolt targets a 24GB GPU: quantization-aware training to 2 bit puts weights at 22GB plus a 1.7GB vision projector, leaving little room for KV cache. Beam's smallest official checkpoint is 294GB in NVFP4. Sizes are from the post, not LLMScale estimates.
vLLM added a specialized Kimi K3 decode path for NVIDIA GB300. It uses agent-assisted CuTe DSL kernels that fuse attention, MoE and communication across 8 or 16 GPUs in one NVLink domain. Per-layer decode time falls from 130 to 69 microseconds.
Without speculation, throughput rises 1.62–1.92x on TP16 and 1.31–1.58x on TP8 at batch 1–64. With DSpark (synthetic AL=6), batch 1 on TP16 hits 842 tokens/s, against 488 on stock vLLM. For long contexts, decode context parallelism splits the KV cache across GPUs.
Intel has named its Crescent Island GPU the D-Series. The D270 has 32 Xe3P cores, 256 XMX engines and 32 MB of L2. It is rated at up to 1.3 PFLOPS FP8 and supports formats from FP4 to FP64, on an air-cooled 350 W PCIe Gen5 x16 card. Intel's own version has 160 GB of LPDDR5X, ODM designs go up to 480 GB, and bandwidth is rated at 1.5 TB/s.
That capacity could hold large weights plus long-context KV cache on one card. However, 1.5 TB/s limits decode speed, which is bandwidth-bound. No pricing yet.
LLMScale