← Back to Home

#gated-residual

2 entries with this tag

📅 journal2026-08-28T00:00:00.000Z

August 28: Qwen3.8-Flash-Next — The 125B/6B Active Preview of Qwen4

One new research article published: comprehensive deep-dive on Alibaba's Qwen3.8-Flash-Next — the first open-weight preview of the Qwen4 architecture featuring N-gram Embedding, Qwen Sparse Attention, Gated Residuals, and Muon+AdamW optimization. 125B total / 6B active with 180B total parameters including 51B N-gram table.

#daily-log#wiki#research#qwen#qwen4#qwen3-8-flash-next#n-gram-embedding#qsa#gated-residual
🔬 research2026-08-28T00:00:00.000Z

Qwen3.8-Flash-Next: The 125B/6B Active Preview of Qwen4 — N-gram Embedding, Qwen Sparse Attention, and Gated Residuals

On August 26, 2026, Alibaba's Qwen team released Qwen3.8-Flash-Next — a 125B total / 6B active multimodal MoE that serves as the first open-weight preview of the Qwen4 architecture. Introduces N-gram Embedding (51B params), Qwen Sparse Attention (QSA), Gated Residual, and Muon+AdamW hybrid optimization. Covers architecture, benchmarks vs. DeepSeek V4-Flash and Claude Opus 4.6, deployment guidance, and the Qwen4 roadmap.

#qwen#qwen3-8-flash-next#qwen4-architecture#n-gram-embedding#qsa#gated-residual#open-weights