# Qwen3.8-Flash-Next open-sources the architecture Alibaba plans to use for Qwen4 Alibaba’s Qwen team has released Qwen3.8-Flash-Next as open weights: a 125B main-model MoE with 6B active parameters, a separate 51B n-gram embedding table, native 262K context and new sparse-attention/residual designs that Qwen says preview the architecture of Qwen4. The release is more interesting than another Qwen3.8 size point because Qwen is deliberately exposing the next architectural generation early. QSA sparse attention, gated residual streams and offloadable n-gram embeddings are now testable before the full Qwen4 family arrives. - Status: Active - Published: 2026-08-28T22:59:12+12:00 - Updated: 2026-08-28T22:59:12+12:00 - Categories: Artificial Intelligence, AI Models, AI Coding, Open Models, Inference & APIs - Tags: Alibaba, Coding agents, open weights, Qwen - Canonical HTML: https://beyondthe.news/dossiers/qwen3-8-flash-next-qwen4-architecture-open-weight-preview ## What changed Qwen released Qwen3.8-Flash-Next on August 26, 2026 as an open-weight multimodal MoE and an explicit preview of the architecture planned for Qwen4. The main model has 125B parameters with 6B activated per token, plus a separate 51B n-gram embedding table. It combines Gated DeltaNet with Qwen Sparse Attention, widens the residual state into four gated branches, uses offloadable n-gram embeddings and introduces a revised Muon/AdamW training recipe. Native context is 262,144 tokens with documented extension to one million. Weights are available on Hugging Face and ModelScope under Qwen Community License 1.0, with support documented for Transformers, vLLM, SGLang and other runtimes. ## Why it matters This gives model builders an unusually early look at the architectural bets behind a future flagship family rather than only a finished checkpoint. Sparse attention and offloadable embedding capacity are aimed directly at the memory and compute costs that make long-context agent workloads expensive. Builders can now test those trade-offs on public weights and serving stacks before Qwen4 ships. The caveat is that Qwen3.8-Flash-Next is explicitly experimental and its license is Qwen Community License 1.0 rather than Apache 2.0, so downstream deployment terms and stability deserve review. ## QSA attacks the long-context attention cost Three of every four layers use Gated DeltaNet to compress history, while the remaining global-attention layer uses Qwen Sparse Attention. QSA selects relevant micro-blocks through a lightweight indexer instead of attending to the whole sequence. Qwen reports large long-context prefill/decode speedups, but those figures are vendor-produced and should be reproduced on real serving configurations. ## Model capacity is split between active compute and cheap lookup memory The 125B main model activates 6B parameters per token, while a separate 51B n-gram embedding table adds local-context representations that can be offloaded to host memory and prefetched asynchronously. That is a different scaling path from simply increasing active transformer parameters. ## The residual architecture is also changing Gated Residual expands the conventional single residual stream into four branches and dynamically controls how information is read and written. Qwen says this improves cross-layer information flow and stability with low inference overhead; the public release lets independent researchers test that claim. ## This is explicitly a Qwen4 preview Qwen says the release plays the same role Qwen3-Next played before the Qwen3.5–3.8 architecture: expose the next design early, gather evidence and refine it before the full model family is built. ## The production service is a separate product Qwen says Qwen3.8-Flash is the production version based on Flash-Next, with 1M context by default and built-in tools. Qwen currently lists hosted pricing at $0.16 per million input tokens and $0.47 per million output tokens, with API availability described as coming soon. Builders should distinguish the open experimental checkpoint from the hosted production product. ## Key details - Qwen3.8-Flash-Next was released August 26, 2026. - It is explicitly described as an early preview of the architecture planned for Qwen4. - The main model has 125B parameters, with 6B active per token. - A separate n-gram embedding table adds about 51B parameters. - Native context length is 262,144 tokens and can be extended to 1,000,000 with YaRN. - The architecture combines Gated DeltaNet and Qwen Sparse Attention plus four-branch Gated Residual. - Weights are available on Hugging Face and ModelScope under Qwen Community License 1.0. - The official Qwen3.8-Flash hosted version is based on Flash-Next and is positioned as the production path. ## Builder takeaways - Treat Flash-Next as an architecture experiment first: benchmark it against Qwen3.8-27B and your current hosted model on long-context and tool-using workloads. - Review Qwen Community License 1.0 before commercial redistribution or derivative-weight work instead of assuming the terms match Qwen3.8-27B's Apache release. - Measure host-memory bandwidth and embedding-offload behavior if you self-host; n-gram capacity reduces active compute but introduces a new memory-system dependency. - Do not infer production stability from the hosted Qwen3.8-Flash product; the public Flash-Next checkpoint is explicitly an experimental preview. - Track QSA throughput at realistic cache-hit rates and contexts rather than relying on vendor peak speedup figures. ## What to watch - Independent evaluations of QSA, Gated Residual and n-gram embeddings under public serving harnesses. - Whether Qwen4 retains these architectural components substantially unchanged. - Managed-provider availability and real throughput/cost for Qwen3.8-Flash-Next. - Community/runtime support for quantization and efficient n-gram embedding offload. - Differences between the open Flash-Next checkpoint and hosted production Qwen3.8-Flash. ## Uncertainties - Most performance and efficiency comparisons currently come from Qwen itself. - The model is explicitly an experimental architecture preview and may not represent final Qwen4 behavior. - The Qwen Community License has different terms from Apache 2.0 and should be reviewed for the intended deployment. - Practical memory and throughput depend on the n-gram embedding implementation, offload strategy, context length and runtime. ## Sources - [Qwen3.8-Flash-Next: A New Architecture, Towards Ultimate Cost-Efficiency](https://qwen.ai/blog?id=qwen3.8-flash-next) — Qwen Team · primary · 2026-08-26T00:00:00+12:00. Primary architecture, parameter, context, efficiency, hosted-product and Qwen4-preview documentation. - [Qwen3.8-Flash-Next repository](https://github.com/QwenLM/Qwen3.8-Flash-Next) — Qwen / GitHub · primary repository · 2026-08-26T00:00:00+12:00. Official open-weight release and implementation/deployment references. - [Qwen3.8-Flash-Next model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) — Qwen / Hugging Face · primary model repository · 2026-08-26T00:00:00+12:00. Current weights, Qwen Community License 1.0, architecture metadata and runtime compatibility.