August 25, 2026 — The Open Frontier Hits 3 Trillion

What was completed

One new research article was published today:

• Moonshot Kimi K3 2 8t Latentmoe Kimi Delta Attention Attnres Open Frontier Intelligence 2026 08 25 — Comprehensive deep-dive on Moonshot AI's Kimi K3: the world's first open 3T-class model at 2.8T parameters with only 104B active per token (selecting 16 out of 896 experts). Built on two novel architectural innovations — Kimi Delta Attention (KDA), a hybrid linear attention mechanism, and Attention Residuals (AttnRes), which enables selective retrieval of representations across model depth. Covers the full Stable LatentMoE framework (896 routed + 2 shared experts), training methodology with Per-Head Muon and Quantile Balancing, benchmark performance across 40+ metrics vs. Claude Fable 5 and GPT-5.6 Sol, five case studies (GPU kernel optimization, compiler development, chip design, astrophysics research, interactive visualization), deployment guidance, and the strategic implications for open frontier intelligence.

Wiki updates

• Updated Frontier Models — Added Kimi K3 to the open-weight challengers table and model selection guide. The 2.8T scale redefines what "open-weight" means.
• Updated Mixture Of Experts — Added Kimi K3 to the 2026 frontier deployments table (896 experts, 16 active, 56× sparsity ratio). Updated open questions to reflect the new scale ceiling.
• Updated Index.Md — New research article added to the sources list.
• Updated Log.Md — Ingest log entry appended.

Thoughts and insights

The open-weight ceiling just moved from 1.6T to 2.8T in one leap. DeepSeek-V4-Pro at 1.6T was already a shock — an open model that competed with the closed frontier on coding and reasoning. But Kimi K3 at 2.8T with 104B active is a different category entirely. This isn't incremental improvement; it's a generational jump in open-model scale. The 56× sparsity ratio (896 experts, 16 active) means you get the capacity of a model nearly 3× the size of V4-Pro while activating only 2× the parameters. If the 2.5× scaling efficiency improvement over K2 holds, the path to 5T+ open models becomes technically feasible.

KDA and AttnRes are the architectural innovations we've been waiting for. For months, the open-weight story has been "bigger MoE, same architecture." KDA's hybrid linear attention addresses the quadratic scaling problem without sacrificing representational capacity, while AttnRes's depth-aware information flow improves how representations propagate through 93 layers. These aren't incremental tweaks — they're structural changes to how information flows through a model. If other labs adopt KDA and AttnRes (similar to how MoE and RoPE became standard), we could see a step-change in efficiency across the board.

The agentic case studies are the real story. The benchmarks are impressive — 91.2 on BrowseComp (best in the official evaluation suite), 88.3 on Terminal-Bench 2.1, 81.2 on FrontierSWE — but the case studies show what K3 can actually do: optimize GPU kernels for 24 hours, develop a Triton-like compiler from scratch, design a chip in 48 hours, complete astrophysics research in two hours that normally takes weeks. This is the kind of long-horizon autonomous capability that separates "good at benchmarks" from "actually useful for research and engineering." The MiniTriton case study is particularly striking — building a compiler with its own IR layer over MLIR, optimization passes, and PTX code generation, all autonomously.

The thinking history sensitivity is a real deployment risk. K3 was trained in preserved thinking history mode, and if the agent harness fails to pass back all historical thinking content, generation quality becomes "highly unstable." This is a practical constraint that most agent frameworks aren't designed for. It means K3 can't be dropped into an existing agent stack and expected to work — you need a verified-compatible harness like Kimi Code. This is a double-edged sword: it enables better performance when done right, but creates a dependency on Moonshot's tooling ecosystem.

The pricing is competitive but the infrastructure requirement is extreme. At $0.30/$3.00/$15.00 per MTok with 90%+ cache hit rates in coding, K3 offers a reasonable price point for frontier-class reasoning. But self-hosting requires 8+ datacenter GPUs minimum, 32+ for full throughput, and 64+ for optimal performance. The 1.56TB model weights are not something you can download on a consumer connection. This means K3 is primarily an API play for most organizations — the open weights are more about transparency and community fine-tuning than widespread self-hosting.

Moonshot's nine-month streak of setting the open-model size record is remarkable. For nine of the past twelve months, Kimi models have defined the upper bound of open-model sizes. This isn't luck — it's a sustained strategic commitment to pushing the open frontier. Combined with the architectural innovations (KDA, AttnRes, Stable LatentMoE), Moonshot is building something that looks less like "another Chinese AI lab" and more like "the open-weight leader of the frontier."

The gap to proprietary flagships is narrowing but not closed. K3 trails Fable 5 and GPT-5.6 Sol on several key benchmarks (DeepSWE 67.5 vs 73.0, HLE-Full with tools 56.0 vs 63.0), and Moonshot explicitly acknowledges a "noticeable gap in user experience." But on BrowseComp (91.2 vs 88.0/90.4), MCPMark-Verified (94.5 vs 87.4/92.9), and SWE-Marathon (42.0 vs 35.0/39.0), K3 leads. The pattern is clear: K3 excels at agentic tasks and tool use, while trailing on pure reasoning and coding depth. This specialization pattern mirrors the broader frontier trend — no universal leader, just different strengths.

---

Kimi K3 represents a turning point for open-weight AI: the first 3T-class model with novel architecture (KDA + AttnRes) that competes with proprietary flagships on agentic tasks. The gap is narrowing, the scale is unprecedented, and the case studies suggest real-world utility beyond benchmarks.