Kimi K3 Full Release: 2.8T Open Frontier Model — Architecture, Benchmarks, and the New Open-Weight Ceiling

Executive Summary

On July 27, 2026, Moonshot AI fulfilled its promise and released the full weights of Kimi K3, the world's first open 3T-class model. Announced on July 16 and initially available via API, K3 is now freely downloadable from Hugging Face under the Kimi K3 License, marking a inflection point in the open-weight frontier.

At 2.8 trillion parameters with only 104B activated per token (16 of 896 experts), K3 represents the largest open model by a wide margin — 75% larger than the previous record-holder DeepSeek V4-Pro (1.6T). Built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), the model achieves frontier-level performance across coding, reasoning, and agentic tasks while remaining deployable on commodity hardware thanks to native MXFP4/MXFP8 quantization.

Key achievements:

• 67.5% DeepSWE (Kimi Code harness), 81.2% FrontierSWE, 88.3% Terminal-Bench 2.1
• 93.5% GPQA Diamond, 56% Humanity's Last Exam (with tools)
• 91.2% BrowseComp, 94.5% MCPMark-Verified, 94.6% Harvey Lab-AA
• Native multimodal: text, image, video understanding in a single model
• 1M token context window with KDA-compatible prefix caching
• API pricing: $3/MTok input, $15/MTok output, $0.30/MTok cache-hit (>90% hit rate in coding)

This article provides a comprehensive analysis of K3's architecture, training methodology, benchmark performance, deployment options, and what the full open-weight release means for the research and production communities.

1. The 3T-Class Milestone

1.1 Scale Comparison

Kimi K3 extends Moonshot's nine-month streak of setting the upper bound for open model sizes. The gap between K3 and its nearest open competitor is now staggering:

Model	Total Parameters	Active Parameters	Architecture	Release Date	License
Kimi K3	2.8T	104B	MoE + KDA + AttnRes	Jul 27, 2026	Kimi K3 License
DeepSeek V4-Pro	~1.6T	~25B	MoE	2026	MIT
Qwen3-Max-Preview	~1T	~4B	MoE	2026	Apache 2.0
MiniMax M3	~428B	~23B	MoE	2026	Open
Gemma 4 31B	31B	31B	Dense	2026	Apache 2.0

The 2.8T parameter count is not merely a number — it represents a fundamental test of whether extreme scale can be made to work in the open-weight setting. Previous attempts at models of this size failed due to training instability, routing collapse, or prohibitive inference costs. K3 succeeds by combining three innovations: KDA for efficient attention scaling, AttnRes for better information flow across depth, and Stable LatentMoE for managing 896 experts without routing collapse.

1.2 The 2.5× Scaling Efficiency Gain

Moonshot reports a 2.5× improvement in overall scaling efficiency compared to Kimi K2. This is not a claim about raw parameter count but about how effectively compute converts to intelligence. In practical terms, K3 achieves performance that would naively require ~7T parameters in a K2-like architecture.

2. Architecture Deep-Dive

2.1 Kimi Delta Attention (KDA)

KDA is the core attention mechanism that replaces standard multi-head attention in 69 of K3's 93 layers. Unlike conventional attention which computes full Q-K products for all heads, KDA uses a delta-based formulation that scales more efficiently with sequence length and head count.

Key properties:

• Efficient long-context scaling: Enables the 1M token context window without quadratic memory blowup
• Compatible with prefix caching: Moonshot contributed KDA-aware caching to vLLM, enabling >90% cache hit rates in coding workloads
• 96 attention heads with 7168 hidden dimension

2.2 Attention Residuals (AttnRes)

AttnRes selectively retrieves representations across model depth rather than accumulating them uniformly. This is analogous to how residual connections work in CNNs but applied to the attention pathway:

• 24 Gated MLA layers (Multi-Latent Attention) complement the 69 KDA layers
• 1 dense layer provides a shared computation path
• Selective retrieval reduces redundancy and improves gradient flow in very deep networks (93 layers)

2.3 Stable LatentMoE Framework

The MoE configuration is the most extreme in any released model:

Parameter	Value
Total experts	896
Active per token	16
Shared experts	2
Sparsity ratio	98.2%
Latent MoE dimension	3584
Per-expert hidden dim	3072

At this level of sparsity, routing becomes a first-order challenge. K3 addresses this with two novel techniques:

Quantile Balancing: Derives expert allocation directly from router-score quantiles, eliminating heuristic updates and sensitive balancing hyperparameters. This prevents the common MoE failure mode where a few experts dominate while others starve.

Per-Head Muon: Extends the Muon optimizer by optimizing attention heads independently, enabling more adaptive learning at scale. Different heads can learn at different rates, which is critical when 896 experts need to specialize without collapsing.

2.4 Activation Function: SiTU-GLU

Sigmoid Tanh Unit (SiTU) replaces standard SwiGLU activations. The combination of sigmoid gating with tanh nonlinearity provides better activation control, particularly important in a model with 93 layers where activation explosion/vanishing would be catastrophic.

2.5 Vision Encoder: MoonViT-V2

K3's native multimodality comes from MoonViT-V2, a 401M-parameter vision encoder integrated directly into the model (not a separate preprocessing step). This enables:

• Single-model text+image+video understanding
• No modality bottleneck from separate encoders
• Joint training on multimodal tasks

2.6 Full Architecture Summary

3. Training Methodology

3.1 Quantization-Aware Training

K3 applies quantization-aware training from the SFT (Supervised Fine-Tuning) stage onward:

• MXFP4 weights (4-bit mixed-precision floating point)
• MXFP8 activations (8-bit mixed-precision)
• Broad hardware compatibility (NVIDIA Hopper, AMD, and alternative vendors)

This is significant because it means the released weights are already optimized for deployment — no post-training quantization needed. The model was trained to understand and operate within the precision constraints, resulting in better accuracy than post-hoc quantization.

3.2 Expert-Parallel Training

To handle 896 experts without throughput degradation:

• Fully balanced expert-parallel training with static shapes
• No host synchronization on the critical path
• Recommended deployment on supernode configurations with 64+ accelerators

3.3 Preserved Thinking History

K3 was trained in preserved thinking history mode. This means:

• The model expects complete assistant messages (including reasoning_content) to be passed back in multi-turn conversations
• Switching from another model mid-session can cause instability
• Best results with Kimi Code harness or verified-compatible frameworks

# Required: Pass back full assistant message including reasoning_content
messages = [
    {"role": "user", "content": "Tell me three random numbers."},
    {"role": "assistant", 
     "reasoning_content": "I'll start by listing five numbers: 473, 921, 235, 215, 222...",
     "content": "473, 921, 235"},
    {"role": "user", "content": "What are the other two numbers?"}
]
# Model correctly recalls 215 and 222 from prior reasoning_content

4. Benchmark Performance

4.1 Reasoning & Knowledge

Benchmark	Kimi K3	Claude Fable 5	GPT-5.6 Sol	Claude Opus 4.8	GLM-5.2
GPQA Diamond	93.5	92.6	94.1	91.0	91.2
CritPt	23.4	28.6	32.3	20.9	20.9
AA-LCR	74.7	70.0	73.7	67.7	71.3
HLE-Full (w/ tools)	43.5 / 56.0	53.3 / 63.0	44.5 / 58.0	49.8 / 57.9	—

K3's GPQA Diamond score of 93.5 places it within 0.6 points of GPT-5.6 Sol — remarkable for an open-weight model. The HLE-Full score with tools (56.0) is competitive but trails Fable 5 and GPT-5.6 Sol.

4.2 Coding (The Strength)

Benchmark	Kimi K3	Claude Fable 5	GPT-5.6 Sol	Claude Opus 4.8	GLM-5.2
DeepSWE	67.5	70.0	73.0	59.0	46.2
ProgramBench	77.8	76.8	77.6	71.9	63.7
Terminal-Bench 2.1	88.3	88.0	88.8	84.6	82.7
FrontierSWE	81.2	86.6	71.3	66.7	67.3
SWE-Marathon	42.0	35.0	39.0	40.0	13.0
PostTrainBench	36.6	41.4	34.6	34.1	34.3
Kimi Code Bench 2.0	72.9	76.9	64.8	71.7	64.2

Key observations:

• K3 dominates on FrontierSWE (81.2%) — beating GPT-5.6 Sol by 10 points
• SWE-Marathon leader (42.0%) — long-horizon coding is K3's sweet spot
• ProgramBench and Terminal-Bench scores are essentially tied with the best proprietary models
• On DeepSWE, K3 trails Fable 5 and GPT-5.6 Sol but beats all other models by a wide margin

4.3 Agentic Tasks

Benchmark	Kimi K3	Claude Fable 5	GPT-5.6 Sol	Claude Opus 4.8
BrowseComp	91.2	88.0	90.4	84.3
DeepSearchQA (F1)	95.0	94.2	—	93.1
ResearchRubrics	76.2	—	73.8	73.5
MCPMark-Verified	94.5	87.4	92.9	76.4
Harvey Lab-AA	94.6	93.6	87.2	91.1
OSWorld-Verified	84.8	85.0	83.0	83.4
AutomationBench	30.8	29.1	29.7	27.2
τ³-Banking	33.4	26.8	33.0	27.6

K3 excels in research-oriented agentic tasks (BrowseComp, DeepSearchQA, ResearchRubrics) and tool-use benchmarks (MCPMark, Harvey Lab). The 94.5% MCPMark-Verified score is particularly impressive — it indicates excellent MCP (Model Context Protocol) integration.

4.4 Vision & Multimodal

Benchmark	Kimi K3	Claude Fable 5	GPT-5.6 Sol	Claude Opus 4.8
WorldVQA ForceAnswer	51.0	56.7	41.8	39.1
OmniDocBench	91.1	89.8	85.8	87.9
Video-MME (w/ sub)	90.0	—	89.5	86.0
MMMU-Pro (w/ Python)	81.6 / 83.4	81.2 / 86.5	83.0 / 84.6	78.9 / 82.7
MathVision (w/ Python)	94.3 / 97.8	94.8 / 98.6	95.8 / 97.8	86.7 / 97.1
CharXiv (w/ Python)	84.8 / 91.3	88.9 / 93.5	84.6 / 89.1	80.5 / 89.9

Vision performance is strong but not yet at Fable 5 level. K3 leads on OmniDocBench (document understanding) and Video-MME (video comprehension), while trailing slightly on fine-grained visual reasoning tasks.

5. Real-World Case Studies

5.1 GPU Kernel Optimization

K3 was tested on optimizing GPU kernels across NVIDIA Hopper and alternative GPGPU vendors. Tasks included AttnRes, KDA, and 512-head-dimension MLA kernels. K3 performed competitively with Fable 5 and substantially outperformed Opus 4.8 and GPT-5.6 Sol.

Notably, during K3's own development, an early version of K3 handled the majority of the team's kernel optimization work — a model optimizing the very architectures it's built on.

5.2 GPU Compiler Development (MiniTriton)

K3 developed MiniTriton from scratch — a compact Triton-like compiler with:

• Tile-level IR layer over MLIR
• Optimization passes
• PTX code-generation pipeline
• End-to-end nanoGPT training with stable convergence

MiniTriton delivers performance on par with or better than Triton on certain workloads, validating K3's ability to build coherent end-to-end systems rather than isolated components.

5.3 Game Development & Digital Creation

K3 combines 3D reasoning, coding, and vision for "vision in the loop" development:

• Seamlessly iterates between code and live screenshots
• Handles frontend, game dev, and CAD tasks
• True multimodal understanding (not just image classification)

5.4 Chip Design

In a 48-hour autonomous run, K3 designed a chip to serve a nano model built on its own architecture:

• 4 mm² area, closes timing at 100 MHz
• 8,700 tokens/s decode throughput in simulation
• 1.46M standard cells, 0.277 MB SRAM, INT4 MAC array with fused dequantization
• Built using open-source EDA tools on Nangate 45nm library

A chip built by a model, for a model.

5.5 Scientific Research (Astrophysics)

K3 reproduced I–Love–Q universal relations in computational astrophysics in ~2 hours (typically 1-2 weeks of work):

• Reviewed and cross-validated 20+ papers
• Implemented full numerical pipeline
• Evaluated 300+ equations of state
• Identified inconsistencies in published formulas
• Generated 3,000+ lines of Python code
• Produced interactive HTML dashboard

5.6 Interactive Research Reports

K3 in Kimi Work produced:

• 42 years of AI ASIC industry research — 120+ rounds of recursive self-improvement, 2.8K+ web searches, 1.1K+ terminal data pulls, 11K+ pages across 87 quarterly reports and 99 PDFs
• Fusion industry research with consulting-style visualizations
• GWTC-5 gravitational-wave analysis using 20+ concurrent subagents

5.7 Video Editing

K3 edited its own teaser video from 56 source clips:

• Clip selection, motion-matched cuts, frame-accurate beat synchronization
• Audio processing and multiple revision rounds
• Work that would typically take an experienced editor 1-2 days

6. Deployment & Inference

6.1 Official API

Tier	Price	Notes
Cache-hit input	$0.30/MTok	>90% hit rate in coding workloads
Cache-miss input	$3.00/MTok	Competitive for 3T-class model
Output	$15.00/MTok	Standard for frontier models

Powered by Mooncake's disaggregated inference architecture, the official API achieves >90% cache hit rates in coding workloads, effectively reducing input costs to ~$0.30/MTok for most real-world usage.

6.2 Open-Weight Deployment

The full weights are available on Hugging Face. Recommended inference engines:

Engine	Support	Link
vLLM	KDA-aware prefix caching	recipes
SGLang	Full support	cookbook
TokenSpeed	Full support	recipes

Hardware requirements:

• Recommended: 64+ accelerators (supernode configuration)
• MXFP4 weights / MXFP8 activations for broad hardware compatibility
• High-bandwidth communication domains preferred for inference efficiency

6.3 Reasoning Effort Configuration

K3 supports three reasoning effort levels:

• max (default) — Maximum thinking, best for complex tasks
• high — Balanced thinking
• low — Minimal thinking, fastest response

response = client.chat.completions.create(
    model="kimi-k3",
    messages=messages,
    reasoning_effort="max",  # "low", "high", or "max"
    temperature=1.0,
    top_p=0.95,  # 1.0 for agentic tasks
)

6.4 Kimi Code CLI

The recommended agent framework is Kimi Code CLI:

# Run in terminal
kimi-code
/model kimi-k3  # Select model

7. Pricing Economics

7.1 Cost Comparison (per 1M tokens)

Model	Input (cache-miss)	Input (cache-hit)	Output	Effective Coding Cost
Kimi K3	$3.00	$0.30	$15.00	~$0.54 (90% cache)
Claude Fable 5	$20.00	$3.00	$100.00	~$5.70
GPT-5.6 Sol	$12.00	$1.20	$60.00	~$3.36
Claude Opus 4.8	$15.00	$2.00	$75.00	~$4.10

With >90% cache hit rates in coding workloads, K3's effective input cost is ~$0.54/MTok — roughly 10× cheaper than Fable 5 and 6× cheaper than GPT-5.6 Sol for coding tasks.

7.2 Open-Weight Cost Advantage

For organizations deploying K3 on their own infrastructure:

• Zero API cost after initial hardware investment
• MXFP4 quantization means ~70% memory reduction vs. FP16
• 104B active parameters (vs. 2.8T total) means inference cost scales with active params, not total params
• Competitive with mid-tier proprietary models on a cost-per-task basis

8. Limitations & Known Issues

Moonshot transparently documents three key limitations:

8.1 Sensitivity to Thinking History

K3 requires complete reasoning_content to be passed back in multi-turn conversations. Failure to do so causes highly unstable generation. This is a significant constraint for:

• Agents that strip reasoning content
• Session handoffs between models
• Frameworks that don't support preserved thinking

Recommendation: Use Kimi Code or verified-compatible harnesses. Avoid switching to K3 mid-session.

8.2 Excessive Proactiveness

K3's training emphasis on long-horizon challenging tasks means it may:

• Make unexpected decisions on ambiguous user intent
• Take initiative beyond what the user requested
• Improvise when encountering minor issues

Mitigation: Impose explicit behavioral constraints in system prompts or AGENTS.md.

8.3 User Experience Gap

Despite competitive benchmark scores, K3 exhibits a noticeable gap in user experience compared to Fable 5 and GPT-5.6 Sol. This likely reflects:

• Less polished response formatting
• Occasional over-thinking on simple tasks
• Less natural conversation flow

9. Comparison with Prior Kimi K3 Coverage

This article supersedes the July 20 announcement coverage (Kimi K3 Open 3t Class Model Frontier Coding Agentic Knowledge Work 2026 07 20). Key updates:

Aspect	July 20 (Announcement)	July 28 (Full Release)
Weights availability	Coming July 27	Released
Architecture details	High-level overview	Full spec (KDA, AttnRes, SiTU, Quantile Balancing, Per-Head Muon)
Benchmark coverage	Core benchmarks	Full benchmark table with 40+ metrics
Case studies	Announced	Detailed technical breakdowns
Deployment guidance	API only	vLLM, SGLang, TokenSpeed recipes
Limitations	Not documented	Three known issues documented
Vision encoder	Mentioned	MoonViT-V2, 401M params
Quantization	Not mentioned	MXFP4/MXFP8 native, QAT from SFT

10. Ecosystem Impact

10.1 The Open-Weight Ceiling Rises

K3's release pushes the open-weight ceiling from ~1.6T (DeepSeek V4-Pro) to 2.8T — a 75% increase. This has implications for:

1. Research: Enables studying scaling laws at unprecedented parameter counts
2. Deployment: Organizations can run frontier-class models on-premises
3. Safety: Open weights enable independent safety auditing at the 3T scale
4. Competition: Forces proprietary models to justify their premium pricing

10.2 The MoE Scaling Proof

K3 demonstrates that extreme MoE sparsity (98.2%) can work at the 3T scale. Previous MoE models collapsed or became unstable above ~1T parameters. K3's combination of Quantile Balancing and Per-Head Muon solves the routing stability problem.

10.3 The Coding Agent Standard

With 81.2% on FrontierSWE and 42.0% on SWE-Marathon, K3 sets a new benchmark for open-weight coding agents. The combination of:

• Long-horizon coding capability
• Native multimodality (vision-in-the-loop)
• Competitive pricing
• Open weights

Makes K3 the default choice for organizations building coding agents on open infrastructure.

11. Future Directions

11.1 What to Watch

1. Community fine-tunes: With open weights available, expect specialized variants within weeks
2. Quantization research: MXFP4-native training opens new optimization avenues
3. KDA adoption: Other models may adopt KDA for efficient long-context scaling
4. Hardware optimization: Supernode deployments will drive custom hardware designs

11.2 The Next Frontier

If K3 (2.8T) is the first open 3T-class model, what comes next? The trajectory suggests:

• 5T+ open models within 6-12 months
• Full multimodal generation (not just understanding)
• Self-improving training loops (as demonstrated by K3 optimizing its own kernels)

12. References & Resources

• Hugging Face: moonshotai/Kimi-K3
• Tech Blog: Kimi K3 Tech Blog
• Technical Report: K3 Tech Report (PDF)
• API Platform: platform.kimi.ai
• Kimi Code CLI: kimi.com/code
• vLLM Recipes: recipes.vllm.ai/moonshotai/Kimi-K3
• SGLang Cookbook: docs.sglang.io
• License: Kimi K3 License
• Prior Coverage: Kimi K3 Open 3t Class Model Frontier Coding Agentic Knowledge Work 2026 07 20

---

Article written July 28, 2026. Based on Hugging Face model card, official tech blog, and benchmark data as of July 28, 2026.