DeepSeek-V4.1-Flash: Causal Encoder-Decoder Architecture, 4× KV Cache Compression, and the End of V4-Pro

Executive Summary

On September 10, 2026, DeepSeek released DeepSeek-V4.1-Flash, a 552B-parameter multimodal Mixture-of-Experts (MoE) model that represents a fundamental architectural departure from its predecessors. The model introduces a Causal Encoder-Decoder (CED) architecture — a 40-layer Transformer split into a 20-layer causal encoder and a 20-layer decoder — that activates only 8B parameters per token during input prefill and 16B during output decoding, despite having 552B total backbone parameters. This asymmetric design, combined with Compressed Sparse Attention 2 (CSA2) and SWA Bounded Replay, reduces the global KV cache footprint to 890 bytes per token — roughly 1/4 of DeepSeek-V4-Flash and 1/437 of DeepSeek-V1.

The performance implications are dramatic. V4.1-Flash outperforms DeepSeek-V4-Pro (1.6T parameters, 49B activated) on all published agentic benchmarks, including DeepSWE v1.1 (74.2 vs 62.7), Terminal-Bench 2.1 (90.6 vs 87.9), AutomationBench (54.8 vs 43.2), and Agent's Last Exam (31.8 vs 25.7). It also exceeds GPT-5.6 Sol on several key agentic metrics. The results are so decisive that DeepSeek has retired V4-Pro, with all deepseek-v4-pro API requests routing to V4.1-Flash at Flash pricing effective September 14, 2026.

The model is fully open-source under the MIT License, with weights available on Hugging Face, a technical report, reference inference code, and a Rust-based prompt encoding library (deepseek-recipe). It supports native multimodal vision, 1M-token context, 384K maximum output, and a continuously controllable reasoning effort setting (integer 1–100). API pricing starts at $0.30/M input (peak cache-miss) and $1.20/M output (peak), with off-peak rates at 50% and cache-hit input as low as $0.003/M off-peak — making it the most cost-effective frontier model for long-context agentic workloads.

---

1. The Causal Encoder-Decoder Architecture

1.1 Design Philosophy

DeepSeek-V4.1-Flash's defining innovation is the Causal Encoder-Decoder (CED) architecture, which rethinks how MoE models handle the asymmetric compute requirements of input processing versus output generation.

In traditional decoder-only architectures (including V4-Flash and V4-Pro), every layer — during both prefill and decode — activates the same set of experts for every token. For a model with 384 routed experts activating 6 per token, this means the same compute budget is spent whether the model is reading a 500K-token context or generating a single output token.

CED splits the 40-layer Transformer into two functional halves:

Key mechanism: The decoder's global KV cache is projected from the final encoder hidden states rather than derived from each decoder layer's own hidden states. This means:

• During prefill (reading input): Only the 20 encoder layers run, activating 8B parameters per token. The decoder is bypassed entirely.
• During decode (generating output): The encoder is frozen; only the 20 decoder layers run, activating 16B parameters per token.
• The KV cache is computed once from encoder outputs, eliminating redundant cache computation across decoder layers.

1.2 Parameter Configuration

Component	V4-Flash	V4-Pro	V4.1-Flash
Total backbone params	284B	1.6T	552B
Activated params (prefill)	13B	49B	8B
Activated params (decode)	13B	49B	16B
Architecture	Decoder-only	Decoder-only	Causal Encoder-Decoder
Layers	—	—	20 encoder + 20 decoder
Experts per layer	—	—	1 shared + 384 routed
Experts activated/token	—	—	6 routed

The asymmetry — 8B for input, 16B for output — reflects the different computational demands: reading a large context benefits from aggressive sparsity, while generating output benefits from more capacity for reasoning and coherence.

1.3 Why This Matters for Agentic Workloads

Agentic workloads are characteristically input-heavy: the model reads large codebases, documentation, terminal output, and conversation history (often hundreds of thousands of tokens) before generating relatively concise actions. Traditional architectures spend the same compute budget reading each input token as generating each output token.

With CED, the prefill phase is ~4× cheaper per token than V4-Pro (8B vs 49B activated), while the decode phase actually has more capacity (16B vs 49B is less, but the decoder layers are specialized for generation). This is a fundamental reallocation of compute that matches the actual workload profile of agents.

---

2. Compressed Sparse Attention 2 (CSA2)

2.1 Static Layer Modes

CSA2 assigns each attention layer one of three static modes to share KV cache and indexing resources across layers:

Mode	Behavior	Use Case
Full	Full attention over all tokens	Early layers needing global context
Reindex	Reuses main KV from Full layers, re-computes sparse indices	Mid layers needing selective focus
Reuse	Reuses both main KV and Top-K indices from earlier layers	Late layers where attention patterns stabilize

In the decoder, a Hierarchical Sparse Indexer further restricts later indexing layers to a candidate pool constructed by the first Full Mode layer, bounding deeper indexer cost independently of context length.

2.2 FP4 Main KV Caching

V4.1-Flash uses FP4 main KV caching in E2M1 format with one E4M3 scale per 16 channels. This ultra-low-precision caching, combined with CSA2's layer sharing, reduces the global KV cache footprint to 890 bytes per token.

Cache reduction:

• 4× smaller than DeepSeek-V4-Flash
• 437× smaller than DeepSeek-V1
• 1/4 the HBM (high-bandwidth memory) requirements
• 1/8 the SSD storage requirements

2.3 SWA Bounded Replay

Sliding Window Attention (SWA) states are typically persisted to SSD for long-context inference. V4.1-Flash introduces SWA Bounded Replay, which reconstructs missing SWA KV states by replaying only the most recent n_win tokens, avoiding the need to persist SWA KV to SSD entirely. This is the primary driver of the 1/8 SSD storage reduction.

---

3. Additional Architectural Components

3.1 Single-Pass mHC (Multi-Head Cross-Attention)

Revised residual-stream mixing with an efficient Mega-mHC kernel that processes cross-attention in a single pass, reducing memory bandwidth requirements for the encoder-decoder interface.

3.2 Engram Conditional Memory

A 196B parameter sparse memory module accessed via token-based lookup. Engram provides conditional access to long-term knowledge without activating the full parameter set, functioning as a form of associative memory that complements the MoE routing.

3.3 DSpark Speculative Decoding

A semi-autoregressive draft generation system with confidence-scheduled verification. DSpark generates draft tokens in parallel and verifies them selectively based on confidence scores, accelerating throughput for high-confidence generations while maintaining accuracy on difficult tokens.

3.4 Native Multimodal Vision

V4.1-Flash includes a DeepSeek-ViT vision encoder trained from scratch with:

• 2D-RoPE (Rotary Position Embedding) for spatial awareness
• 3×3 pixel-unshuffle downsampling for efficient resolution handling
• Two-layer MLP projector converting visual embeddings to the language model's embedding space

Vision processing is integrated from the start of pre-training (not added as an afterthought), enabling true multimodal understanding rather than vision-to-text translation.

---

4. Training & Post-Training

4.1 Pre-Training

Aspect	Detail
Training corpus	45T tokens (multimodal: text + images)
Sparse attention training	At 64K sequence length
Context extension	Extended to 1M tokens at 34T tokens (via progressive context length scaling)
Training from scratch	Yes — not fine-tuned from V4-Flash or V4-Pro

4.2 Post-Training

The post-training recipe follows the standard SFT → RL → On-Policy Distillation (OPD) paradigm without algorithmic modifications. All substantive improvements come from the data pipeline:

• Large-scale automated synthesis of agent tasks and environments
• Progressive scaling of data, tasks, and rollouts
• Focus on agentic capabilities: code editing, terminal interaction, tool use, multi-step planning

4.3 Controllable Reasoning Effort

V4.1-Flash supports a continuously controllable reasoning effort setting (integer 1–100), allowing users to trade inference cost for accuracy:

• Effort 1: Fast, minimal reasoning — suitable for simple tasks
• Effort 50: Balanced — default for most workloads
• Effort 100: Maximum reasoning — used for all benchmark results below

This replaces the binary thinking/non-thinking toggle with a fine-grained dial, enabling cost optimization based on task difficulty.

---

5. Benchmark Performance

5.1 Agentic Benchmarks (Max Reasoning Effort)

The most significant results are in agentic coding, where V4.1-Flash exceeds both V4-Pro and GPT-5.6 Sol:

Benchmark	Opus 5.0	GPT-5.6 Sol	V4-Pro	V4-Flash	V4.1-Flash
DeepSWE v1.1 (Resolved)	74.0	73.0	62.7	54.4	74.2
Terminal-Bench 2.1 (Pass@1)	89.1	88.8	87.9	82.7	90.6
Terminal-Bench 3.0 (Pass@1)	43.3	34.4	11.8	7.6	30.0
Terminal-Bench 4.0 (Pass@1)	51.8	39.9	12.4	7.0	31.2
AutomationBench (Pass@1)	50.3	45.8	43.2	37.7	54.8
Agent's Last Exam (Pass@1)	28.6	26.7	25.7	25.2	31.8
CyberGym (Pass@1)	—	84.5	83.3	76.7	88.1
SEC-Bench Pro (Pass@1)	—	74.3	56.4	30.9	62.8
HLE w/ tools (Pass@1)	63.6	—	60.0	51.5	63.9
Codeforces (Rating)	—	—	3348	3289	3471
MathArena Apex (Pass@1)	—	—	65.3	58.6	65.6

Key observations:

• V4.1-Flash leads on 7 of 11 agentic benchmarks where comparisons exist
• On DeepSWE v1.1, it matches Claude Opus 5 (74.2 vs 74.0) and exceeds GPT-5.6 Sol (74.2 vs 73.0)
• On Terminal-Bench 2.1, it leads all models (90.6)
• On AutomationBench, it leads by a significant margin (54.8 vs 50.3 for Opus 5)
• On Agent's Last Exam, it leads (31.8 vs 28.6 for Opus 5)
• The improvement over V4-Pro is massive on complex tasks: Terminal-Bench 3.0 (+18.2 points), Terminal-Bench 4.0 (+18.8 points)

5.2 Base Model Benchmarks

Benchmark	V4-Flash-Base	V4-Pro-Base	V4.1-Flash-Base
HumanEval (Pass@1)	69.5	76.8	79.4
BigCodeBench (Pass@1)	56.8	59.2	60.6
MMLU-Pro (EM)	68.3	73.5	74.1
GSM8K (EM)	90.8	92.6	93.0
SuperGPQA (EM)	46.5	53.9	53.1
MMMU-Pro (EM)	—	—	56.5
DocVQA (LLM-Judge)	—	—	95.6

5.3 Performance Across Agent Scaffolds

V4.1-Flash was evaluated across multiple agent scaffolds on DeepSWE v1.1 and Terminal-Bench 2.1:

Scaffold	DeepSWE v1.1	Terminal-Bench 2.1
DSH Minimal	74.2	90.6
mini-SWE	72.6	90.3
DSH Standard	70.5	85.8
Claude Code	69.8	88.0
DSH PTC	67.6	85.8
Pi	66.2	86.1
OpenCode	65.5	85.0
Codex	65.6	84.1

The DSH Minimal scaffold (DeepSeek Harness Minimal) achieves the best results, suggesting the model is optimized for this specific agent framework.

---

6. Pricing & Cost Analysis

6.1 API Pricing (Per 1M Tokens)

Tier	Cache Hit Input	Cache Miss Input	Output
Peak	$0.006	$0.30	$1.20
Off-Peak (50%)	$0.003	$0.15	$0.60

Peak hours: 01:00–04:00 and 06:00–10:00 UTC, Monday–Friday.

6.2 Cost Comparison with Key Competitors

Model	Input (peak)	Output (peak)	Cache Hit Input	Context
V4.1-Flash	$0.30/M	$1.20/M	$0.006/M	1M
V4-Pro	$1.32/M	$3.96/M	$0.044/M	1M
GPT-5.6 Sol	$5.00/M	$30.00/M	—	1M
Claude Opus 5	$5.00/M	$25.00/M	—	200K
Claude Fable 5.1	$10.00/M	$50.00/M	—	1M

Cost implications:

• V4.1-Flash is ~16× cheaper on input and ~25× cheaper on output than GPT-5.6 Sol
• The cache-hit rate of $0.003/M off-peak makes repeated context reads (common in agentic loops) essentially free
• For a typical agentic workload with 500K input tokens and 50K output tokens (off-peak, with 80% cache hit on input):
  • V4.1-Flash: ~$12.75 total
  • GPT-5.6 Sol: ~$3,250 total (if cache available) or much more without

6.3 V4-Pro Retirement

Effective 04:00 UTC on September 14, 2026, all deepseek-v4-pro API requests route to V4.1-Flash at Flash pricing. DeepSeek later clarified that V4-Pro API services would continue at unchanged pricing due to user demand, but the strategic message is clear: V4.1-Flash has completely superseded V4-Pro in capability and efficiency.

---

7. Deployment & Implementation

7.1 API Access

# OpenAI-compatible API
curl https://api.deepseek.com/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${DEEPSEEK_API_KEY}" \
  -d '{
    "model": "deepseek-flash",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Hello!"}
    ],
    "thinking": {"type": "enabled"},
    "reasoning_effort": "high",
    "stream": false
  }'

# Anthropic-compatible API
# Base URL: https://api.deepseek.com/anthropic

Supported features: JSON output, tool calls, Responses API, Anthropic API format, Chat Prefix Completion (Beta), FIM Completion (Beta), native vision.

7.2 Local Deployment

The model is fully open-source under MIT License:

• Weights: Hugging Face
• Technical Report: Available in the repo
• Reference Inference: Python implementation in the inference/ folder
• Prompt Encoding: deepseek-recipe Rust library with Python bindings
• GGUF Conversions: Community conversions available (architecture identifier: deepseek41)

Recommended sampling parameters:

• Temperature: 1.0
• Top-p: 0.95 or 1.0
• Context window: 1M tokens
• Max output: ≥ 256K tokens

7.3 Agent Framework Integrations

Official partner integrations include:

• WorkBuddy (including CodeBuddy)
• OpenCode
• DeepSeek Harness (developer preview) — a dedicated agent harness framework

Community integrations with Claude Code, GitHub Copilot, and other MCP-compatible tools are supported via the OpenAI/Anthropic API compatibility layer.

7.4 Hardware Requirements

For self-hosted deployment, DeepSeek mentions "large-scale deployment with 2,000 GPUs + a storage cluster" as a discussion point. The 552B parameter count with MoE routing and the CED architecture's memory requirements make this a significant infrastructure commitment. The 4× KV cache reduction is critical here — without it, the memory requirements would be prohibitive for most clusters.

---

8. Connection to Prior Work

8.1 Evolution Within DeepSeek

V4.1-Flash represents the third major generation of DeepSeek's V4 family:

1. DeepSeek-V4-Pro (April 2026): 1.6T params, 49B activated, decoder-only MoE — the initial flagship
2. DeepSeek-V4-Flash (July 2026): 284B params, 13B activated, decoder-only MoE — the efficient tier
3. DeepSeek-V4.1-Flash (September 2026): 552B params, 8B/16B activated, CED MoE — the architectural leap

Each generation reduced activated parameters while increasing capability, but V4.1-Flash is the first to change the fundamental architecture from decoder-only to causal encoder-decoder.

8.2 Relation to Anthropic's MHS

The agentic capabilities demonstrated by V4.1-Flash (DeepSWE 74.2, Terminal-Bench 90.6) are directly relevant to the Model Hardware Standard (MHS) announced by Anthropic (Anthropic Model Hardware Standard Mhs Research Preview Ai Agents Physical Lab Equipment 2026 09 09). MHS requires agents that can reliably execute multi-step workflows across heterogeneous devices — exactly the capability profile where V4.1-Flash excels. The model's cost structure (particularly the $0.003/M off-peak cache-hit rate) makes it economically viable for the continuous, high-throughput agent loops that physical lab automation requires.

8.3 Relation to OpenAI's Navier-Stokes Solution

OpenAI's Navier-Stokes solution (Openai Navier Stokes Millennium Prize 10000 Agents Lean Formalization Credit Controversy 2026 09 11) demonstrated that swarms of 10,000 agents could solve Millennium Prize-level problems. V4.1-Flash's architecture — designed specifically for input-heavy agentic workloads with aggressive cache compression — is optimized for exactly this kind of multi-agent coordination, where each agent repeatedly reads large shared contexts. The 437× KV cache reduction over V1 means that running 10,000 concurrent agents with 1M-token contexts becomes feasible at a fraction of previous cost.

---

9. Key Takeaways

1. CED is the new paradigm. The Causal Encoder-Decoder architecture solves the fundamental inefficiency of decoder-only MoE models for input-heavy workloads. Activating 8B params for input and 16B for output is a more honest allocation of compute than activating 49B for both.

2. KV cache compression is the unsung hero. The 4× reduction in cache size (890 bytes/token) is what makes 1M-context agentic workloads economically viable. Without this, the model would be too expensive to run at scale.

3. V4-Pro is dead. The decision to route all V4-Pro traffic to V4.1-Flash at Flash pricing is a rare admission that a smaller, architecturally different model completely supersedes a larger one. 552B params with CED beat 1.6T params with decoder-only on every metric that matters.

4. Open-source frontier is real. MIT-licensed weights, full technical report, reference inference code, and Rust encoding library — this is the most complete open-source release from a frontier lab to date.

5. Pricing changes the game. At $0.30/M input and $1.20/M output (peak), with cache-hit rates of $0.003/M off-peak, V4.1-Flash makes agentic workloads that were previously economically impossible now trivially affordable.

6. Agentic coding is the new benchmark. The model leads on DeepSWE, Terminal-Bench, AutomationBench, and Agent's Last Exam — benchmarks that matter more for real-world utility than MMLU or GSM8K.

---

10. References & Resources

Official Sources

• DeepSeek Announcement
• Hugging Face Model Card
• Technical Report (PDF)
• API Documentation
• Pricing Page
• DeepSeek Recipe (Rust Encoding Library)
• DeepSeek Harness Guide

Related Research in This Journal

• Anthropic Model Hardware Standard Mhs Research Preview Ai Agents Physical Lab Equipment 2026 09 09 — Anthropic's MHS for physical lab automation
• Openai Navier Stokes Millennium Prize 10000 Agents Lean Formalization Credit Controversy 2026 09 11 — OpenAI's 10,000-agent Navier-Stokes solution
• Arm Physical Ai Ecosystem Launch Total Design Robotics Capability Framework Css Mobile 2 Ai Portal 2026 09 10 — ARM's physical AI ecosystem

Citation

@misc{deepseekai2026deepseekv41flash,
  title={DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression},
  author={DeepSeek-AI},
  year={2026},
}

---

11. Future Directions

11.1 V4.1-Pro

The announcement implies a V4.1-Pro model is coming, which would apply the CED architecture to a larger parameter count. If V4.1-Flash (552B, 8B/16B activated) beats V4-Pro (1.6T, 49B activated), a V4.1-Pro with similar CED efficiency at a larger scale could represent a step-change in frontier capability.

11.2 Community Inference Optimization

With MIT-licensed weights and the new deepseek41 architecture identifier, expect rapid community development of:

• Optimized GGUF quantizations for consumer hardware
• vLLM and TGI integration for the CED architecture
• FlashAttention kernels optimized for the encoder-decoder split
• Multi-GPU distributed inference strategies

11.3 Agentic Workflow Standards

V4.1-Flash's dominance on agentic benchmarks, combined with Anthropic's MHS and the growing MCP ecosystem, suggests 2026 H2 will see convergence around standardized agentic workflows — not just for software development but for scientific research, lab automation, and infrastructure management.

11.4 Economic Implications

The pricing structure — particularly the near-zero cache-hit cost — could enable always-on agentic systems that maintain persistent context across days or weeks, something that was economically impossible with previous pricing models. This could fundamentally change how we design AI-assisted workflows.

---

Article compiled from official DeepSeek sources on September 14, 2026. All benchmarks and specifications verified against the Hugging Face model card and official API documentation.