# Kimi K3 Kimi K3 is a 2.8 trillion parameter open-weight [[Large Language Models (LLMs)|LLM]] from [[Moonshot AI]], announced on 16 July 2026 with weights published on Hugging Face on 27 July. It is the successor to [[Kimi K2.6]] and the first open-weight model to be argued, credibly and by people with no stake in it, as sitting inside the closed frontier rather than one tier below it. Moonshot's own positioning is careful: K3 "still trails" [[Claude Fable 5]] and [[GPT-5.6|GPT-5.6 Sol]], and beats every open model tested. Available via kimi.com, Kimi Work, [[Kimi Code]], and the [[Kimi|Kimi API]] (platform.kimi.ai). ## Architecture This is the part worth reading closely, because K3 is not K2.6 with more parameters. Nearly every conventional component was swapped for a cheaper one. - **Stable LatentMoE**: 896 experts, 16 active per token. The "latent" part compresses the large linear layers the same way multi-head latent attention compresses attention - **Multi-head latent attention** replaces standard attention - **Kimi Delta Attention (KDA)**, a hybrid mechanism, at roughly 3 KDA layers per full attention layer - **NoPE everywhere**: no rotary embeddings, no positional embeddings at all - **Attention residuals**: residuals connected across layers with attention-based weighting. Costs about 4% more to train and 2% more to run, and improves validation loss - **Native multimodal**, new to this generation - **1M token context**, native **KDA is not attention, whatever the name says.** It is closer to an RNN that can be parallelized during training, a small modification of Gated DeltaNet: a recurrent hidden state that behaves like a small editable memory. That matters for the next point. **The NoPE choice is the interesting one.** Everyone else hedges by keeping RoPE in the local layers. Moonshot dropped positional embeddings entirely, and the tech report says why: position is encoded implicitly through KDA's recurrent gating and decay. The recurrence knows X came before Y, so the transformer does not need to be told. Causal masking does the rest. It works, and whether it keeps working at larger scale is the open question the architecture people are actually watching. The overall direction is efficiency by substitution rather than by novelty, and it is the same direction Nemotron 3 and DeepSeek V4 took. Component-for-component replacement, each one cheaper, none of them individually dramatic. Moonshot also open sourced the surrounding infrastructure: MoonEP, AgentEnv, and FlashKDA. ## Pricing - Input, cache hit: **$0.30** / MTok - Input, cache miss: **$3.00** / MTok - Output: **$15.00** / MTok - Cache hit rate above 90% on coding workloads That is 1:1 with Anthropic's Sonnet series and very close to GPT-5.6 Terra. For a Chinese open-weight model it is startling, and it is the single most-argued number in every thread about K3. GLM 5.2 is under a third of the price. DeepSeek V4 is cheaper still. Moonshot's implicit argument is that if you are competitive with the frontier, you price like the frontier. Whether the market agrees is a 2026 question, not a settled one. ## What people actually say about it The reception splits cleanly along two lines, and both lines are worth holding at once. **It is genuinely good.** Fireworks published an analysis titled "Kimi K3 is competitive with Fable; Kimi K3 and Fable is SoTA". Practitioners report running K3 alongside Claude on normal coding work and not being able to tell them apart. That claim drew the obvious pushback in the thread ("which Claude? at what effort level?"), which is fair, and it still gets made often enough to mean something. **It thinks too much.** The most consistent complaint about K3, and about Chinese models generally right now, is that they burn reasoning tokens on trivial tasks. The commercial read is that this is benchmaxxing: over-saturating reasoning buys benchmark points and costs you daily usability. When you pay per token, a model that overthinks is expensive in a way its price card does not show. **Serving it is not cheap.** Native mxfp4, roughly 1.5TB of VRAM. That is just at the limit of 8×B200, and realistically you want 16 for context and throughput. K3 is open-weight in the license sense far more than in the run-it-yourself sense. **KV caching has a rough edge.** Providers implement the cache in fixed 1024-token blocks rather than caching up to the most recent prompt, so you can eat up to 1023 extra cache-miss tokens per inference. That is the price of the recurrent state, and Qwen's GDN-series models have the same shape of problem. ## Why it matters **The distillation argument got weaker.** The standard Western-lab framing is that Chinese open models are downstream of Claude and GPT, distilled rather than invented. KDA, NoPE-everywhere, and LatentMoE are hard to fit into that story. Both things are probably true: heavy synthetic-data distillation *and* real architectural work. The claim that it is only the first one is now the harder one to defend. **Sparsity is the real gap, not scale.** K3 benchmarks in the same range as Opus and Fable, which invites the conclusion that everyone is in the 2-4T range and only training quality separates them. Active parameter count is where the difference lives, and DeepSeek's CEO has hinted at an order of magnitude. Benchmarks compress that gap; sustained human usage does not. **It reset the size race in public.** Within three days of the K3 announcement, Alibaba announced [[Qwen 3.8]] at 2.4T. Whether that was always the plan or a reaction, the effect is the same: two open-weight models at frontier scale, announced in the same week, both from labs operating under export restrictions. **Commoditizing the model is the strategy.** If [[Moonshot AI]] and Alibaba both believe the value has moved to what gets built on top, giving away the weights is not generosity, it is positioning. It happens to be very good for everyone else. ## Caveats - The quickstart docs point at a tech blog that had not covered K3 at all; the last entry was K2.6, two releases back. Benchmark sourcing was messy at launch - "Competitive with Fable" comes from Fireworks, who serve the model. Directionally supported elsewhere, but not disinterested - Refusals on politically sensitive topics are real and reported. If that matters for your use, test it rather than assuming - Token-per-token pricing comparisons across labs are close to meaningless given how differently tokenizers encode the same text, and how differently models spend reasoning tokens. Price per page of real work is the number you want, and nobody publishes it - Fine-tuning is not a weekend project. LoRA over a 2.8T MoE needs multiple GPUs and nodes; waiting for a provider fine-tuning API is the sane path ## References - Official announcement: https://www.kimi.com/blog/kimi-k3 - Technical report (PDF): https://github.com/MoonshotAI/Kimi-K3/blob/main/k3_tech_report.pdf - Weights: https://huggingface.co/moonshotai/Kimi-K3 - Sebastian Raschka, "Kimi K3 Architecture Overview and Notes" (2026-07-28) — https://sebastianraschka.com/blog/2026/kimi-k3-architecture-notes.html - Fireworks, "Kimi K3 is competitive with Fable; Kimi K3 and Fable is SoTA" — https://fireworks.ai/blog/kimik3-fable - Benchmarks and price analysis — https://artificialanalysis.ai/models/kimi-k3 - Stephen Bochinski, "The Kimi K3 Moment" (2026-07-18) — https://stephen.bochinski.dev/blog/2026/07/18/the-kimi-k3-moment/ - Open sourced infra: https://github.com/MoonshotAI/MoonEP · https://github.com/kvcache-ai/AgentEnv · https://github.com/MoonshotAI/FlashKDA ## Related - [[Moonshot AI]] - [[Yang Zhilin]] - [[Kimi]] - [[Kimi K2.6]] - [[Kimi K2.5]] - [[Kimi Code]] - [[Kimi CLI]] - [[Qwen 3.8]] - [[Qwen]] - [[Deepseek]] - [[Large Language Models (LLMs)]] - [[AI Mixture of Experts (MoE)]] - [[AI Open Weight Models]] - [[AI Expert Offloading]] - [[AI Frontier Model]] - [[AI Foundation Models]] - [[AI Reasoning Models]] - [[AI Agents]] - [[AI Tool Use]] - [[How Coding Agents Work]] - [[Agentic Engineering]] - [[Claude Fable 5]] - [[GPT-5.6]] - [[Claude Opus 4.7]] - [[Gemini 3]] - [[OpenRouter]] - [[Artificial Analysis]] - [[SWE-Bench]]