Kimi K3 Dissected
What Makes a 2.8-Trillion-Parameter Open-Weight Model Tick

On July 27, 2026, Moonshot AI released the weights for Kimi K3. At 2.8 trillion (2.8T) total parameters, it is the largest open-weight model ever shipped — a moment analysts have called a signal that "near-frontier performance you can hold in your own hands" has arrived. This piece takes the spec sheet apart against primary sources and explains what's actually new, and why it can run at all.

Kimi K3 by the numbers

Item	Value
Total parameters	2.8 trillion (2.8T) — largest open weights to date
Active params / token	~50B equivalent (16 of 896 experts fire) — sources vary
Context length	1,048,576 tokens (~1M)
Input	Text + image + video (native multimodal)
Quantization	MXFP4 weights (4-bit) / MXFP8 activations (8-bit), QAT from the SFT stage
Weight footprint	~1.4 TB (vs ~5.6 TB at FP16)
License	Open weights (commercial use allowed). Training data & code not released

An honest note on the discrepancies. Active-parameter counts differ by outlet: 32B / 50B / 104B. Since "16 of 896 experts fire" is consistent across write-ups, ~50B is our best read; Moonshot's own tech blog doesn't state an exact active count, so we won't assert one. The license name also varies — some report "Modified MIT," others "Kimi K3 License (with MaaS clauses)." Commercial use is permitted either way, but check the actual license in the public repo before you ship on it.

Three architectural innovations

K3's roughly 2.5× scaling-efficiency gain over its predecessor K2 comes down to three new mechanisms.

Kimi Delta Attention (KDA) — a hybrid linear attention that makes decoding ~6.3× faster at million-token contexts, so feeding an entire codebase in becomes economically viable. This is the core of its long-context cost story.
Attention Residuals (AttnRes) — a residual mechanism that lets each layer selectively pull representations from any earlier layer, keeping information flow from thinning out as depth grows and stabilizing deep training.
Stable LatentMoE — the MoE backbone that routes 16 of 896 experts in latent space, using Quantile Balancing (curbing load skew) and Per-Head Muon (per-head adaptive learning) to suppress the routing collapse that plagues large MoEs.

In short: make long context cheap (KDA), preserve depth-wise information flow (AttnRes), and keep a 2.8T-scale MoE stable (LatentMoE). That trio is what makes "huge but actually runnable" hold together.

Why you can self-host it — MXFP4 quantization

Holding 2.8T parameters at FP16 would be ~5.6 TB. K3 instead carries weights in MXFP4 (4-bit) and activations in MXFP8 (8-bit), and — crucially — does so via quantization-aware training (QAT) from the SFT stage, not post-hoc. Because it was trained assuming 4-bit, the accuracy loss from quantization is small.

Weight footprint compresses to ~1.4 TB (about a quarter of FP16).
MXFP4/MXFP8 are natively supported on NVIDIA Blackwell and AMD MI400.
A realistic self-host minimum is 8× 80 GB GPUs × 8 nodes (64 total), with headroom for KV-cache and activations.

The point. We move from "frontier-class only behind a closed API" to "hold the weights and run them in your own datacenter." Out of reach for an individual, but for a provider it means the cost structure itself can be rebuilt. We sketch the self-host break-even in the GPU-cost article.

Benchmarks — top-tier on coding

Benchmark	K3	Standing
SWE Marathon	42.0	Best overall
Program Bench	77.8	Best overall
DeepSWE (coding)	67.5	~0.5 pt behind GPT-5.6 Sol
GDPval-AA v2 (general)	1,687	Below Fable 5 Max (1,815)

The pattern is clear: top-tier on coding and agentic work, leading on some benchmarks, while general intelligence still trails the closed frontier (Fable 5, GPT-5.6 Sol) by a step. Treat the numbers as reference — they move with evaluation conditions.

We actually ran it through teai.io

Beyond quoting numbers, we hit the Kimi K3 API through teai.io to watch its behavior (2026-07-28).

"Always-on thinking," observed. Set max_tokens to 20 and the body comes back empty — K3 spends reasoning tokens before answering, so a small cap gets consumed by thought alone. Widen it to 2,000 and it correctly returns K3-LIVE-OK, with roughly 28 of 38 tokens spent on pre-answer reasoning. Practical takeaway: don't starve K3 on max_tokens.
Coding via the agent. Using teai.io's official CLI "Sente" in its maximum mode (te max = Kimi K3), we asked it in one shot to write an integer→Roman-numeral function with self-checks and run it. We then executed the generated roman.py independently in Python to confirm correctness (result folded into the summary below).

# Launch Kimi K3 at maximum performance (teai.io's official CLI, Sente)
curl -fsSL https://teai.io/te | sh
te max run "write an int->Roman-numeral function with self-checks and run it"

What open weights actually mean

Analysts framed this as "the open-weights escalation." Once near-frontier performance is something anyone can hold, weights and all, the moat shifts from raw model quality toward infrastructure, UX, and distribution. The model edges toward commodity; the edge comes from how you deliver it, how cheaply you run it, and what experience you wrap it in.

The same is true for us. K3 API pricing is roughly flat across vendors, so teai.io competes on delivery: drop-in compatibility with Claude Code and Codex CLI, billing that fits Japanese business practice, and an official CLI (Sente). Self-hosting for cost optimization is on the table down the road.

Summary. Kimi K3 makes "huge (2.8T) yet runnable" real via the combination of KDA, AttnRes, LatentMoE, and MXFP4 QAT — the largest open weights to date. Top-tier on coding and agentic work; a step behind the closed frontier on general intelligence. With the weights open, the contest moves from "performance" to "delivery." Through teai.io you can try this 2.8T model from your usual tools in minutes.

Try it free → Meet Sente, the official CLI

Sources: Moonshot AI official tech blog / Hugging Face (MXFP4 quantization overview) / Tom's Hardware / Interconnects (Nathan Lambert) / Forbes. Benchmark figures and active-parameter counts are reference values from public reporting and vary with evaluation conditions.

Kimi K3 DissectedWhat Makes a 2.8-Trillion-Parameter Open-Weight Model Tick