Kimi K3

Kimi K3 is a frontier mixture-of-experts model with 2.8 trillion total parameters, activating 16 of 896 experts — roughly 104 billion parameters — per token, making it the largest open-weight model released to date. It is built on two architectural innovations developed at Moonshot AI: Kimi Delta Attention, a hybrid linear attention mechanism, and Attention Residuals, a drop-in replacement for residual connections that delivers consistent scaling gains. With a 1 million token context, native visual understanding, and an always-on thinking mode, it is particularly strong at navigating large repositories, using tools, and iterating against images, logs, tests, and runtime feedback.