Hugging Face Journal Club: Kimi K3
Kimi K3 is trained to be comparable to Claude Opus 4.8 using novel 3x3 domain expert distillation.
“they've trained a model that is probably comparable to Opus 4.8”
Hugging Face journal club reviewed Kimi K3, a model Moonshot AI claims reaches Claude Opus 4.8-level performance through a novel post-training pipeline: SFT followed by RL-trained specialist models across three domains (general tasks, general agents, coding agents) at three reasoning effort levels, yielding nine domain experts that are then merged back via on-policy distillation. The approach demonstrates a systematic, scalable way to bake in both domain specialization and compute-adaptive reasoning into a single model, which is a meaningful engineering signal for frontier training methodology.