The Base Model Is Dead — Varun Singh, Arcee AI
The traditional base-model paradigm of web-scale pre-training is being displaced by post-training
“RL was mostly just a cherry on top, shaping the flavor of the interactions more than conferring extra knowledge or quality onto the base model itself.”
Arcee AI's pre-training lead argues that the classical base model — defined by massive web-text pre-training as the primary source of model quality — is giving way to a paradigm where post-training stages carry increasing weight. Historically, RL and instruction-tuning were cosmetic layers on top of a pre-trained foundation; the talk suggests that relationship is inverting. This is a meaningful practitioner signal about where compute and research focus is shifting in frontier model development, though the transcript is incomplete.