Cursor |Why Online RL Is Just the Cherry on Top
Cursor uses online (real-time) RL only to polish already-shipped models, not build them from scratch.
“That's kind of the paradox of online RL or how we like to call it real time is that, you know, we can't use this to really create the model from scratch because users need to be using the model.”
A Cursor team member explains their RL training pipeline: offline RL teaches reasoning and tool-calling to get a model good enough to ship, after which online (real-time) RL from live user feedback only incrementally improves it. The 'cherry on top' framing highlights that online RL can refine but not bootstrap a model, since users won't engage with a model that isn't already strong.