User Signal Dies at the Retrieval Boundary - Sonam Pankaj, StarlightSearch
Agent failures stem from static retrieval that never learns from eval and observability signals.
“We made wrong answers appear faster and cheaper, but we forgot to make retrieval learn.”
StarlightSearch's CEO argues that most agent failures (citing 73% of pipelines) happen at retrieval, not generation, because retrieval is static and eval signals die in dashboards instead of informing future agent runs. The talk proposes a missing feedback layer that converts traces and evals into retrieval guidance so agents learn from outcomes. It matters as a framing of the retrieval-learning gap in production agent systems, though it leans on vendor framing and unverified stats.