The Hallway Track
22 tracked signals on human-in-the-loop.
AI agents confidently misdiagnose production failures due to lack of verification signals
“The agent is very confident in returning a response. He says, "You know, you have a memory leak from the code you just deployed." And it still takes a person to go and check if it really happened.”
AWS launches AgentCore Runtime for event-driven ambient agents with built-in human-in-the-loop support
“The event itself is the prompt. That is an ambient agent: it responds to event streams, pauses for human input through a single ask_human tool when it needs to, and resumes from where it left off once the human answers.”
Best Buy reports 45% of AI agent sessions on major platforms involve shopping queries.
“right now about 45% of all agent sessions that happen within major providers like chat.gbt.com and Google Gemini are related to shopping”
Coding agent phones developer autonomously only for non-reversible decision points while working AFK
“I'm Echo, your coding assistant working on Tempo in Cloud Code. We're at a fork in the road and I need your decision.”
A coding agent autonomously phones the developer when it hits a judgment call it cannot make itself.
“I can't decide that for you.”
AI agents given broad API access can autonomously cause irreversible damage without adequate oversight
“it goes ahead and it drops the database and then it doesn't have a way to check to see if it was backed up”
Stripe built a production-grade AI agent system on Amazon Bedrock that cut compliance review handling time by 26 percent.
“skilled analysts were spending up to 80% of their time navigating fragmented systems to gather documentation rather than performing high-value risk assessments”
Cisco built an agentic feedback loop turning production thumbs-down signals into merged PR fixes with human oversight.
“every thumbs up is a lead. Every trace with an error is a potential regression. And every confused user is a description of a gap that needs to be addressed.”
Temporal enables human-in-the-loop approval flows in multi-agent systems without crashing the workflow
“sometimes we need to bring a human into the process”
AI agents are taking over end-to-end corporate procurement, starting with human-in-the-loop trust-building
“We're going to take control of this entire process from start to finish.”
Best production agents use deliberate human checkpoints rather than pursuing full autonomy from day one
“The best teams, sort of like shipping the best agents, just aren't really going for a full autonomy on day one.”
Microsoft Copilot UX components reach general availability in October 2026
“AI for intent. UX for action. Humans in control.”
A coding agent uses voice calls to escalate irreversible decisions to humans before acting
“Instead of guessing, it recognizes this is irreversibly my decision, loads the voice escalation skill, and calls me.”
New term 'meat proxy' describes people who blindly relay AI output without validation
“By all means, prompt AI. But don't just relay the output. Read it, understand it, validate it, and then write a response in your own words (a decent certificate that you've done the prior steps). Making that effort is value you can add.”
Core Automation aims to maximize human agency, not remove humans from AI research loops
“most automated lab in the in the world”
Trisca built a 4-agent AI pipeline automating oncology prior authorizations with zero human review
“confidence is a key metric that we were working towards”
Jon Udell argues we should reframe 'human in the loop' as inviting agents into our development loop, not joining theirs.
“I dislike the phrase “human in the loop” because it cedes authority to the machines.”
Hugging Face ships huggingface_hub weekly using AI agents with a human reviewer in the loop.
Google quietly removed its public commitment to keeping 'humans in the loop' from an AI statement.
“The new statement no longer stated that "it's critical that we maintain humans in the loop."”
AgentCraft applies real-time strategy game mechanics to reduce human bottleneck in multi-agent orchestration
“really, we are a bottleneck”
datasette-agent 0.2a0 adds mid-execution user questions and human-approved query saving.
“Tools that declare a context parameter receive a ToolContext object, and await context.ask_user(...) can ask a yes/no, multiple-choice (options=[...]) or free-text (free_text=True) question.”
uniopen fine-tuned Amazon Nova 2 Lite for proprietary two-axis retail content moderation in production