Harrison Chase argues probability thresholds beat LLM judgment in agents
> feels way more shippable than "the LLM decided" probabilities are the underrated part of decis...
The point reframes agent reliability as an engineering problem: convert model judgment into numeric scores so thresholds, logs and eval tests become possible. Teams building production agents can adopt this pattern to make decisions auditable and testable. It contrasts with opaque end-to-end prompting where behavior is hard to regression-test.
LangChain founder Harrison Chase posted on X that probabilities are the underrated part of decision models in LLM systems.
He argues that numeric probabilities can be thresholded, logged and used to write evals, unlike raw LLM decisions.
In a reply, Hiếu Nguyễn Trung agreed that thresholding and testing against probabilities makes systems more shippable than saying "the LLM decided".
> feels way more shippable than "the LLM decided" probabilities are the underrated part of decis...
> feels way more shippable than "the LLM decided" probabilities are the underrated part of decision models you can threshold them, log them, write evals against them https://t.co/k4eRrMgNV0 Hiếu Nguyễn Trung @Hiumonday @hwchase17 honestly the probabilities are the best part. you can set a threshold, log it, write tests for it. feels way more shippable than "the LLM decided" 🔗 View Quoted Tweet
This report is categorized under AITOP's "How-to" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include LangChain, Harrison Chase, agents, evals.
Source attribution: curated from Harrison Chase (@hwchase17).
