Chain-of-Self-Questioning for LLMs to Abstain
When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control
This research provides a concrete method for LLMs to be more reliable by allowing them to abstain from answering when they lack sufficient information, which is a critical safety feature.
This paper introduces Chain-of-Self-Questioning (CoSQ), a prompt framework that makes LLMs commit to an answer only after assessing the information needed.
It tested three CoSQ variants on the TruthfulQA dataset with 11 model families.The best variant, Grounded-CoSQ, reduced the wrong-commitment rate from 13.
1% to 8.9% on a balanced protocol, while maintaining high accuracy.
When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control
Large language models can produce fluent answers when their factual support is weak. This paper introduces Chain-of-Self-Questioning (CoSQ), a prompt-only framework that makes answer commitment conditional on an explicit assessment of the information required to answer a question. We evaluate three CoSQ variants under seventeen conditions on the 817-item TruthfulQA multiple-choice validation set using eleven open-weight and hosted model families. In the final balanced-option protocol, Grounded-CoSQ at τ=0.90 reduces the mean unconditional wrong-commitment rate from 13.1% under chain-of-thought prompting to 8.9%, a 32.1% relative reduction, while increasing answered accuracy from 86.9% to 89.7% and answering 87.6% of questions. Both improvements hold for all eleven models and at every evaluated threshold. Critical-CoSQ and Adaptive-CoSQ provide neighboring operating points with 88.6% and 86.5% coverage, respectively, while remaining more reliable than the baseline. A secondary Natural Questions Short-Answer evaluation provides convergent open-form evidence. These findings show that self-assessment can support explicit, tunable answer-or-abstain decisions when an unsupported commitment is more costly than referral or review.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Chain-of-Self-Questioning, prompt engineering, risk control.
Source attribution: curated from arXiv cs.AI (@Ali Şenol).
