加提示词后同一模型失败率从 75.1% 降至 6.3%
Perplexity 放出实测数据:不换模型只加提示词,失败率就从 75.1% 降到 6.3%,做工具调用的值得对比下自己的数字。
Perplexity 公布了一组测试数据:不给模型做任何更新,仅在推理时加入提示词(hints),模型在原本会失败的案例中 93.
7% 的情况下成功避开失败,此前为 75.1%。
另一项独立实测中,两版模型在不加推理时提示的条件下,工具调用失败率从 2.24% 降到 1.
77%。数据说明提示词设计与模型训练都能降低工具调用错误,两条路径可分别量化。
With hints, the unchanged model avoided the original failure in 93.7% of cases, up from 75.1% In a separate live test, tool-call failures fell from 2.24% to 1.77% between trained versions, without hints at inference time.📊 1 ⚡
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Perplexity, 提示词工程, 工具调用.
Source attribution: curated from Perplexity (@perplexity_ai).