研究测试七个开源 LLM 估算食物热量:称重越大误差越明显
Energy estimates of open-weight large language models across stated food weights: a dose-response evaluation
一篇挺有意思的评测:七个开源模型算食物热量时会'往平均值靠',分量翻倍它却不跟着翻。文中给出的两步法——让模型报密度、代码里乘重量——能把误差因子从最高 2.43 压到 1.38。做食物记录 App 或做过 LLM 数值评测的人应该看看这个实验设计。
arXiv 论文测试了七个开源权重 LLM 估算食物热量的能力:模型需对来自 USDA Food and Nutrient Database for Dietary Studies 的 168 种食物,在 0.25 到 4 倍默认份量的九个设定重量下给出能量估计。结果显示所有模型的估计弹性介于 0.548 到 0.979,全部显著低于 1,说明估算没有随重量等比例缩放;其中四个模型压缩严重,在 0.25 倍份量时高估 47% 到 130%,在 4 倍份量时低估 20% 到 34%。
机制分析发现,模型其实掌握了正确的能量密度数值,只是习惯按典型份量回答,类似膳食回顾调查中的 flat-slope syndrome。研究者让代码将模型自己给出的能量密度乘以设定重量做两步计算,四个严重压缩的模型中有三个恢复了近等比例缩放,极端份量下的典型误差因子从 1.47-2.43 降到 1.20-1.38。压缩现象在满精度设置下依然存在,用朴素提问方式时七个模型中五个还进一步加重。
对开发者与企业的含义是:食物记录类产品不应直接采信 LLM 对带重量条目的热量输出,而应在代码中用密度乘重量计算;模型在常见份量上的表面准确率掩盖了这个缺陷,评估方案应固定食物并扫描多个设定重量。
Energy estimates of open-weight large language models across stated food weights: a dose-response evaluation
We tested whether energy estimates of open-weight large language models (LLMs) scale proportionally with an exactly stated food weight, and whether computing energy density times weight removes the error. Under a prespecified protocol, seven open-weight LLMs estimated the energy of 168 randomly drawn foods from the USDA Food and Nutrient Database for Dietary Studies at nine stated weights, from 0.25 to 4 times the default amount. The primary outcome was the elasticity of the estimate with respect to weight (1: proportional scaling). A two-step estimate multiplied each model's own stated energy density by the weight in code. All seven models compressed their estimates, with elasticities from 0.548 to 0.979 (every 95% CI below 1); four compressed severely (0.548 to 0.709). These four overestimated a quarter of the default amount by 47% to 130% and underestimated four times it by 20% to 34%. Supplying the density restored near-proportional scaling in three of the four, and the two-step estimate cut the typical error factor at these extremes from 1.47-2.43 to 1.20-1.38. Compression persisted at full precision and grew in five of seven models under a plain question. The models hold the energy densities needed for proportional scaling and answer near a typical serving, reproducing the flat-slope syndrome of dietary recall. Accuracy at usual portions does not reveal it. Food-logging tools should compute the energy of weighed entries as density times weight, and evaluations should test fixed foods across stated weights.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include LLM, open-source models, model evaluation, USDA.
Source attribution: curated from arXiv cs.AI (@Donato Riccio, Fabrizio Maturo, Iulia Brezeanu).