Papers

研究测试七个开源 LLM 估算食物热量:称重越大误差越明显

Energy estimates of open-weight large language models across stated food weights: a dose-response evaluation

Why we picked it

一篇挺有意思的评测:七个开源模型算食物热量时会'往平均值靠',分量翻倍它却不跟着翻。文中给出的两步法——让模型报密度、代码里乘重量——能把误差因子从最高 2.43 压到 1.38。做食物记录 App 或做过 LLM 数值评测的人应该看看这个实验设计。

arXiv 论文测试了七个开源权重 LLM 估算食物热量的能力:模型需对来自 USDA Food and Nutrient Database for Dietary Studies 的 168 种食物,在 0.25 到 4 倍默认份量的九个设定重量下给出能量估计。结果显示所有模型的估计弹性介于 0.548 到 0.979,全部显著低于 1,说明估算没有随重量等比例缩放;其中四个模型压缩严重,在 0.25 倍份量时高估 47% 到 130%,在 4 倍份量时低估 20% 到 34%。

Body

机制分析发现,模型其实掌握了正确的能量密度数值,只是习惯按典型份量回答,类似膳食回顾调查中的 flat-slope syndrome。研究者让代码将模型自己给出的能量密度乘以设定重量做两步计算,四个严重压缩的模型中有三个恢复了近等比例缩放,极端份量下的典型误差因子从 1.47-2.43 降到 1.20-1.38。压缩现象在满精度设置下依然存在,用朴素提问方式时七个模型中五个还进一步加重。

对开发者与企业的含义是:食物记录类产品不应直接采信 LLM 对带重量条目的热量输出,而应在代码中用密度乘重量计算;模型在常见份量上的表面准确率掩盖了这个缺陷,评估方案应固定食物并扫描多个设定重量。

Original · arXiv cs.AI

Energy estimates of open-weight large language models across stated food weights: a dose-response evaluation

We tested whether energy estimates of open-weight large language models (LLMs) scale proportionally with an exactly stated food weight, and whether computing energy density times weight removes the error. Under a prespecified protocol, seven open-weight LLMs estimated the energy of 168 randomly drawn foods from the USDA Food and Nutrient Database for Dietary Studies at nine stated weights, from 0.25 to 4 times the default amount. The primary outcome was the elasticity of the estimate with respect to weight (1: proportional scaling). A two-step estimate multiplied each model's own stated energy density by the weight in code. All seven models compressed their estimates, with elasticities from 0.548 to 0.979 (every 95% CI below 1); four compressed severely (0.548 to 0.709). These four overestimated a quarter of the default amount by 47% to 130% and underestimated four times it by 20% to 34%. Supplying the density restored near-proportional scaling in three of the four, and the two-step estimate cut the typical error factor at these extremes from 1.47-2.43 to 1.20-1.38. Compression persisted at full precision and grew in five of seven models under a plain question. The models hold the energy densities needed for proportional scaling and answer near a typical serving, reproducing the flat-slope syndrome of dietary recall. Accuracy at usual portions does not reveal it. Food-logging tools should compute the energy of weighed entries as density times weight, and evaluations should test fixed foods across stated weights.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include LLM, open-source models, model evaluation, USDA.

Source attribution: curated from arXiv cs.AI (@Donato Riccio, Fabrizio Maturo, Iulia Brezeanu).