Latest

Every published item, in order · 1029 items
Oct 16
Oct 15
PapersBuildersMulti-source13:00
Chinese content研究拆解 LLM 网络威胁情报实体抽取失败模式:主要问题不是幻觉

DeepSeek-V4-Flash 和 GPT-6.1-Sol 抽实体都不靠谱,但坏的原因不是幻觉,而是把背景信息当成攻击实体。用置信度分层排审核优先级,别当真值。

Event
arXiv: DeepSeek@Jiahong Qian5 sourcesSource
ModelsUS vs ChinaMulti-sourceSelected13:00
DeepSeek V4.1 Flash Tops Artificial Analysis AgentPerf Leaderboard

Serving efficiency, not raw scale, is where V4.1 Flash wins: it delivers 1.3x to 6.2x the throughput per MW of V4 Pro 0813 across Blackwell hardware while scoring higher on intelligence. Teams running agentic coding workloads can use the AA-AgentPerf data to pick hardware — B300 nodes stay within about 20% of rack-scale GB300 NVL72, so single 8-GPU nodes are viable for this model. As an open-weights release from DeepSeek, it widens cheap self-hosting options for Southeast Asian inference providers competing with rented frontier APIs.

Event
官方账号Artificial Analysis@ArtificialAnlys11 sourcesSource
ModelsBuildersMulti-sourceSelected71°12:54
AMD MI355X joins AA-AgentPerf benchmark against NVIDIA Blackwell

The MI355X's 288GB of HBM3E per GPU gives it an edge on very large models like the 1.6T-parameter DeepSeek V4 Pro, but serving-stack maturity on ROCm remains the bottleneck at higher loads. Teams evaluating accelerator purchases for agentic inference can now compare AMD and NVIDIA on realistic coding-agent workloads rather than synthetic throughput numbers. All results run on ROCm nightly builds of vLLM and SGLang, so scores should improve as the software stack matures.

Event
官方账号Artificial Analysis@ArtificialAnlys11 sourcesSource
PapersBuildersSelected12:52
BizBench Benchmark Tests Agent Execution Validity in Payments Workflows

The benchmark shows that outcome-only evaluation masks invalid execution, since models can hit the final goal while violating constraints mid-run — a gap that matters most in regulated domains like payments where intermediate errors carry financial or compliance consequences. Agent builders can use the 50-task suite and deterministic checks to measure constraint compliance and tool-call overhead, not just task success. Teams choosing models for workflow automation get a direct comparison of frontier and open models under identical conditions.

arXiv: DeepSeek@Enhan Zhao et al. (9)Source
Chinese contentarXiv 论文:证伪驱动的模型无关智能体协议,输出纪律决定决策下限

用中医辨证基准证明了件事:很多模型翻车不是能力不够,是输出纪律不行。他们设计的证伪协议能把 Kimi K3 的方差从 0.218 压到 0.007,写智能体的值得看看。

Event
arXiv: DeepSeek@Jing Liu et al. (4)2 sourcesSource
Oct 14
ModelsBuildersMulti-sourceSelected03:07
Fish Audio Drama 3 Preview Tops Open-Weights TTS Leaderboard

An open-weights model beating closed leaders like Eleven v4 on pronunciation while costing a fifth of the price signals real pressure on commercial TTS pricing. Builders producing audiobooks, dubbed content or multi-speaker agents can self-host the weights from Hugging Face or call the API, with 83-language support covering major Southeast Asian languages. The Elo gain of 101 points over Fish Audio's own S2.1 Pro shows the jump is internal as well as competitive.

Event
官方账号Artificial Analysis@ArtificialAnlys10 sourcesSource
Oct 13
ModelsBuildersMulti-sourceSelected04:17
FrontierCode 1.1 Leaderboard Launches on Artificial Analysis

The benchmark penalises behaviours conventional evals ignore, such as oversized diffs, out-of-scope file edits and unhelpful tests, so its ranking offers a different picture of coding ability than SWE-style test suites. Even the best model fails over 40% of tasks, meaning maintainer cleanup is still unavoidable. Teams choosing models by cost-performance should note GPT-6.1 Sol matching Claude Fable 5.1 at under a tenth of the per-task price.

Event
Cognition@cognition_labs11 sourcesSource
ModelsUS vs ChinaMulti-sourceSelected04:15
Artificial Analysis launches leaderboard for Cognition's FrontierCode 1.1

The benchmark's merge-focused grading penalises the messy diffs and unhelpful tests that cheaper evaluations ignore, so leaderboard gaps here reflect real maintainer acceptance rather than raw test-passing. Leading models still fail over 40% of tasks, and nine tasks went unsolved by every model evaluated. Teams choosing coding agents can weigh GPT-6.1 Sol's 49% score at US$0.81 per task against Claude Opus 5.5's 52% at far higher cost, with open-weights options like MiMo-V2.6-Pro at 45% close behind.

Event
官方账号Artificial Analysis@ArtificialAnlys11 sourcesSource
Oct 12
ModelsBuildersMulti-sourceSelected10:30
Adaptive DHiSparse Speculative Decoding

This approach solves a critical bottleneck in long-context AI serving by enabling more efficient memory usage while maintaining high throughput. The technology allows models to handle significantly larger contexts without hitting memory limits, which directly impacts real-world applications requiring extensive context windows. Developers working with large language models should consider this optimization technique to improve performance in production environments.

Event
arXiv: DeepSeek@Naji Alazhar, Wassel Alazhar6 sourcesSource
Oct 11
PapersBuildersSelected14:42
Internalizer maps context to parameters for DeepSeek V4-Flash

The shared-expert placement means one hypernetwork can steer every layer's routed experts at once, which sidesteps the usual per-layer adapter sprawl and could make context injection cheaper at inference time. Teams running Mixture-of-Experts models such as DeepSeek V4-Flash or Qwen variants should watch whether the mapping survives fine-tuning and long-context loads before adopting it over LoRA.

Teortaxes@teortaxesTexSource
PapersUS vs ChinaSelected04:29
Tsinghua, Oxford, Stanford Paper Finds LLMs Reason Even When Told Not To

The finding that a missing think tag or short reply does not mean a model skipped reasoning undermines a common evaluation shortcut. Teams benchmarking reasoning models or building answer-only constraints should re-check how compliance is measured, since suppressing visible reasoning costs roughly 15 accuracy points. The result spans 5 models, making it a broad behavioral pattern rather than a single-vendor quirk.

Event
rohanpaul_ai@rohanpaul_ai2 sourcesSource
How-toBuildersMulti-sourceSelected01:35
Open SWE routes tasks to cheaper models, cutting cost 64%

Cost-aware routing in the agent harness, rather than a single frontier model everywhere, is becoming a practical default for coding agents. Teams building multi-step agents can benchmark cheaper models per task type and keep premium models for escalation only. The DeepSeek V4.1 Flash versus Opus split shows a concrete tiering pattern worth copying.

Event
Harrison Chase@hwchase177 sourcesSource