Papers00:51
Chinese contentStanford HAI 分析 25 万条对话：多数用户把 AI 当学习工具而非捷径斯坦福团队扒了 25 万条真实对话，发现大家用 AI 不是偷懒，是学习。数据挺有意思，做产品的可以看看。
Latest
Every published item, in order · 1470 items
Oct 13
ANCHOR Benchmark Measures Persona Collapse in AI Companions
Memory settings that look fine in short chats fail when a companion must track a relationship over months, so persistence claims in companion apps deserve scrutiny. Builders of consumer AI products can use ANCHOR to benchmark state tracking before shipping retention features. The gap between event recall (44.4%) and situation-change tracking (21–25%) suggests temporal memory, not raw context length, is the real engineering bottleneck.
Oct 12
Nvidia paper scores base models by first passing edit
Base-model choice before expensive post-training has lacked a reliable signal, and this proxy gets it at 0.964 correlation with SWE-bench Verified outcomes. Teams running RL or SFT on coding agents can apply it with existing benchmark traces instead of full fine-tuning trials. It compares directly with running standard agent benchmarks, which barely predict downstream agent skill on base models.
Paper Tests Multi-Agent Systems With Persistent Memory on Long-Context Work
The finding suggests viral multi-agent memory demos may not transfer to standard evaluation setups, since benchmarks are structured in ways that neutralise recall benefits. Engineers deciding whether to add persistent memory to agent pipelines should weigh the near-zero measured accuracy gain against its efficiency advantages. The paper's benchmark-design critique is a useful checklist when validating similar architectures.
MiniCorp Office Simulator Generates Enterprise Data for Agents
The checkpoint-and-replay design lets researchers run counterfactual experiments on identical market states, which no real business dataset supports. Teams building or benchmarking agents for long-running business workflows can use MiniCorp to generate controlled training and evaluation data. Compared with hand-curated agent benchmarks, it offers a closed loop where decisions have measurable market consequences.
Google Publishes Agent Tools & Interoperability Whitepaper
The non-obvious takeaway is the paper's stance that expert agents should be collaborators, not tools — forcing a peer agent into a tool wrapper is like introducing a GOTO statement. Developers building multi-agent systems in production can use its protocol-by-protocol guidance to pick the right integration layer, from MCP tool calls to AP2-secured payments.
Double Ratchet evolves agent graders from 10 labeled examples
The non-obvious result is that removing safeguards lets the grader rubber-stamp outputs while losing most of the training signal, so the 10-example check is what preserves real skill gains. Teams building self-improving or RL-style agent loops without a budget for human grading can apply this to automate evaluation. The HSBC involvement hints at finance-grade use cases where graded outputs like SQL and reports matter.
Gary Marcus defends symbolic verification against 'basic' dismissal
The post reframes a common dismissal in AI engineering circles by grounding it in the actual history of proof theory and type checking. Practitioners designing coding agents or theorem-proving pipelines may find the co-design framing useful when deciding how much investment goes into the harness versus the model itself. The SWE-bench sensitivity point is a concrete argument to cite in benchmark design debates.
Stanford CS329A: Self-Improving AI Agents Course Overview Notes
The takeaway is that small models can beat larger ones when given enough parallel samples and a verifier, which reframes inference-time compute as the practical lever for builders without trillion-parameter budgets. Engineers designing agent pipelines should note the coverage-versus-pass@k distinction when choosing between repeated sampling and reasoning models like o1. Stanford publishing the full lecture makes a frontier-lab curriculum freely accessible to self-taught practitioners.
Tomorrow 08:00, the daily lands in your inboxTelegram channel or email. Same screening as this page, no ads.
UC San Diego Paper Adds Theory of Mind to 7B Agent
The reward-shaping trick means theory of mind gets baked into the weights rather than bolted on at inference, so negotiation and cooperation gains come with zero added serving cost. Agent builders working on multi-agent negotiation or role-play can borrow this training recipe directly. Compared with the usual memory-summary heuristics, an explicit partner-model reward targets the exact failure mode of misreading intent.
Papers16:42
Chinese content研究发现近似 SGD 的生物学可行机制，bio-backprop 无需硬编码讨论了生物合理反向传播的论文，说近似 SGD 的机制可以自己涌现出来，不用硬编码，概念挺有意思的。
JetBrains Paper Introduces SWE-TaskFlow Benchmark for Coding Agents
The finding that multi-turn benchmarking doubles cost without improving success rates gives teams a concrete cost signal when designing agent evaluations. Anyone building or selecting coding agents should audit their own session logs first, then mirror that mix of questions, planning and edits in benchmarks. The paper is on arXiv (2610.09633), so the methodology is freely available for replication.
Oct 11
Meta Superintelligence Labs GitSwarm
The research demonstrates how persistent memory enables agents to build on previous work, with 94.7% of later contributions referencing prior commits and 70.1% of failed commits being cited by accepted work. Developers working on complex multi-agent systems can implement this approach to improve problem-solving efficiency and reduce redundant computation.
Tsinghua AI Agents Study Game Bots
The research reveals a critical limitation: AI agents excel with detailed replays but struggle with complex rules, even with increased resources. This has practical implications for game developers and AI researchers in Asia, where game development is a significant industry. The findings suggest that while AI can analyze past performances effectively, it still lacks the adaptability for rule-intensive environments without human-like reasoning capabilities.
Amazon paper: LLM predicts training gains before GPU runs
The result suggests experiment history matters more than reasoning budget: low-effort runs with records beat max-effort runs without them, so teams may get better experiment selection by curating logs than by paying for longer inference. Groups running automated research agents or hyperparameter sweeps can start by standardising failure logging now, since the gain depends on records from the same setup rather than general model strength.
Meta paper: GitSwarm agents share Git repo, hit 79.4% on ProgramBench
The result suggests that persistent shared state, not more compute per run, is what lifts multi-agent performance, since GitSwarm beat a single Codex agent at comparable cost by preserving failures that isolated runs discard. Teams building coding agents or evaluation harnesses can test this by giving parallel runs a common commit history instead of separate sandboxes. It also offers a concrete benchmark reference, ProgramBench, for comparing swarm-style orchestration against single-agent baselines.
Chinese contentPangram 漏检 79.8% 的 Muse-Glimmer 改写论文摘要
同一篇摘要换个模型改写，Pangram 的漏检率就从 6.5% 跳到 79.8%，说明检测器认的是模型指纹而不是 AI 痕迹。
Chinese content研究：harness自进化在DeepPlanning任务上收益有限
有人拿DeepPlanning这个多日旅行规划基准来讨论：harness自己改自己，到哪一步就不管用了。做agent评测的可以看看这个思路。
Internalizer maps context to parameters for DeepSeek V4-Flash
The shared-expert placement means one hypernetwork can steer every layer's routed experts at once, which sidesteps the usual per-layer adapter sprawl and could make context injection cheaper at inference time. Teams running Mixture-of-Experts models such as DeepSeek V4-Flash or Qwen variants should watch whether the mapping survives fine-tuning and long-context loads before adopting it over LoRA.
MemoType paper routes agent memory by type, lifts Recall@1
The upper-bound proof matters more than the 16.18% gain: it says no amount of embedding tuning fixes a mixed-type memory store, so teams should budget for classification and per-type indexes before scaling agent memory. Anyone building long-running agents with vector databases can test this by labelling existing memories and comparing a type-routed retriever against their current single-index baseline.
HarnessSQL Training Lifts Qwen3 on Spider 2.0-SQLite
The core insight is that evaluation harnesses create a train-deploy mismatch: models trained to emit one static query underperform when deployed agents must probe schemas and revise queries interactively. Teams building agentic SQL or tool-use systems should consider matching training and inference environments rather than adding capabilities at fine-tuning time. The method uses open Qwen models, so results are reproducible without proprietary systems.
Stanford paper splits agent failures between harness fixes and fine-tuning
The actionable shift is diagnostic: the same failure mode responds to different interventions, so prompt and tooling edits can waste effort on problems only weight training solves. Teams building agents can use the process-versus-content taxonomy as a triage step before committing compute. The Qwen3.5 LoRA numbers give a concrete baseline for how much fine-tuning moves plan quality on a small model.
Tsinghua, Oxford, Stanford Paper Finds LLMs Reason Even When Told Not To
The finding that a missing think tag or short reply does not mean a model skipped reasoning undermines a common evaluation shortcut. Teams benchmarking reasoning models or building answer-only constraints should re-check how compliance is measured, since suppressing visible reasoning costs roughly 15 accuracy points. The result spans 5 models, making it a broad behavioral pattern rather than a single-vendor quirk.
Sakana AI Researcher Discusses Digital Red Queen Evolution Work
The interview connects Sakana's evolutionary research to its broader thesis on agentic AI, suggesting arms-race dynamics could drive continual improvement without human-labelled data. Researchers interested in open-endedness, automated search or agent self-improvement can hear the reasoning directly rather than reading only the paper. Core War offers a concrete, low-cost testbed others can replicate.
Sakana AI paper proposes MASS for recursive self-improvement
The non-obvious idea is that multi-agent self-supervision can replace the external verifier that most self-improvement loops depend on, extending the approach to open-ended tasks with no ground-truth checker. Researchers working on self-improving agents and efficient training should read the paper and the accompanying Chat with Paper walkthrough to see whether the workflow-search setup transfers to their own models.
Meta's IdeaScientist: 27B open model tops autoresearch baselines
Splitting ideation into three RL-trained roles rather than one monolithic agent appears to be what drives the novelty gains, a design choice others building AI-scientist pipelines can copy with a 27B open model. Builders of autonomous research or agent workflows should compare it against their Claude Code or Codex SDK stacks. The paper and chat-with-paper links are public, so the evaluation methodology can be inspected directly.
Oct 10
Chinese contentEpoch AI 数据显示数学论文中 AI 使用率超 35%
Epoch AI 扒了 5.6 万多篇 arXiv 数学论文，发现承认用 AI 的比例从个位数涨到 35%，还细分了用在哪一环。
Tsinghua's TokenRouter serves token-level LLM routing at 64.15x throughput
Token-level routing turns model cascades from a latency trick into an infrastructure decision: serving small and large models on separate servers only pays off if the KV cache survives the handoff, which is what TokenRouter engineers. Teams running mixed-model inference on vLLM or SGLang can use the 2.01x-64.15x range to judge whether their traffic mix justifies a rewrite, and API providers billing per token have the clearest margin case.
NYU and Amazon skill selection paper
This research reveals that quality trumps quantity in skill distillation, potentially reducing computational costs by over 90% while maintaining performance. For developers and researchers working with large language models, this method offers a practical way to optimize training efficiency and improve model performance on specific tasks like math contests.
Meta Superintelligence Labs Agent Plasticity Study
The research reveals that the best-performing models aren't always the most efficient learners, challenging assumptions about AI self-improvement. Developers and researchers can use agent plasticity metrics to optimize AI training investments, particularly for complex tasks requiring artifact reuse. The findings have practical implications for resource allocation in AI development across industries.
Microsoft CABRA paper links coding agent failures to code reading
The practical shift is in evaluation: diff size is a weak proxy for agent difficulty, so teams benchmarking coding agents should instrument and measure reading-heavy workloads instead. Engineering leads choosing between agents can rerun their evals with CABRA-style isolated difficulty ramps to see whether a candidate degrades on comprehension rather than editing volume. The reading-versus-editing distinction also suggests where tooling like grep integration pays off most.
AI use exceeds half of arXiv papers in 3 math subfields
AI adoption in mathematics is measured here through paper acknowledgments rather than surveys, giving a harder signal of how embedded these tools have become in research workflows. Mathematicians, journal editors and research funders should watch which subfields cross the majority threshold next. Differential geometry's jump from 8% to 57% in two months is among the fastest adoption shifts documented in academic publishing.
