MIT 与 Meta 论文：ReHarness 用任务结果修正 agent 记忆笔记

Most memory systems judge lessons only when they are written, never after the outcome; ReHarness closes that loop by treating each finished trajectory as a review of the reflections that shaped it. Builders maintaining agent memory banks can apply the idea to reduce stale or overgeneralized lessons accumulating over long-running tasks. The evaluation spans three backbones, so the gains appear consistent across models rather than tied to one.

DAIR.AI@dair_aiSource

The practical takeaway is that expert granularity is the wrong unit for MoE compression — neuron-level pruning keeps skills intact while cutting memory by over a third. Teams running self-hosted MoE models on constrained GPU budgets, from inference startups to enterprise ML platforms, can apply RAMPr-style scoring to cut serving costs without retraining. The results give open-source MoE deployments a concrete alternative to expert-dropping methods.

rohanpaul_ai@rohanpaul_aiSource

The key shift is moving evaluation from fixed, easily saturated datasets to generated environments with ground truth derived from simulation logs, which makes skill-targeted benchmarking reproducible at scale. Teams building enterprise data agents or text-to-SQL systems can use the seven estates and 256 tasks to identify exactly where multi-event chaining breaks their pipelines. The finding that state-of-the-art agents still fail on chained tasks gives labs a concrete target for the next round of agentic improvements.

DAIR.AI@dair_aiSource

Static rankings on human-preference benchmarks may hide a moving baseline: voter tolerance is shrinking even when models are identical. Teams that track model quality via Chatbot Arena or run their own human evals should segment results by time period before drawing conclusions. The stability of relative rankings versus the rise in negative votes is also useful evidence for anyone designing preference-based leaderboards or RLHF data pipelines.

rohanpaul_ai@rohanpaul_aiSource

The counterintuitive finding is that models judge trajectories they cannot execute, making a strong evaluator a better reward teacher than the actor's own rollouts. Teams training small open models for web or embodied agents can plug a decision model into GRPO instead of self-critique and expect gains on hard tasks. A Chat with Paper walkthrough is available via dair.ai for those wanting a quick read of the method.

elvis@omarsar0Source