Nathan Lambert Adds Score-Centering to the RLHF Book
Score-centering moving from a niche trick to an industry default signals how RL post-training recipes are converging, and the book chapter offers a derivation that clarifies why training-inference mismatch hurts policy optimisation. Practitioners implementing GRPO-style or StableReinforce-style pipelines can use it to stabilise advantage estimates. It pairs well with the book's existing chapters on reward modelling and policy gradients.
