Evaluation and Alignment: The Seminal Papers (new book + 50% code)

Hi r/MachineLearning,

I'm Stjepan from Manning, and I'm posting on behalf of Manning with the mods' approval.

We’ve just released a book that focuses on a part of ML systems that tends to get less attention than model design, but ends up driving a lot of the hard decisions in practice: evaluation and alignment.

Evaluation and Alignment: The Seminal Papers by Hanchung Lee
https://www.manning.com/books/evaluation-and-alignment-the-seminal-papers

Evaluation and Alignment, The Seminal Papers

A lot of current work in LLMs and applied ML ends up circling the same set of questions: what does “good” actually mean for this system, how do we measure it, and what do we do when the metrics don’t match user expectations? This book approaches those questions by going back to the research that shaped how we evaluate and adapt models.

It walks through the progression from surface-level metrics to semantic similarity approaches and then into more judgment-based evaluation methods. The interesting part is how those ideas connect to real system design. Evaluation is treated as something you define upfront, based on what your system needs to get right, rather than something you tack on at the end.

The book also introduces a working cycle that shows up a lot in production settings: define what matters, evaluate against it, analyze failures, and then align the system accordingly. That loop is where most of the practical work happens, especially when you’re balancing things like helpfulness, safety, and consistency of outputs.

If you’ve ever had a model that looked good on paper but didn’t behave the way you expected in practice, this book spends time in that gap between metrics and behavior.

For the r/MachineLearning community:
You can get 50% off with the code MLLEE450RE.

If there’s interest, I’d be happy to invite the author to join the discussion and answer questions about the papers and evaluation approaches covered in the book.

Thanks for having us here.

Cheers,

Stjepan