After T9.1 you know whether an answer is usable: it matches the schema, the score is in range, and there is a reasoning. Whether it is
right, you don’t know yet. In this lesson you measure that with the library Microsoft.Extensions.AI.Evaluation. You write an evaluator of
your own that compares suitability and score with the test set, let a second model act as a judge of the reasonings, and in the end get a
table over all 20 cases.
The libraries
Microsoft.Extensions.AI.Evaluation builds on the types you have known since T2: ChatMessage, ChatResponse, IChatClient. It consists
of several packages, almost all of which you need in this module:
| Package | Version | What it brings |
|---|---|---|
Microsoft.Extensions.AI.Evaluation |
10.11.0 | the basic types: IEvaluator, EvaluationResult, metrics |
Microsoft.Extensions.AI.Evaluation.Quality |
10.11.0 | evaluators that ask a language model as a judge |
Microsoft.Extensions.AI.Evaluation.NLP |
10.11.0-preview.1.26513.7 | evaluators without a model that compare words |
Microsoft.Extensions.AI.Evaluation.Reporting |
10.11.0 | storing results, caching responses (T9.3) |
Microsoft.Extensions.AI.Evaluation.Console |
10.11.0 | the tool dotnet aieval for reports (T9.3) |
Microsoft.Extensions.AI.Evaluation.Safety |
10.11.0-preview.1.26513.7 | safety checks through a Foundry service (T9.4) |
Two of the packages are a preview. The lessons pin the version the examples ran with; with a newer preview the interface may still change.