In module T4, you cut the Kladde’s help pages, embedded them and made them searchable in PostgreSQL. A search returns sections, but no answer. In this module, that becomes a support chat that answers questions about the Kladde, and only from those pages. This lesson builds the first version. The next one has it name its sources, the third declines where the help is silent, and the fourth improves the search itself.
The pattern is called retrieval-augmented generation, RAG for short: first you search (retrieval), then the model gets the finds along with the question (augmented) and writes the answer from them (generation). An answer that rests only on such finds is called grounded. This lesson’s examples ran with these packages:
| Package | Version |
|---|---|
Microsoft.Extensions.AI |
10.11.0 |
Microsoft.Extensions.VectorData.Abstractions |
10.11.0 |
Microsoft.ML.Tokenizers, Microsoft.ML.Tokenizers.Data.O200kBase |
2.0.1 |
CommunityToolkit.VectorData.InMemory (only in the tests) |
1.0.1 |
Microsoft.Extensions.Hosting (only in the file-based app) |
10.0.14 |
xunit.v3.mtp-v2 |
4.0.2 |
Why the model doesn’t know the Kladde
Ask the local model from T2 “How do I call off an appointment in the Kladde?” and you get a fluent answer. It sounds like instructions, with menu items and a deadline. But none of it comes from the Kladde: the model has never seen it. It continues what usually follows such a question (T1.1). For a support chat, that’s worse than no answer at all, because whoever believes it looks for a menu item that doesn’t exist or misses a deadline that is different.
Grounding turns that around. The model isn’t supposed to know, but to read: your code searches for the matching sections of the help, puts them in front of the model and demands that the answer come only from them. That turns the question “Does the model know this?” into two questions you can answer yourself:
- Did the search find the right sections?
- Did the model answer only from them?
Your test set for the search (T5.4) answers the first question; your code (T5.2 and T5.3) and the test set with the
GroundednessEvaluator (T5.3) check the second. A model that knows nothing about the Kladde is even good for this: it can’t rely on
knowledge it doesn’t have.