The support chat from T5.1 answers from the help, but nobody can see where an answer comes from. The user has to believe it, and your code can’t check anything. In this lesson, the chat names, for every answer, the sections it comes from. Those become links to the help pages, and your code checks every one of them first. At the end, the Kladde shows the chat with sources in the browser.

The examples ran with the same packages as in T5.1. For the interface, no package is added: it uses Blazor from K2, and Markdig for the help pages has been there since T4.3 through Microsoft.Extensions.DataIngestion.Markdig (as Markdig.Signed 0.43.0, MIT license).

What sources are for

A source citation has three readers:

  • The user. They can look up whether the answer is right, and on the page they often find more than fits into four sentences.
  • Your code. It can check whether the named sections were in the message at all. From T5.3 on, the Kladde doesn’t show an answer without a valid source any more.
  • Your test set. It can measure whether the chat names the right sections. That is a number you can compare when the prompt, the model or the search changes.

A source citation can be wrong in two ways:

  1. Invented: The model names an id it was never given, such as en-prices-cancellation-1, because such a page sounds plausible. Your code recognizes that reliably, and does so in this lesson.
  2. Wrongly assigned: The id exists, but the answer isn’t in that section. No simple comparison recognizes that. In T5.3, your code removes numbers and addresses that are in no source, and the GroundednessEvaluator measures the rest in the test set.