The support chat now names its sources. But it still answers everything: if the search finds nothing, it asks the model with an empty fence, and an answer without a single valid source it shows without a link. In this lesson, it learns to say no. The rule is simple: a question that the help doesn’t answer gets no answer, but a way to a person. Who decides whether that’s the case is the more important part: your code, not the model.
The examples ran with the packages from T5.1, plus two that the evaluation project has had since T9 for the test set:
| Package | Version |
|---|---|
Microsoft.Extensions.AI.Evaluation.Quality (since T9.2) |
10.11.0 |
Microsoft.Extensions.Hosting (since T9.1) |
10.0.14 |
Why no is the better answer
An invented answer does harm twice. The user acts on it, for example with a deadline that doesn’t exist. And they learn that the chat is sometimes wrong, so they don’t believe it even when it’s right. An honest “The help says nothing about this, our team will help you”, on the other hand, costs only a message to support. For the Kladde, that means:
- Better to hand an answerable question to a person than to invent an answer for an unanswerable one. In T5.1, you chose the threshold rather too high for this reason.
- Whether an answer is shown is decided by the code, by rules that can be read and tested. The prompt asks the model for the same thing, but a request is not a rule. The house rules put it this way: what must never happen is in the code.