The Kladde has two AI functions that read text you did not write. The assessment reads tenders from the TED interface, and the support chat reads the questions of the people who use it, plus the help pages that the search finds. Any of these texts can contain something that looks like an instruction. A language model cannot reliably tell whether a sentence is an instruction to it or just data it is supposed to talk about. That is what prompt injection is based on, the best-known attack on AI applications.

This module attacks the Kladde first and builds the defense afterward:

  • T6.1 (this lesson): what attacks look like, what an attacker wants, and a test set with the first five attacks.
  • T6.2: foreign text enters a message only fenced in as data, and the model's answer is treated like foreign text.
  • T6.3: the content filters and Prompt Shields of Microsoft Foundry, and how the code deals with a refusal.
  • T6.4: what must never happen is written in code; the attack test set runs with every build.

After that, the rule from the house rules applies: what must never happen is in the code, not in the prompt. Before the offer agent in T7 gets tools that change something in the world, this defense has to be in place.

What this module assumes of the Kladde

The examples use the names from the modules before. If something is named differently in yours, use your names:

  • The assessment from T2.3 and T3: TenderNotice (id, title, contracting authority, place, deadline, description) and TenderAssessor with AssessAsync(TenderNotice, CancellationToken) and the result AssessmentOutcome, plus the check rules TenderAssessmentRules. Since T3.3, the service gets the model, AiOptions, PromptCatalog, settings with the name of the office and the prompt version, and a TimeProvider; tests build it with PromptCatalog.Embedded and Options.Create. The tender stands in the last user message between <documents> and </documents>.
  • The support chat from T5: IHelpSearch returns sections of the help as HelpPassage(Id, Source, Title, Text, Score). SupportChat gets a search, SupportChatPrompt, IChatClient, SupportChatOptions, and a logger, and is asked with AskAsync(question, language, cancellationToken). Which version of its prompt it takes is in SupportChatOptions.PromptVersion; the structured answer came with v2 from T5.2. Its SupportAnswer has an outcome (SupportOutcome: Answered, NoHits, Declined, Withheld) and a text only for Answered; for every other outcome, the interface shows a fixed text and the way to the team (T5.3). The model answers in a structured way with answered, answer, and the ids of its sources in sources. The code accepts only ids that were really delivered, builds the links to the sources itself (T5.2), and strikes every sentence with a number or address that is in no source (ClaimCheck, T5.3).
  • The tests: tests/Kladde.Ai.Tests with the FakeChatClient from T2 and FakeHelpSearch and SampleHelp from T5.1, plus tests/Kladde.Ai.Evaluation from T9 with EvaluationPaths, EvaluationSettings (with Live for KLADDE_EVAL_LIVE=1, CreateClient, and the prompt version from evaluation.json) and the RecordingChatClient.