In this track, the Kladde gets features that talk to people, generate texts and prepare decisions. Before the first of them is built, this lesson settles two questions: What can go wrong, and what does the law require? You answer the first with a procedure that Microsoft describes for its own teams. For the second you learn the basics of the EU’s AI Act, as far as they apply to an application like the Kladde.
A preliminary remark: this lesson is teaching material, not legal advice. You learn to ask the right questions and to document your answers. Whether a particular application meets an obligation is, in case of doubt, for a lawyer to judge.
Six principles
Microsoft bases the development of AI on six principles. They aren’t law, but they make a good checklist, because every principle asks a question of your feature:
| Principle | Question to a feature of the Kladde |
|---|---|
| Fairness | Does it treat equal cases equally, for example a request in English as well as one in German? |
| Reliability and safety | Does it do the right thing even with unusual input, and does it fail safely? |
| Privacy and security | Does the model get only the data it needs, and can nobody extract other people’s data? |
| Inclusiveness | Can everyone use it, including with a screen reader, including with little experience? |
| Transparency | Do people know that an AI is answering, what it can do and where its answer comes from? |
| Accountability | Who is responsible when it says something wrong, and who can step in? |
Finding, measuring and mitigating harms
Microsoft’s guidance on Foundry organizes the work on risks in three steps, modeled on its own Responsible AI Standard:
- Discover: Which harms can the feature cause, and how often do they occur? That includes targeted attacks, such as inputs meant to push the model into something forbidden. What you find, you measure with a test set (T9) and with attacks (T6).
- Protect: Countermeasures at the model and in the application: content filters, rules in code, approvals by humans.
- Govern: Watch operation, with traces, metrics and alerts, so that new risks stand out after launch (T10).
For you as a developer, this becomes a work step that you repeat for every AI feature: a list of possible harms, for each harm a countermeasure, and a way to check that it works. You write this list for the support chat in exercise 1.
The AI Act in outline
Regulation (EU) 2024/1689, the AI Act for short, regulates AI systems in the EU. It sorts them by their risk and distributes obligations among roles.
The roles. The provider develops an AI system and places it on the market under its own name or puts it into service. The deployer uses
an AI system under its own responsibility. An IT office that builds the Kladde itself and uses it itself is both for its support chat. Whoever makes
the models themselves, here for example the providers of gpt-5.4-mini or qwen2.5-0.5b, has obligations of its own as the provider of a
general-purpose AI model; those don’t concern you if you merely use such a model.
The risk classes.
| Class | What falls under it | Consequence |
|---|---|---|
| prohibited practices (Art. 5) | for example manipulative techniques that harm people, scoring people by their social behavior, emotion recognition in the workplace and in education | may not be used |
| high risk (Art. 6 with Annex I and III) | AI as a safety component of certain products (Annex I) and AI in certain areas (Annex III), for example in selecting job applications, access to education, creditworthiness, critical infrastructure | strict obligations: risk management, data, documentation, human oversight, accuracy |
| transparency obligations (Art. 50) | AI that talks to people or generates content | people must be able to recognize it |
| low risk | everything else | no special obligations under the regulation |
The classes don’t exclude one another: a high-risk system that talks to people also has the obligations from Art. 50.
What applies. The prohibitions and the obligations on AI literacy (Art. 4) already apply, and so do the obligations for general-purpose AI models and the transparency obligations from Art. 50. The obligations for high-risk systems have been postponed by an amending regulation, the “Digital Omnibus on AI”. The same omnibus reworded Art. 4: providers and deployers take measures to promote the AI literacy of their people. It also gives providers of generative systems that were already on the market before Art. 50 began to apply a transition period of four months for the marking under Art. 50(2). Read the exact deadlines in the regulation and in the omnibus; they may change again.
Where does the Kladde stand? The assessment of tenders supports an office’s decision about which contracts to bid on. That’s not an area from Annex III. The support chat talks to people and falls under Art. 50. If the Kladde instead assessed applications for a job, it would be a high-risk system under Annex III. The difference lies not in the technology but in the purpose.
Article 50: people must notice
Art. 50 contains the obligations that concern the Kladde most. In our own words:
- Whoever talks to an AI must know it. Providers design AI systems that interact directly with people so that these people are informed that they’re talking to an AI, unless it’s obvious from the circumstances.
- Generated content is marked in a machine-readable way. Providers of systems that generate text, images, audio or video make sure that the outputs are marked in a machine-readable format as artificially generated, as far as technically feasible. There are exceptions for assistive functions such as ordinary editing.
- Emotion recognition and biometric categorization must be disclosed to the people affected.
- Deepfakes are disclosed by deployers. So are texts that are meant to inform the public on matters of public interest, unless a human has reviewed them and takes editorial responsibility.
- When and how: at the latest at the first interaction or exposure, clearly and distinguishably, and accessibly.
Exactly what a machine-readable marking should look like is described by codes of practice that the Commission’s AI Office encourages (Art. 50(7)). For the Kladde this already means today: every answer of the chat carries a marker in its data that an AI generated it, and the page passes this marker on. Which form is enough belongs on the list of questions for the lawyer.
The rule is in the code
You could put the notice about the AI into the prompt: “Point out at the start that you are an AI.” That would be a wish addressed to the model, and you know from T1.1 how reliable wishes to a model are. The Kladde’s house rules therefore say: what must never happen is in the code. What must never happen here is that an answer appears without the notice.
// notice.cs – the notice about the AI is in the code: no answer before the notice.
var chat = new SupportChat();
try
{
chat.Answer("Choose a free time on the \"Appointments\" page.");
}
catch (InvalidOperationException e)
{
Console.WriteLine($"Rejected: {e.Message}");
}
Console.WriteLine(chat.Start());
ChatMessage answer = chat.Answer("Choose a free time on the \"Appointments\" page.");
Console.WriteLine($"{answer.Text} (AI: {answer.AiGenerated})");
// A conversation in the support chat. The rule is here, not in the prompt: the notice comes before the first answer.
sealed class SupportChat
{
private bool noticeShown;
public string Start()
{
noticeShown = true;
return "You are chatting with an AI.";
}
public ChatMessage Answer(string modelText)
{
if (!noticeShown)
{
throw new InvalidOperationException("The notice about the AI must be shown before the first answer.");
}
return new ChatMessage(modelText, AiGenerated: true);
}
}
// Every message carries, in a machine-readable way, whether an AI generated it.
sealed record ChatMessage(string Text, bool AiGenerated);
dotnet run notice.cs
Rejected: The notice about the AI must be shown before the first answer.
You are chatting with an AI.
Choose a free time on the "Appointments" page. (AI: True)
The text of the answer is fixed here; in the real chat it would come from the model. The class doesn’t care where it comes from. It enforces two
rules: before the notice there is no answer, and every answer carries AiGenerated. In the support chat from T5 this becomes an attribute in the
HTML and a field in the interface’s answer, and both rules get a test there.
No personal data without a legal basis
A prompt is a transfer of data. If it contains a name, an email address or a customer’s question, you’re processing personal data, and the General Data Protection Regulation (GDPR) applies, locally as in the cloud. Four rules for the Kladde follow:
- A legal basis for every purpose. Every processing needs a basis under Art. 6 GDPR, such as performance of a contract or a legitimate interest. Which one applies is recorded in the record of processing activities and isn’t decided while programming.
- Only what’s necessary. Assessing a tender needs no contact data. What the model doesn’t need doesn’t go in.
- Know where the data goes. In the cloud, Microsoft processes the data on your behalf; Microsoft’s data protection terms cover that, and the deployment type determines the place of processing (T1.3). Locally, the data doesn’t leave the computer.
- Say what happens. The Kladde’s privacy notice names the chat, its purpose and the place of processing, and the notice in the chat asks people not to enter sensitive details.
For this course the house rules also apply: no real people, companies or customer data to a model, not even a local one. All test data is invented.
Exercise 1: The support chat’s risks
From T5 on, the Kladde’s support chat is to answer questions from the IT office’s customers, for example about booking appointments, and only from its own help pages. Write a list of at least eight possible harms. For each harm, note:
- which of the six principles it touches,
- a countermeasure, preferably one in code,
- how you check that the countermeasure works,
- in which lesson of the track the countermeasure is built, if you can already say.
How to check your list: each of the six principles appears at least once, and at most two countermeasures say “it’s in the prompt”.
Solution 1
One possible list. Yours may look different, as long as the countermeasures can be checked.
| Harm | Principle | Countermeasure | Check | Lesson |
|---|---|---|---|---|
| The chat invents an answer that is on no help page. | Reliability and safety | Answer only from retrieved sections; with no match, a fixed answer without the model | Test set with questions the help doesn’t answer | T5.1, T5.3 |
| The chat names a source that doesn’t exist. | Transparency | The code checks every named source against the sections supplied | Test with a fake client that returns an invented source | T5.2 |
| The chat states a number, an amount or an address without a source. | Reliability and safety | The code removes numbers, amounts and addresses that are in no source | Test with an answer full of invented numbers | T5.3, T6.4 |
| Someone takes the chat for a human. | Transparency | Notice before the first answer, enforced in code; every answer marked as an AI answer | Test: no answer without the notice | T1.4, T5 |
| Someone enters health or banking data. | Privacy and security | Notice in the chat; no storing of the conversation beyond its purpose; no prompts in logs | Check the logs for content, test the deletion | T2.4, T7.5 |
| A prepared help page makes the chat follow instructions. | Privacy and security | Foreign text enters the message only fenced off, as data; rules in code | Attack test set in the build | T6 |
| The chat promises a refund or gives legal advice. | Accountability | For refunds, legal questions and a locked account, the code hands over to a human | Test set with such questions | T6.4 |
| English questions get worse answers than German ones. | Fairness | Help pages in both languages; test set in both languages | Compare the hit rate per language | T5, T9 |
| The chat can’t be used with a screen reader. | Inclusiveness | Live region for new answers, keyboard operation, notice as text, not only as an icon | Check with a screen reader and an automated accessibility check | T2.2 (with K2 2.5.13) |
| A success of the chat lets the cost rise unnoticed. | Accountability | Monthly cap; at the cap the chat shows the form | Test across the month boundary | T10.3 |
| A local model produces abusive answers. | Reliability and safety | in the cloud guardrails; locally no filter, so test set and attacks before any operation | Attack test set | T6.3 |
What stands out is how little of this is in the prompt. A prompt can help, but it replaces none of the checks. That’s why the track spends so much time on test sets, attacks and rules in code.
Exercise 2: The notice in both languages
The Kladde is bilingual, so the chat needs the notice in German and in English. Extend notice.cs:
- The notice says more than “You are chatting with an AI”: that no human is answering, where the answers come from, that they may contain mistakes, which details not to enter and how to reach a human. At most five short sentences.
SupportChatgets the language in its constructor. For a language without a notice, the chat doesn’t start.- The program prints both notices and shows that a third language is rejected.
How to check it: read both notices aloud. Does someone without prior knowledge understand that a machine is answering and how to reach a human?
Solution 2
// notice.cs – the notice about the AI in both languages of the Kladde, before the first answer.
foreach (string language in new[] { "de", "en" })
{
var chat = new SupportChat(language);
Console.WriteLine(chat.Start());
Console.WriteLine();
}
try
{
_ = new SupportChat("fr");
}
catch (ArgumentException e)
{
Console.WriteLine($"Rejected: {e.Message}");
}
// A conversation in the support chat. The rules are here, not in the prompt:
// without a notice in the chat's language it doesn't start, and the notice comes before the first answer.
sealed class SupportChat
{
private static readonly Dictionary<string, string> Notices = new()
{
["de"] = "Sie schreiben mit einer KI, nicht mit einem Menschen. Die KI antwortet nur aus unseren Hilfeseiten und nennt "
+ "die Seite, aus der die Antwort stammt. Die Antworten können Fehler enthalten. Geben Sie bitte keine Passwörter, "
+ "Bankdaten oder Gesundheitsdaten ein. Einen Menschen erreichen Sie über das Kontaktformular.",
["en"] = "You are chatting with an AI, not with a person. It answers only from our help pages and names the page "
+ "its answer comes from. Its answers may contain mistakes. Please do not enter passwords, bank details or health "
+ "information. You can reach a person through the contact form.",
};
private readonly string notice;
private bool noticeShown;
public SupportChat(string language)
{
notice = Notices.TryGetValue(language, out string? text)
? text
: throw new ArgumentException($"There is no notice about the AI for the language '{language}'.", nameof(language));
}
public string Start()
{
noticeShown = true;
return notice;
}
public ChatMessage Answer(string modelText)
{
if (!noticeShown)
{
throw new InvalidOperationException("The notice about the AI must be shown before the first answer.");
}
return new ChatMessage(modelText, AiGenerated: true);
}
}
// Every message carries, in a machine-readable way, whether an AI generated it.
sealed record ChatMessage(string Text, bool AiGenerated);
Sie schreiben mit einer KI, nicht mit einem Menschen. Die KI antwortet nur aus unseren Hilfeseiten und nennt die Seite, aus der die Antwort stammt. Die Antworten können Fehler enthalten. Geben Sie bitte keine Passwörter, Bankdaten oder Gesundheitsdaten ein. Einen Menschen erreichen Sie über das Kontaktformular.
You are chatting with an AI, not with a person. It answers only from our help pages and names the page its answer comes from. Its answers may contain mistakes. Please do not enter passwords, bank details or health information. You can reach a person through the contact form.
Rejected: There is no notice about the AI for the language 'fr'. (Parameter 'language')
The German notice stays German on purpose: it’s the text that the Kladde’s German-speaking users see, and the Kladde is bilingual (German and English). The dictionary is here only so the file-based app runs on its own. In the Kladde, the texts belong in the resource files, like all other texts for users since K2 2.5.12, and the rule “no notice, no start” is then checked by the test from K2 that demands both languages for every resource.
Exercise 3: Assigning risk classes
Assign these invented systems to a risk class of the AI Act and justify each assignment in one sentence:
- The assessment of tenders in the Kladde.
- The support chat of the Kladde.
- A tool that reads incoming job applications for a company and suggests a ranking of the applicants.
- A camera in the office that detects employees’ mood from their faces and reports it to management.
- A generator that turns one sentence into a deceptively real photo of a well-known politician.
Solution 3
- Low risk. The assessment prepares an office’s decision about which contract to bid on; that’s not an area from Annex III, and the system holds no conversation with people. Its reasons are, however, text that an AI generated. Whether the machine-readable marking under Art. 50(2) applies to them belongs on the list for the lawyer; the Kladde marks them anyway. The principles still apply: a human decides, and the assessment names its reasons.
- Transparency obligation under Art. 50. The chat interacts directly with people; they must be informed that they’re talking to an AI, and its answers are to be marked in a machine-readable way.
- High risk. Selecting applications is an area from Annex III (employment). The strict obligations apply as soon as they become applicable.
- Prohibited. Emotion recognition in the workplace is one of the prohibited practices under Art. 5, except for medical or safety reasons.
- Transparency obligation under Art. 50. The provider must mark the image in a machine-readable way as artificially generated, and whoever publishes it as a deepfake must disclose that it’s generated. Depending on the use, other laws may apply, such as those protecting personality rights.
The assignment depends on the purpose, not on the technology: the same language model is in case 1 and in case 3.
The Kladde: risk list and notice texts
Two additions in your Kladde repository, as one pull request:
docs/ai-risks.md with your list from exercise 1 and the assignment from exercise 3 for the Kladde’s three features:
# AI risks
As of: <date>. Teaching material, not legal advice; open legal questions are below.
## Classification under the AI Act
| Feature | Role of the Kladde | Class | Justification |
| --- | --- | --- | --- |
| Tender assessment | provider and deployer | low risk | ... |
| Support chat | provider and deployer | transparency obligation (Art. 50) | ... |
| Offer agent | provider and deployer | ... | ... |
## Risks of the support chat
| Harm | Principle | Countermeasure | Check | Lesson |
| --- | --- | --- | --- | --- |
| ... | ... | ... | ... | ... |
## Open questions for the lawyer
- Which form of machine-readable marking is enough for the chat’s answers?
- Does the marking under Art. 50(2) also apply to the assessment’s reasons, which only the office reads?
- Which legal basis supports processing the questions in the chat?
The notice texts in the resource files that your Kladde has kept for German and English since K2 2.5.12, for example in a resource
SupportChat for the chat that comes in T5. In the file for German:
<data name="AiNotice" xml:space="preserve">
<value>Sie schreiben mit einer KI, nicht mit einem Menschen. Die KI antwortet nur aus unseren Hilfeseiten und nennt die Seite, aus der die Antwort stammt. Die Antworten können Fehler enthalten. Geben Sie bitte keine Passwörter, Bankdaten oder Gesundheitsdaten ein. Einen Menschen erreichen Sie über das Kontaktformular.</value>
</data>
In the file for English:
<data name="AiNotice" xml:space="preserve">
<value>You are chatting with an AI, not with a person. It answers only from our help pages and names the page its answer comes from. Its answers may contain mistakes. Please do not enter passwords, bank details or health information. You can reach a person through the contact form.</value>
</data>
Then run the Kladde’s tests. The test from K2 that demands both languages for every resource must keep passing, with exit code 0 from dotnet test.
That ends the module. The module exam asks about tokens, costs and model choice and has you build a small cost calculation against tests.
What you can do now
- You know Microsoft’s six principles and use them to ask questions of a feature.
- You work through risks following Discover, Protect, Govern: find, protect, govern.
- You explain the roles and risk classes of the AI Act and classify a system by its purpose.
- You know the obligations from Art. 50 and enforce the notice about the AI in code, not in the prompt.
- You send no personal data to a model without a legal basis and record open legal questions for the lawyer.
Read more
- Responsible AI for Microsoft Foundry: Discover, Protect, Govern for agents and AI applications.
- Implement a responsible generative AI solution in Microsoft Foundry: a module on Microsoft Learn with exercises on finding and mitigating harms.
- Responsible AI with .NET: the six principles and the safety evaluators you meet in T9.
- Regulation (EU) 2024/1689: the text of the AI Act, Art. 50 and Annex III.