A prompt is everything a language model receives as input. The quality of the answer depends heavily on how good this input is. This lesson shows what a good prompt consists of, and practices it on tasks of the made-up Nordlicht Reisen GmbH. The playground from lesson 2.1 is for trying things out. The lesson is pure text work and needs no access to Azure.

The answers of a model are not printed in this lesson. They would be made up. What a model answers to a prompt, you try out yourself; here you learn how to ask.

Roles in a conversation

According to Learn, chat models receive a series of messages with roles:

  • system: instructions for the whole conversation. The system message normally stands at the very beginning and, according to Learn, is the top level of the instructions. People also say system prompt.
  • user: what the user writes (the user prompt).
  • assistant: what the model has answered so far. You can also insert examples of your own as assistant messages (see few-shot below).

The split has a purpose: The system prompt determines how the model behaves always. The user prompt contains what is to be done now. What this looks like in C# is shown by the AI track (T2.1, "The IChatClient"); it goes deeper into the prompts themselves in T3.1 ("System prompts") and T3.2 ("Examples in the prompt"). Here it stays with the basics.

What a system prompt can do

According to Learn, a system message serves to

  1. set the role and the limits of the assistant,
  2. determine tone and language style,
  3. name the output format, for example JSON,
  4. set safety and quality rules for its use.

It can be a single sentence or many lines of rules. Learn gives three hints: keep instructions unambiguous, avoid contradictory rules and state the behavior for the emergency explicitly, that is, what the assistant does when information is missing or the question does not belong to its task.

And a warning: A system message influences the model, but does not guarantee compliance. You have to test, improve and put further safeguards around it, such as filters and evaluation. It is the same idea as in lesson 1.1: A rule in the prompt never replaces a human check.

Example: system prompt for reise-chat

Nordlicht wants the model behind reise-chat to prepare customer requests. A possible system prompt (an example, not a template to copy):

You are an assistant in customer service at Nordlicht Reisen GmbH.
Task: Answer questions about trips, hotels and booking processes in a factual way.
Language: Answer in the language of the request (German or English).
Tone: friendly, short, no technical terms.
Limits: You never decide on refunds, cancellations or contract questions.
For such questions you write: "An advisor will clarify that."
Way out: If you lack information, say so and do not make anything up.

Every line follows Learn's list: role, task, tone, limit, way out. The limit matches the pilot's limits: No model decides on refunds, cancellations or contract questions. The prompt does not enforce this limit, it asks the model to keep it. That a person checks every answer before it goes to customers stays true all the same (lessons 1.1 and 2.4).

Craft for the user prompt

The following techniques come from Learn's page "Prompt engineering techniques". Learn expressly calls them techniques to try out, not guarantees.

Clear instruction first

Say at the beginning what the model is to do, and only afterwards the context or the examples. Learn calls this generally advisable, but notes that in one example the tested chat models showed no difference. So check it on your case.

Separate the context cleanly

With clear syntax, such as delimiters like --- and headings in capital letters, you show where one part of the prompt ends and the next begins. When unsure, Learn advises Markdown or XML, because models have seen many web pages in these formats.

Example (instruction and data separated):

Summarize the review in one sentence.
---REVIEW
The hotel was clean, but breakfast only arrived after an hour.
---SUMMARY

The review is made up, like all sample data of the course (the files bewertung-1.txt to bewertung-3.txt follow in lesson 2.6).

Specify the output format

Whoever names the structure of the answer gets answers that are easier to process further. Learn shows a JSON pattern in the prompt for this. A note on reliability: Learn writes that sentences like "don't make anything up" alone are not always enough. It can help more to demand evidence, because models then have to make two mistakes when they invent something: the false statement and the false evidence.

Include examples (few-shot)

Without examples this is called zero-shot. With few-shot you put one or more input-output pairs into the prompt. The model learns nothing permanently from this; the examples only tune it better for this one request. In a chat, according to Learn, you give them as pairs of a user and an assistant message after the system message.

Nordlicht sorts customer reviews by topic. Two made-up example pairs:

Review: "Room was loud, but staff were very nice."
Topic: Noise

Review: "The transfer from the airport came very late."
Topic: Transfer

Review: "Breakfast was cold and the coffee was gone."
Topic:

The last block ends with the beginning of the answer. Learn calls this a cue: a hint at the end that prepares the form of the answer.

Repeat what matters

According to Learn, models may give more weight to the end of a prompt (recency bias). It can be worth repeating the most important instruction at the end. Here too: try it out.

Improving in the playground

In the playground (lesson 2.1) you enter the system prompt in the field provided for it and put the user prompts in the chat. Proceed in small steps:

  1. Begin with the simplest prompt.
  2. Make one change (role, format, example).
  3. Send the same user prompt again and compare.
  4. Use Compare to look at two or three models or versions side by side (up to three, Learn).

That way you see which change achieves something. As typical questions for the playground, Learn names: Which system prompt structure delivers the best quality? How do examples affect reliability? How do temperature, top_p and max_tokens act?

Summary table

Technique Effect Example at Nordlicht
Role and limits in the system prompt consistent behavior no decision on refunds
Name tone and language fitting answers language of the customers
Name a way out less made-up content "If information is missing, say so"
Delimiters instruction and data stay separate ---REVIEW
Output format machine-readable answer JSON with fixed fields
Few-shot model recognizes the pattern two sample reviews with a topic

Exercise 1

An advisor types into the playground: "Write something about Mr. Lund's mail." The answer is too long, in the wrong tone and unusable. Improve the prompt in three steps and assign a technique from the summary table to each step:

  1. What is the model supposed to do and not do, permanently?
  2. What should the answer look like?
  3. How do you separate the mail from the task?

Solution 1

One possible solution (an example, not measured):

  1. System prompt with role, tone and limit: "You prepare draft replies for customer service at Nordlicht Reisen GmbH, friendly and short. You never decide on refunds."
  2. Output format: "Write at most five sentences and begin with a salutation."
  3. Delimiters: The mail sits between ---MAIL and ---END, the task comes before it: "Write a draft reply to the mail."

Other words are just as good. What matters is that each change removes one problem and that a person reads the draft before it goes out.

Further reading