This lesson answers a single question: What does a language model do when it "answers"? You need neither math nor code for it. You need a picture in your head that explains why models write so well and why they can still talk nonsense. The AI-901 exam asks for exactly this picture. The depth, with counting tokens and costs in C#, is offered by the AI track (AI track, T1.1 and T1.2). Here you get the overview.
In the example of Nordlicht Reisen GmbH, a helper in customer service is meant to prepare answers. Ms. Berg, the head of customer service, wants to know whether it can be trusted. The picture of this lesson gives her the answer: not blindly.
Generative AI creates new content
Generative AI creates content such as text, images, audio or code. The models are trained on large amounts of such content, recognize patterns in it and from them form new content that statistically resembles the training data. From your input, the model builds an output. In doing so, it deliberately creates something new and therefore does not always return the same output for identical inputs. Where language is concerned, a large language model (LLM) is at work. For images or audio, according to Learn, other models come in as well.
Step 1: Text becomes tokens
A model does not read text but tokens. A tokenizer splits the text into units from a fixed vocabulary. A token can be a whole word, a part of a word, a single character or a punctuation mark. Each token has a number in the vocabulary, and the model calculates only with these numbers. At the end, it translates the generated numbers back into text.
Tokens are the unit for everything that follows: billing is usually per token (input and output), and the length of what a model can process at once is called the context window and is measured in tokens. Input and output share this space. A long prompt leaves less room for the answer. Figures for this depend on the model and appear in lesson 1.3 only as a pointer to where to look them up.
Nordlicht: A customer review such as bewertung-1.txt is split up before the model "sees" it. A long word can break into several tokens. That explains why the length of a message in tokens does not equal its number of words.
Step 2: Tokens become embeddings
So that a model can calculate with tokens, each token gets a vector, a list of numbers. This list is called an embedding. According to Learn, the numbers describe semantic features of the token and result from how often tokens occur together or in similar contexts. As a result, words with similar meaning lie close together in this number space.
Embeddings are not just an intermediate stage inside the model. You can also compute them for whole texts and then compare texts by meaning, for example for search. That is the idea behind vector search (AI track, T4.1).
Step 3: Attention and probabilities
The model is supposed to predict the next token. To do so, it weights the tokens of the sequence so far: each gets a weight that expresses how strongly it influences the next token. That is the basic idea of attention: the model pays attention to the parts of the text that matter for the next position. The architecture that implements this is usually called a transformer; Learn names it as a typical architecture of large language models. From the weights and embeddings, the model calculates which tokens come into question next and how probable each one is.
Then one is selected, usually a probable one. This selection can be influenced through settings, for example the temperature (lesson 1.3). The chosen token is appended to the sequence, and the game starts over. In this way the output is produced one token after another, similar to the word prediction on a keyboard, only far more far-reaching.
Prompt and completion
Everything you send to the model is called the prompt: the system's instructions, the conversation so far and your input. What the model then produces is called the completion or simply the answer. The model has no memory between two calls. Whoever wants a conversation sends the history along each time. How to write prompts well is covered in lesson 2.2.
| Term | What it is | Where you meet it |
|---|---|---|
| Token | unit made of a word, part of a word or character | billing, context window |
| Embedding | list of numbers for the meaning of a token or text | vector search, similarity |
| Attention | weighting of which tokens matter for the next one | transformer |
| Prompt | everything that goes to the model | system instruction, question, history |
| Completion | what the model produces | the answer |
Why answers can be wrong
The model chooses what fits statistically, not what is verified as true. It does not look up facts, it continues text. That is why it can give answers that sound convincing but are wrong. This is called hallucination. As a related measure, Learn describes ungroundedness: the model produces information that is not true or that deviates from what is in the provided sources. On top of that, its knowledge has a cutoff date, and the model does not know more recent events or your own data.
Nordlicht: The helper is supposed to answer until when a guest at Hotel Seeblick may cancel free of charge. Without the booking confirmation in the prompt, it has no basis. It would still name a deadline that sounds plausible.
Grounding: give the model the source
The remedy is called grounding. Along with the request, you give the model the information on which the answer should rest, for example the text of the booking confirmation rechnung-seeblick.pdf. Learn describes this as a technique for supplementing a model's knowledge with further data: information that was not part of training is added to the prompt. A grounded answer stays close to the given information and avoids speculation. The common form is called retrieval-augmented generation (RAG): first search for matching texts, then have the model answer with them. The AI track shows the flow in C# (AI track, T5.1). Grounding is not a guarantee. That is why in the pilot project a person reviews every answer (lesson 1.1).
Exercise 1
The Nordlicht helper writes to a guest: "You can cancel free of charge until shortly before arrival." The booking confirmation says nothing about this. Answer each question in one sentence:
- How did this answer come about?
- What is the phenomenon called?
- What could Nordlicht change about the request?
- Which part of the prompt could have supplied the deadline: system instruction, question or provided text?
Solution 1
- The model chose the statistically most probable token, one after another. The statement sounds like usual cancellation rules, but it rests on no source.
- Hallucination, in Learn's words an ungrounded answer.
- Include the booking confirmation or the hotel's cancellation terms in the prompt (grounding) and instruct the model not to name a deadline without a source.
- The provided text, that is, the confirmation. The system instruction sets behavior, the question poses the task, but the fact must come from the source.