Every assessment of the Kladde sends almost a thousand tokens that were exactly the same in the last request of the same day: the system message and the three examples. Only the tender at the end is new. The providers of language models take advantage of that: They remember the processing of a start they have just seen and don’t compute it again the next time. That makes requests faster and, in the cloud, cheaper. But it only works if the start is exactly the same, character for character. In this lesson, you find out that the assessment prompt has been wasting this start, build version 3, and secure the stable start with a test.

How the prompt cache works

Microsoft describes the prompt cache of Azure OpenAI like this:

  • From 1,024 tokens. A request uses the cache only if it’s at least 1,024 tokens long and its first 1,024 tokens match an earlier request exactly. A single different character in them results in a miss.
  • Then in steps. Up to the GPT-5.5 model family, a hit grows after the first 1,024 tokens in steps of 128 tokens; from GPT-5.6 on, this rounding no longer applies.
  • Everything at the front counts: system, user and assistant messages, images, the descriptions of tools and the schema of a structured answer. The service puts the schema ahead of the system message. So if you change TenderAssessment, the start changes too.
  • It doesn’t live long. In memory, the cache without requests usually empties after five to ten minutes, and at the latest an hour after the last use. For some models up to GPT-5.5, prompt_cache_retention gives a longer retention of up to 24 hours, for gpt-5.5 by default; Microsoft’s page names the models. From GPT-5.6 on, prompt_cache_options.ttl sets a minimum duration, 30 minutes if nothing is specified.
  • It belongs to one subscription. Nothing is shared between Azure subscriptions.
  • You can see it in the answer. The number cached_tokens under prompt_tokens_details says how many input tokens came from the cache. Microsoft.Extensions.AI stores it in UsageDetails.CachedInputTokenCount.
  • Reading is cheaper, writing can cost. Tokens from the cache are billed at a discount for standard deployments. Up to GPT-5.5, writing to the cache costs nothing extra. From GPT-5.6 on, it can cost; the answer then reports cache_write_tokens.
  • It changes nothing about the content of the answer. The cache saves compute time; it doesn’t make the answer different.

Locally, none of this applies in the same sense. Foundry Local costs no money per token, so there’s nothing to save; whether a local server reports anything like cached_tokens at all depends on the server. What you build in this lesson pays off in the cloud. Locally, you check it with tests and the tokenizer.