Choosing models

In T1.2 you calculated what a model costs for assessing tenders. Which model it should be was already settled there. In this lesson you make that decision yourself, for all three AI features that the Kladde gets over the course of the track: the assessment of tenders (T2), the support chat (T5) and the agent that drafts offers (T7). At the end, docs/model-choice.md holds a table with a justification for each choice.

A preliminary remark that holds for the whole lesson: whether a model is good enough for a task is decided in the end by a measurement on your own test set (T9), not by a ranking and not by a data sheet. What you settle today is a reasoned first choice that the measurement may confirm or overturn.

Large or small

Models differ above all in their size, measured in parameters, the numbers learned during training. qwen2.5-0.5b has half a billion of them, large models in the cloud many times more. As a rule of thumb:

• Larger models solve harder tasks better: summarizing long texts, planning several steps, reading between the lines. They are slower and more expensive per token.
• Smaller models are fast and cheap and are enough for narrow, clearly described tasks: assigning a text to one of three categories, reading fields from a form, wording a short answer from a section that comes with the request.

Many providers therefore offer families in several sizes. In Microsoft Foundry, such a family is for example gpt-5.4, gpt-5.4-mini and gpt-5.4-nano. The usual way: start with the smallest model that passes the task on the test set, and go bigger only when a measurement demands it.

Reasoning models

Reasoning models think before they answer. For that they produce tokens of their own, which don’t appear in the answer but cost time and are billed like output (T1.2). With the parameter reasoning_effort you set how much the model may think; which levels exist depends on the model, and for the newer models they include none, that is, no thinking at all. These models don’t accept the temperature from T1.1.

Reasoning pays off for tasks with several steps that build on one another: drafting a plan, weighing conditions against each other, understanding code. For simple tasks you mainly pay with wait time. The models of the gpt-5.4 family are reasoning models; for assessing a tender a low level would be a sensible start, for the agent that assembles an offer from several sources, rather a medium one.

Local or cloud

Foundry Local and Microsoft Foundry speak the same interface, so the Kladde can do both. Which environment suits which feature is decided by these points:

For the Kladde that means: development and testing happen locally, so nobody pays for experiments. Whether a feature runs locally or in the cloud in operation is a decision per feature. A feature with content that must never leave the computer can stay local on purpose, even if a cloud model would be better.

Reading a model card

For every model there is a description, the model card. In the model catalog of the Foundry portal, every model has a page of its own, and on Microsoft Learn the page “Foundry Models sold by Azure” summarizes the Azure OpenAI models in tables. For local models, foundry model info <alias> shows the details, and foundry model list --verbose names the license. Look out for:

• Capabilities: Can the model call tools? Does it deliver structured responses according to a JSON schema? Does it understand images?
• Context window and longest output: how many tokens go in and how many come out at most.
• Training data cutoff: What happened afterward, the model doesn’t know.
• Version and retirement: models in the cloud have a version and an announced end. After that you can’t use them any more. Whoever chooses a model plans the switch too (T10.4).
• License: for models with open weights that you run locally, the license says whether and how you may use them in a product.
• Deployment types and regions: not every model exists in every region and every deployment type.

An example from Microsoft Learn: gpt-5.4-mini is a reasoning model with Chat Completions and the Responses API, structured responses, text and image input and tool calls. Its context window has 400,000 tokens, of which at most 272,000 are for the input and 128,000 for the output, and its training data runs to August 2025. Always look such details up at the source; they change with every new version.

Deployment types in Foundry

You use a model in the cloud through a deployment. Its type determines where your requests are processed and how you pay. The most important types:

Two details from Microsoft’s description matter for the Kladde:

• Stored data stays in the Azure geography of your resource for all types. The deployment type affects where the request is processed.
• The EU data zone follows Azure’s EU Data Boundary. According to Microsoft, that can include EFTA countries such as Norway and Switzerland.

An office that processes tenders and customer inquiries and wants to keep them in the EU therefore usually chooses Data Zone Standard in a region of the EU. For gpt-5.4-mini, according to T1.2, that costs about 10 percent more than Global Standard. Whether a deployment of this type exists for your model in your region is shown by Microsoft’s page on availability by region (under “Read more”); whether your subscription has quota there, only the portal shows. The price list proves only that a price exists.

Capabilities first

The Kladde’s three AI features need different capabilities, and you have to check them before you talk about quality and price:

• The assessment needs structured responses: the Kladde wants a suitability, reasons and a score in fixed fields, not free text (T2.3). In the cloud, models with “Structured outputs” enforce a JSON schema strictly. Whether Foundry Local does too isn’t in the documentation; T2.3 checks it, and until then the code checks every answer itself.
• The support chat needs a context window large enough for the help pages it sends along, a short wait time and good German and English.
• The offer agent needs tool calls (T7). Locally, the Task column in foundry model list shows with tools which models can do that. Small models often call tools unreliably; Microsoft explicitly describes forcing a tool call in Foundry Local only as “best effort”.

A model without the necessary capability drops out, however cheap it is.

Comparing costs

For the comparison, you extend the calculation from T1.2 to three features and three models. The volumes are assumptions for a small office; they stand as data at the start of the program so you can change them easily:

// comparison.cs – what the Kladde's three AI features would cost per month with three models, at today's prices.
using System.Globalization;
using System.Text.Json;

const string Region = "swedencentral";
const string PricesApi = "https://prices.azure.com/api/retail/prices";

// The names exactly as the price list writes them (case matters), all Data Zone Standard.
(string Name, string InputSku, string OutputSku)[] models =
[
    ("gpt-5.4", "5.4 inp Dz", "5.4 opt Dz"),
    ("gpt-5.4-mini", "5.4 mini Inp Dz", "5.4 mini Opt Dz"),
    ("gpt-5.4-nano", "5.4 nano Inp Dz", "5.4 nano Opt Dz"),
];

// Assumptions per feature: calls per month, input and output tokens per call. Measured from T2.4 on.
(string Name, int Calls, int Input, int Output)[] features =
[
    ("Assessment", 1_000, 430, 400),
    ("Support chat", 2_000, 3_000, 250),
    ("Offer agent", 300, 4_000, 600),
];

using var http = new HttpClient { Timeout = TimeSpan.FromSeconds(15) };
var prices = new Dictionary<string, decimal>();
try
{
    foreach (var model in models)
    {
        prices[model.InputSku] = await PricePerMillionAsync(http, model.InputSku);
        prices[model.OutputSku] = await PricePerMillionAsync(http, model.OutputSku);
    }
}
catch (Exception e) when (e is HttpRequestException or TaskCanceledException or JsonException or KeyNotFoundException or InvalidDataException)
{
    Console.Error.WriteLine($"No price read ({e.Message}). Without current prices I don't compare.");
    return 1;
}

var usd = CultureInfo.InvariantCulture;
Console.WriteLine($"USD per month, Data Zone Standard, {Region}, read {DateTime.UtcNow:yyyy-MM-dd HH:mm} UTC");
Console.WriteLine($"{"",-16}" + string.Concat(models.Select(m => $"{m.Name,14}")));
foreach (var feature in features)
{
    string row = string.Concat(models.Select(m =>
    {
        decimal perCall = feature.Input * prices[m.InputSku] / 1_000_000m + feature.Output * prices[m.OutputSku] / 1_000_000m;
        return (perCall * feature.Calls).ToString("0.00", usd).PadLeft(14);
    }));
    Console.WriteLine($"{feature.Name,-16}{row}");
}

return 0;

static async Task<decimal> PricePerMillionAsync(HttpClient http, string sku)
{
    string filter = $"serviceName eq 'Foundry Models' and armRegionName eq '{Region}' "
        + $"and priceType eq 'Consumption' and skuName eq '{sku}'";
    string url = $"{PricesApi}?api-version=2023-01-01-preview&$filter={Uri.EscapeDataString(filter)}";
    using JsonDocument json = JsonDocument.Parse(await http.GetStringAsync(url));
    JsonElement[] items = [.. json.RootElement.GetProperty("Items").EnumerateArray()];
    if (items.Length != 1
        || items[0].GetProperty("currencyCode").GetString() != "USD"
        || items[0].GetProperty("unitOfMeasure").GetString() != "1M")
    {
        throw new InvalidDataException($"did not get exactly one price in USD per 1M for '{sku}'");
    }

    return items[0].GetProperty("retailPrice").GetDecimal();
}

dotnet run comparison.cs

With the prices of the day this lesson was written; in the first line you’ll see the time of your run, and the amounts follow today’s prices:

USD per month, Data Zone Standard, swedencentral, read YYYY-MM-DD hh:mm UTC
                       gpt-5.4  gpt-5.4-mini  gpt-5.4-nano
Assessment                7.78          2.33          0.64
Support chat             24.75          7.43          2.01
Offer agent               6.27          1.88          0.51

PricePerMillionAsync is the same method as in costs.cs. Only the two tables at the start are new. That the names of the prices are written inconsistently (5.4 inp Dz versus 5.4 mini Inp Dz) is no typo but is how the price list has it; that’s why the comment sits above them.

Between the three sizes there’s a factor of about three to four each time. In absolute amounts, all three features are cheap for a small office. The choice of model is therefore decided here less by price than by quality, wait time and data location. That changes as soon as the volumes grow, as exercise 2 shows.

    ("Support chat", 20_000, 3_000, 250),
    ("Offer agent", 300, 12_000, 600),

USD per month, Data Zone Standard, swedencentral, read YYYY-MM-DD hh:mm UTC
                       gpt-5.4  gpt-5.4-mini  gpt-5.4-nano
Assessment                7.78          2.33          0.64
Support chat            247.50         74.25         20.08
Offer agent              12.87          3.86          1.04

The Kladde: justify the model choice

Extend docs/model-choice.md from T1.1 with your table from exercise 1 and the assumptions the costs are based on:

## Models per feature (first choice, measured in T9)

| Feature | Model | Deployment | Reasoning | Justification | next candidate |
| --- | --- | --- | --- | --- | --- |
| Assessment | ... | Data Zone Standard | ... | ... | ... |
| Support chat | ... | Data Zone Standard | ... | ... | ... |
| Offer agent | ... | Data Zone Standard | ... | ... | ... |

## Assumptions for the costs

- Assessment: 1,000 calls a month, 430 tokens input, 400 output
- Support chat: 2,000 questions a month, 3,000 tokens input, 250 output
- Offer agent: 300 steps a month, 4,000 tokens input, 600 output
- Prices: Azure Retail Prices API, region swedencentral, read on <date>

## Local

- Development and tests: Foundry Local, alias qwen2.5-0.5b
- For the agent a local model with Task "tools": <alias from foundry model list>

Put comparison.cs in your exercise folder. Submit the change as a pull request. In the review, one question is worth asking: Would someone who doesn’t know the Kladde understand from the justification why this choice was made?

What you can do now

• You tell large, small and reasoning models apart and start with the smallest model that can pass a measurement.
• You weigh local against cloud: data location, cost, quality, speed, operation and protection.
• You read a model card and find capabilities, context window, training data cutoff, version and license.
• You know the deployment types in Foundry and justify Data Zone Standard for processing in the EU.
• You check first the capabilities a feature needs, then quality and price.
• You have a reasoned first choice for every AI feature of the Kladde in docs/model-choice.md.

Read more

• Deployment types for Microsoft Foundry Models: all deployment types, with data location and payment.
• Foundry Models sold by Azure: the model cards of the Azure OpenAI models in tables.
• Region availability for Foundry Models sold by Azure: which model exists in which region and deployment type.
• Azure OpenAI reasoning models: how reasoning models work, what reasoning_effort does and which models can do what.
• How to use tool calling with Foundry Local: which local models call tools and how reliably.