The travel assistant is running, and the questions of the engineers and of the boss are the same: What does a path through the system cost, where does it get slow, and does every question have to wake the large model? On top of that come the house rules: no model call without a counter (R-11), refunds, legal questions and blocked accounts always go to a person (R-02).
Prior knowledge. "AI track, lesson T10.2" shows traces, spans and the GenAI attributes, "T2.4" the middleware that measures and counts, "T10.3" budgets and caps, "T6.4" hard rules in code, "T7.4" workflows. "Lesson 1.3" of this course covers quotas, costs and monitoring of the platform. Here: what Foundry and Application Insights bring along, your own switch and Foundry's model router next to it.
New in this lesson. The application has a switch with three paths: Mensch (rule, no model), Klein (deployment chat-klein) and Gross (deployment chat). Every path carries its name as a tag on the span.
The requirement
All numbers are assumptions of the task.
| No. | Requirement | Type | Measure | Weight | Source |
|---|---|---|---|---|---|
| 2.8-01 | A request can be followed from the application to the model and the tool in one trace. | F | 1 trace per request | Must | Operations |
| 2.8-02 | Tokens are visible per deployment and per path. | Q | query per week | Must | Cost (R-09, R-11) |
| 2.8-03 | Safety signals (blocked requests) are visible. | Q | warning on increase | Should | Security |
| 2.8-04 | Latency is visible per path. | Q | 95th percentile per path | Should | Customers |
| 2.8-05 | Refund, legal question, blocked account: no model call. | R | 0 calls | Must | Legal (R-02) |
| 2.8-06 | The text of questions is not in traces in operation. | R | 0 contents | Must | Data protection (R-12) |
Cost frame. Application Insights per gigabyte, tokens per million and deployment (for the model router, those of the model it chose). Prices: the pricing pages of Azure Monitor and Foundry.