The draft of the group-travel team is to be ready in under 10 minutes (M-02, 1.4-02), and the trip planner is to be able to rely on it. Two problems are rarely loud: A run is slow without an error occurring, and a draft is wrong without an agent having failed, because a fact got lost on the way. The lesson breaks down the time and diagnoses the context, using what the team itself owns: the excerpts and the team state from lesson 2.3.
Prior knowledge. Quotas, 429 and measurement plan: "AI-103, lesson 1.3". Gate, call counter and spans with mandantId, laufId, agent: "lesson 1.4". Excerpts, fields in the team state and the limit of compaction: "lesson 2.3". Hybrid search: "AI-103, lesson 5.2". Traces: "AI track, lesson T10.2". The improvement itself follows from lesson 3.2. Here we measure and recognize.
New in this lesson. (1) The time amounts in the table below are assumptions of the task, not a measurement. (2) A retry after 429 counts as a model call of the run (1.4: at most 25). (3) Entities in the team state carry identifiers: hotels the form HT-017, supplier contracts LV-0017 (invented). (4) A diagnosis reads the fields of the team state and the texts of the agents per laufId (lesson 1.3 stores results and versions there); it changes nothing.
The requirement
All numbers are assumptions of the task.
| No. | Requirement | Type | Measure | Weight | Source |
|---|---|---|---|---|---|
| 3.3-01 | For every stage of a run the time at the 95th percentile is known. | F | 7 roles | Must | M-02 |
| 3.3-02 | The draft is ready in under 10 minutes, waiting time in the queue and at rate limits included. | Q | 95% of runs | Must | 1.4-02 |
| 3.3-03 | A retry after 429 follows retry-after-ms and counts against the call limit. |
R | 0 retries without waiting time | Must | R-09 |
| 3.3-04 | Number of people, destination and period are in every excerpt and in every output that needs them. | R | 0 missing facts in the test set | Must | 2.3-03 |
| 3.3-05 | An identifier from the request (contract, hotel) is found among the hits, or the diagnosis reports it. | Q | 0 silent misses | Should | M-06 |
Cost frame. Cost drivers: tokens of the runs with diagnosis (the diagnosis itself needs no model), Application Insights per gigabyte. Prices: the pricing pages of Foundry and Azure Monitor.