Quotas, scaling, cost and monitoring – Azure AI applications (AI-103, lesson 6)

What you learn in this lesson

derive the capacity of a deployment from quota, load and the ratio of tokens to requests

choose between Standard, provisioned throughput and batch processing by the shape of the load

set up a measurement plan with metrics, alerts and people responsible for throttling, tokens, quality and safety events

monitor the health of ingestion and the search index through indexer status and metrics

sketch a metric alert for 429 responses as Bicep

In summer, the customers of Nordlicht Reisen GmbH (R-01) call and write in the portal at the same time. The travel assistant must not stall then, and nobody should learn from complaints that requests are being throttled or that the answers have gotten worse. This lesson calculates how much capacity is needed, chooses the scaling form and sets up the measurement plan. Everything is paper and a compiled Bicep sketch; you need an Azure account only for the optional run, and costs arise by cost driver: tokens per million, capacity per hour, ingestion of telemetry per gigabyte.

Prior knowledge. Tokens and costs are covered by "AI track, lesson T1.2", capacity in TPM and RPM and the deployment types by "AI track, lesson T10.1", traces by "AI track, lesson T10.2", the monthly cap, 429 and Retry-After by "AI track, lesson T10.3", drift and the switch of models in the shadow by "AI track, lesson T10.4", index ingestion by "AI track, lesson T5.4". Alerts and KQL are in "K3, lessons 3.4.1 to 3.4.5". None of that is repeated here: the lesson says where in Foundry the numbers come from and how to put them together into a plan.

The requirement

All numbers are assumptions of the task.

No.	Requirement	Type	Measure	Weight	Source
1.3-01	In the summer peak (40 simultaneous conversations) requests are not throttled.	Q	share of 429 under 1%	Must	Operations
1.3-02	Operations learns about throttling.	Q	notification within 15 min	Must	Operations
1.3-03	Token consumption per deployment is visible (R-09, R-11).	F	report per day	Must	Finance
1.3-04	Falling grounding or relevance quality is noticed.	Q	within 1 day	Should	Quality
1.3-05	The index shows the state of the documents (R-03).	Q	indexer run failing for at most 24 h	Must	Business unit
1.3-06	Blocked and flagged inputs can be audited.	F	weekly report	Should	Security

This lesson belongs to course AI-103

You will see the rest of this lesson and its video once you have bought the course. The goals above show what you learn here.

€129.00

incl. VAT, one-time payment

Access to all of the course's lessons with their exercises, solutions and videos on this site, for 24 months from the purchase. A self-study course: no tutoring, no exams or grading by us, and no certificate.

Withdraw from contract here