In module 3.4 the Kladde learned to send telemetry, count its own metrics, answer queries and send a first alert. In the new subscription, everything that wasn't in Bicep is missing: the action group you may have created in the portal, the availability test, the alert rules. This lesson brings them back as code, adds the alerts the targets from 3.10.1 demand, and proves that every single one arrives.
3.10.1 holds the principle for this: an alert that has never fired is a guess. It could land at the wrong address, have a filter that finds nothing (3.4.4), or never fire because its condition never becomes true.
Which alerts the Kladde needs
Every alert belongs to a target and says what to do. An alert without a next step trains people to ignore alerts.
| Alert | Signal | Condition | Target from 3.10.1 | First step |
|---|---|---|---|---|
| Availability | standard availability test | 2 of 3 locations fail | booking reachable | read console logs and revisions of the web app |
| Error rate | requests |
more than 5% in 5 minutes, from 20 requests | few errors | failures view, most frequent exception (3.4.6) |
| Dead letter | metric DeadletteredMessages |
greater than 0 in tender-import or contact-import |
tenders up to date | read the reason with the tool from 3.5.5 |
| Import missing | counter kladde.import.completed |
no import for 26 hours | tenders up to date | last execution of the job, console logs |
| Test reports nothing | availabilityResults |
no results in 30 minutes | booking reachable (the measurement itself) | check the state of the availability test |
| Budget | Cost Management | 50, 80, 100% | costs | cost analysis by resource group |
Two of the alerts fire when something fails to happen. They are the most important, because they close the gaps of the others: if the availability test fails, the availability alert reports nothing, the overview stays green, and nobody measures any more. And an import that doesn't start at all at night produces no error that the error rate could see. That is the same thought as in Exercise 1 of 3.10.1: nothing measured doesn't mean "all good".
The budget comes from 3.10.2 and reports through Cost Management, not through Azure Monitor. It can't be triggered at the push of a button. Its proof is the check script from 3.10.2 that reads the four notifications.