In module 1.9, you learned what async and await do: A method waits for something slow, a file, an answer from the network, without
holding on to a thread. In a console app, that was mostly a good habit, because only one person is waiting there. On a web server, many
people wait at the same time, and await decides whether the app stays fast under load or comes to a standstill for everyone.
In this lesson, you measure what a single blocking call costs. Then you get to know the request’s CancellationToken, with which an
endpoint notices that nobody is waiting for its answer anymore, and time limits, which end a request after a fixed time. At the end, all
the Kladde’s endpoints pass the token on down to the store.
Few threads for many requests
Kestrel accepts requests and has them handled by threads from the thread pool, a reserve of threads that the whole app shares. The pool starts with as many threads as the machine has processor cores, so on a small server or in a container, with two or four. That is enough for hundreds of simultaneous requests, as long as each one uses its thread only for as long as it is really computing.
That is exactly what await does: When a handler waits for the file or the database, it returns its thread to the pool. When the answer is
there, any free thread carries on. A blocking call, by contrast, holds on to the thread while it waits and does nothing with it. You
recognize such calls by these forms:
| blocks | instead |
|---|---|
task.Result, task.Wait(), task.GetAwaiter().GetResult() |
await task |
Thread.Sleep(…) |
await Task.Delay(…, cancellationToken) |
File.ReadAllText(…), File.WriteAllText(…) |
await File.ReadAllTextAsync(…, cancellationToken) |
| a synchronous method that waits for something inside | the …Async version |
If there are too few free threads, the pool adds new ones, but slowly, so that not every short spike creates hundreds of threads. If many requests block at the same time, all the others line up behind them. This is called thread pool starvation, and it is one of the most common reasons why a web app suddenly becomes slow for everyone under load.