The Kladde runs in Azure, it is observed, and since module 3.8 you know how fast it answers under load. This module is about the question of what happens when something doesn't go as it did in the load test: A foreign interface answers slowly or not at all, a service sends an unexpected answer, many visitors come at once, a page is slow and nobody knows why. The module has five lessons: time limits (this one), retries, caching, rate limiting and measuring. At the end comes an exam with a deliberately unreliable service.

For this module, two rules are added to the Kladde's house rules:

  • Every call to the outside has a time limit that somebody chose deliberately. A library's default value is not a choice.
  • An error is reported honestly. A redirect is not a success, an expired time limit is not a "nothing found", and an unclear outcome is called "unclear".

The examples are file-based apps again. Next to the Kladde's repository, create a folder k3-exercises/3-9 and work in it. You don't need a new package in this lesson.

How long does a call wait?

An HttpClient waits until an answer comes, until the time limit expires or until someone cancels. The following program starts a small server in the same file for that: It answers /slow only after five seconds. As in lesson 3.8.5, it binds port 0 and then reads which port it got (app.Urls). Create timeout.cs:

C#
// timeout.cs: timeout or cancellation?
#:sdk Microsoft.NET.Sdk.Web
#:property PublishAot=false

var builder = WebApplication.CreateBuilder(args);
builder.Logging.ClearProviders();
builder.WebHost.UseUrls("http://127.0.0.1:0");
var app = builder.Build();

app.MapGet("/slow", async (CancellationToken cancellationToken) =>
{
    await Task.Delay(TimeSpan.FromSeconds(5), cancellationToken);
    return "finally";
});

await app.StartAsync();

using var standard = new HttpClient();
Console.WriteLine($"Without a setting, an HttpClient waits {standard.Timeout.TotalSeconds} seconds.");

using var client = new HttpClient { BaseAddress = new Uri(app.Urls.First()), Timeout = TimeSpan.FromSeconds(1) };

await CallAsync("Client timeout", CancellationToken.None);

using var caller = new CancellationTokenSource(TimeSpan.FromMilliseconds(300));
await CallAsync("Cancellation by the caller", caller.Token);

await app.StopAsync();

async Task CallAsync(string name, CancellationToken cancellationToken)
{
    try
    {
        await client.GetStringAsync("/slow", cancellationToken);
        Console.WriteLine($"{name}: answer received");
    }
    catch (TaskCanceledException ex) when (ex.InnerException is TimeoutException)
    {
        Console.WriteLine($"{name}: timed out");
        Console.WriteLine($"  {ex.Message}");
    }
    catch (TaskCanceledException) when (cancellationToken.IsCancellationRequested)
    {
        Console.WriteLine($"{name}: cancelled, the caller's token was triggered");
    }
}
Bash
dotnet run timeout.cs

Without a setting, an HttpClient waits 100 seconds.
Client timeout: timed out
  The request was canceled due to the configured HttpClient.Timeout of 1 seconds elapsing.
Cancellation by the caller: cancelled, the caller's token was triggered

The output shows three things:

  • 100 seconds is the default value of HttpClient.Timeout. For that long, a call without a setting of its own holds on to a connection, part of the memory and the visitor who is waiting for the page. The visitor is gone after five seconds, the call keeps running.
  • A timeout and a cancellation look the same at first. Both end with a TaskCanceledException. Since .NET 5, a timeout has a TimeoutException in InnerException; with a cancellation, the caller's token is triggered. That is how you tell the two apart, and you have to: A timeout is an error of the dependency, which you count and perhaps report. A cancellation only means that nobody is waiting for the result any more, say because the visitor closed the page. It is not an error and does not belong in the error rate from lesson 3.4.5.
  • The server notices the cancellation. The handler gets its CancellationToken and stops waiting as soon as the client gives up the connection. That works only because the handler passes the token on to Task.Delay. Every method of the Kladde that can wait therefore gets the token, as you have done since K1 1.9.

Which time limit is right? Don't guess. In Application Insights, the durations of all calls to the outside are in the table dependencies (lesson 3.4.4). Take the 99th percentile of the last weeks, add some headroom, and check whether the caller can wait that long at all. A call that takes 800 milliseconds at the 99th percentile might get three seconds. What takes longer is very likely broken, and then a quick, honest error message is better than a slow one.

Why is a time limit that is too long dangerous? Under load, the waiting calls pile up. Each holds a connection and part of the memory; new requests wait for free connections, and the Kladde gets slow although it does nothing wrong itself. A slow dependency pulls the whole application down with it. A short time limit cuts this chain off.