With this module you go one level deeper: what actually happens when a hundred visitors call the Kladde at the same time? So far ASP.NET Core has taken care of that for you, and most of the time that is the right way. But two field reports in K3 had their cause exactly here: the two timers that fell due at the same moment (lesson 3.8.4), and the blocking call that brings a small replica to its knees (lesson 3.9.5). This module does thoroughly what was only touched on there. It begins with the building blocks: threads and the thread pool, on which almost everything in .NET runs.

For the examples, create a folder k3-exercises/3-11. Each example is a file that you start with dotnet run <file>.cs. Copy the load generator load.cs from lesson 3.8.6 into the same folder.

Process, thread, core

  • A process is a running program with memory of its own. The Kladde in a replica of Container Apps is a process.
  • A thread is a strand of execution in a process. All threads of a process share its memory; each has its own stack for local variables and calls.
  • A core (more precisely: a logical processor) runs exactly one thread at any moment. The operating system switches between the threads, many times a second. More threads than cores means: they take turns.

Environment.ProcessorCount gives the number of logical processors that .NET sees. In a container that is what has been assigned to it: a replica with half a core sees one processor.

What does a thread cost? The operating system creates a stack and management data of its own for each one, and every switch between threads costs time. A comparison: a thousand waits of one second, once as a thousand threads, once as a thousand tasks with await.

C#
// threads.cs: a thousand waits, once as threads and once as tasks
#:property PublishAot=false
using System.Diagnostics;
using System.Globalization;

CultureInfo.CurrentCulture = CultureInfo.GetCultureInfo("en-US");
Console.WriteLine($"Logical cores: {Environment.ProcessorCount}");

using var process = Process.GetCurrentProcess();
long memoryBefore = PrivateMemory(process);
var watch = Stopwatch.StartNew();
var threads = Enumerable.Range(0, 1000).Select(_ => new Thread(() => Thread.Sleep(1000))).ToList();
threads.ForEach(thread => thread.Start());
int threadsWhileWaiting = ThreadCount(process);
long memoryWhileWaiting = PrivateMemory(process);
threads.ForEach(thread => thread.Join());
Console.WriteLine($"1,000 threads: {watch.Elapsed.TotalSeconds:0.0} s, {threadsWhileWaiting,5:N0} threads in the process, " +
    $"{(memoryWhileWaiting - memoryBefore) / 1024.0 / 1024:N0} MB more memory");

memoryBefore = PrivateMemory(process);
watch.Restart();
Task all = Task.WhenAll(Enumerable.Range(0, 1000).Select(_ => Task.Delay(1000)));
threadsWhileWaiting = ThreadCount(process);
memoryWhileWaiting = PrivateMemory(process);
await all;
Console.WriteLine($"1,000 tasks:   {watch.Elapsed.TotalSeconds:0.0} s, {threadsWhileWaiting,5:N0} threads in the process, " +
    $"{(memoryWhileWaiting - memoryBefore) / 1024.0 / 1024:N0} MB more memory");

static int ThreadCount(Process process)
{
    process.Refresh();
    return process.Threads.Count;
}

static long PrivateMemory(Process process)
{
    process.Refresh();
    return process.PrivateMemorySize64;
}

On a test machine with 32 logical cores; your numbers are different:

Logical cores: 32
1,000 threads: 1.3 s, 1,008 threads in the process, 59 MB more memory
1,000 tasks:   1.0 s,     9 threads in the process, 0 MB more memory

A thousand threads that only wait occupy a thousand stacks and dozens of megabytes. A thousand tasks that wait with await occupy no thread at all: while Task.Delay runs there is nothing to do, and only when the time is up does a thread continue the task. This insight carries the whole module: waiting needs no thread, only work needs one.