Most of the Kladde's tests need no port: WebApplicationFactory starts the app in memory, and the test's HttpClient talks to it directly. Some tests do need a real socket, though, for example a replayed server of a third-party interface or the app for the browser in the end-to-end test. For that, many projects have a small helper method FreePort(), and that very method is one of the most persistent sources of tests that are occasionally red. In this lesson you see why, and how to do without it.

Field report: the free port that no longer is

From practice, without names: on a Linux agent, five runs of the pipeline failed within a few days, each time a different test and each time with the same message when binding a socket: “Address already in use.” All the affected tests started a server on a port that a helper method had determined to be free shortly before. The method looked like this, and this is how it looks almost everywhere:

C#
static int FreePort()
{
    using var listener = new TcpListener(IPAddress.Loopback, 0);
    listener.Start();
    var port = ((IPEndPoint)listener.LocalEndpoint).Port;
    listener.Stop();
    return port;
}

It binds port 0, the operating system gives it a free port, it reads the number and releases the port again. Only afterward does the actual server bind. Between Stop() and the bind there is a window in which the port belongs to nobody. When many tests run at the same time, another test binds its own port 0 or opens an outgoing connection exactly then, and the operating system hands out the just-freed port a second time.

The window can be shown by having an “other test” take the port in it:

C#
// window.cs
#:sdk Microsoft.NET.Sdk.Web

using System.Net;
using System.Net.Sockets;

var port = FreePort();

// In the window between finding and binding, another test was faster.
using var otherTest = new TcpListener(IPAddress.Loopback, port);
otherTest.Start();

var builder = WebApplication.CreateSlimBuilder();
builder.Logging.ClearProviders();
builder.WebHost.UseUrls($"http://127.0.0.1:{port}");
await using var app = builder.Build();
try
{
    await app.StartAsync();
    Console.WriteLine("started");
}
catch (IOException ex)
{
    Console.WriteLine($"{ex.GetType().Name}: {ex.Message.Replace(port.ToString(), "<Port>")}");
}

static int FreePort()
{
    using var listener = new TcpListener(IPAddress.Loopback, 0);
    listener.Start();
    var port = ((IPEndPoint)listener.LocalEndpoint).Port;
    listener.Stop();
    return port;
}
Bash
dotnet run window.cs

IOException: Failed to bind to address http://127.0.0.1:<Port>: address already in use.

#:sdk Microsoft.NET.Sdk.Web turns the file-based app into a web app, with Kestrel and WebApplication. The program replaces the port number with <Port>, because it is different on every run.

How often this happens without intent depends on the operating system, load and luck. In the exercise below a test starts 400 replayed servers at the same time; on Windows test machines that made one or two of five runs red. Rare enough never to see it locally, and frequent enough to disturb a pipeline with many runs a day regularly.