DNS

In lesson 3.0.2 you worked with addresses. People and configuration files use names, though: learn.microsoft.com, the database server, the address of the TED interface. The Domain Name System (DNS) translates names into addresses and into a few other things. When DNS is wrong, nothing is right, and the error messages rarely say so clearly. In this lesson you follow a resolution step by step, read the records of real domains and see how Azure uses DNS so that the same database has a private address from the inside and a public one from the outside.

A tree of zones

DNS is a tree. At the very top stands the root, written as a single dot .. Below it lie the top-level domains (TLD) such as com, de or org, below them the domains such as microsoft.com, below them any number of levels such as learn.microsoft.com. A name is read from right to left. Strictly speaking, every name ends with the dot of the root: learn.microsoft.com.. Most tools leave it out, in zone files and in the output of dig it is there.

The tree is managed in zones. A zone is a part of the tree for which certain servers know the binding answer, the authoritative servers. The root knows only who is responsible for com. The servers of com know only who is responsible for microsoft.com. Only the servers of microsoft.com know learn.microsoft.com. This passing on is called delegation.

Who asks whom?

When the Kladde calls the TED interface, two kinds of questions run:

1. Your program asks the resolver. The operating system sends the question to a recursive resolver and asks: “Find it out for me.” At home that is usually your router, in the Codespace a resolver of the platform, in a Docker network the built-in DNS server at 127.0.0.11 and in a virtual network of Azure the service at 168.63.129.16.
2. The resolver asks its way through. If it doesn’t know the answer from its cache, it asks a root server, which refers it to the servers of com. They refer it to the servers of microsoft.com, and those give the answer. These questions are iterative: every server answers only with what it knows itself.

The resolver remembers every answer for as long as the answer allows (the TTL, more on that in a moment). That is why the second question for the same name goes much faster than the first.

A small tool that asks

The standard tool for DNS is dig. So that you see there is nothing magical behind it, first write a small dig of your own in C#. It builds a DNS question by hand, sends it over UDP to port 53 and reads the answer. Without a server it asks your system’s resolver and asks for recursion. With a server it asks exactly that server and asks for nothing, the way a resolver does.

// minidig.cs: asks one DNS server one question over UDP and prints the answer's sections, like a very small dig.
// dotnet run minidig.cs -- <name> [type] [server]   without a server it asks the system's resolver, with recursion.
using System.Buffers.Binary;
using System.Net;
using System.Net.NetworkInformation;
using System.Net.Sockets;
using System.Text;

Dictionary<string, ushort> types = new() { ["A"] = 1, ["NS"] = 2, ["CNAME"] = 5, ["SOA"] = 6, ["MX"] = 15, ["TXT"] = 16, ["AAAA"] = 28, ["CAA"] = 257 };
var name = args.Length > 0 ? args[0].TrimEnd('.') : "learn.microsoft.com";
var type = args.Length > 1 ? args[1].ToUpperInvariant() : "A";
var recursive = args.Length < 3;
var server = recursive ? SystemResolver() : await ServerAddressAsync(args[2]);

var query = Query(name, types[type], recursive);
using var udp = new UdpClient(AddressFamily.InterNetwork);
await udp.SendAsync(query, new IPEndPoint(server, 53));
var reply = (await udp.ReceiveAsync().WaitAsync(TimeSpan.FromSeconds(3))).Buffer;

var flags = BinaryPrimitives.ReadUInt16BigEndian(reply.AsSpan(2));
Console.WriteLine($";; {name} {type} at {server}, {(recursive ? "with" : "without")} recursion, status {Rcode(flags & 0xF)}{((flags & 0x0200) != 0 ? ", truncated" : "")}");
var counts = Enumerable.Range(0, 4).Select(i => BinaryPrimitives.ReadUInt16BigEndian(reply.AsSpan(4 + 2 * i))).ToArray();
var offset = 12;
for (var i = 0; i < counts[0]; i++)
{
    ReadName(reply, ref offset);
    offset += 4; // type and class of the question
}

foreach (var (section, count) in new[] { ("ANSWER", counts[1]), ("AUTHORITY", counts[2]), ("ADDITIONAL", counts[3]) })
{
    var lines = new List<string>();
    for (var i = 0; i < count; i++)
    {
        var owner = ReadName(reply, ref offset);
        var recordType = BinaryPrimitives.ReadUInt16BigEndian(reply.AsSpan(offset));
        var ttl = BinaryPrimitives.ReadUInt32BigEndian(reply.AsSpan(offset + 4));
        var length = BinaryPrimitives.ReadUInt16BigEndian(reply.AsSpan(offset + 8));
        offset += 10;
        var data = offset;
        offset += length;
        if (recordType == 41)
        {
            continue; // the OPT pseudo record of EDNS, not data
        }

        var typeName = types.FirstOrDefault(t => t.Value == recordType).Key ?? $"TYPE{recordType}";
        lines.Add($"{owner,-36} {ttl,7}  {typeName,-5}  {Data(reply, data, recordType, length)}");
    }

    if (lines.Count > 0)
    {
        Console.WriteLine($";; {section}");
        lines.ForEach(Console.WriteLine);
    }
}

// A query: header (id, flags, one question, one additional record), the question, and an EDNS record that allows 1232-byte answers.
static byte[] Query(string name, ushort type, bool recursive)
{
    var message = new List<byte>();
    message.AddRange(BitConverter.GetBytes((ushort)Random.Shared.Next()));
    message.AddRange([(byte)(recursive ? 0x01 : 0x00), 0x00, 0, 1, 0, 0, 0, 0, 0, 1]);
    foreach (var label in name.Split('.'))
    {
        message.Add((byte)label.Length);
        message.AddRange(Encoding.ASCII.GetBytes(label));
    }

    message.AddRange([0, (byte)(type >> 8), (byte)type, 0, 1]);
    message.AddRange([0, 0, 41, 0x04, 0xD0, 0, 0, 0, 0, 0, 0]);
    return [.. message];
}

// A name as labels; a byte with the two top bits set points to a name earlier in the message (compression).
static string ReadName(byte[] message, ref int offset)
{
    var labels = new List<string>();
    var position = offset;
    var jumped = false;
    while (message[position] != 0)
    {
        if ((message[position] & 0xC0) == 0xC0)
        {
            if (!jumped)
            {
                offset = position + 2;
            }

            position = BinaryPrimitives.ReadUInt16BigEndian(message.AsSpan(position)) & 0x3FFF;
            jumped = true;
            continue;
        }

        labels.Add(Encoding.ASCII.GetString(message, position + 1, message[position]));
        position += message[position] + 1;
    }

    if (!jumped)
    {
        offset = position + 1;
    }

    return string.Join('.', labels) + ".";
}

static string Data(byte[] message, int at, ushort type, int length)
{
    var position = at;
    switch (type)
    {
        case 1 or 28:
            return new IPAddress(message.AsSpan(at, length)).ToString();
        case 2 or 5:
            return ReadName(message, ref position);
        case 15:
            position += 2;
            return $"{BinaryPrimitives.ReadUInt16BigEndian(message.AsSpan(at))} {ReadName(message, ref position)}";
        case 16:
            var parts = new List<string>();
            while (position < at + length)
            {
                parts.Add(Encoding.UTF8.GetString(message, position + 1, message[position]));
                position += message[position] + 1;
            }

            return $"\"{string.Concat(parts)}\"";
        case 257:
            var tagLength = message[at + 1];
            var tag = Encoding.ASCII.GetString(message, at + 2, tagLength);
            var value = Encoding.ASCII.GetString(message, at + 2 + tagLength, length - 2 - tagLength);
            return $"{message[at]} {tag} \"{value}\"";
        case 6:
            var primary = ReadName(message, ref position);
            var mailbox = ReadName(message, ref position);
            return $"{primary} {mailbox} {BinaryPrimitives.ReadUInt32BigEndian(message.AsSpan(position))}";
        default:
            return $"({length} bytes)";
    }
}

static string Rcode(int code) => code switch { 0 => "NOERROR", 2 => "SERVFAIL", 3 => "NXDOMAIN", 5 => "REFUSED", _ => $"RCODE {code}" };

static IPAddress SystemResolver() => NetworkInterface.GetAllNetworkInterfaces()
    .Where(i => i.OperationalStatus == OperationalStatus.Up)
    .SelectMany(i => i.GetIPProperties().DnsAddresses)
    .First(a => a.AddressFamily == AddressFamily.InterNetwork);

static async Task<IPAddress> ServerAddressAsync(string server) => IPAddress.TryParse(server, out var address)
    ? address
    : (await Dns.GetHostAddressesAsync(server)).First(a => a.AddressFamily == AddressFamily.InterNetwork);

You don’t have to know every line by heart. What matters is what a DNS message contains:

• Header (12 bytes): an ID so that the answer matches the question; switches such as “recursion desired” and, in the answer, the status (NOERROR, NXDOMAIN for “does not exist”, SERVFAIL); four counters for the sections that follow.
• Question: the name as a sequence of parts, each preceded by its length (5learn9microsoft3com0), the type, the class (1 for internet).
• ANSWER: the answer, if the server knows it.
• AUTHORITY: who is responsible, if it doesn’t know it (the delegation), or, for “does not exist”, the SOA record of the zone.
• ADDITIONAL: helpful extras, such as the addresses of the servers from AUTHORITY, so that you don’t have to look them up first. They are called glue.

The last entry of the question (41, 0x04 0xD0) is EDNS: it tells the server that answers of up to 1232 bytes may come over UDP. Without it, the limit would be 512 bytes, and larger answers would come only truncated.

If the program ever aborts with a TimeoutException, the question or the answer got lost on the way. With UDP that happens, there is no delivery guarantee (lesson 3.0.2). Just start it again. Real resolvers ask again after a short time, or ask another server of the same zone.

The resolution, step by step

Play resolver. First step: ask one of the 13 root servers, without recursion.

dotnet run minidig.cs -- learn.microsoft.com A a.root-servers.net

;; learn.microsoft.com A at 198.41.0.4, without recursion, status NOERROR
;; AUTHORITY
com.                                  172800  NS     l.gtld-servers.net.
com.                                  172800  NS     j.gtld-servers.net.
com.                                  172800  NS     h.gtld-servers.net.
;; ADDITIONAL
l.gtld-servers.net.                   172800  A      192.41.162.30
l.gtld-servers.net.                   172800  AAAA   2001:500:d937::30
j.gtld-servers.net.                   172800  A      192.48.79.30

Shortened: the answer names all 13 servers of com with their addresses. The root server doesn’t know learn.microsoft.com, but says who is responsible for com and hands over the addresses right away. Second step, one of those servers:

dotnet run minidig.cs -- learn.microsoft.com A a.gtld-servers.net

;; learn.microsoft.com A at 192.5.6.30, without recursion, status NOERROR
;; AUTHORITY
microsoft.com.                        172800  NS     ns1-39.azure-dns.com.
microsoft.com.                        172800  NS     ns2-39.azure-dns.net.
microsoft.com.                        172800  NS     ns3-39.azure-dns.org.
microsoft.com.                        172800  NS     ns4-39.azure-dns.info.
;; ADDITIONAL
ns1-39.azure-dns.com.                 172800  A      150.171.10.39

The zone microsoft.com lives at Azure DNS. There is glue only for the first server, because only it lies under com itself; the addresses of the others would have to be asked for by a resolver at net, org and info. Third step, the authoritative server:

dotnet run minidig.cs -- learn.microsoft.com A ns1-39.azure-dns.com

;; learn.microsoft.com A at 150.171.10.39, without recursion, status NOERROR
;; ANSWER
learn.microsoft.com.                    3600  CNAME  learn-public.trafficmanager.net.

Finally an answer, but not an address: learn.microsoft.com is a CNAME, another name for learn-public.trafficmanager.net. The resolver would now have to start at the root again with this name. For comparison, ask your own resolver, which does all the work:

dotnet run minidig.cs -- learn.microsoft.com A

;; learn.microsoft.com A at 10.0.0.1, with recursion, status NOERROR
;; ANSWER
learn.microsoft.com.                    1432  CNAME  learn-public.trafficmanager.net.
learn-public.trafficmanager.net.          30  CNAME  learn.microsoft.com.edgekey.net.
learn.microsoft.com.edgekey.net.       20250  CNAME  e13636.dscb.akamaiedge.net.
e13636.dscb.akamaiedge.net.               20  A      23.37.194.77

Your resolver’s address, the names in the chain and the address at the end will be different on your machine: Microsoft distributes its pages over Traffic Manager and a network of servers around the world, and the answer depends on where you ask. You still see two things:

• A chain of three CNAMEs is nothing unusual. Each stage has a job of its own: a fixed name, a distribution by load and health, the choice of a nearby server.
• The TTL on the first record is 1432, not 3600 as at the authoritative server. Your resolver already had the answer in its memory and counts down: it may use it for another 1432 seconds.

dig, nslookup and Resolve-DnsName

In daily work you use ready-made tools. In the Codespace or in an Ubuntu container you install dig with sudo apt-get install -y dnsutils (without sudo in the container). The most important calls:

dig learn.microsoft.com              # A, with all sections
dig +short learn.microsoft.com       # just the answers
dig MX example.com                   # a particular type
dig @1.1.1.1 example.com             # ask a particular resolver
dig +trace learn.microsoft.com       # the whole chain from the root, like by hand above

dig +trace does exactly what you just did with minidig.cs: it starts at the root and follows every delegation. nslookup exists on all systems and is enough for a quick question, but can do less.

In PowerShell on Windows, the tool is called Resolve-DnsName:

Resolve-DnsName -Name example.com -Type MX -DnsOnly

Name                                     Type   TTL   Section    NameExchange                              Preference
----                                     ----   ---   -------    ------------                              ----------
example.com                              MX     300   Answer     .                                         0

-DnsOnly asks only DNS and not, in addition, the hosts file or NetBIOS. You should know two limits: Resolve-DnsName doesn’t know the type CAA (-Type CAA is an error, and nslookup on Windows also reports unknown query type), and with -Server and -NoRecursion it fails on some servers with “DNS server failure” where dig and your minidig.cs get an answer. For CAA and for following a chain, use dig or your own tool.

The records

On top of that comes PTR for the reverse direction, from the address to the name; mail servers check it on senders (lesson 3.0.7).

Three rules about CNAMEs that save a lot of trouble:

• A CNAME stands alone. A name with a CNAME may have no other records, no MX, no TXT, nothing.
• No CNAME at the zone apex. The apex, that is kladde.example itself without www, always has SOA and NS. Whoever wants to set “a CNAME to Front Door” there breaks the first rule. Azure DNS solves this with alias records: an A record that points to an Azure resource and keeps track of its addresses itself.
• An MX never points to a CNAME, but to a name with an A or AAAA record.

TTL and caching

Every record has a TTL (time to live) in seconds: that is how long resolvers may remember it. A long TTL relieves the servers and makes answers fast. A short TTL means a change arrives everywhere quickly. A “does not exist” is remembered too, for as long as the zone’s SOA record says. Whoever creates a name only after the first failed attempt therefore sometimes waits minutes until it shows up everywhere.

From this follows the most important rule for a move: Lower the TTL first, then move. If the Kladde’s A record has a TTL of 86400 (one day) and you change the address, some visitors land on the old address for up to a day. So lower the TTL at least one old TTL before the move to, say, 300, move, and raise it again afterwards.

There are caches on several levels: in the resolver, in the operating system (on Windows, Get-DnsClientCache shows the contents and Clear-DnsClientCache empties it) and in programs. .NET itself has no DNS cache of its own and asks the operating system on every new connection. The problem lies one level higher: an HttpClient keeps its connections open and reuses them, and an open connection never asks DNS again. An HttpClient that lives as a static for a lifetime keeps talking to the old address after the other side has moved. Two ways out:

• The clients from IHttpClientFactory, which the Kladde has used since K2 (AddHttpClient<TedClient>), renew their connections on their own; the factory swaps out its handlers every two minutes by default.
• Whoever keeps an HttpClient alive for a long time on their own limits the life of the connections:

var handler = new SocketsHttpHandler { PooledConnectionLifetime = TimeSpan.FromMinutes(2) };
var client = new HttpClient(handler);

Names from the hosts file and from Docker

Before the operating system asks DNS, it looks in a file: /etc/hosts on Linux, C:\Windows\System32\drivers\etc\hosts on Windows. What is written there wins. That is why localhost can have different answers on two machines. This program asks the operating system the way every .NET application does:

// resolve.cs
using System.Net;
using System.Net.Sockets;

foreach (var name in new[] { "localhost", "learn.microsoft.com", "kladde.invalid" })
{
    try
    {
        var addresses = await Dns.GetHostAddressesAsync(name);
        Console.WriteLine($"{name}: {string.Join(", ", addresses.Select(a => a.ToString()))}");
    }
    catch (SocketException e)
    {
        Console.WriteLine($"{name}: {e.SocketErrorCode}");
    }
}

On Windows:

localhost: ::1, 127.0.0.1
learn.microsoft.com: 23.37.194.77
kladde.invalid: HostNotFound

On Ubuntu:

localhost: 127.0.0.1
learn.microsoft.com: 23.37.194.77, 2a02:26f0:3380:a96::3544, 2a02:26f0:3380:ab9::3544
kladde.invalid: HostNotFound

The addresses of learn.microsoft.com will be different on your machine. On Ubuntu, localhost in /etc/hosts appears only with 127.0.0.1 (::1 is called ip6-localhost there), on Windows the system returns both. In a container, Docker writes the file itself; what applies there is shown by cat /etc/hosts. .invalid is an ending that is guaranteed never to exist; it is suited for examples and tests.

Docker has a small DNS server of its own. In every network you create yourself, and every compose.yaml creates one, it answers at 127.0.0.11 the names of the containers and services. That is why the Kladde in the compose.yaml from K2 can reach its database under the name of the service, such as Server=sql,1433, without ever knowing an address. In the default network bridge, where a docker run without --network ends up, these names don’t exist.

Private zones in Azure

Azure DNS hosts public zones such as microsoft.com above, and it knows private zones: zones that answer only in the virtual networks they are linked to. That is how private endpoints work, which you build for the Kladde in K4. This is what it looks like when the Kladde’s database has a private endpoint in the subnet snet-endpoints from lesson 3.0.2:

Question from the internet:
  kladde-sql-dev.database.windows.net                CNAME  kladde-sql-dev.privatelink.database.windows.net
  kladde-sql-dev.privatelink.database.windows.net    CNAME  (a public name of Azure SQL)
  …                                                  A      a public address

Question from vnet-kladde-dev (resolver 168.63.129.16, private zone privatelink.database.windows.net linked):
  kladde-sql-dev.database.windows.net                CNAME  kladde-sql-dev.privatelink.database.windows.net
  kladde-sql-dev.privatelink.database.windows.net    A      10.20.2.4

The Kladde keeps its connection string with the public name. Which address it gets is decided by DNS: in its own network the private zone answers with the private address from snet-endpoints, from outside the public chain answers. If you turn off the database’s public access, from outside only a rejection comes, although the name is still resolved. Every service has its own private zone, such as privatelink.vaultcore.azure.net for Key Vault, privatelink.blob.core.windows.net for Blob Storage and privatelink.servicebus.windows.net for Service Bus.

The most common error here: the private zone exists, but isn’t linked to the network, or a DNS server of your own in the network doesn’t forward the questions to 168.63.129.16. Then the Kladde gets the public address and is turned away. The first question on “private endpoint doesn’t work” is therefore always: Which address does the name resolve to from where the app runs?

Typical errors

example.com.               187  A      104.20.23.154
example.com.               187  A      172.66.147.243
example.com.               187  AAAA   2606:4700:10::6814:179a
example.com.             86400  NS     hera.ns.cloudflare.com.
example.com.             86400  NS     elliott.ns.cloudflare.com.
example.com.               300  MX     0 .
example.com.               300  TXT    "_k2n1y4vw3qtb4skdx9e7dxt97qrmmq9"
example.com.               300  TXT    "v=spf1 -all"
_dmarc.example.com.        300  TXT    "v=DMARC1;p=reject;sp=reject;adkim=s;aspf=s"
microsoft.com.            3600  MX     10 microsoft-com.mail.protection.outlook.com.
microsoft.com.            3600  CAA    0 contactemail "caarecordaware@microsoft.com"
microsoft.com.             300  SOA    ns1-39.azure-dns.com. azuredns-hostmaster.microsoft.com. 1

docker network create k3-net
docker run --rm -d --name kladde-web --network k3-net kladde-web
docker run --rm -it --network k3-net ubuntu:24.04 bash

getent hosts kladde-web
cat /etc/resolv.conf
apt-get update && apt-get install -y --no-install-recommends curl
curl -sS -o /dev/null -w '%{http_code}\n' http://kladde-web:8080/health

172.18.0.2      kladde-web
nameserver 127.0.0.11
options ndots:0
200

Wednesday 5:00 PM  Lower the TTL of app.kladde.example to 300 (address unchanged).
                   From Thursday 5:00 PM on, every resolver has dropped the old record with the long TTL.
Thursday 6:00 PM   Set the address to the new one. After 300 seconds at the latest, all resolvers ask anew.
Thursday 6:05 PM   Check the new address: dig +short app.kladde.example @1.1.1.1 and from a second network.
Thursday 7:00 PM   The old server may go off. An hour of buffer instead of five minutes to be safe,
                   because not every resolver follows the TTL exactly.
Friday             Raise the TTL again to 3600 or 86400.

The Kladde after this lesson

The Kladde changes in this lesson only if exercise 3 found an IP address in the compose.yaml. What you take away you will need again right away: in lesson 3.0.4, TLS checks whether the name in the server’s identity matches the name you called, and in lesson 3.0.7 SPF, DKIM and DMARC are nothing other than TXT records.

What you can do now

• You explain how a resolver finds its way from the root through the TLD to the authoritative server, and you follow this chain yourself.
• You know what is in a DNS message: header, question, ANSWER, AUTHORITY, ADDITIONAL.
• You read A, AAAA, CNAME, MX, TXT, CAA, NS and SOA and know the rules for CNAMEs.
• You plan a move with a lowered TTL and know why a long-lived HttpClient needs a limit on its connections.
• You explain how a private zone makes sure a private endpoint gets a private address in your own network.
• You reach services in Docker by name and know where the names come from.

Further reading

• Introduction to Azure DNS: a module from Microsoft Learn on public and private zones in Azure.
• Azure DNS alias records overview: alias records, also at the zone apex.
• Azure Private Endpoint private DNS zone values: the private zones of all services and how the chain of CNAMEs comes about.
• What is IP address 168.63.129.16?: the resolver in virtual networks of Azure.
• HttpClient guidelines for .NET: DNS, connections and PooledConnectionLifetime.
• Resolve-DnsName: parameters and outputs.