When a quote is accepted, sales should hear about it. The quote is in the database, the message goes over Service Bus. Two systems, two writes, and the process can die in between. In K2 you got to know the outbox for the confirmation mails of a booking. In this lesson it becomes a tool of architecture: You see why there is no shortcut, get to know three forms of the outbox for three kinds of storage, and publish QuoteAccepted so that neither a crash nor a lost reply loses or duplicates anything.

Two systems, no common commit

A database transaction makes several changes one: all or none. Across two systems that doesn't exist. Service Bus can bundle several operations into one transaction, but only within its own namespace; a common transaction with SQL Server doesn't exist. So an order has to be chosen, and every order has a gap.

The following program plays through three cases, with a database and a bus in memory. A “crash” here is simply the end of a run at a certain point. Save it as two-systems.cs:

C#
// two-systems.cs
// State in the database, message on the bus: what a crash does at each point.
var db = new Database();
var bus = new Bus();

// A: save first, then send. The process dies in between.
db.Commit(tx => tx.SetStatus("q-1", "accepted"));
// <- Crash: Send is never reached.
Console.WriteLine($"A  Quote q-1: {db.Status("q-1")}, messages on the bus: {bus.Accepted.Count}");

// B: send first, then save. Saving fails (write conflict).
bus.Send(new Message(Guid.NewGuid(), "q-2 accepted"));
try
{
    db.Commit(tx => throw new InvalidOperationException("Write conflict"));
}
catch (InvalidOperationException)
{
}
Console.WriteLine($"B  Quote q-2: {db.Status("q-2")}, messages on the bus: {bus.Accepted.Count}");

// C: outbox. State and message in one transaction; a relay sends later.
var bus2 = new Bus();
db.Commit(tx =>
{
    tx.SetStatus("q-3", "accepted");
    tx.AddToOutbox(new Message(Guid.Parse("0b7d6c2a-3f51-4e8b-9c1d-5a2e7f9b0c44"), "q-3 accepted"));
});
// <- Crash before sending: The message waits in the outbox.
var relay = new Relay(db, bus2);
relay.RunOnce(crashAfterSend: true); // sends, dies before checking it off
relay.RunOnce(crashAfterSend: false); // after the restart: sends once more and checks it off
Console.WriteLine($"C  Quote q-3: {db.Status("q-3")}, sent: {bus2.Sends}, accepted: {bus2.Accepted.Count}, discarded as duplicate: {bus2.Duplicates}, open in the outbox: {db.Pending().Count}");

public sealed record Message(Guid MessageId, string Body);

public sealed class Database
{
    private readonly Dictionary<string, string> statuses = [];
    private readonly List<(Message Message, bool Sent)> outbox = [];

    public string Status(string quote) => statuses.GetValueOrDefault(quote, "unchanged");

    public List<Message> Pending() => [.. outbox.Where(o => !o.Sent).Select(o => o.Message)];

    public void MarkSent(Guid messageId)
    {
        var index = outbox.FindIndex(o => o.Message.MessageId == messageId);
        outbox[index] = (outbox[index].Message, true);
    }

    // All or nothing: Only when the work runs through without an exception is it taken over.
    public void Commit(Action<Transaction> work)
    {
        var tx = new Transaction();
        work(tx);
        foreach (var (quote, status) in tx.Statuses)
        {
            statuses[quote] = status;
        }

        outbox.AddRange(tx.Outbox.Select(m => (m, false)));
    }
}

public sealed class Transaction
{
    public List<(string Quote, string Status)> Statuses { get; } = [];

    public List<Message> Outbox { get; } = [];

    public void SetStatus(string quote, string status) => Statuses.Add((quote, status));

    public void AddToOutbox(Message message) => Outbox.Add(message);
}

// A bus with duplicate detection on the MessageId, like a Service Bus queue from the Standard tier up.
public sealed class Bus
{
    private readonly HashSet<Guid> seen = [];

    public int Sends { get; private set; }

    public int Duplicates { get; private set; }

    public List<Message> Accepted { get; } = [];

    public void Send(Message message)
    {
        Sends++;
        if (seen.Add(message.MessageId))
        {
            Accepted.Add(message);
        }
        else
        {
            Duplicates++;
        }
    }
}

public sealed class Relay(Database db, Bus bus)
{
    public void RunOnce(bool crashAfterSend)
    {
        foreach (var message in db.Pending())
        {
            bus.Send(message);
            if (crashAfterSend)
            {
                return; // Crash: sent, but not checked off
            }

            db.MarkSent(message.MessageId);
        }
    }
}
Bash
dotnet run two-systems.cs
A  Quote q-1: accepted, messages on the bus: 0
B  Quote q-2: unchanged, messages on the bus: 1
C  Quote q-3: accepted, sent: 2, accepted: 1, discarded as duplicate: 1, open in the outbox: 0
  • A loses the message: The quote is accepted, sales never finds out.
  • B sends a message about something that never happened. Sales wins a deal for a quote that is still open in the quotes.
  • C is the outbox. The message is stored together with the state, and a relay sends it afterward. If the relay dies between sending and checking off, it sends once more after the restart. The bus recognizes the second copy by its MessageId and discards it.