Docker Swarm across UK and Sweden: multi-region redundancy that actually holds

You have racks in two cities. Coventry and Stockholm, say. You run Docker Swarm and you want one cluster that survives losing either site. So you bring up three managers, put two in the UK and one in Sweden, join the workers, and call it multi-region. That cluster will fail at the worst possible moment, and it fails in a way most tutorials skip over entirely.

The problem is not Docker. It is Raft, the consensus protocol Swarm uses to keep manager state consistent. Raft was designed for machines on the same LAN, and the geometry of two sites breaks its one hard rule.

Why three managers across two sites is a trap

Swarm managers form a Raft cluster. Every change to cluster state has to be agreed by a majority of managers before it commits. The Docker documentation states the rule plainly: a quorum of (N/2)+1 managers must agree, and the cluster can tolerate (N-1)/2 failures.

Now split three managers across two sites. You cannot split three evenly, so one site holds two and the other holds one. Quorum is two.

Lose the site with two managers, which is exactly the disaster you built this to survive, and one manager is left. One is below the quorum of two. The survivor cannot elect a leader, cannot commit changes, and drops into a read-only state. Running containers keep running, but you cannot deploy, scale, reschedule, or drain a node. The admin guide confirms it: when quorum is lost, "swarm nodes cannot be added, updated, or removed, and new or existing tasks cannot be started, stopped, moved, or updated."

So the two-site swarm survives losing the small site and dies when it loses the big one. That is not redundancy. It is a coin flip weighted against you.

The usual reflex is to add a manager. Four managers, two per site, feels balanced. It is worse. Docker's guidance is blunt here: "An odd number of managers is recommended, because the next even number does not make the quorum easier to keep." Four managers need a quorum of three and still tolerate only one failure. Lose either site of two and you are down to two managers, below three. You bought a fourth machine to make the cluster more fragile.

There is a latency cost on top of the topology problem. More managers means more nodes that must acknowledge each write to cluster state, so write performance drops as you add them. Stretch those acknowledgements over a link with real round-trip time and every scheduling decision slows down. Between the UK and Sweden that link is not free. A common rule of thumb for fibre is roughly 10 ms of round trip per 1,000 km of route, and real paths are longer than the great-circle distance, so plan for something in the 20 to 40 ms range rather than the sub-millisecond RTT Raft was tuned for.

The two topologies that work

There are two honest ways to run Docker across two datacentres. Pick based on what you are actually protecting against.

Option A: managers in one region, workers in both

Keep all your managers in one site, ideally three of them on separate hosts so you tolerate one manager failure locally. Join workers from both sites into the same swarm. This gives you a single cluster, one docker service API, and workloads scheduled anywhere.

The trade is stark and you should say it out loud: the manager site is a single point of failure for control. Lose it and every running container in both regions keeps serving traffic, but you lose the ability to schedule, scale, or heal until the managers come back. Data-plane survives, control-plane does not.

This is the right choice when your two sites are one primary and one overflow or DR target, and you can tolerate a control-plane freeze during a primary-site outage while you promote by hand.

Option B: two independent single-region swarms behind DNS failover

Run a complete, self-contained swarm in each region. Each has its own managers, its own workers, its own local quorum. Neither depends on the other to make decisions. In front of both you put health-checked DNS failover, so the name resolves to the UK site while it is healthy and flips to Sweden when it is not.

Nothing shares Raft state across the link, so the 30 ms RTT never touches consensus. Each site heals itself. The cost is that you now operate two clusters and deploy to both, and your data layer has to exist independently in each region, which is the hard part covered further down.

Which one to pick

For a two-datacentre operator who wants each site to survive the other going dark, choose Option B. Independent swarms with DNS failover is the design that has no shared failure mode across the link. Reach for Option A only when you genuinely have a primary and a satellite, and a control-plane freeze during a primary outage is acceptable.

What almost nobody should build is the stretched three-manager swarm you started with.

The network: an encrypted overlay over WireGuard

If you go with Option A, or you want a private path between the two independent swarms for replication, you need an encrypted link between regions. Swarm's own control traffic is already encrypted with AES-GCM, but the overlay data plane is not encrypted by default.

Docker offers --opt encrypted on overlay networks, which turns on IPsec at the VXLAN layer. It works, and it carries a performance penalty that the community has flagged for years, so test it under load before you trust it on a cross-region link. Many operators put WireGuard underneath instead, because its overhead is low enough to run alongside a production cluster spanning datacentres, per discussion in the Docker community forums.

The pattern is a WireGuard tunnel between the two sites, then Swarm gossip and overlay traffic pinned to the tunnel interface. Bring up WireGuard first:

Then initialise the swarm advertising the WireGuard address, so gossip and VXLAN ride the tunnel rather than the public interface:

Lock the firewall so Swarm's ports (2377/tcp for management, 7946 for gossip, 4789/udp for VXLAN) are only reachable inside the tunnel. If you are still building that base, our notes on securing a VPS with a firewall, SSH hardening and fail2ban cover the groundwork.

The routing mesh will drag traffic across the link

Here is the surprise that shows up in your latency graphs after everything looks healthy. Swarm's ingress routing mesh makes every node accept connections on a published port, whether or not a task for that service runs locally. A request hitting a UK node for a service whose only healthy replica sits in Sweden gets transparently forwarded over the tunnel and back. You added 30 ms of RTT to a request that had no business leaving the country.

The fix is to take the port off the ingress mesh and bind it to the host, then constrain where the service runs. Publishing in host mode is the documented way to bypass the routing mesh. The long-form syntax:

With mode=host the container port binds directly to the host, so a request to a UK node is served by a UK container or by nothing. That last part matters: the docs warn that on a node not running the task, "it is possible that nothing is listening, or that a completely different application is listening." So you pair host-mode publishing with a placement constraint and a per-region entry point, run the service in each region, and let your DNS or edge load balancer decide which region a client reaches. If you have wrestled with published ports not answering, our write-up on a Docker container port that is not accessible walks the same debugging path.

Volumes do not replicate. Your data layer has to.

This is where a two-region swarm quietly stops being redundant. A Docker volume is local to the node that holds it. Swarm will not copy it to another node, let alone another country. Reschedule a stateful task to the other region and it comes up with an empty volume. The second site looks like a warm standby and holds none of your data.

You have two real choices. Put a replicated network filesystem underneath, which over a 30 ms link punishes every write, or let the data engine replicate itself. For Postgres, that means streaming replication. The PostgreSQL documentation describes it: "the standby connects to the primary, which streams WAL records to the standby as they are generated." Run the primary in one region and a hot standby in the other.

Keep replication asynchronous across regions. Synchronous replication makes every commit wait for the standby to acknowledge, and the docs are explicit that "the minimum wait time is the round-trip time between primary and standby." At 30 ms that tax lands on every write transaction. Asynchronous replication has a delay typically under a second, at the cost of a small window of possible data loss if the primary dies before shipping its last records. For cross-region redundancy that trade is usually right. Watch the lag directly:

If those locations drift far apart, your standby is falling behind and a failover would lose more than you think.

The failure test: kill a region and watch

None of this is real until you have pulled the plug and read the output. Take down the primary site, or block its WireGuard peer, and observe.

On a surviving Option B swarm in Sweden, nothing dramatic happens locally. Its own managers still have local quorum. docker node ls answers instantly and shows only its own nodes. DNS failover notices the UK health check going red and flips the record. The window your users feel is the DNS TTL plus health-check interval, so keep both short.

On a stretched Option A swarm where you lost the manager site, the output sends people down the wrong path. Run docker node ls on the surviving worker-turned-manager and you do not get a clean error about quorum. You get this:

People read "deadline exceeded" as a network timeout and start poking at the tunnel, when the truth is the manager cannot reach quorum and the API is refusing to answer. If you truly cannot bring the lost managers back, the documented recovery is to force a new single-manager cluster from the survivor:

That command removes every manager except the one you run it from and restores your ability to administer the swarm. It is a recovery of last resort, not a failover you want to lean on. The fact that you need it at all is the argument against the stretched topology.

Write down what happened in each test: how long until traffic recovered, whether the standby had current data, whether any manual step was needed. If the honest answer is "the cluster went read-only and I had to run --force-new-cluster by hand," you have not built redundancy. You have built a cluster that needs you awake at 2am.

The decision rule

For two datacentres where each must survive the other, run two independent single-region swarms behind health-checked DNS failover, replicate state with your database's own tooling, and keep asynchronous replication across the link. Reserve the single-manager-region design for a genuine primary-and-satellite setup where a control-plane freeze is acceptable. And whatever you do, never stretch three Raft managers across two sites, because the site holding two will be the one you lose.

Sources

1. Raft consensus in swarm mode | Docker Docs (unknown)
2. Administer and maintain a swarm of Docker Engines | Docker Docs (unknown)
3. Use Swarm mode routing mesh | Docker Docs (unknown)
4. PostgreSQL: Log-Shipping Standby Servers (unknown)
5. Using Swarm with WireGuard instead of Overlay Network? - Docker Community Forums (unknown)
6. It's Time To Learn About Latency - TeleGeography (unknown)

Frequently asked questions

Can I run Docker Swarm managers across two regions?

You can, but you should not for high availability. Three managers split across two sites end up two-and-one, and losing the site with two managers drops you below quorum, freezing all cluster changes. Keep managers in one region, or run two independent single-region swarms behind DNS failover.

Why does losing quorum only break deploys and not running containers?

Raft quorum governs the control plane. Without a majority of managers, Swarm cannot commit state changes, so you cannot deploy, scale, or reschedule. Containers already running keep serving traffic on the data plane, which does not depend on live consensus.

Should I use --opt encrypted or WireGuard for cross-region overlay traffic?

Both encrypt the link. Docker's --opt encrypted enables IPsec at the VXLAN layer with a measurable performance penalty. WireGuard runs underneath the overlay with low overhead and is a common choice for production clusters spanning datacentres. Test either under real load before trusting it.

How do I stop Swarm sending requests to a container in the other region?

Publish the port in host mode with the long-form syntax (mode=host) to bypass the ingress routing mesh, then add a placement constraint so the service only runs in the intended region. Requests to a node are then served locally or not at all, never forwarded over the inter-region link.

Do Docker volumes replicate between regions?

No. A volume is local to its node and Swarm never copies it elsewhere. For redundancy, either put a replicated filesystem underneath, which is slow over a high-latency link, or let the data engine replicate itself, such as PostgreSQL streaming replication with an asynchronous standby in the second region.