← Notes

pathosd: what it took to announce an address only while the service is up

October 7, 2026 · bgp, bird, frr, anycast, go, openwrt

Some services in my network run in several places at once — a DNS resolver on every hub, the S3 gateway on more than one storage node. Clients should see a single stable address, and routing should send them to the nearest instance that actually works. In a routed network that’s anycast: every instance announces the same address over BGP. The catch is the word works — a route has no idea whether the service behind it is alive.

pathosd closes that gap. It is a health checker and a BGP route originator in one Go process: it checks the service locally (HTTP, DNS, ping, TCP, UDP, gRPC) and announces the address only while the check passes, or keeps it with a lower priority so others are preferred. keepalived/VRRP solves a similar problem, but on one layer-2 segment with a shared MAC — not across a routed mesh of tunnels.

The README covers what it can do. This note is about how it got there — mostly the things that didn’t work the first time.

Addresses that never went away

March 2026. My hubs got shared anycast addresses — the DNS resolver first. Each hub put them on a dummy interface and FRR announced them into the mesh. Simple, and fine until a service on one hub stopped: the route stayed, the hub kept attracting traffic, and the traffic went nowhere.

The fix is obvious on paper — announce only while the service is healthy — and I didn’t want that split across a routing daemon and a check script that could disagree. So pathosd embeds GoBGP and does both. If it dies, its sessions drop and the routes go with them; addresses start withdrawn and need a few good checks in a row before they appear.

FRR on the same box

April. The natural setup is pathosd talking to the FRR on the same router. It never really settled: first a session over loopback addresses, then over dedicated /32s on a dummy interface — and by the end of the same day I gave up on it.

So I stopped fighting it: each hub’s pathosd peered with the FRR on the other hubs instead, and those only accepted the VIP prefixes from it. That was stable. The DNS address, the one that really mattered, went back to a plain static announcement for the time being. One small thing from that period I kept: every hub’s pathosd got its own AS number, which makes an AS path tell you at a glance which hub an announcement came from.

One day with BIRD

August 11. When the mesh moved from FRR to BIRD, I wanted the local session back — the VIP should be announced by the hub that actually serves it. That took most of a day:

1. Both sides waiting for each other. pathosd dialled BIRD on 179 while BIRD dialled pathosd on its own port; nobody listened where the other was calling, and the session sat in Active.
2. GoBGP refused to dial itself. With pathosd as the active side and the same loopback address on both ends, GoBGP’s protection against connecting to itself kicked in. pathosd became the passive side; BIRD dials it.
3. “Invalid next hop.” pathosd announced the VIP with 127.0.0.1 as next hop. It now announces the router’s LAN address instead.
4. “Invalid remote address.” BIRD looks up a direct BGP neighbour on a connected network, and loopback isn’t one. multihop skips the lookup; in the end the session moved to a second address on the LAN bridge.
5. “Unreachable.” Even with a real next hop, BIRD won’t route to an address that is its own, so it marked the route unreachable. I tried simply exporting the VIP as a connected route instead — reverted it, re-applied it — but that loses the point: no health control and no way to step back.

The fix that stuck is a few lines of BIRD’s import filter:

import filter {
    if net ~ [ 192.0.2.53/32 ] then {   # only our VIPs
        bgp_next_hop = net.ip;          # next hop = the VIP itself
        preference = 250;               # beat the connected route (240)
        accept;
    }
    reject;
};

BIRD still considers the route unreachable locally, but with a preference above the connected route it selects it and exports it to the mesh peers — and that is all it needs to do.

With that in place the DNS address came back under pathosd, and it now steps back instead of disappearing: when the local check fails, the route stays with a prepended AS path, so the mesh prefers another hub but still has this one if everything else is worse. The prepend passes through BIRD untouched, no iBGP or communities needed.

The same week pathosd learned HTTP checks through a forward proxy, so a shared proxy address is announced only while the proxy really forwards. And to see what was going on inside, I switched on its embedded GoBGP API for the gobgp CLI — on localhost TCP, because the CLI can’t talk to a unix socket.

A storage node that answered from the wrong place

Late April, and again in October. The S3 gateway runs on plain Debian storage nodes, so pathosd got a Debian package and the nodes announce the gateway address to the homelab router, checked over TLS.

The surprise came later. A client on the same LAN segment as a storage node sent its request to the gateway address through the router, as it should — but the node answered directly over the segment, from the gateway address. The client saw a reply from an address it had never talked to on that link and dropped it. The fix is on the node, not in pathosd: replies sourced from the gateway address are policy-routed back through the router.

And one cleanup: pathosd only announces, it never needs routes back, so the router now exports nothing to such advertiser-only peers.