2023-12-14 - OpenVPN and fast reconnects

i use VPN for all my outgoing traffic. there are however times, when VPN connection dies for some random reasons and we need to reconnect. of course – tough luck – this usually happens when you're busy doing other stuff (eg. telko). while i can go to router and restart and / or SSH to kick-restart OpenVPN service, this ain't perfect as it takes time and it can be done by me only – rest of the family is at my mercy.

my first approach to auto detect failed connect and restart was just using cron, to ping some should-always-be-there servers, and if all fail, restart OpenVPN. this however was limited, as job was running once per 10 minutes + it took a moment to determine network is down (a couple timeouts typically). while in theory i could go down to 1min interval in cron, this often buffered more “instances” when connection was down, causing cascade of restarts, even when connection was already back. it did stabilize, but it was annoying enough that i disabled it.

there are obviously ping options for OpenVPN, so that connection can self-monitor. cool, but by provider pushes these settings for me, and it's in many-minutes range, while i really want it to happen swiftly. fortunately turned out that starting with OpenVPN 2.4.0, there's a pull-filter ignore <command> syntax, that allows you to selectively ignore pushed configs, so:

pull-filter ignore ping
pull-filter ignore ping-restart
ping 2
ping-restart 13
ping-timer-rem

it was quite good, BUT restarts were never successful, if remote end dies (one can simulate this with explicit DROP rule in iptables). TL;DR – turned out that persist-tun option was the issue. so another fix:

#persist-key
#persist-tun
pull-filter ignore persist-key
pull-filter ignore persist-tun

so now we have OpenVPN that correctly detects failed connection, drops it, restarts it… or sometimes gives up and exits client. since it was exit 0, systemd does not restart VPN… and connection is permanently dead.

btw:

ping-exit 13

had the same outcome - exit 0, and systemd in this is fine mode. ok – systemd magic time, then…

while obvious part was to just edit OpenVPN service settings in systemd, the problem is that this will evaporate with next OpenVPN update (that comes with service definition). fortunately there's a lesser-known systemd trick, to create overrides for a given service! just do this:

mkdir /etc/systemd/system/openvpn@myvpn.service.d
cd $_
vi override.conf
systemctl daemon-reload

where override.conf goes like this:

[Service]
RestartSec=1s
Restart=always

so now we restart every 1s, if service turns off, for whatever reason.

one last thing, is to disable default rate-limiting in systemd, to make sure we try to connect as hard as possible. full override.conf then looks like this:

[Service]
RestartSec=1s
Restart=always
StartLimitIntervalSec=0
StartLimitBurst=0

another systemctl daemon-reload + one more systemctl restart openvpn@myvpn and we're golden! :)