Who would have thunk it? (Not me, that's for certain.)
Anyhoo if you're running a VPN on your SBC and add Docker to experiment with some other stuff (safe in the knowledge that you're less likely to smeg up DietPi!) Docker doesn't start.
This has something to do with the bridge networking and I won't pretend I understand it - perhaps the guys can figure it out. However, if you don't need the bridge (which is meant for inter-container communication) you can disable it in /etc/docker/daemon.json by adding the line:
Yeah I guess when the docker containers try to communicate through the bridge, the VPN route rules catch the requests and tunnels them through the VPN where nothing listens to. So another possibility would be to add docker specific routes.
But generally good to know and at least there is an easy solution if the bridge is not required.