Solved!
Solution was to remove everything from AllowedIPs and change it back to 0.0.0.0/0. Now everything works.
For some reason it didn’t kill my local network connection and SSH this time… No idea why it did last time. Maybe fail2ban was to blame? Either way I moved to ufw which I like much more (uncomplicated firewall)