Use a different port for WG server, say 51821.
Then create a rule for the traffic originating from port 51821 to use ISP.
echo '10 isp' >> /etc/iproute2/rt_tables
ip route add to default via 192.168.1.1 table isp
ip rule add iif lo sport 51821 to default lookup isp prio 15010