Help remotely connecting using wireguard while NordVPN is running on PI

DaringDougler · 7 February 2022 01:13

PROBLEM
My Pi is primarily used for Pihole and Home Automation (over Nordvpn). I need HELP remotely connecting to my network using Wireguard, while NordVPN is running on my Pi. I’ve tried a bunch of different routing rules, but I cant seem to get it. My wireguard client is unable to complete a handshake. Also I am ‘listening to all interfaces/all orgins’ on PIhole. If I chose one hop, i need to restart the pihole DNS after each reboot. I’ve seen a lot of threads on that as well, but I’m not sure what the right answer is (I’m really only using eth0, wg0 and nordlynx)

REQUIREMENTS

All clients on LAN can use Pihole to filter adds (working)
All clients on LAN can ping each other even when NordVPN is running on both the client and on the PI (working – whitelisted subnet in Nord)
Ability to remotely connect from WAN to home network IPs using Wireguard (partial. Only works when i turn Nordvpn off on the PI)
Ability to remotely connect to home network through Wireguard for DNS/Internet accesss using Pihole Ad filtering (same as 2 - works only when NordVpn is off on PI)

NETWORK

Gateway 192.168.1.1 (192.168.1.0/24)
PI 192.168.1.28
Wireguard Interface: (wg0)         ---udp connection on port 51820
NordVPN Interface:  (nordlynx)   ---udp connection also on port 51820
NordVPN Whitelist IP 192.168.1.0/24

DEFAULT IP TABLES - after powering on (nordvpn is autostarted) (I x’d out the public IP. I think that is Nord?)

    0     0 ACCEPT     all  --  wlan0  any     99-177-xx-xxx.reverse-dns  anywhere            
  349  154K ACCEPT     all  --  eth0   any     99-177-xx-xxx.reverse-dns  anywhere            
    0     0 ACCEPT     all  --  wlan0  any     192.168.1.0/24       anywhere            
 1725  277K ACCEPT     all  --  eth0   any     192.168.1.0/24       anywhere            
    0     0 DROP       all  --  wlan0  any     anywhere             anywhere            
   57 25192 DROP       all  --  eth0   any     anywhere             anywhere            

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 939 packets, 121K bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  any    wlan0   anywhere             99-177-xx-xxx.reverse-dns 
  351 57724 ACCEPT     all  --  any    eth0    anywhere             99-177-xx-xxx.reverse-dns 
    0     0 ACCEPT     all  --  any    wlan0   anywhere             192.168.1.0/24      
  576 65434 ACCEPT     all  --  any    eth0    anywhere             192.168.1.0/24      
    0     0 DROP       all  --  any    wlan0   anywhere             anywhere            
   10  1020 DROP       all  --  any    eth0    anywhere             anywhere

ROUTE

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.5.0.2        128.0.0.0       UG    0      0        0 nordlynx
0.0.0.0         192.168.1.1     0.0.0.0         UG    202    0        0 eth0
10.6.0.0        0.0.0.0         255.255.255.0   U     0      0        0 wg0
128.0.0.0       10.5.0.2        128.0.0.0       UG    0      0        0 nordlynx
172.93.yyy.yy   192.168.1.1     255.255.255.255 UGH   0      0        0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U     202    0        0 eth0

Joulinar · 7 February 2022 12:56

basically quite an old request to have some kind of split tunnel feature. So far no real solution from our side.

https://github.com/MichaIng/DietPi/issues/2758#issuecomment-1020349622

trendy · 7 February 2022 13:36

Use a different port for WG server, say 51821.
Then create a rule for the traffic originating from port 51821 to use ISP.

echo '10 isp' >> /etc/iproute2/rt_tables 
ip route add to default via 192.168.1.1 table isp
ip rule add iif lo sport 51821 to default lookup isp prio 15010

DaringDougler · 8 February 2022 05:34

Hi again!

I changed my WG server to listen on 51821 and reconfirmed with Nord that they are using 51820. I also added your firewall rules below (though they are way beyond my comprehension ).

On the first /etc/iproute2/rt_tables —Initially I was not able to execute that, even when prefacing it with sudo. I had to sudo su to get it to run without giving me a permissions failure (but not sure if it actually did what it was supposed to do). I don’t completely understand why it didn’t fail permissions when I logged in as root, versus when I used sudo.

Unfortunately, I am having the same end result. If I turn OFF Nordvpn, I can connect to wireguard. As soon as I turn ON Nord, my connection to my wireguard client drops and is unable to complete a handshake.

I’ve seen a number of threads where a lot of people where trying to do this, but they were using OpenVPN. I really want to get this working with wireguard/Nordlynx. The performance difference is dramatic. My pi averages about 400mbps download speeds over Nordlynx with an SSL connection to boot. OpenVpn performance is like half of that.

trendy · 8 February 2022 07:21

I don’t completely understand why it didn’t fail permissions when I logged in as root, versus when I used sudo.

Because you sudo the echo, when the part that needs it is the output redirect to the rt_table file.
Post the following so we can troubleshoot:

ss -tunlp | grep 51821; ip -4 addr; ip -4 ro list table all; ip -4 ru; iptables-save -c

DaringDougler · 8 February 2022 07:29

I rebooted a few times since. Do I need to renter the firewall rules before I run that, or do they persist?

trendy · 8 February 2022 08:56

They are not firewall rules, they are ip rules. Except the first one, the other two won’t survive a reboot.

DaringDougler · 9 February 2022 04:43

I really appreciate your help on this! See results of your query below…

root@pivpn:/etc/samba# ss -tunlp | grep 51821; ip -4 addr; ip -4 ro list table all; ip -4 ru; iptables-save -c
udp   UNCONN 0      0            0.0.0.0:51821      0.0.0.0:*                                                          
udp   UNCONN 0      0                  *:51821            *:*                                                          
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    inet 192.168.1.28/24 brd 192.168.1.255 scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
4: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.6.0.1/24 scope global wg0
       valid_lft forever preferred_lft forever
10: nordlynx: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.5.0.2/32 scope global nordlynx
       valid_lft forever preferred_lft forever
default via 192.168.1.1 dev eth0 table isp 
0.0.0.0/1 via 10.5.0.2 dev nordlynx 
default via 192.168.1.1 dev eth0 src 192.168.1.28 metric 202 
10.6.0.0/24 dev wg0 proto kernel scope link src 10.6.0.1 
128.0.0.0/1 via 10.5.0.2 dev nordlynx 
185.203.219.132 via 192.168.1.1 dev eth0 
192.168.1.0/24 dev eth0 proto dhcp scope link src 192.168.1.28 metric 202 
local 10.5.0.2 dev nordlynx table local proto kernel scope host src 10.5.0.2 
broadcast 10.6.0.0 dev wg0 table local proto kernel scope link src 10.6.0.1 
local 10.6.0.1 dev wg0 table local proto kernel scope host src 10.6.0.1 
broadcast 10.6.0.255 dev wg0 table local proto kernel scope link src 10.6.0.1 
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1 
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1 
broadcast 192.168.1.0 dev eth0 table local proto kernel scope link src 192.168.1.28 
local 192.168.1.28 dev eth0 table local proto kernel scope host src 192.168.1.28 
broadcast 192.168.1.255 dev eth0 table local proto kernel scope link src 192.168.1.28 
0:	from all lookup local
15010:	from all iif lo sport 51821 lookup isp
32766:	from all lookup main
32767:	from all lookup default
# Generated by iptables-save v1.8.7 on Tue Feb  8 22:41:15 2022
*filter
:INPUT ACCEPT [137106:427632728]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [53372:156389407]
[0:0] -A INPUT -s 185.203.219.132/32 -i wlan0 -j ACCEPT
[363419:460182616] -A INPUT -s 185.203.219.132/32 -i eth0 -j ACCEPT
[0:0] -A INPUT -s 192.168.1.0/24 -i wlan0 -j ACCEPT
[256690:22840095] -A INPUT -s 192.168.1.0/24 -i eth0 -j ACCEPT
[0:0] -A INPUT -i wlan0 -j DROP
[283:52551] -A INPUT -i eth0 -j DROP
[0:0] -A OUTPUT -d 185.203.219.132/32 -o wlan0 -j ACCEPT
[143252:168256296] -A OUTPUT -d 185.203.219.132/32 -o eth0 -j ACCEPT
[0:0] -A OUTPUT -d 192.168.1.0/24 -o wlan0 -j ACCEPT
[107097:2026616021] -A OUTPUT -d 192.168.1.0/24 -o eth0 -j ACCEPT
[0:0] -A OUTPUT -o wlan0 -j DROP
[342:26732] -A OUTPUT -o eth0 -j DROP
COMMIT
# Completed on Tue Feb  8 22:41:15 2022
# Generated by iptables-save v1.8.7 on Tue Feb  8 22:41:15 2022
*nat
:PREROUTING ACCEPT [13050:1146492]
:INPUT ACCEPT [12837:1129794]
:OUTPUT ACCEPT [4303:307241]
:POSTROUTING ACCEPT [3977:281980]
[0:0] -A POSTROUTING -s 10.6.0.0/24 -o eth0 -m comment --comment wireguard-nat-rule -j MASQUERADE
COMMIT
# Completed on Tue Feb  8 22:41:15 2022

trendy · 9 February 2022 11:14

Who set up the firewall rules? You are dropping everything coming to eth0

[283:52551] -A INPUT -i eth0 -j DROP

and the wg clients are not allowed to communicate to the server.

DaringDougler · 9 February 2022 17:32

We are making progress! The firewall rules are auto-generated by Nord. I have two variables to play with (Killswitch, Firewall). I reran your query and retested for the following permutations 1) Firewall and Killswitch disabled. 2) Firewall enabled Killswitch disabled. 3 NordVPN disconnected. When I add your iptable rules I can remotely connect to LAN IP addresses as long as the KILLSWITCH is DISABLED (both scenario 1 and 2). However, I have no access to internet outside my LAN. When I disconnect Nord, I have access to LAN and Internet through wireguard.

If we can get the internet working (with the NordVPN KillSwitch disabled), I’d also like to lock down the pi so that it has no internet connectivity without Nord (a custom killswitch firewall rule). I want to prevent automatic downloads from occurring without VPN connection. In terms of internet connectivity, my preference would be a double-hop (inbound wireguard connections get forwarded to Nord)

pi@pivpn:~ $ nordvpn settings
Technology: NORDLYNX
Firewall: disabled
Kill Switch: disabled
CyberSec: disabled
Notify: disabled
Auto-connect: enabled
IPv6: disabled
DNS: 192.168.1.28
Whitelisted subnets:
	192.168.1.0/24
pi@pivpn:~ $ nordvpn connect chicago
Connecting to United States #9310 (us9310.nordvpn.com)
You are connected to United States #9310 (us9310.nordvpn.com)!

pi@pivpn:~ $ sudo su
root@pivpn:/home/pi# ss -tunlp | grep 51821; ip -4 addr; ip -4 ro list table all; ip -4 ru; iptables-save -c
udp   UNCONN 0      0            0.0.0.0:51821      0.0.0.0:*                                                         
udp   UNCONN 0      0                  *:51821            *:*                                                         
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    inet 192.168.1.28/24 brd 192.168.1.255 scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
4: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.6.0.1/24 scope global wg0
       valid_lft forever preferred_lft forever
20: nordlynx: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.5.0.2/32 scope global nordlynx
       valid_lft forever preferred_lft forever
default via 192.168.1.1 dev eth0 table isp 
0.0.0.0/1 via 10.5.0.2 dev nordlynx 
default via 192.168.1.1 dev eth0 src 192.168.1.28 metric 202 
10.6.0.0/24 dev wg0 proto kernel scope link src 10.6.0.1 
128.0.0.0/1 via 10.5.0.2 dev nordlynx 
185.203.219.130 via 192.168.1.1 dev eth0 
192.168.1.0/24 dev eth0 proto dhcp scope link src 192.168.1.28 metric 202 
local 10.5.0.2 dev nordlynx table local proto kernel scope host src 10.5.0.2 
broadcast 10.6.0.0 dev wg0 table local proto kernel scope link src 10.6.0.1 
local 10.6.0.1 dev wg0 table local proto kernel scope host src 10.6.0.1 
broadcast 10.6.0.255 dev wg0 table local proto kernel scope link src 10.6.0.1 
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1 
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1 
broadcast 192.168.1.0 dev eth0 table local proto kernel scope link src 192.168.1.28 
local 192.168.1.28 dev eth0 table local proto kernel scope host src 192.168.1.28 
broadcast 192.168.1.255 dev eth0 table local proto kernel scope link src 192.168.1.28 
0:	from all lookup local
15010:	from all iif lo sport 51821 lookup isp
32766:	from all lookup main
32767:	from all lookup default
# Generated by iptables-save v1.8.7 on Wed Feb  9 11:15:34 2022
*filter
:INPUT ACCEPT [16819454:73736317088]
:FORWARD ACCEPT [20575:10602192]
:OUTPUT ACCEPT [5472150:953778801]
[0:0] -A INPUT -s 185.203.219.130/32 -i wlan0 -j ACCEPT
[101:37144] -A INPUT -s 185.203.219.130/32 -i eth0 -j ACCEPT
[0:0] -A INPUT -s 192.168.1.0/24 -i wlan0 -j ACCEPT
[1706:207709] -A INPUT -s 192.168.1.0/24 -i eth0 -j ACCEPT
[0:0] -A INPUT -i wlan0 -j DROP
[202:33554] -A INPUT -i eth0 -j DROP
[0:0] -A OUTPUT -d 185.203.219.130/32 -o wlan0 -j ACCEPT
[68:12052] -A OUTPUT -d 185.203.219.130/32 -o eth0 -j ACCEPT
[0:0] -A OUTPUT -d 192.168.1.0/24 -o wlan0 -j ACCEPT
[941:138809] -A OUTPUT -d 192.168.1.0/24 -o eth0 -j ACCEPT
[0:0] -A OUTPUT -o wlan0 -j DROP
[94:10884] -A OUTPUT -o eth0 -j DROP
COMMIT
# Completed on Wed Feb  9 11:15:34 2022
# Generated by iptables-save v1.8.7 on Wed Feb  9 11:15:34 2022
*nat
:PREROUTING ACCEPT [40971:3576797]
:INPUT ACCEPT [39555:3396272]
:OUTPUT ACCEPT [15935:1091185]
:POSTROUTING ACCEPT [15774:1123291]
[404:82821] -A POSTROUTING -s 10.6.0.0/24 -o eth0 -m comment --comment wireguard-nat-rule -j MASQUERADE
COMMIT
# Completed on Wed Feb  9 11:15:34 2022

root@pivpn:/home/pi# nordvpn set firewall enabled
Firewall is set to 'enabled' successfully.
root@pivpn:/home/pi# nordvpn settings
Technology: NORDLYNX
Firewall: enabled
Kill Switch: disabled
CyberSec: disabled
Notify: disabled
Auto-connect: enabled
IPv6: disabled
DNS: disabled

root@pivpn:/home/pi# nordvpn connect chicago
Connecting to United States #9045 (us9045.nordvpn.com)
You are connected to United States #9045 (us9045.nordvpn.com)!

root@pivpn:/home/pi# ss -tunlp | grep 51821; ip -4 addr; ip -4 ro list table all; ip -4 ru; iptables-save -c
udp   UNCONN 0      0            0.0.0.0:51821      0.0.0.0:*                                                         
udp   UNCONN 0      0                  *:51821            *:*                                                         
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    inet 192.168.1.28/24 brd 192.168.1.255 scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
4: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.6.0.1/24 scope global wg0
       valid_lft forever preferred_lft forever
22: nordlynx: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.5.0.2/32 scope global nordlynx
       valid_lft forever preferred_lft forever
default via 192.168.1.1 dev eth0 table isp 
0.0.0.0/1 via 10.5.0.2 dev nordlynx 
default via 192.168.1.1 dev eth0 src 192.168.1.28 metric 202 
10.6.0.0/24 dev wg0 proto kernel scope link src 10.6.0.1 
108.62.107.241 via 192.168.1.1 dev eth0 
128.0.0.0/1 via 10.5.0.2 dev nordlynx 
192.168.1.0/24 dev eth0 proto dhcp scope link src 192.168.1.28 metric 202 
local 10.5.0.2 dev nordlynx table local proto kernel scope host src 10.5.0.2 
broadcast 10.6.0.0 dev wg0 table local proto kernel scope link src 10.6.0.1 
local 10.6.0.1 dev wg0 table local proto kernel scope host src 10.6.0.1 
broadcast 10.6.0.255 dev wg0 table local proto kernel scope link src 10.6.0.1 
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1 
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1 
broadcast 192.168.1.0 dev eth0 table local proto kernel scope link src 192.168.1.28 
local 192.168.1.28 dev eth0 table local proto kernel scope host src 192.168.1.28 
broadcast 192.168.1.255 dev eth0 table local proto kernel scope link src 192.168.1.28 
0:	from all lookup local
15010:	from all iif lo sport 51821 lookup isp
32766:	from all lookup main
32767:	from all lookup default
# Generated by iptables-save v1.8.7 on Wed Feb  9 11:16:53 2022
*filter
:INPUT ACCEPT [16820518:73736540247]
:FORWARD ACCEPT [20583:10602613]
:OUTPUT ACCEPT [5472965:953918279]
COMMIT
# Completed on Wed Feb  9 11:16:53 2022
# Generated by iptables-save v1.8.7 on Wed Feb  9 11:16:53 2022
*nat
:PREROUTING ACCEPT [41122:3589950]
:INPUT ACCEPT [39706:3409425]
:OUTPUT ACCEPT [15998:1095667]
:POSTROUTING ACCEPT [15837:1127773]
[404:82821] -A POSTROUTING -s 10.6.0.0/24 -o eth0 -m comment --comment wireguard-nat-rule -j MASQUERADE
COMMIT
# Completed on Wed Feb  9 11:16:53 2022

root@pivpn:/home/pi# nordvpn disconnect
You are disconnected from NordVPN.
How would you rate your connection quality on a scale from 1 (poor) to 5 (excellent)? Type 'nordvpn rate [1-5]'.
root@pivpn:/home/pi# nordvpn status
Status: Disconnected

root@pivpn:/home/pi# ss -tunlp | grep 51821; ip -4 addr; ip -4 ro list table all; ip -4 ru; iptables-save -c
udp   UNCONN 0      0            0.0.0.0:51821      0.0.0.0:*                                                         
udp   UNCONN 0      0                  *:51821            *:*                                                         
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    inet 192.168.1.28/24 brd 192.168.1.255 scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
4: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.6.0.1/24 scope global wg0
       valid_lft forever preferred_lft forever
default via 192.168.1.1 dev eth0 table isp 
default via 192.168.1.1 dev eth0 src 192.168.1.28 metric 202 
10.6.0.0/24 dev wg0 proto kernel scope link src 10.6.0.1 
192.168.1.0/24 dev eth0 proto dhcp scope link src 192.168.1.28 metric 202 
broadcast 10.6.0.0 dev wg0 table local proto kernel scope link src 10.6.0.1 
local 10.6.0.1 dev wg0 table local proto kernel scope host src 10.6.0.1 
broadcast 10.6.0.255 dev wg0 table local proto kernel scope link src 10.6.0.1 
broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1 
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1 
broadcast 192.168.1.0 dev eth0 table local proto kernel scope link src 192.168.1.28 
local 192.168.1.28 dev eth0 table local proto kernel scope host src 192.168.1.28 
broadcast 192.168.1.255 dev eth0 table local proto kernel scope link src 192.168.1.28 
0:	from all lookup local
15010:	from all iif lo sport 51821 lookup isp
32766:	from all lookup main
32767:	from all lookup default
# Generated by iptables-save v1.8.7 on Wed Feb  9 11:17:14 2022
*filter
:INPUT ACCEPT [16820933:73736596170]
:FORWARD ACCEPT [20583:10602613]
:OUTPUT ACCEPT [5473190:953953903]
COMMIT
# Completed on Wed Feb  9 11:17:14 2022
# Generated by iptables-save v1.8.7 on Wed Feb  9 11:17:14 2022
*nat
:PREROUTING ACCEPT [41149:3591922]
:INPUT ACCEPT [39733:3411397]
:OUTPUT ACCEPT [16002:1095944]
:POSTROUTING ACCEPT [15841:1128050]
[404:82821] -A POSTROUTING -s 10.6.0.0/24 -o eth0 -m comment --comment wireguard-nat-rule -j MASQUERADE
COMMIT
# Completed on Wed Feb  9 11:17:14 2022

trendy · 10 February 2022 10:46

However, I have no access to internet outside my LAN.

You need to masquerade also the traffic towards VPN iptables -t nat -A POSTROUTING -s 10.6.0.0/24 -o nordlynx -m comment --comment wireguard-nat-rule -j MASQUERADE

I’d also like to lock down the pi so that it has no internet connectivity without Nord

Before the

  -A INPUT -i eth0 -j DROP

and

 -A OUTPUT -o eth0 -j DROP

there need to be 2 rules to accept traffic towards and from port 51821 respectively.

-A INPUT -i eth0 -p udp --dport 51821 -j ACCEPT
-A OUTPUT -i eth0 -p udp --sport 51821 -j ACCEPT

DaringDougler · 10 February 2022 21:21

THANK YOU! Still having trouble with 3 commands though. Putting it all together we have:

echo ‘10 isp’ >> /etc/iproute2/rt_tables
ip route add to default via 192.168.1.1 table isp
ip rule add iif lo sport 51821 to default lookup isp prio 15010
iptables -A POSTROUTING -s 10.6.0.0/24 -o nordlynx -m comment --comment wireguard-nat-rule -j MASQUERADE
iptables -A INPUT -i eth0 -j DROP
iptables -A OUTPUT -o eth0 -j DROP
iptables -A INPUT -i eth0 --dport 51821 -j ACCEPT
iptables -A OUTPUT -i eth0 --sport 51821 -j ACCEPT

HELP
**When I attempt command #4 (MASQUERADE) I get the following error:
iptables: No chain/target/match by that name

When I attempt commands 7 and 8 I get the following errors:
iptables v1.8.7 (nf_tables): unknown option “–dport”
iptables v1.8.7 (nf_tables): unknown option “–sport” Questions:
\

Are the original ip table rules and routes still applicable (lines 1-3)?
NordVPN has a WHITELIST option that allows local network devices to talk to each other. I whitelisted 192.168.1.0/24. Is there anything that we did that will prevent that? Should we create our own rule for that or rely on Nord’s? I guess my preference would be to have all custom rules and turn off all of Nord’s firewalling if possible.
Will this act as a gateway for LAN devices if I ever decide to do that (eg change the gateway on a streaming device to the PI IP (192.168.1.28) and have the traffic route through NordVPN?
Are any of the rules order dependent. And if so are the rules in the correct order?
What’s the best way to run at startup. Should i put these in a systemd daemon? If so, is there any dependency or something this needs to run after? I’m a bit hesitant to make them persistent as I don’t have a lot of knowledge on networking. I like the idea of a daemon because, if i run into connectivity issues with Nord, I can just disable it and reboot. What are your thoughts, is there another best practice?

FYI – THis is what my iptables look like (without the failures noted above. I dont remember the destination of Nord being a hostname before, seemed it was always an IP address. Doubt that it is a problem, but I thought I’d call it out

pi@pivpn:~ $ sudo iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all – unn-89-187-183-162.cdn77.com anywhere
ACCEPT all – unn-89-187-183-162.cdn77.com anywhere
ACCEPT all – 192.168.1.0/24 anywhere
ACCEPT all – 192.168.1.0/24 anywhere
DROP all – anywhere anywhere
DROP all – anywhere anywhere
DROP all – anywhere anywhere
DROP all – anywhere anywhere

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all – anywhere unn-89-187-183-162.cdn77.com
ACCEPT all – anywhere unn-89-187-183-162.cdn77.com
ACCEPT all – anywhere 192.168.1.0/24
ACCEPT all – anywhere 192.168.1.0/24
DROP all – anywhere anywhere
DROP all – anywhere anywhere
DROP all – anywhere anywhere
DROP all – anywhere anywhere

Thanks again for all your help!**

trendy · 11 February 2022 11:20

I forgot to assign the masquerade rule to the nat table, I have fixed it, you can try it again.
Also I forgot the ‘-p udp’ and that is why it was giving you the error in s/dport.

The first line is permanent. The rest will not survive a reboot or interface flap.
If you are going to whitelist some subnets, add also there the vpn subnet. This should allow the vpn users to access the tunnel and the local subnet.
yes
order matters, the accept rules must be before the final drop all.
making a daemon seems like a good idea to me. Better to add them in the nordvpn up/down scripts though, so they can be executed together.
better use ‘iptables-save -c’

DaringDougler · 11 February 2022 15:20

YOU ARE THE MAN! WORKING BEAUTIFULLY! I did have to change

iptables v1.8.7 (nf_tables): Can’t use -i with OUTPUT

to
“iptables -A OUTPUT -o eth0 -p udp --sport 51821 -j ACCEPT”

I assume that’s correct?

After playing with it a while, the only thing I couldn’t make work was using my pi as an internet gateway for local devices. Based on reading other posts, I thought I could just set a static IP on my laptop’s Wifi Connection to —192.1.168.1.100 and set the Gateway for that connection to 192.168.1.28---- and that the traffic would be routed through Nord. What am I missing?

Everything else works. I can have Nord Running on the pi and connect remotely via wireguard. When connected through my wireguard VPN, I am able to access internal IPs, and my external internet requests are being forwarded through Nord. FANTASTIC!

Also, Can you help me write the equivalent of the whitelist rule for both my Local LAN and the NordVPN? I want to turn off Nord’s ‘whitelist feature’ and have all the rules in one place. I dont want things happening out of order and getting messed up. Would it be whitelisting the 10.5.0.0 and 10.6.0.0 subnet or whitelisting ports 51821 and 51820? I would feel better getting your help with the syntax and putting that in the right spot in our script, to prevent problems and ensure that I havent opened something up I shouldn’t. This is what we have so far, incorporating the changes you provided today, including the reorder.

Finally, is there anything else you would recommend hardening for security? I think I’ve seen elsewhere some extra conditions on the Nord inbound rules that doesn’t allow it unless its an existing connection? I dont remember the syntax or where I saw it. I dont want to open up SSH etc. Anything I do on the LAN, I will do after connecting through wireguard. Based on my understanding of what you have architected, the only thing we are allowing in and out over the PI is just wireguard and nord.

OUR CURRENT IPTABLES AND FIREWALL RULES
echo ‘10 isp’ >> /etc/iproute2/rt_tables
ip route add to default via 192.168.1.1 table isp
ip rule add iif lo sport 51821 to default lookup isp prio 15010
iptables -t nat -A POSTROUTING -s 10.6.0.0/24 -o nordlynx -m comment --comment wireguard-nat-rule -j MASQUERADE
iptables -A INPUT -i eth0 -p udp --dport 51821 -j ACCEPT
iptables -A OUTPUT -o eth0 -p udp --sport 51821 -j ACCEPT
iptables -A INPUT -i eth0 -j DROP
iptables -A OUTPUT -o eth0 -j DROP

trendy · 11 February 2022 19:31

“iptables -A OUTPUT -o eth0 -p udp --sport 51821 -j ACCEPT”

I assume that’s correct?

Right.

What am I missing?

masquerade for the 192.168.1.0/24 via nordlynx.

Also, Can you help me write the equivalent of the whitelist rule for both my Local LAN and the NordVPN? I want to turn off Nord’s ‘whitelist feature’ and have all the rules in one place. I dont want things happening out of order and getting messed up. Would it be whitelisting the 10.5.0.0 and 10.6.0.0 subnet or whitelisting ports 51821 and 51820? I would feel better getting your help with the syntax and putting that in the right spot in our script, to prevent problems and ensure that I havent opened something up I shouldn’t. This is what we have so far, incorporating the changes you provided today, including the reorder.

I don’t know what the nord whitelisting does exactly, so I cannot suggest anything. Also I don’t know where does 10.5.0.0 belong. You could do the same things for any network as you did for the wireguard. You shouldn’t need to open any more ports.

Finally, is there anything else you would recommend hardening for security? I think I’ve seen elsewhere some extra conditions on the Nord inbound rules that doesn’t allow it unless its an existing connection? I dont remember the syntax or where I saw it. I dont want to open up SSH etc. Anything I do on the LAN, I will do after connecting through wireguard. Based on my understanding of what you have architected, the only thing we are allowing in and out over the PI is just wireguard and nord.

Not really, you have already opened up holes in the airtight solution provided by nord to accommondate your needs, so you need to decide if you want security over comfort. This also depends on the usecase. If you are trying to hide from a suppressive regime, then I’d suggest you rethink about it. If you just want to browse without trackers and advertisements then it’s not a huge deal if it breaks.

DaringDougler · 11 February 2022 20:17

The whitelisting of 192.168.1.0/24, supposedly created a rule that allowed my local lan devices to reach the pie and vice-versa. Nord’s nomenclature defines whitelisting as bypassing the VPN for local traffic. Do we already have that covered? I should be able to turn their whitelisting feature off. When I mentioned it before you said something about making sure I whitelist wireguard too (that’s where I got confused)

Also when I just tryed the drop commands for the first time. When I dropped eth0 and eth1, i suddently was unable to SSH into the pi on the LAN (not connected to wireguard). Thoughts? Isnt this the piece that protects me from the evil overlords tracking me? You mentioned that I opened up a hole in Nord’s airtight solution. I want to understand that just a bit more. I think when I set their killswitch, it just creates something similar to this. As long as this firewall rule is activated, isnt all my traffic guaranteed to go through Nord (because eth0 is shut down). The only way into create a new connection is though wireguard and the only way out to the internet is through Nord? The only way I can access local servers is by first establishing a wireguard VPN connection. Unless I’m missing something, seems like nirvana.

So this is what I’m missing for the local gateway access (another postrouting rule)?

iptables -A POSTROUTING -s 192.166.1.0/24 -o nordlynx -m comment --comment wireguard-nat-rule -j MASQUERADE

trendy · 12 February 2022 21:21

Nord’s nomenclature defines whitelisting as bypassing the VPN for local traffic. Do we already have that covered?

Kind of, wg is a different interface than eth0 and is by default allowed in from the firewall policy.

i suddently was unable to SSH into the pi on the LAN (not connected to wireguard). Thoughts?

The rules I provided were supposed to be used supplementary to the ones provided by nord, not standalone. Sorry if I didn’t make it clear.

Isnt this the piece that protects me from the evil overlords tracking me? You mentioned that I opened up a hole in Nord’s airtight solution. I want to understand that just a bit more. I think when I set their killswitch, it just creates something similar to this

I didn’t provide any killswitch, just a hole so you can connect from a wg client. If you don’t use the killswitch the traffic will use the isp in case nord goes down.

The only way into create a new connection is though wireguard and the only way out to the internet is through Nord? The only way I can access local servers is by first establishing a wireguard VPN connection. Unless I’m missing something, seems like nirvana.

Seems to me like this.

iptables -A POSTROUTING -s 192.166.1.0/24 -o nordlynx -m comment --comment wireguard-nat-rule -j MASQUERADE

needs the -t nat after iptables. And you need to do the same for every subnet you want to route via nord.

DaringDougler · 15 February 2022 20:44

I was finally able to get this working (ALMOST). I had to add/tweak a couple of additional rules, but now I’m able to use my PI as a NordVPN Gateway. I can access all my internal LAN IP addresses when I’m remote -even while Nord is running on the PI. I can double hop from a remote location through wireguard on the PI, and back out to the internet on Nord VPN. All of my internal LAN devices can communicate with the PI. I can use the PI as a NordVPN gateway for LAN devices by pointing them to the IP address of the PI. I am able to use Pihole to add-filter on the LAN as well as via remote Wireguard connections. And most importantly, if Nord disconnects, the pi has no internet connectivity. THANK YOU SO MUCH! This hits all of my requirements and the result is IS BETTER THAN I COULD HAVE IMAGINED. I’m averaging 600mbps in speed tests and 420mbps sustained download speeds. Very impressive for a PI downloading through an external VPN .

HELPI
I am not able to get this launched at startup. I tried iptables-save -c, and although it shows committed on my screen, my etc/iptables/rules.v4 does not change. Eventually I had to log in as root and issue an iptables-save > /etc/iptables/rules.v4 command. It saved, however, when I restarted the pi, Nord would not connect due to a socket error. I notice that Nord does add ALLOW entries for specific servers to my iptables when it connects, and removes the entries when it disconnects. I’m reluctant to save my iptables when Nord is connected, as I don’t want to be hardcoded to the servers. I’ve attached the original rules.v4 (that works at startup), as well as the final commands that I’m able to successfully append to the iptables from the Terminal.

Questions:

Should all of my commands be covered by iptables-save? Some are routing rules, are they picked up in iptables-save?
Should my iptables-save command default to the path /etc/iptables (some of what I’ve read online indicate that it should–mine does not)?
Is there a way to append my commands versus replacing the whole rules file (which we know works from the command prompt)?
I tried installing iptables-persistent, but ended up having to restore my pi from a backup. I’m not sure if it was caused by my experimenting and getting the rules mucked up, of if iptables-persistent caused the issue.
Even though I’m NOT currently using iptables-persistent, and my iptables-save isn’t pointed to /etc/iptables/rules.v4, clearly something is loading /etc/iptables/rules.v4 at startup, because when I mess around with it I get different results.
When you suggested iptables-save, how did you anticipate they would load on reboot?
Do you know what the INPUT ACCEPT, FORWARD ACCEPT etc entries in brackets are doing under filter and NAT in the working rules.v4 file below? I assumed it was something dynamic put in their by Nord?7.
I clearly never issued a iptable-save command. I assume the save from Feb 7 was done automatically by Nord or wireguard?
What’s your recommendation on how to get this working at startup (including if not all the entries will get picked up by save)

Working rules.v4 file# Generated by iptables-save v1.8.7 on Mon Feb 7 22:28:37 2022
*filter
:INPUT ACCEPT [582195:1163499661]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [216673:468280139]
COMMIT

Completed on Mon Feb 7 22:28:37 2022

Generated by iptables-save v1.8.7 on Mon Feb 7 22:28:37 2022

*nat
:PREROUTING ACCEPT [2038:158822]
:INPUT ACCEPT [2031:158216]
:OUTPUT ACCEPT [3228:210793]
:POSTROUTING ACCEPT [2084:134452]
-A POSTROUTING -s 10.6.0.0/24 -o eth0 -m comment --comment wireguard-nat-rule -j MASQUERADE
COMMIT

Completed on Mon Feb 7 22:28:37 2022

Our CUSTOM IPTABLES AND ROUTING RULES

ip route add to default via 192.168.1.1 table isp
ip rule add iif lo sport 51821 to default lookup isp prio 15010
iptables -t nat -A POSTROUTING -s 10.6.0.0/24 -o nordlynx -m comment --comment wireguard-nat-rule -j MASQUERADE
iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -o nordlynx -m comment --comment gateway-nat-rule -j MASQUERADEiptables -A INPUT -i eth0 -p udp --dport 51821 -j ACCEPT
iptables -A OUTPUT -o eth0 -p udp --sport 51821 -j ACCEPT
iptables -A INPUT -i eth0 -p all -d 192.168.1.0/24 -j ACCEPT
iptables -A OUTPUT -o eth0 -p all -d 192.168.1.0/24 -j ACCEPT
iptables -A INPUT -i eth0 -j DROP
iptables -A OUTPUT -o eth0 -j DROP

IPTABLES-L when Nord Connects

ACCEPT all – 83-177-xy-za2.reverse-dns anywhere
ACCEPT all – 83-177-xy-za2.reverse-dns anywhere
DROP all – anywhere anywhere
ACCEPT udp – anywhere anywhere udp dpt:51821
ACCEPT all – anywhere 192.168.1.0/24
DROP all – anywhere anywhere

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all – anywhere 83-177-xy-za2.reverse-dns
ACCEPT all – anywhere 83-177-xy-za2.reverse-dns
DROP all – anywhere anywhere
ACCEPT udp – anywhere anywhere udp spt:51821
ACCEPT all – anywhere 192.168.1.0/24
DROP all – anywhere anywhere

trendy · 16 February 2022 13:33

iptables-save shows a snapshot of the running configuration for iptables. Useful to see what are the hits too, if you invoke -c too. ip route and ip rule are not connected to iptables, you can see them with ip route show
It is unusual for an executable to reside in etc, usually /usr/sbin is the place.
You can place the commands in specific positions to make sure that the rules are with the correct order. See -I option https://www.frozentux.net/iptables-tutorial/iptables-tutorial.html#COMMANDS
In your case that Nord is adding/removing her own rules, it would make more sense to also add/remove the extra rules you have with the up/down scripts.
Maybe some startup script, @reboot in cron, rc.local, some service
Again, I asked you to use iptables-save to see what is running. Should you want to use it for save and restore, you’d need to dump it on a file, then use iptables-restore to read that file and configure the firewall.
[packet_count:byte_count]
I don’t know, could be.
What I am doing is running my vpn server and client on my OpenWrt router. Then using Policy Based Routing package to separate the traffic depending on the desired upstream interface. It does all you are trying to do and much more in a more user friendly way with a web interface.

DaringDougler · 23 October 2022 04:49

Hello Again! Everything has been working beautifully (except it seems that Nord changes their firewall rules with every new release, and I need to make some minor mods to my script.) However, I have a new wrinkle that I could use your help with. I recently installed docker. There is a container that I want to use that needs to communicate between my PI and my smartthings HUB on my local LAN. When I installed docker it added a bunch of firewall rules. I’m not sure how to allow docker activity (before I drop all the activity that isn’t LOCAL LAN, WIREGUARD and NORD). As before, one of my key requirements is that I dont want to allow activity in and out of my PI that isn’t secured by a VPN connection. However I need docker to communicate with the rest of my LAN. See below…

-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A INPUT -i wlan0 -m connmark --mark 0xe1f1 -m comment --comment nordvpn -j ACCEPT
-A INPUT -i eth0 -m connmark --mark 0xe1f1 -m comment --comment nordvpn -j ACCEPT
-A INPUT -s 192.168.1.0/24 -i eth0 -j ACCEPT
-A INPUT -i eth0 -p udp -m udp --dport 51821 -j ACCEPT
-A INPUT -i wlan0 -m comment --comment nordvpn -j DROP
-A INPUT -i eth0 -m comment --comment nordvpn -j DROP
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A OUTPUT -o wlan0 -m mark --mark 0xe1f1 -m comment --comment nordvpn -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A OUTPUT -o wlan0 -m connmark --mark 0xe1f1 -m comment --comment nordvpn -j ACCEPT
-A OUTPUT -d 192.168.1.0/24 -o eth0 -j ACCEPT
-A OUTPUT -o eth0 -p udp -m udp --sport 51821 -j ACCEPT
-A OUTPUT -o eth0 -m mark --mark 0xe1f1 -m comment --comment nordvpn -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A OUTPUT -o eth0 -m connmark --mark 0xe1f1 -m comment --comment nordvpn -j ACCEPT
-A OUTPUT -o wlan0 -m comment --comment nordvpn -j DROP
-A OUTPUT -o eth0 -m comment --comment nordvpn -j DROP
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN