Inter-VLAN issues VLAN interface not able to ping itself, route or NAT to internet

[dpsguard]

Hello everyone,

I am hoping someone already ran into inter vlan routing issues and resolved it and I can get to know the fix or missing configuration.

\

1. RPI 4B running latest dietPi version. Have wireless disabled, so only eth0 is working. Usual stuff like Pi-hole and PiVPN works as well. DHCP scope thru the Pi-Hole works as well.

2. Added a subinterface (VLAN) eth.120 and corresponding dhcp scope via dnsmasq to listen to this tagged dhcp request on vlan 120. This works as well. Client connected to this vlan ( via a wireless AP with SSID mapped to this vlan 120) obtains the IP address from this vlan 120 dhcp scope set up on the RPi.

3. default route is thru the eth0 to get to internet. VLAN interface has only IP and subnet mask.

Here are the steps that I followed for this vlan part:

apt install vlan
echo 8021q >> /etc/modules

lsmod | grep 8021q (checks out good)

nano /etc/sysctl.conf

remove comment from (this is already done on my box as I am using IP forwarding for VPN client to be able to traverse the RPi and get to inside network, but listing here for completeness of vlan setup)

#net.ipv4.ip_forward=1

save

nano /etc/network/interfaces.d/vlans

add below

auto eth0.120
iface eth0.120 inet static
address 10.10.120.1
netmask 255.255.254.0
vlan-raw-device eth0

Save

service networking restart

verify presence of new interface and IP via following:

Hostname -I
ifconfig

route -n (shows default route thru eth0 and two connected routes for eth0 and eth0.120 subnet as should be the case).

Add dhcp scope via dnsmasq

nano /etc/dnsmasq.d/120-vlan-dhcp.conf

dhcp-range=set:120,10.10.120.2,10.10.121.254,255.255.254.0,20m
dhcp-option=tag:120,option:router,10.10.120.1
dhcp-option=tag:120,6,192.168.2.34

save

At this time, I attach RPi to a managed VLAN switch with switch port set to tag vlan 120 and untag on default vlan 1 (default vlan 1 is where the main eth0 IP of RPi is). I then create another similar port on switch and connect the Wireless AP. Wireless AP obtains dhcp IP address from the vlan 1 and comes online. I then create a SSID on the AP with the egress traffic mapped to vlan 120. The client connects to this SSID and is successful in getting IP address in vlan 120 from dhcp scope for this vlan set up on the RPi. So we know VLANs are working.

Then I try to do some pings:

ping 192.168.2.34 (works and this is the RPI eth0 / vlan 1 untagged Ip address).

ping 10.10.120.1 (works and this is the RPI eth0.120 / vlan ip address)

ping -I eth0 192.168.2.34 ( works of course)

ping -I eth0.120 10.10.120.1 (does not work).

The client gets ip address in 10.10.120.0/23 subnet but also cannot ping its gateway 10.10.120.1.

So something is broken that client on the same subnet / vlan cannot ping the gateway and the router (RPi) itself cannot ping its own vlan interface.

I have ufw firewall installed but is disabled for now.

My next step after making this local (inter vlan) routing / forwarding work is to then approach the NAT side via ufw / iptables masquerading. eth0.120 will be the LAN side and eth0 will be acting as WAN / Internet side to which 10.10.120.x traffic will be natted.

MichaIng and Joulinar , your expertise is needed here please.

Thanks and very best

[dpsguard]

Just looked at this document and everything is set up correctly in my RPi.

https://www.sbprojects.net/projects/raspberrypi/vlan.php

[Joulinar]

Hi,

sorry I never used VLAN’s or have any hardware that is capable to do so. Therefore I don`t have enough knowledge on this

[dpsguard]

Thanks Joulinar . I have two of Dlink DGS VLAN managed switches at home. They are DGS-1100-05 and I have 3 VLANs that I use. There is a 802.1Q trunk between the two. Look for this low cost on the eBay. I purchased open box from ebay these for only 25 Canadian dollars each. There is also a DGS-1100-8. I can help fund purchase of one for you for DietPi development. Lots of applications for VLANs for segregation and isolation of traffic for things like Guest, IOT devices, users and servers, storage etc.

Meanwhile, to test the RPi for vlans, you can use any linux PC / laptop by using the exact same configuration for its NIC (to add vlans) as I did for eth0 of RPi and you can see if you get IP address on the vlan interface and if you can ping. I have to go to work and I am going to test later today the whole thing on a Ubuntu machine.

[Joulinar]

let’s see if MichaIng has an idea on this.

[trendy]

Make sure that nothing is blocking traffic

iptables-save -c

I don’t believe it is a routing issue, but it doesn’t hurt to verify with tcpdump that echo requests reach the interface, but no reply is sent back.

tcpdump -i any -evn icmp

[dpsguard]

trendy and Joulinar , you folks are very correct. There was no issue with the configuration or the routing. I had flawed test switch. I was using a Mikrotik managed switch 250GS and I was reading the documentation for 260GS and there is subtle difference between the two and 250GS was a bad design, that they fixed in 260GS. The 250GS does not support any untagged / native vlans on the trunks (260GS does support it) and I was trying to set up traditional way with an untagged vlan that was also then used for the eth0 (my design does require it). So i replaced that with my home lab Dlink router and everything started working, Results with 250GS were weired, making me believe that VLAN interface is working as attached laptop will get correct Ip address (and Pihole will show it as a lease, even if it is not set up as a scope under main pi-hole provided scope). This scope defined out of bound of the pi-hole, makes it survive any change into the Pi-hole settings and allows changing the lease time to less than 1 hour ( in my example, I used 20 minutes).

For the benefit of others, who may try to use VLANs (and thus making the RPi as a router on a stick), I will like to add that in the dhcp scope configuration in my original post, I forgot to add DNS server. I will update that post also, but under /etc/dnsmasq.d/120-vlan-dhcp.conf, we need to add , dhcp-option=tag:120,6,. option number 6 is for a custom DNS server address(es).

I am out of town tomorrow and I will test in a day and two to add the masquerade rule in iptables for this subinterface / vlan for its traffic to get to the internet. and then come back here to probably add the configuration for that portion.

trendy , will you recommend changing the firewall rules using ufw or directly touching the tables and chains? My RPi is behind the home router and I need to just have iptables rules just for isolating the guest traffic to internet and to NAT the traffic to the uplink (eth0) interface which then goes into home network.

[Joulinar]

dpsguard
if you have time and mood, you could create a small how-to on our tutorial section. It might help other dealing with this topic as it’s quite specific and not that common

[dpsguard]

Great idea Joulinar . I will sure write few tutorials based on my experience. Hopefully start putting it up in a day or two.

[trendy]

@trendy , will you recommend changing the firewall rules using ufw or directly touching the tables and chains? My RPi is behind the home router and I need to just have iptables rules just for isolating the guest traffic to internet and to NAT the traffic to the uplink (eth0) interface which then goes into home network.

It depends how comfortable you are with direct iptables rules.
ufw is a failsafe choice in this case.

[dpsguard]

Thanks trendy . I know more than basics of iptables, chains and especially that even ufw is frontend for iptables. I generally use ufw as you mentioned to be safe and easier to add rules and understand. What I meant was for the NAT, is this better to directly add the rule in the iptables or rather use the rule populated inside the ufw before.rules file, which then gets referenced by iptables.

I think, I will use this latter method, unless it could cause other issues, and I already did that and tested NAT working fine on my subinterface / vlan. Essentially with this, I am able to use eth0.120 as the LAN and eth0 as the WAN. The whole point is that if someone does not use the built-in Wireless AP and instead needs to use the same eth0 for in and out of traffic (with one or multiple external Wireless APs connected to RPi via POE switch, then all this is needed and is a valid use- case in certain situations. I am going to write some small tutorials on this subject and maybe related items, with due credit to you folks for expert help and guidance.

When I installed ufw, I set the logging to high, just to troubleshoot few things. But I have yet to find the log file location under dietPi. In Ubuntu, I could get it under /var/log/messages or /etc/var/ufw.log but I cannot find any of these. I then installed rsyslog and then I could see the usual messages and syslog file under var/log, but still no ufw.log.

[trendy]

Thanks @trendy . I know more than basics of iptables, chains and especially that even ufw is frontend for iptables. I generally use ufw as you mentioned to be safe and easier to add rules and understand. What I meant was for the NAT, is this better to directly add the rule in the iptables or rather use the rule populated inside the ufw before.rules file, which then gets referenced by iptables.

In this case, better use the ufw altogether. A restart of ufw service or interface restart will wipe the custom masquerade rule and won’t be added back in.

[Joulinar]

dpsguard
In DietPi logging is reduced to a minimum to reduce r/w operations on SD cards. As well logs are saved within ramlog, means they are not boot persistent. Next to this logs are cleared once an hour. If needed, logging methode can be switched to a full log mode using dietpi-software. This will install rsyslog.

[dpsguard]

Thank you Joulinar and it makes perfect sense as to why logging is reduced in dietPi. I only installed rsyslog as I was not sure as to why teh syslog and messages are missing under var/log.

And thanks trendy for your valuable suggestions. I actually save rules via iptables-save >somefile and then use rc.local to do iptables-restore for this exact reason that Iptables don’t persist the reboots or crash (did not explore iptables persistence however). And since rc.local is not supported, I had to add support for it as this might be required by few people (though you can use cron as well, I guess to run at boot time to do this). Appreciate again all help and support.

[Joulinar]

dpsguard
You can have a look to iptables-persistent. It’s a service to restore iptables during boot

apt install iptables-persistent