DietPi v6.28.0 Wifi drop update: freezed networking

[kriznik]

small update
I've installed 6.28 on another rpi for testing, with different usb wifi and same software.
again 1:17am system is unreachable via wifi, but this time i do have monitor and shell as well directly.

interestingly, system is not freezed, even wifi has got IP, but networking is ded. no ping in no ping out at all.

systemctl restart networking solved it

so I'm wondering what can cause networking to stall thanks to hourly crons, which seems to be only doing some logg rotating if enabled?
as seen from previous debug, hourly cron somehow restarts networking for some uknown reason (maybe thanks to wifi manager?) but sometimes it got stuck.

How we can debug more around networking what is happening and why?

as quick fix, i probably just edit dietpi-wifi-monitor.sh and instead of ifdown/ifup I'll restart whole networking service

[Joulinar]

hi,

your system seems to reboot completely time by time. Probably some issues with power plug or device (hardware). Maybe SD card or file system corruption. There is no reason on the logs for the reboot . Unfortunately we don't know if there are any memory issues / overflow at the time your system get stuck.

As well it seems your system is not booting correctly once it crashed or it's hanging during boot. or it has issues to connect to WiFi. Because 1 hour for time sync it a huuuuugggge amount of time.

pls can you check these 2 values

Code: Select all

cat /DietPi/dietpi.txt |grep CONFIG_NTP_MODE
cat /DietPi/dietpi.txt |grep CONFIG_BOOT_WAIT_FOR_NETWORK

[kriznik]

Code: Select all

dietpi@garden:~$ cat /DietPi/dietpi.txt |grep CONFIG_NTP_MODE
CONFIG_NTP_MODE=4
dietpi@garden:~$ cat /DietPi/dietpi.txt |grep CONFIG_BOOT_WAIT_FOR_NETWORK
# - It is HIGHLY recommended to also set CONFIG_BOOT_WAIT_FOR_NETWORK=2, to force infinite wait for network connection during boot and prevent connection timeout errors.
CONFIG_BOOT_WAIT_FOR_NETWORK=1
dietpi@garden:~$

and second device

Code: Select all

root@DietPi:~# cat /DietPi/dietpi.txt |grep CONFIG_NTP_MODE
CONFIG_NTP_MODE=2
root@DietPi:~# cat /DietPi/dietpi.txt |grep CONFIG_BOOT_WAIT_FOR_NETWORK
# - It is HIGHLY recommended to also set CONFIG_BOOT_WAIT_FOR_NETWORK=2, to force infinite wait for network connection during boot and prevent connection timeout errors.
CONFIG_BOOT_WAIT_FOR_NETWORK=1
root@DietPi:~#

Hi, no ... as I posted in previous post, it is happening on two completely different raspberries (both model 2b) with usb wifi dongles from different vendors. And uptime is not changed eventho it looks like it's rebooted. Power is ok, sd/usb is indeed ok, memory is fine.

Logs on the second pi (which i have monitor attached to it and keyboard) seems exactly the same as on the first one. I was able to gain network access by simply restarting whole networking -> systemctl restart networking

on first machine there is gap I assume because networking freezed as well, but as i can't observe it as it is headless, hard to tell.

[MichaIng]

Hmm, strange issue. When you have some Pi up, could you paste output of:

Code: Select all

systemctl status networking
systemctl status ifup@wlan0
cat /etc/network/interfaces

[kriznik]

sure i can
first one

Code: Select all

dietpi@garden:~$ sudo systemctl status networking
● networking.service - Raise network interfaces
   Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
   Active: active (exited) since Thu 2020-02-06 20:17:27 CET; 17h ago
     Docs: man:interfaces(5)
  Process: 317 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=0/SUCCESS)
 Main PID: 317 (code=exited, status=0/SUCCESS)

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
dietpi@garden:~$ sudo systemctl status ifup@wlan0
● ifup@wlan0.service - ifup for wlan0
   Loaded: loaded (/lib/systemd/system/ifup@.service; static; vendor preset: enabled)
   Active: active (exited) since Thu 2020-02-06 20:17:25 CET; 17h ago
  Process: 318 ExecStart=/bin/sh -ec ifup --allow=hotplug wlan0; ifquery --state wlan0 (code=exited, status=0/SUCCESS)
 Main PID: 318 (code=exited, status=0/SUCCESS)
   Memory: 5.4M
   CGroup: /system.slice/ifup@wlan0.service
           ├─396 /sbin/wpa_supplicant -s -B -P /run/wpa_supplicant.wlan0.pid -i wlan0 -D nl80211,wext -c /etc/wpa_supplicant/wpa_suppl
icant.conf
           └─424 /sbin/dhclient -4 -v -i -pf /run/dhclient.wlan0.pid -lf /var/lib/dhcp/dhclient.wlan0.leases -I -df /var/lib/dhcp/dhcl
ient6.wlan0.leases wlan0

Feb 07 13:27:00 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:29:34 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:32:10 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:32:11 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:32:12 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:34:46 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:34:47 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:37:20 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:37:24 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:45:10 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
dietpi@garden:~$ sudo cat /etc/network/interfaces
# Location: /etc/network/interfaces
# Please modify network settings via: dietpi-config
# Or create your own drop-ins in: /etc/network/interfaces.d/

# Drop-in configs
source interfaces.d/*

# Local
auto lo
iface lo inet loopback

# Ethernet
#allow-hotplug eth0
iface eth0 inet dhcp
address 0.0.0.0
netmask 0.0.0.0
gateway 0.0.0.0
#dns-nameservers 0.0.0.0

# WiFi
allow-hotplug wlan0
iface wlan0 inet dhcp
address 10.10.1.55
netmask 255.255.255.0
gateway 10.10.1.1
#dns-nameservers 10.10.1.5
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
dietpi@garden:~$

second one

Code: Select all

root@DietPi:~# systemctl status networking
● networking.service - Raise network interfaces
   Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
   Active: active (exited) since Fri 2020-02-07 12:15:21 CET; 1h 40min ago
     Docs: man:interfaces(5)
  Process: 6304 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=0/SUCCESS)
 Main PID: 6304 (code=exited, status=0/SUCCESS)

Feb 07 12:15:21 DietPi systemd[1]: Starting Raise network interfaces...
Feb 07 12:15:21 DietPi systemd[1]: Started Raise network interfaces.
root@DietPi:~# systemctl status ifup@wlan0
● ifup@wlan0.service - ifup for wlan0
   Loaded: loaded (/lib/systemd/system/ifup@.service; static; vendor preset: enabled)
   Active: active (exited) since Fri 2020-02-07 12:15:41 CET; 1h 40min ago
  Process: 6376 ExecStart=/bin/sh -ec ifup --allow=hotplug wlan0; ifquery --state wlan0 (code=exited, status=0/SUCCESS)
 Main PID: 6376 (code=exited, status=0/SUCCESS)
   Memory: 1.7M
   CGroup: /system.slice/ifup@wlan0.service
           ├─6391 /sbin/wpa_supplicant -s -B -P /run/wpa_supplicant.wlan0.pid -i wlan0 -D nl80211,wext -c /etc/wpa_supplicant/wpa_supplicant.conf
           └─6397 /sbin/dhclient -4 -v -i -pf /run/dhclient.wlan0.pid -lf /var/lib/dhcp/dhclient.wlan0.leases -I -df /var/lib/dhcp/dhclient6.wlan0.leases wlan0

Feb 07 12:15:46 DietPi sh[6376]: DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 6
Feb 07 12:15:49 DietPi dhclient[6397]: DHCPOFFER of 10.10.1.182 from 10.10.1.1
Feb 07 12:15:49 DietPi sh[6376]: DHCPOFFER of 10.10.1.182 from 10.10.1.1
Feb 07 12:15:49 DietPi sh[6376]: DHCPREQUEST for 10.10.1.182 on wlan0 to 255.255.255.255 port 67
Feb 07 12:15:49 DietPi dhclient[6397]: DHCPREQUEST for 10.10.1.182 on wlan0 to 255.255.255.255 port 67
Feb 07 12:15:49 DietPi sh[6376]: DHCPACK of 10.10.1.182 from 10.10.1.1
Feb 07 12:15:49 DietPi dhclient[6397]: DHCPACK of 10.10.1.182 from 10.10.1.1
Feb 07 12:15:50 DietPi dhclient[6397]: bound to 10.10.1.182 -- renewal in 33495 seconds.
Feb 07 12:15:50 DietPi sh[6376]: bound to 10.10.1.182 -- renewal in 33495 seconds.
Feb 07 12:15:50 DietPi sh[6376]: wlan0=wlan0
root@DietPi:~# cat /etc/network/interfaces
# Location: /etc/network/interfaces
# Please modify network settings via: dietpi-config
# Or create your own drop-ins in: /etc/network/interfaces.d/

# Drop-in configs
source interfaces.d/*

# Local
auto lo
iface lo inet loopback

# Ethernet
allow-hotplug eth0
iface eth0 inet dhcp
address 10.10.1.170
netmask 255.255.255.0
gateway 10.10.1.1
#dns-nameservers 10.10.1.5

# WiFi
allow-hotplug wlan0
iface wlan0 inet dhcp
address 10.10.1.182
netmask 255.255.255.0
gateway 10.10.1.1
#dns-nameservers 10.10.1.5
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
root@DietPi:~#

[kriznik]

btw MichaIng is there some reason that hourly cron restarts network (or usb dongle ath)? As far as I can see in /etc/cron.hourly/dietpi it does not look like anything should mess with networking at all.
And as it is the only cron ran at 17th minute I assume there have to be something in there.

on top of it, not every hour tho, but 17th minute is indicating me hourly cron

debug log from first machine

Code: Select all

Feb  6 07:17:19 garden kernel: [    0.000000] On node 0 totalpages: 61440
Feb  6 07:17:19 garden kernel: [    0.000000]   Normal zone: 540 pages used for memmap
Feb  6 07:17:19 garden kernel: [    0.000000]   Normal zone: 0 pages reserved
Feb  6 07:17:19 garden kernel: [    0.000000]   Normal zone: 61440 pages, LIFO batch:15
Feb  6 07:17:19 garden kernel: [    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
Feb  6 07:17:19 garden kernel: [    0.000000] pcpu-alloc: [0] 0
Feb  6 07:17:19 garden kernel: [    0.825715] dwc_otg: Microframe scheduler enabled
Feb  6 07:17:19 garden kernel: [    0.833875] dwc_otg: FIQ enabled
Feb  6 07:17:19 garden kernel: [    0.833887] dwc_otg: NAK holdoff enabled
Feb  6 07:17:19 garden kernel: [    0.833893] dwc_otg: FIQ split-transaction FSM enabled
Feb  6 07:17:19 garden kernel: [    0.833911] Module dwc_common_port init
Feb  6 07:17:19 garden kernel: [    3.289831] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
Feb  6 07:17:19 garden kernel: [   27.646858] ath: EEPROM regdomain: 0x809c
Feb  6 07:17:19 garden kernel: [   27.646865] ath: EEPROM indicates we should expect a country code
Feb  6 07:17:19 garden kernel: [   27.646873] ath: doing EEPROM country->regdmn map search
Feb  6 07:17:19 garden kernel: [   27.646880] ath: country maps to regdmn code: 0x52
Feb  6 07:17:19 garden kernel: [   27.646889] ath: Country alpha2 being used: CN
Feb  6 07:17:19 garden kernel: [   27.646894] ath: Regpair used: 0x52
Feb  6 18:17:18 garden kernel: [    0.000000] On node 0 totalpages: 61440
Feb  6 18:17:18 garden kernel: [    0.000000]   Normal zone: 540 pages used for memmap
Feb  6 18:17:18 garden kernel: [    0.000000]   Normal zone: 0 pages reserved
Feb  6 18:17:18 garden kernel: [    0.000000]   Normal zone: 61440 pages, LIFO batch:15
Feb  6 18:17:18 garden kernel: [    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
Feb  6 18:17:18 garden kernel: [    0.000000] pcpu-alloc: [0] 0
Feb  6 18:17:18 garden kernel: [    0.825647] dwc_otg: Microframe scheduler enabled
Feb  6 18:17:18 garden kernel: [    0.833821] dwc_otg: FIQ enabled
Feb  6 18:17:18 garden kernel: [    0.833833] dwc_otg: NAK holdoff enabled
Feb  6 18:17:18 garden kernel: [    0.833839] dwc_otg: FIQ split-transaction FSM enabled
Feb  6 18:17:18 garden kernel: [    0.833856] Module dwc_common_port init
Feb  6 18:17:18 garden kernel: [    3.289928] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
Feb  6 18:17:18 garden kernel: [   27.006839] ath: EEPROM regdomain: 0x809c
Feb  6 18:17:18 garden kernel: [   27.006847] ath: EEPROM indicates we should expect a country code
Feb  6 18:17:18 garden kernel: [   27.006855] ath: doing EEPROM country->regdmn map search
Feb  6 18:17:18 garden kernel: [   27.006862] ath: country maps to regdmn code: 0x52
Feb  6 18:17:18 garden kernel: [   27.006871] ath: Country alpha2 being used: CN
Feb  6 18:17:18 garden kernel: [   27.006876] ath: Regpair used: 0x52
Feb  6 20:17:21 garden kernel: [    0.000000] On node 0 totalpages: 61440
Feb  6 20:17:21 garden kernel: [    0.000000]   Normal zone: 540 pages used for memmap
Feb  6 20:17:21 garden kernel: [    0.000000]   Normal zone: 0 pages reserved
Feb  6 20:17:21 garden kernel: [    0.000000]   Normal zone: 61440 pages, LIFO batch:15
Feb  6 20:17:21 garden kernel: [    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
Feb  6 20:17:21 garden kernel: [    0.000000] pcpu-alloc: [0] 0
Feb  6 20:17:21 garden kernel: [    0.825680] dwc_otg: Microframe scheduler enabled
Feb  6 20:17:21 garden kernel: [    0.833834] dwc_otg: FIQ enabled
Feb  6 20:17:21 garden kernel: [    0.833846] dwc_otg: NAK holdoff enabled
Feb  6 20:17:21 garden kernel: [    0.833852] dwc_otg: FIQ split-transaction FSM enabled
Feb  6 20:17:21 garden kernel: [    0.833870] Module dwc_common_port init
Feb  6 20:17:21 garden kernel: [    3.289808] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
Feb  6 20:17:21 garden kernel: [   27.232844] ath: EEPROM regdomain: 0x809c
Feb  6 20:17:21 garden kernel: [   27.232851] ath: EEPROM indicates we should expect a country code
Feb  6 20:17:21 garden kernel: [   27.232860] ath: doing EEPROM country->regdmn map search
Feb  6 20:17:21 garden kernel: [   27.232867] ath: country maps to regdmn code: 0x52
Feb  6 20:17:21 garden kernel: [   27.232875] ath: Country alpha2 being used: CN
Feb  6 20:17:21 garden kernel: [   27.232881] ath: Regpair used: 0x52

I as well suspecting node-red might be part of the issue. I've disabled it on first machine and left running on second one, for observation.
It seems to hang somehow when network being restarted and node-red wants to communicate with mqtt server and then network hangs as well.

[Joulinar]

hi,

one small question. on the second device, you have Ethernet (10.10.1.170) as well as WiFi (10.10.1.182) connect at the same time?

[kriznik]

it was yes (because i needed to compile driver for that edimax from other thread )
ethernet was disconnected before night yesterday, and stalled at 1:17am which had only wifi connection

[Joulinar]

yep /etc/cron.hourly/dietpi did not contain anything related to network. so would be good to find out what happen at the 17th minute of a hour on your system. Did you checked crontab? If not you could run the following. It will display active crontab only. If there are no crontab entries, it will report nothing.

Code: Select all

for user in $(cut -f1 -d: /etc/passwd); do crontab -u $user -l 2>/dev/null | grep -v '^#'; done

if you like to see all crontab for all user, even if they are empty, go for this

Code: Select all

for user in $(cut -f1 -d: /etc/passwd); do echo $user; crontab -u $user -l; done

[kriznik]

nothing is running at 17th minute via crons for any users