DietPi v6.28.0 Wifi drop update: freezed networking

Have some feedback, questions, suggestions, or just fancy a chat? Pop it in here.
kriznik
Posts: 22
Joined: Mon Feb 03, 2020 8:03 pm

Re: DietPi v6.28.0 Wifi drop, updated: freezed networking

Post by kriznik »

small update
I've installed 6.28 on another rpi for testing, with different usb wifi and same software.
again 1:17am system is unreachable via wifi, but this time i do have monitor and shell as well directly.

interestingly, system is not freezed, even wifi has got IP, but networking is ded. no ping in no ping out at all.

systemctl restart networking solved it

so I'm wondering what can cause networking to stall thanks to hourly crons, which seems to be only doing some logg rotating if enabled?
as seen from previous debug, hourly cron somehow restarts networking for some uknown reason (maybe thanks to wifi manager?) but sometimes it got stuck.

How we can debug more around networking what is happening and why?

as quick fix, i probably just edit dietpi-wifi-monitor.sh and instead of ifdown/ifup I'll restart whole networking service
User avatar
Joulinar
Posts: 2072
Joined: Sat Nov 16, 2019 12:49 am

Re: DietPi v6.28.0 Wifi drop update: freezed networking

Post by Joulinar »

hi,

your system seems to reboot completely time by time. Probably some issues with power plug or device (hardware). Maybe SD card or file system corruption. There is no reason on the logs for the reboot :? . Unfortunately we don't know if there are any memory issues / overflow at the time your system get stuck.

As well it seems your system is not booting correctly once it crashed or it's hanging during boot. or it has issues to connect to WiFi. Because 1 hour for time sync it a huuuuugggge amount of time.

pls can you check these 2 values

Code: Select all

cat /DietPi/dietpi.txt |grep CONFIG_NTP_MODE
cat /DietPi/dietpi.txt |grep CONFIG_BOOT_WAIT_FOR_NETWORK
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
kriznik
Posts: 22
Joined: Mon Feb 03, 2020 8:03 pm

Re: DietPi v6.28.0 Wifi drop update: freezed networking

Post by kriznik »

Code: Select all

dietpi@garden:~$ cat /DietPi/dietpi.txt |grep CONFIG_NTP_MODE
CONFIG_NTP_MODE=4
dietpi@garden:~$ cat /DietPi/dietpi.txt |grep CONFIG_BOOT_WAIT_FOR_NETWORK
# - It is HIGHLY recommended to also set CONFIG_BOOT_WAIT_FOR_NETWORK=2, to force infinite wait for network connection during boot and prevent connection timeout errors.
CONFIG_BOOT_WAIT_FOR_NETWORK=1
dietpi@garden:~$
and second device

Code: Select all

root@DietPi:~# cat /DietPi/dietpi.txt |grep CONFIG_NTP_MODE
CONFIG_NTP_MODE=2
root@DietPi:~# cat /DietPi/dietpi.txt |grep CONFIG_BOOT_WAIT_FOR_NETWORK
# - It is HIGHLY recommended to also set CONFIG_BOOT_WAIT_FOR_NETWORK=2, to force infinite wait for network connection during boot and prevent connection timeout errors.
CONFIG_BOOT_WAIT_FOR_NETWORK=1
root@DietPi:~#
Hi, no ... as I posted in previous post, it is happening on two completely different raspberries (both model 2b) with usb wifi dongles from different vendors. And uptime is not changed eventho it looks like it's rebooted. Power is ok, sd/usb is indeed ok, memory is fine.

Logs on the second pi (which i have monitor attached to it and keyboard) seems exactly the same as on the first one. I was able to gain network access by simply restarting whole networking -> systemctl restart networking

on first machine there is gap I assume because networking freezed as well, but as i can't observe it as it is headless, hard to tell.
User avatar
MichaIng
Site Admin
Posts: 2294
Joined: Sat Nov 18, 2017 6:21 pm

Re: DietPi v6.28.0 Wifi drop update: freezed networking

Post by MichaIng »

Hmm, strange issue. When you have some Pi up, could you paste output of:

Code: Select all

systemctl status networking
systemctl status ifup@wlan0
cat /etc/network/interfaces
kriznik
Posts: 22
Joined: Mon Feb 03, 2020 8:03 pm

Re: DietPi v6.28.0 Wifi drop update: freezed networking

Post by kriznik »

sure i can
first one

Code: Select all

dietpi@garden:~$ sudo systemctl status networking
● networking.service - Raise network interfaces
   Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
   Active: active (exited) since Thu 2020-02-06 20:17:27 CET; 17h ago
     Docs: man:interfaces(5)
  Process: 317 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=0/SUCCESS)
 Main PID: 317 (code=exited, status=0/SUCCESS)

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
dietpi@garden:~$ sudo systemctl status ifup@wlan0
● ifup@wlan0.service - ifup for wlan0
   Loaded: loaded (/lib/systemd/system/ifup@.service; static; vendor preset: enabled)
   Active: active (exited) since Thu 2020-02-06 20:17:25 CET; 17h ago
  Process: 318 ExecStart=/bin/sh -ec ifup --allow=hotplug wlan0; ifquery --state wlan0 (code=exited, status=0/SUCCESS)
 Main PID: 318 (code=exited, status=0/SUCCESS)
   Memory: 5.4M
   CGroup: /system.slice/ifup@wlan0.service
           ├─396 /sbin/wpa_supplicant -s -B -P /run/wpa_supplicant.wlan0.pid -i wlan0 -D nl80211,wext -c /etc/wpa_supplicant/wpa_suppl
icant.conf
           └─424 /sbin/dhclient -4 -v -i -pf /run/dhclient.wlan0.pid -lf /var/lib/dhcp/dhclient.wlan0.leases -I -df /var/lib/dhcp/dhcl
ient6.wlan0.leases wlan0

Feb 07 13:27:00 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:29:34 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:32:10 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:32:11 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:32:12 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:34:46 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:34:47 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:37:20 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:37:24 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
Feb 07 13:45:10 garden wpa_supplicant[396]: wlan0: CTRL-EVENT-BEACON-LOSS
dietpi@garden:~$ sudo cat /etc/network/interfaces
# Location: /etc/network/interfaces
# Please modify network settings via: dietpi-config
# Or create your own drop-ins in: /etc/network/interfaces.d/

# Drop-in configs
source interfaces.d/*

# Local
auto lo
iface lo inet loopback

# Ethernet
#allow-hotplug eth0
iface eth0 inet dhcp
address 0.0.0.0
netmask 0.0.0.0
gateway 0.0.0.0
#dns-nameservers 0.0.0.0

# WiFi
allow-hotplug wlan0
iface wlan0 inet dhcp
address 10.10.1.55
netmask 255.255.255.0
gateway 10.10.1.1
#dns-nameservers 10.10.1.5
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
dietpi@garden:~$
second one

Code: Select all

root@DietPi:~# systemctl status networking
● networking.service - Raise network interfaces
   Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
   Active: active (exited) since Fri 2020-02-07 12:15:21 CET; 1h 40min ago
     Docs: man:interfaces(5)
  Process: 6304 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=0/SUCCESS)
 Main PID: 6304 (code=exited, status=0/SUCCESS)

Feb 07 12:15:21 DietPi systemd[1]: Starting Raise network interfaces...
Feb 07 12:15:21 DietPi systemd[1]: Started Raise network interfaces.
root@DietPi:~# systemctl status ifup@wlan0
● ifup@wlan0.service - ifup for wlan0
   Loaded: loaded (/lib/systemd/system/ifup@.service; static; vendor preset: enabled)
   Active: active (exited) since Fri 2020-02-07 12:15:41 CET; 1h 40min ago
  Process: 6376 ExecStart=/bin/sh -ec ifup --allow=hotplug wlan0; ifquery --state wlan0 (code=exited, status=0/SUCCESS)
 Main PID: 6376 (code=exited, status=0/SUCCESS)
   Memory: 1.7M
   CGroup: /system.slice/ifup@wlan0.service
           ├─6391 /sbin/wpa_supplicant -s -B -P /run/wpa_supplicant.wlan0.pid -i wlan0 -D nl80211,wext -c /etc/wpa_supplicant/wpa_supplicant.conf
           └─6397 /sbin/dhclient -4 -v -i -pf /run/dhclient.wlan0.pid -lf /var/lib/dhcp/dhclient.wlan0.leases -I -df /var/lib/dhcp/dhclient6.wlan0.leases wlan0

Feb 07 12:15:46 DietPi sh[6376]: DHCPDISCOVER on wlan0 to 255.255.255.255 port 67 interval 6
Feb 07 12:15:49 DietPi dhclient[6397]: DHCPOFFER of 10.10.1.182 from 10.10.1.1
Feb 07 12:15:49 DietPi sh[6376]: DHCPOFFER of 10.10.1.182 from 10.10.1.1
Feb 07 12:15:49 DietPi sh[6376]: DHCPREQUEST for 10.10.1.182 on wlan0 to 255.255.255.255 port 67
Feb 07 12:15:49 DietPi dhclient[6397]: DHCPREQUEST for 10.10.1.182 on wlan0 to 255.255.255.255 port 67
Feb 07 12:15:49 DietPi sh[6376]: DHCPACK of 10.10.1.182 from 10.10.1.1
Feb 07 12:15:49 DietPi dhclient[6397]: DHCPACK of 10.10.1.182 from 10.10.1.1
Feb 07 12:15:50 DietPi dhclient[6397]: bound to 10.10.1.182 -- renewal in 33495 seconds.
Feb 07 12:15:50 DietPi sh[6376]: bound to 10.10.1.182 -- renewal in 33495 seconds.
Feb 07 12:15:50 DietPi sh[6376]: wlan0=wlan0
root@DietPi:~# cat /etc/network/interfaces
# Location: /etc/network/interfaces
# Please modify network settings via: dietpi-config
# Or create your own drop-ins in: /etc/network/interfaces.d/

# Drop-in configs
source interfaces.d/*

# Local
auto lo
iface lo inet loopback

# Ethernet
allow-hotplug eth0
iface eth0 inet dhcp
address 10.10.1.170
netmask 255.255.255.0
gateway 10.10.1.1
#dns-nameservers 10.10.1.5

# WiFi
allow-hotplug wlan0
iface wlan0 inet dhcp
address 10.10.1.182
netmask 255.255.255.0
gateway 10.10.1.1
#dns-nameservers 10.10.1.5
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
root@DietPi:~#
kriznik
Posts: 22
Joined: Mon Feb 03, 2020 8:03 pm

Re: DietPi v6.28.0 Wifi drop update: freezed networking

Post by kriznik »

btw MichaIng is there some reason that hourly cron restarts network (or usb dongle ath)? As far as I can see in /etc/cron.hourly/dietpi it does not look like anything should mess with networking at all.
And as it is the only cron ran at 17th minute I assume there have to be something in there.

on top of it, not every hour tho, but 17th minute is indicating me hourly cron

debug log from first machine

Code: Select all

Feb  6 07:17:19 garden kernel: [    0.000000] On node 0 totalpages: 61440
Feb  6 07:17:19 garden kernel: [    0.000000]   Normal zone: 540 pages used for memmap
Feb  6 07:17:19 garden kernel: [    0.000000]   Normal zone: 0 pages reserved
Feb  6 07:17:19 garden kernel: [    0.000000]   Normal zone: 61440 pages, LIFO batch:15
Feb  6 07:17:19 garden kernel: [    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
Feb  6 07:17:19 garden kernel: [    0.000000] pcpu-alloc: [0] 0
Feb  6 07:17:19 garden kernel: [    0.825715] dwc_otg: Microframe scheduler enabled
Feb  6 07:17:19 garden kernel: [    0.833875] dwc_otg: FIQ enabled
Feb  6 07:17:19 garden kernel: [    0.833887] dwc_otg: NAK holdoff enabled
Feb  6 07:17:19 garden kernel: [    0.833893] dwc_otg: FIQ split-transaction FSM enabled
Feb  6 07:17:19 garden kernel: [    0.833911] Module dwc_common_port init
Feb  6 07:17:19 garden kernel: [    3.289831] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
Feb  6 07:17:19 garden kernel: [   27.646858] ath: EEPROM regdomain: 0x809c
Feb  6 07:17:19 garden kernel: [   27.646865] ath: EEPROM indicates we should expect a country code
Feb  6 07:17:19 garden kernel: [   27.646873] ath: doing EEPROM country->regdmn map search
Feb  6 07:17:19 garden kernel: [   27.646880] ath: country maps to regdmn code: 0x52
Feb  6 07:17:19 garden kernel: [   27.646889] ath: Country alpha2 being used: CN
Feb  6 07:17:19 garden kernel: [   27.646894] ath: Regpair used: 0x52
Feb  6 18:17:18 garden kernel: [    0.000000] On node 0 totalpages: 61440
Feb  6 18:17:18 garden kernel: [    0.000000]   Normal zone: 540 pages used for memmap
Feb  6 18:17:18 garden kernel: [    0.000000]   Normal zone: 0 pages reserved
Feb  6 18:17:18 garden kernel: [    0.000000]   Normal zone: 61440 pages, LIFO batch:15
Feb  6 18:17:18 garden kernel: [    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
Feb  6 18:17:18 garden kernel: [    0.000000] pcpu-alloc: [0] 0
Feb  6 18:17:18 garden kernel: [    0.825647] dwc_otg: Microframe scheduler enabled
Feb  6 18:17:18 garden kernel: [    0.833821] dwc_otg: FIQ enabled
Feb  6 18:17:18 garden kernel: [    0.833833] dwc_otg: NAK holdoff enabled
Feb  6 18:17:18 garden kernel: [    0.833839] dwc_otg: FIQ split-transaction FSM enabled
Feb  6 18:17:18 garden kernel: [    0.833856] Module dwc_common_port init
Feb  6 18:17:18 garden kernel: [    3.289928] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
Feb  6 18:17:18 garden kernel: [   27.006839] ath: EEPROM regdomain: 0x809c
Feb  6 18:17:18 garden kernel: [   27.006847] ath: EEPROM indicates we should expect a country code
Feb  6 18:17:18 garden kernel: [   27.006855] ath: doing EEPROM country->regdmn map search
Feb  6 18:17:18 garden kernel: [   27.006862] ath: country maps to regdmn code: 0x52
Feb  6 18:17:18 garden kernel: [   27.006871] ath: Country alpha2 being used: CN
Feb  6 18:17:18 garden kernel: [   27.006876] ath: Regpair used: 0x52
Feb  6 20:17:21 garden kernel: [    0.000000] On node 0 totalpages: 61440
Feb  6 20:17:21 garden kernel: [    0.000000]   Normal zone: 540 pages used for memmap
Feb  6 20:17:21 garden kernel: [    0.000000]   Normal zone: 0 pages reserved
Feb  6 20:17:21 garden kernel: [    0.000000]   Normal zone: 61440 pages, LIFO batch:15
Feb  6 20:17:21 garden kernel: [    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
Feb  6 20:17:21 garden kernel: [    0.000000] pcpu-alloc: [0] 0
Feb  6 20:17:21 garden kernel: [    0.825680] dwc_otg: Microframe scheduler enabled
Feb  6 20:17:21 garden kernel: [    0.833834] dwc_otg: FIQ enabled
Feb  6 20:17:21 garden kernel: [    0.833846] dwc_otg: NAK holdoff enabled
Feb  6 20:17:21 garden kernel: [    0.833852] dwc_otg: FIQ split-transaction FSM enabled
Feb  6 20:17:21 garden kernel: [    0.833870] Module dwc_common_port init
Feb  6 20:17:21 garden kernel: [    3.289808] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
Feb  6 20:17:21 garden kernel: [   27.232844] ath: EEPROM regdomain: 0x809c
Feb  6 20:17:21 garden kernel: [   27.232851] ath: EEPROM indicates we should expect a country code
Feb  6 20:17:21 garden kernel: [   27.232860] ath: doing EEPROM country->regdmn map search
Feb  6 20:17:21 garden kernel: [   27.232867] ath: country maps to regdmn code: 0x52
Feb  6 20:17:21 garden kernel: [   27.232875] ath: Country alpha2 being used: CN
Feb  6 20:17:21 garden kernel: [   27.232881] ath: Regpair used: 0x52
I as well suspecting node-red might be part of the issue. I've disabled it on first machine and left running on second one, for observation.
It seems to hang somehow when network being restarted and node-red wants to communicate with mqtt server and then network hangs as well.
User avatar
Joulinar
Posts: 2072
Joined: Sat Nov 16, 2019 12:49 am

Re: DietPi v6.28.0 Wifi drop update: freezed networking

Post by Joulinar »

hi,

one small question. on the second device, you have Ethernet (10.10.1.170) as well as WiFi (10.10.1.182) connect at the same time?
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
kriznik
Posts: 22
Joined: Mon Feb 03, 2020 8:03 pm

Re: DietPi v6.28.0 Wifi drop update: freezed networking

Post by kriznik »

it was yes (because i needed to compile driver for that edimax from other thread ;) )
ethernet was disconnected before night yesterday, and stalled at 1:17am which had only wifi connection
User avatar
Joulinar
Posts: 2072
Joined: Sat Nov 16, 2019 12:49 am

Re: DietPi v6.28.0 Wifi drop update: freezed networking

Post by Joulinar »

yep /etc/cron.hourly/dietpi did not contain anything related to network. so would be good to find out what happen at the 17th minute of a hour on your system. Did you checked crontab? If not you could run the following. It will display active crontab only. If there are no crontab entries, it will report nothing.

Code: Select all

for user in $(cut -f1 -d: /etc/passwd); do crontab -u $user -l 2>/dev/null | grep -v '^#'; done
if you like to see all crontab for all user, even if they are empty, go for this

Code: Select all

for user in $(cut -f1 -d: /etc/passwd); do echo $user; crontab -u $user -l; done
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
kriznik
Posts: 22
Joined: Mon Feb 03, 2020 8:03 pm

Re: DietPi v6.28.0 Wifi drop update: freezed networking

Post by kriznik »

nothing is running at 17th minute via crons for any users
Post Reply