Sparky Update Stalls

Yatsushiro · August 4, 2017, 11:59am

Hi

Been using this excellent OS to run LMS on an Allo SParky SBC, feeding a network of Pis running Dietpi and PCP.

I noticed there was an update available (153 > 154), so initiated the update process. After stopping the running services, the update process stalls at ‘patching’. The Sparky is still accessible by ssh, but there is minimal CPU activity, and HTOP shows only HTOP consuming any CPU.

A reboot gets everything running again, but still at v153.

Any ideas how to resolve this, or investigate further?

Fourdee · August 5, 2017, 11:43am

Hi,

Thanks for the report.

Would you be able to take a screenshot of the hang, as it occurs?

Also:

Which Sparky setup are you running (eg: Sparky + Kali etc)?
Which Power supply used (eg: 5V/3A barrel official)?
Which SD/EMMC card used?

Yatsushiro · August 5, 2017, 6:29pm

Hi Dan

Struggling with a screen grab, but below shows how we get to the point where the update process stalls. I’ve once left it in this state overnight, but that made no difference. If I open another ssh terminla winodw and run HTOP, it only shows HTOP consuming 0.9% CPU

[paul@Manjaro-Laptop ~]$ ssh root@192.168.1.222
root@192.168.1.222's password: 

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.

 ───────────────────────────────────────
 DietPi     | Update available 
 ───────────────────────────────────────
 V153       | Sparky SBC (armv7l)
 ───────────────────────────────────────
 IP Address | 192.168.1.222
 ───────────────────────────────────────

 Created by : Daniel Knight
 Web        : http://DietPi.com
 Twitter    : http://twitter.com/dietpi_
 Donate     : http://goo.gl/pzISt9 
 DietPi's web hosting is powered by: MyVirtualServer.com

 dietpi-launcher  = All the DietPi programs in one place.
 dietpi-config    = Feature rich configuration tool for your device.
 dietpi-software  = Select optimized software for installation.
 dietpi-update    = Run now to update DietPi (from V153 to V154).
 htop             = Resource monitor.
 cpu              = Shows CPU information and stats.

root@Sparky:~# dietpi-update
 [Info] Checking RootFS R/W access. Please wait...
 [Ok] RootFS R/W access.

 DietPi-Update
─────────────────────────────────────────────────────
 Mode: Checking for DietPi updates
 Please wait...

 [Info] Checking Mirror : http://dietpi.com/downloads/dietpi-update_mirror/master/server_version
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     6  100     6    0     0     69      0 --:--:-- --:--:-- --:--:--    70
 [Ok] Using update server: http://dietpi.com/downloads/dietpi-update_mirror/master/server_version

 [Ok] Update available
 [Info] Current Version : 153
 [Info] Server Version  : 154
 [Info] Detecting drives, please wait...
 [Info] Detected /dev mount: /dev/mmcblk0p1 > /boot
 [Info] Detected /dev mount: /dev/mmcblk0p2 > /
 [Info] Detected UUID mount: /dev/sda1 > /mnt/8ae2b5d6-3f83-4e81-a670-f1d76feb8616
 [Info] Processing drive information, please wait...
 [Info] Checking available free space on RootFS, please wait...
 [Ok] 26307 MB available, 500 MB required

 DietPi-Services
─────────────────────────────────────────────────────
 Mode: stop
 Please wait...

 [Ok] cron
 [Ok] proftpd
 [Ok] nfs-kernel-server
 [Ok] SqueezeBoxr

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  351k  100  351k    0     0  1659k      0 --:--:-- --:--:-- --:--:-- 1664k
Archive:  /tmp/dietpi-update/update.zip
e76ec216e532a1571bfc9c995f65e38517d9c051
   creating: /tmp/dietpi-update/DietPi-master/
  inflating: /tmp/dietpi-update/DietPi-master/.gitattributes  
   creating: /tmp/dietpi-update/DietPi-master/.github/
  inflating: /tmp/dietpi-update/DietPi-master/.github/ISSUE_TEMPLATE.md  
  inflating: /tmp/dietpi-update/DietPi-master/.github/PULL_REQUEST_TEMPLATE.md  
  inflating: /tmp/dietpi-update/DietPi-master/.gitignore  
  inflating: /tmp/dietpi-update/DietPi-master/CHANGELOG.txt  
  inflating: /tmp/dietpi-update/DietPi-master/PREP_SYSTEM_FOR_DIETPI.sh  
  inflating: /tmp/dietpi-update/DietPi-master/README.md  
  inflating: /tmp/dietpi-update/DietPi-master/TESTING-BRANCH.md  
  inflating: /tmp/dietpi-update/DietPi-master/boot_c1.ini  
  inflating: /tmp/dietpi-update/DietPi-master/boot_c2.ini  
  inflating: /tmp/dietpi-update/DietPi-master/boot_xu4.ini  
  inflating: /tmp/dietpi-update/DietPi-master/config.txt  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi.txt  
   creating: /tmp/dietpi-update/DietPi-master/dietpi/
 extracting: /tmp/dietpi-update/DietPi-master/dietpi/.dietpi-autostart_index  
 extracting: /tmp/dietpi-update/DietPi-master/dietpi/.install_stage  
 extracting: /tmp/dietpi-update/DietPi-master/dietpi/.update_stage  
 extracting: /tmp/dietpi-update/DietPi-master/dietpi/.version  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/boot  
   creating: /tmp/dietpi-update/DietPi-master/dietpi/conf/
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/apache2_jessie.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/brutefir.service  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/brutefir_config  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/cava.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/cron.daily_dietpi  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/cron.hourly_dietpi  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/deluge.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/deluge.service  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/deluge_web.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/fail2ban_jail.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/fstab  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/fstab_vm  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/fstab_x86_64-native  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/haproxy_init  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/htoprc  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/kodi_udev  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/minidlna.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/mpd.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/network_interfaces  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/nginx.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/nginx.site-available-default  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/nginx.sites-dietpi.owncloud.config  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/openbazaar-server.service  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/opentyrian_run  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/phpsysinfo.ini  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/proftpd.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/raspimjpeg.service  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/smb.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/squeezeboxserver.service  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/vsftpd.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/xorg_c1.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/xorg_c2.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/xorg_xu4.conf  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/conf/ympd_init  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-autostart  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-backup  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-banner  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-bugreport  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-cleaner  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-cloudshell  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-config  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-cpu_set  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-cpuinfo  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-cron  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-drive_manager  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-funtime  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-launcher  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-letsencrypt  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-logclear  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-morsecode  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-obtain_hw_model  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-process_tool  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-ramdisk  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-ramlog  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-services  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-software  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-survey  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-sync  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/dietpi-update  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/finalise  
   creating: /tmp/dietpi-update/DietPi-master/dietpi/func/
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/change_hostname  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/check_connection  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/create_mysql_db  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/dietpi-led_control  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/dietpi-notify  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/dietpi-optimal_mtu  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/dietpi-set_curlftpfs  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/dietpi-set_dphys-swapfile  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/dietpi-set_hardware  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/dietpi-set_nfsclient  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/dietpi-set_smbclient  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/dietpi-set_software  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/dietpi-set_userdata  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/obtain_network_details  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/func/run_ntpd  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/login  
   creating: /tmp/dietpi-update/DietPi-master/dietpi/misc/
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/misc/dietpi-justboom  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/misc/start_kodi  
  inflating: /tmp/dietpi-update/DietPi-master/dietpi/patch_file  
 extracting: /tmp/dietpi-update/DietPi-master/dietpi/server_version  
  inflating: /tmp/dietpi-update/DietPi-master/uEnv.txt  

 DietPi-Update
─────────────────────────────────────────────────────
 Mode: Updating DietPi
 Please wait...

 [Info] Current Version : 153
 [Info] Server Version  : 154

 [Info] Patching 153 to 154

The Sparky is on it’s own, no Kali or other HAT, ethernet. connected to a 1TB HDD via a powered USB hub. Due to a mishap with the micro-usb power port, the Sparky itself is powered through the GPIO pins, by the standard Allo 5v wallwart. The system has run this way since day one without issue, and has been through previous update cycles without issue (installed at version 149, from memory)

The SD card is a Sandisk Ultra, 32Gb Class 10.

Looking at the changelog, I can see nothing in the update that I need, but I like to keep it current.

Thanks for your help, let me know if you need more info/detail.

Fourdee · August 6, 2017, 11:55am

Thanks.

We’ll need to debug the 153 > 154 patch code, to see where its stalling.

Please can you paste results:

cat /DietPi/dietpi.txt | grep '^NTPD_Mirror='
cat /DietPi/config.txt | grep 'sdtv_mode='
dpkg -l | grep pciutils
cat /DietPi/dietpi/.installed | grep -ci -m1 '^aSOFTWARE_INSTALL_STATE\[58\]=2'

#Some tests we need to run
apt purge -y pciutils

PS. I’ve just updated my Sparky from v152 to v154 without any hanging issues. So this issue must be limited to your current system in some form.

Yatsushiro · August 7, 2017, 9:29am

In order, the commands and output requested:

root@Sparky:~# cat /DietPi/dietpi.txt | grep '^NTPD_Mirror='
root@Sparky:~#

root@Sparky:~# cat /DietPi/config.txt | grep 'sdtv_mode='
#sdtv_mode=2
root@Sparky:~#

root@Sparky:~# dpkg -l | grep pciutils
ii  pciutils                  1:3.2.1-3                 armhf        Linux PCI Utilities
root@Sparky:~#

root@Sparky:~# cat /DietPi/dietpi/.installed | grep -ci -m1 '^aSOFTWARE_INSTALL_STATE\[58\]=2'
0
root@Sparky:~#

root@Sparky:~# apt purge -y pciutils
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  libpci3
Use 'apt-get autoremove' to remove it.
The following packages will be REMOVED:
  pciutils*
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 1,119 kB disk space will be freed.
(Reading database ... 19505 files and directories currently installed.)
Removing pciutils (1:3.2.1-3) ...
Purging configuration files for pciutils (1:3.2.1-3) ...
root@Sparky:~#

Hope this is useful…

Yatsushiro · August 7, 2017, 9:35am

Not sure if this is of any use at all, but since the last upgrade from 152 to 153, other than adding more music files, the only change made was to add a symbolic link to the Music directory on my Linux laptop:

root@Sparky:~# df
Filesystem      1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2   30615212   1965512  27077836   7% /
none               506520      6796    499724   2% /run
udev                10240         0     10240   0% /dev
tmpfs              506520         0    506520   0% /dev/shm
tmpfs                5120         0      5120   0% /run/lock
tmpfs              506520         0    506520   0% /sys/fs/cgroup
/dev/sda1       961302540 592221448 369064708  62% /export/Music
tmpfs               10240      1432      8808  14% /DietPi
tmpfs               20480        24     20456   1% /var/log
tmpfs              506520         0    506520   0% /tmp
/dev/mmcblk0p1      49038     11942     37096  25% /boot
192.168.1.222:/  30615296   1965568  27077888   7% /mnt/nfs_client

[paul@Manjaro-Laptop ~]$ exa -l
drwxr-xr-x  - paul 25 Jul 15:26 Desktop
drwxr-xr-x  - paul 20 Jul 20:36 Documents
drwxr-xr-x  - paul  6 Aug  9:10 Downloads
drwxr-xr-x  - paul  6 Aug  9:15 Music
drwxr-xr-x  - paul 20 Jul 12:47 Pi
drwxr-xr-x  - paul  5 Aug 19:34 Pictures
drwxr-xr-x  - paul  3 Jul 15:10 Public
drwxr-xr-x  - paul 23 Jul 15:22 Recordings
lrwxrwxrwx 17 paul  3 Jul 19:10 Sparky -> /mnt/Music/Music
drwxr-xr-x  - paul  3 Jul 15:10 Templates
drwxr-xr-x  - paul  3 Jul 15:10 Videos
[paul@Manjaro-Laptop ~]$

Fourdee · August 7, 2017, 12:59pm

root@Sparky:~# cat /DietPi/dietpi.txt | grep ‘^NTPD_Mirror=’
root@Sparky:~#

Thanks.

As none of the other patches have been applied, I believe this would of been the command hanging the system, but appears functional now.

Ok, try running the update again, it should be working (in theory).

NFS can cause endless system/command hangs, its a little “flaky” at times. CIFS (samba) is recommended where possible. If problems persist with the update, try dismounting the NFS drive before running the update:

umount /mnt/nfs_client
dietpi-update

If it fails to unmount, you’ll need to disable the /etc/fstab entry, run systemctl daemon-reload and reboot. Then try the update again.

Yatsushiro · August 7, 2017, 5:56pm

Still no go, hangs at the same point.

Tried twice as instructed, second time with “umount /mnt/nfs_client” as this did unmount, as shown with a df command.

Your help is appreciated. As I said previously, it’s all working without the update, but I’d prefer to stay up to date…

Not sure if this is useful, but if I ssh in another terminal when the update process is stalled, this is the HTOP output:

Fourdee · August 8, 2017, 9:11am

Looks like its hanging on obtaining the hardware information on this device.

Please can you run the following commands and paste results:

cat /DietPi/dietpi/.hw_model
/DietPi/dietpi/dietpi-obtain_hw_model

If it hangs on the 2nd command, check for any onscreen errors/info, press CTRL+C to terminate and report back.

Yatsushiro · August 8, 2017, 9:15am

As requested:

root@Sparky:~# cat /DietPi/dietpi/.hw_model
70
Sparky SBC (armv7l)
3
/dev/mmcblk0p2
2936888e-d7c8-43fe-be10-f09c6ab5c2b1
2
armv7l

0
0
0
Unknown
Unknown
0
Unknown
root@Sparky:~#

root@Sparky:~# /DietPi/dietpi/dietpi-obtain_hw_model
root@Sparky:~#

Fourdee · August 8, 2017, 10:31am

Yatsushiro:

As requested:

root@Sparky:~# cat /DietPi/dietpi/.hw_model
70
Sparky SBC (armv7l)
3
/dev/mmcblk0p2
2936888e-d7c8-43fe-be10-f09c6ab5c2b1
2
armv7l

0
0
0
Unknown
Unknown
0
Unknown
root@Sparky:~#

root@Sparky:~# /DietPi/dietpi/dietpi-obtain_hw_model
root@Sparky:~#

Thanks. Appears fine.

Ok lets get a bug report for this, it will contain system information we can use to debug:
https://dietpi.com/forum/t/send-dietpi-a-bug-report/421/1

Yatsushiro · August 8, 2017, 12:14pm

HI Dan

Bug report completed, Reference Code is 2936888e-d7c8-43fe-be10-f09c6ab5c2b1-0

Thanks for your help.

Fourdee · August 20, 2017, 12:00pm

Hi,

Sincere apologies for the delay, this one slipped the net

Are you still experiencing this issue with failed updates?

Yatsushiro · August 20, 2017, 4:09pm

Yep, keep trying every couple of days, but stalls at the same point each time.

Yatsushiro · September 8, 2017, 9:13am

A gentle bump…

With an update to v155 now available, I still can’t upgrade from 153 to 154. I’m happy to sport flares and a permed mullet, but I like my software to be up to date

Yatsushiro · September 23, 2017, 7:05pm

Sorry to be an irritant, but have you had a chance to have a look at this?

Fourdee · September 23, 2017, 7:41pm

Hi,

Apologies again, I have not had a chance to look into this further yet.

Due to a mishap with the micro-usb power port, the Sparky itself is powered through the GPIO pins

I believe this is the culprit, unstable power causing filesystem and stability issues. Although, we cant verify this yet.

To prevent wasting your time, and to provide a solution now:

Everything our end appears to be in order. We cant see any issues in your system setup/configuration that would cause this issue at the moment.
As we believe the issue is related to your current hardware and installation only (we cant replicate the issue, and, this is the sole case), we recommend writing a fresh DietPi image and reinstalling the software you require.

If problems persist after the fresh image write, we may need to look at:

replacing the SD card
verify the GPIO wires are 2A+ capable
verify the GPIO connections are stable
try the Micro USB power. Curious, what actually happened with the USB power port?

With SBC’s, any slight interruption in power stability can cause havoc on the filesystem. Opening a can of endless worms.