Failed upgrade to Buster from Strecth

Torvald · 18 March 2022 10:48

Hello everyone!
Finally decided to upgrade my system to the latest version. Now I have - DietPi v7.9.4 (stretch) .
I follow the manual https://dietpi.com/docs/usage/#how-to-upgrade-to-buster. And can’t get past the “apt upgrade” step.
This is the output of the “apt upgrade” command:

Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  cpp-6 gnupg-agent gvfs-bin libboost-chrono1.62.0 libboost-python1.62.0 libboost-random1.62.0 libboost-system1.62.0
  libcdio-cdda1 libcdio-paranoia1 libcdio13 libcryptsetup4 libdns-export162 libexempi3 libfile-copy-recursive-perl
  libgcr-3-common libgdbm3 libgles2-mesa libglew2.0 libgstreamer-plugins-bad1.0-0 libgtop-2.0-10 libisc-export160
  libisl15 liblogging-stdlog0 liblttng-ust-ctl2 libmpfr4 libnfs8 libpango1.0-0 libpangox-1.0-0 libpangoxft-1.0-0
  libperl5.24 libprocps6 libtinyxml2-4 libunique-3.0-0 liburcu4 libustr-1.0-1 libwayland-egl1-mesa libxfont1
  mate-backgrounds perl-modules-5.24 python-certifi python-idna python-requests python-setuptools python-twisted-web
  python-urllib3 python3-pyasn1 update-inetd
Use 'apt autoremove' to remove them.

I try to execute “apt autoremove” and the output is:

Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 842 not upgraded.

Nothing happened. How do I get through this step and upgrade to a Buster?

Joulinar · 18 March 2022 10:51

do you cancel the apt upgrade? Because running apt autoremove is just an information and would not be needed to be executed. Usually you should be ask with a yes/no question while executing apt upgrade, if you like to proceed.

Torvald · 18 March 2022 10:56

Yep i canceled it. Since after this message nothing happened for several minutes. Should I just wait?

Joulinar · 18 March 2022 10:58

yes pls wait. be patient and get a coffee. You need to update more than 800 packages. Means, it will take time depending on the speed of your device.

Torvald · 18 March 2022 12:47

Thanks for help!
The package update went through, although not very smoothly.
My deluge does not start, and the worst thing is that the nextcloud is gone.
ncc list says “php: command not found”. I suspect that the problem is in php.

Joulinar · 18 March 2022 12:54

Did you continue following the guide? There are probably more steps needed to get PHP correctly working after the update.

Torvald · 18 March 2022 13:00

Yes i did all the steps from the guide. When I ran the php related commands, some packages were removed but not installed back.
In the console was:

root@MyServer:~# apt -y install "${packages[@]}"
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'php-apcu' instead of 'php7.3-apcu'
Note, selecting 'php-igbinary' instead of 'php7.3-igbinary'
Note, selecting 'php-redis' instead of 'php7.3-redis'
Package php8.1-igbinary is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

Package php8.1-redis is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

Package php7.2-cli is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

And several similar posts.

Joulinar · 18 March 2022 13:56

ok let’s check what packages you have installed for PHP

dpkg -l | grep 'php'

and as well, can you share your source list configuration

for i in /etc/apt/sources.list{,.d/*.list}
do
echo "List: $i"
cat "$i"
done

Torvald · 18 March 2022 14:04

ii  libapache2-mod-php7.3                                       7.3.31-1~deb10u1                                          amd64        server-side, HTML-embedded scripting language (Apache 2 module)
ii  php                                                         2:7.3+69                                                  all          server-side, HTML-embedded scripting language (default)
ii  php-common                                                  2:69                                                      all          Common files for PHP packages
rc  php-igbinary                                                3.2.6+2.0.8-6+0~20220117.32+debian9~1.gbp1d540e           amd64        igbinary PHP serializer
rc  php-redis                                                   5.3.5+4.3.0-5+0~20220117.40+debian9~1.gbpf8a972           amd64        PHP extension for interfacing with Redis
rc  php7.2-cli                                                  7.2.34-28+0~20211119.67+debian9~1.gbpf24e81               amd64        command-line interpreter for the PHP scripting language
rc  php7.2-common                                               7.2.34-28+0~20211119.67+debian9~1.gbpf24e81               amd64        documentation, examples and common module for PHP
rc  php7.2-curl                                                 7.2.34-28+0~20211119.67+debian9~1.gbpf24e81               amd64        CURL module for PHP
rc  php7.2-gd                                                   7.2.34-28+0~20211119.67+debian9~1.gbpf24e81               amd64        GD module for PHP
rc  php7.2-json                                                 7.2.34-28+0~20211119.67+debian9~1.gbpf24e81               amd64        JSON module for PHP
rc  php7.2-mbstring                                             7.2.34-28+0~20211119.67+debian9~1.gbpf24e81               amd64        MBSTRING module for PHP
rc  php7.2-mysql                                                7.2.34-28+0~20211119.67+debian9~1.gbpf24e81               amd64        MySQL module for PHP
rc  php7.2-opcache                                              7.2.34-28+0~20211119.67+debian9~1.gbpf24e81               amd64        Zend OpCache module for PHP
rc  php7.2-readline                                             7.2.34-28+0~20211119.67+debian9~1.gbpf24e81               amd64        readline module for PHP
rc  php7.2-xml                                                  7.2.34-28+0~20211119.67+debian9~1.gbpf24e81               amd64        DOM, SimpleXML, WDDX, XML, and XSL module for PHP
rc  php7.2-zip                                                  7.2.34-28+0~20211119.67+debian9~1.gbpf24e81               amd64        Zip module for PHP
ii  php7.3                                                      7.3.31-1~deb10u1                                          all          server-side, HTML-embedded scripting language (metapackage)
rc  php7.3-apcu                                                 5.1.21+4.0.11-7+0~20220117.31+debian9~1.gbp9a887a         amd64        APC User Cache for PHP
ii  php7.3-cli                                                  7.3.31-1~deb10u1                                          amd64        command-line interpreter for the PHP scripting language
ii  php7.3-common                                               7.3.31-1~deb10u1                                          amd64        documentation, examples and common module for PHP
ii  php7.3-curl                                                 7.3.31-1~deb10u1                                          amd64        CURL module for PHP
ii  php7.3-fpm                                                  7.3.31-1~deb10u1                                          amd64        server-side, HTML-embedded scripting language (FPM-CGI binary)
ii  php7.3-gd                                                   7.3.31-1~deb10u1                                          amd64        GD module for PHP
rc  php7.3-igbinary                                             3.2.6+2.0.8-6+0~20220117.32+debian9~1.gbp1d540e           amd64        igbinary PHP serializer
rc  php7.3-intl                                                 7.3.33-1+0~20211119.91+debian9~1.gbp618351                amd64        Internationalisation module for PHP
ii  php7.3-json                                                 7.3.31-1~deb10u1                                          amd64        JSON module for PHP
ii  php7.3-mbstring                                             7.3.31-1~deb10u1                                          amd64        MBSTRING module for PHP
ii  php7.3-mysql                                                7.3.31-1~deb10u1                                          amd64        MySQL module for PHP
ii  php7.3-opcache                                              7.3.31-1~deb10u1                                          amd64        Zend OpCache module for PHP
ii  php7.3-readline                                             7.3.31-1~deb10u1                                          amd64        readline module for PHP
rc  php7.3-redis                                                5.3.5+4.3.0-5+0~20220117.40+debian9~1.gbpf8a972           amd64        PHP extension for interfacing with Redis
rc  php7.3-sqlite3                                              7.3.33-1+0~20211119.91+debian9~1.gbp618351                amd64        SQLite3 module for PHP
ii  php7.3-xml                                                  7.3.31-1~deb10u1                                          amd64        DOM, SimpleXML, WDDX, XML, and XSL module for PHP
ii  php7.3-zip                                                  7.3.31-1~deb10u1                                          amd64        Zip module for PHP
rc  php8.0-cli                                                  8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        command-line interpreter for the PHP scripting language
rc  php8.0-common                                               8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        documentation, examples and common module for PHP
rc  php8.0-curl                                                 8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        CURL module for PHP
rc  php8.0-fpm                                                  8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        server-side, HTML-embedded scripting language (FPM-CGI binary)
rc  php8.0-gd                                                   8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        GD module for PHP
rc  php8.0-igbinary                                             3.2.6+2.0.8-6+0~20220117.32+debian9~1.gbp1d540e           amd64        igbinary PHP serializer
rc  php8.0-intl                                                 8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        Internationalisation module for PHP
rc  php8.0-mbstring                                             8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        MBSTRING module for PHP
rc  php8.0-mysql                                                8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        MySQL module for PHP
rc  php8.0-opcache                                              8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        Zend OpCache module for PHP
rc  php8.0-readline                                             8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        readline module for PHP
rc  php8.0-sqlite3                                              8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        SQLite3 module for PHP
rc  php8.0-xml                                                  8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        DOM, SimpleXML, XML, and XSL module for PHP
rc  php8.0-zip                                                  8.0.15-1+0~20220129.29+debian9~1.gbpc7cb1f                amd64        Zip module for PHP
rc  php8.1-common                                               8.1.2-1+0~20220127.12+debian9~1.gbpe46588                 amd64        documentation, examples and common module for PHP
rc  php8.1-igbinary                                             3.2.6+2.0.8-2+0~20211115.31+debian9~1.gbpfaf284           amd64        igbinary PHP serializer
rc  php8.1-redis                                                5.3.4+4.3.0-5+0~20211118.38+debian9~1.gbpf06028           amd64        PHP extension for interfacing with Redis

List: /etc/apt/sources.list
deb https://deb.debian.org/debian/ buster main contrib non-free
deb https://deb.debian.org/debian/ buster-updates main contrib non-free
deb https://deb.debian.org/debian-security/ buster/updates main contrib non-free
deb https://deb.debian.org/debian/ buster-backports main contrib non-free
List: /etc/apt/sources.list.d/dietpi-mono.list
deb https://download.mono-project.com/repo/debian/ buster main
List: /etc/apt/sources.list.d/plexmediaserver.list
deb https://downloads.plex.tv/repo/deb public main
List: /etc/apt/sources.list.d/sonarr.list
deb https://apt.sonarr.tv/debian/ buster main

I manually installed php and few other packages which ncc says missing.

Joulinar · 18 March 2022 14:12

I manually installed php

Did you installed something before or after the Buster update? Your web server is an Apache2?

MichaIng
I guess PHP packages would need to be removed completely before going to reinstall PHP, right?

Torvald · 18 March 2022 15:53

I had to install php manually when I upgraded Nextcloud to version 21, since php 7.2 does not work with this version. I don’t remember which version I installed. This was before the upgrade to the buster.
Should I remove all php packages and install only one version that fits everything?

I am using nginx.

Joulinar · 18 March 2022 15:54

Hold on, let’s wait for the developer to have a look into before we doing something wrong

MichaIng · 18 March 2022 16:13

Yes, please do this steps for cleanup:

apt autopurge 'php7.2-*' 'php8.0-*' 'php8.1-*'
apt-mark auto libpcre3 libpcre3-dev libpcre32-3 libpcrecpp0v5 libsodium23 libxml2 libxml2-dev libzip4 php php-common php7.3-redis php7.3-sqlite3 php7.3-intl php7.3-igbinary php7.3-apcu php-igbinary php-redis
apt autopurge
dpkg -l | grep -E 'debian9|bpo9'

The last command is to get a list of still present Stretch packages.

Torvald · 18 March 2022 16:45

I did cleanup. Nextcloud get error in web interface and webserver log.

2022/03/18 19:34:00 [error] 873#873: *367 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Uncaught OC\HintException: [0]: Memcache \OC\Memcache\APCu not available for local cache (Is the matching PHP module installed and enabled?)

  thrown in /var/www/nextcloud/lib/private/Memcache/Factory.php on line 98" while reading response header from upstream, client: ip, server: xxx.xxxx.net, request: "GET /remote.php/dav/files/Torvald/Joplin/info.json HTTP/2.0", upstream: "fastcgi://unix:/run/php/php7.3-fpm.sock:", host: "xxx.xxxx.net"
2022/03/18 19:34:04 [error] 873#873: *367 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Uncaught OC\HintException: [0]: Memcache \OC\Memcache\APCu not available for local cache (Is the matching PHP module installed and enabled?)

Looks like missing some php modules.

This is the output of last command:

ii  libargon2-1:amd64                                           0~20190702-0.1+0~20190710.3+debian9~1.gbp2fb167           amd64        memory-hard hashing function - runtime library
ii  libenchant1c2a:amd64                                        1.6.1-1+0~20210212.1+debian9~1.gbp73138d                  amd64        Wrapper library for various spell checker engines (runtime libs)
ii  libgd3:amd64                                                2.2.5-5.2+0~20190808.4+debian9~1.gbp6d9343                amd64        GD Graphics Library
ii  libicu65:amd64                                              65.1-1+0~20200223.8+debian9~1.gbp519cf3                   amd64        International Components for Unicode
ii  libidn2-0:amd64                                             2.2.0-2+0~20200302.4+debian9~1.gbpf85c2e                  amd64        Internationalized domain names (IDNA2008/TR46) library
ii  libpcre2-8-0:amd64                                          10.39-2+0~20211122.14+debian9~1.gbp0d570b                 amd64        New Perl Compatible Regular Expression Library- 8 bit runtime files
ii  libpcre3:amd64                                              2:8.44-2+0~20210301.9+debian9~1.gbpa278ad                 amd64        Perl 5 Compatible Regular Expression Library - runtime files
ii  libxml2:amd64                                               2.9.12+dfsg-0+0~20210621.9+debian9~1.gbp43e861            amd64        GNOME XML library
ii  libzstd1:amd64                                              1.4.8+dfsg-2+0~20210302.3+debian9~1.gbp8effd2             amd64        fast lossless compression algorithm
ii  monodoc-browser                                             4.2-2xamarin12+debian9b1                                  all          MonoDoc GTK+ based viewer
rc  php-igbinary                                                3.2.6+2.0.8-6+0~20220117.32+debian9~1.gbp1d540e           amd64        igbinary PHP serializer
rc  php-redis                                                   5.3.5+4.3.0-5+0~20220117.40+debian9~1.gbpf8a972           amd64        PHP extension for interfacing with Redis
rc  php7.3-apcu                                                 5.1.21+4.0.11-7+0~20220117.31+debian9~1.gbp9a887a         amd64        APC User Cache for PHP
rc  php7.3-igbinary                                             3.2.6+2.0.8-6+0~20220117.32+debian9~1.gbp1d540e           amd64        igbinary PHP serializer
rc  php7.3-intl                                                 7.3.33-1+0~20211119.91+debian9~1.gbp618351                amd64        Internationalisation module for PHP
rc  php7.3-redis                                                5.3.5+4.3.0-5+0~20220117.40+debian9~1.gbpf8a972           amd64        PHP extension for interfacing with Redis
rc  php7.3-sqlite3                                              7.3.33-1+0~20211119.91+debian9~1.gbp618351                amd64        SQLite3 module for PHP

Joulinar · 18 March 2022 20:11

No need to try NextCloud until we fixed PHP. This is still not done fully. There are Stretch packages installed, still

Torvald · 18 March 2022 20:24

Yes, please do this steps for cleanup:

apt autopurge 'php7.2-*' 'php8.0-*' 'php8.1-*'
apt-mark auto libpcre3 libpcre3-dev libpcre32-3 libpcrecpp0v5 libsodium23 libxml2 libxml2-dev libzip4 php php-common php7.3-redis php7.3-sqlite3 php7.3-intl php7.3-igbinary php7.3-apcu php-igbinary php-redis
apt autopurge
dpkg -l | grep -E 'debian9|bpo9'

The last command is to get a list of still present Stretch packages.

I did this several times and the Nextcloud worked. However, I did not notice the line “ncc maintenance mode on/off” when I restart the services. This is fine?

I also encountered that Netdata gave an error “Access to file is not permitted: /usr/share/netdata/web//index.html”.
This was fixed by the command “chown netdata:netdata -R /usr/share/netdata”

The deluge does not work yet. Service fails to start with error “ImportError: cannot import name certificate_transparency”

Joulinar MichaIng Thanks for help!

Joulinar · 18 March 2022 20:35

we set the maintenance mode automatically if you stop/restart all services and we well remove the maintenance mode of you start all services again. It’s a feature of our script to ensure consistency on nextcloud.

Torvald · 18 March 2022 20:46

Yes, I understand that, that’s why I’m worried about the absence of this line.
Below is the console output after restarting the services

root@MyServer:~# dietpi-services restart

 DietPi-Services
─────────────────────────────────────────────────────
 Mode: restart

[  OK  ] DietPi-Services | restart : nmbd
[  OK  ] DietPi-Services | restart : smbd
[  OK  ] DietPi-Services | restart : redis-server
[  OK  ] DietPi-Services | restart : mariadb
[  OK  ] DietPi-Services | restart : php7.3-fpm
[  OK  ] DietPi-Services | restart : nginx
[  OK  ] DietPi-Services | restart : deluged
[  OK  ] DietPi-Services | restart : plexmediaserver
[  OK  ] DietPi-Services | restart : ubooquity
[  OK  ] DietPi-Services | restart : jackett
[  OK  ] DietPi-Services | restart : sonarr
[  OK  ] DietPi-Services | restart : radarr
[  OK  ] DietPi-Services | restart : lidarr
[  OK  ] DietPi-Services | restart : deluge-web
[  OK  ] DietPi-Services | restart : tor
[  OK  ] DietPi-Services | restart : netdata
[  OK  ] DietPi-Services | restart : cron

Joulinar · 18 March 2022 20:50

How does it looks like if you stop all services

dietpi-services stop

Torvald · 18 March 2022 20:52

root@MyServer:~# dietpi-services stop

 DietPi-Services
─────────────────────────────────────────────────────
 Mode: stop

[  OK  ] DietPi-Services | stop : cron
[  OK  ] DietPi-Services | stop : netdata
[  OK  ] DietPi-Services | stop : tor
[  OK  ] DietPi-Services | stop : deluge-web
[  OK  ] DietPi-Services | stop : lidarr
[  OK  ] DietPi-Services | stop : radarr
[  OK  ] DietPi-Services | stop : sonarr
[  OK  ] DietPi-Services | stop : jackett
[  OK  ] DietPi-Services | stop : ubooquity
[  OK  ] DietPi-Services | stop : plexmediaserver
[  OK  ] DietPi-Services | stop : deluged
[  OK  ] DietPi-Services | stop : nginx
[  OK  ] DietPi-Services | stop : php7.3-fpm
[  OK  ] DietPi-Services | stop : mariadb
[  OK  ] DietPi-Services | stop : redis-server
[  OK  ] DietPi-Services | stop : smbd
[  OK  ] DietPi-Services | stop : nmbd

Also fail2ban is not on this list. Although it is in the list of services.