apt and muon are slow

Have some feedback, questions, suggestions, or just fancy a chat? Pop it in here.
Post Reply
srs4511351
Posts: 3
Joined: Wed Aug 18, 2021 2:44 am

apt and muon are slow

Post by srs4511351 »

I installed DietPi v7.5 64 bit bullseye.
It works better than any other 64 bit distribution that I tried. The Debian Raspberry Pi release has audio and video problems.
DietPi works well, loving it.
When installing my usual software, I installed muon which rebuilt the Xapian index. It said "Rebuilding Xapian index... 1%" and incremented to 100%, but it took over 12 hours!
apt list takes about a minute to list a package. muon takes a minute or two to open.
I don't know if apt was slow before the muon install.

I installed muon on the Debian Raspberry Pi release. The Xapian index rebuilt, but it only took about a minute.
The same apt list takes a few seconds and muon loads quickly.
This also happened on a DietPi 7.4.2 installation.

----Steve
User avatar
Joulinar
Posts: 5619
Joined: Sat Nov 16, 2019 12:49 am

Re: apt and muon are slow

Post by Joulinar »

hmm the only thing that might be slow on apt is to run apt update because DietPi is having the index files within a compressed file format. And running apt update takes time to un-compress them. Reason for this is to reduce r/w operation on SD cards to increase life time.

But I need to say I have no idea how these muon/ Xapian is working and if there is a relation or not.

Anyway, if needed you could switch to uncompressed index files as follow

Code: Select all

echo 'Acquire::GzipIndexes "false";' > /etc/apt/apt.conf.d/98dietpi-uncompressed
/boot/dietpi/func/dietpi-set_software apt-cache clean
apt update
apt update should be much faster now.
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
vimmala
Posts: 1
Joined: Wed Mar 18, 2020 12:15 pm

Re: apt and muon are slow

Post by vimmala »

I upgraded yesterday to bullseye and my aptitude really has slowed down a lot. Might be a related issue.
For example, just moving the cursor on different packages and displaying that package description takes a few seconds.

I noticed some high use of "apt-get -s -o" which simply gets called too often. Try to open htop and see if you get any more clues.
srs4511351
Posts: 3
Joined: Wed Aug 18, 2021 2:44 am

Re: apt and muon are slow

Post by srs4511351 »

I ran some tests before and after executing the commands that you gave. It appears that compression was the cause of the slowness.
Thanks!

The second apt-get was slower because it updated more lists than it did with the compressed index files.
However listing packages and rebuilding the Xapian index was dramatically faster.
I ran this command: sudo /usr/sbin/update-apt-xapian-index -f
It completed in less that 2 minutes, where before it was 12 hours.

These are the results of my tests:

Code: Select all

dietpi@DietPi:~$ time apt list -a libusb*
Listing... Done
libusb-0.1-4/stable 2:0.1.12-32 arm64

...blah blah blah...

libusbredirparser1/stable 0.8.0-1+b1 armhf

real	0m28.043s
user	0m27.658s
sys	0m0.348s
---------------------------------

Code: Select all

dietpi@DietPi:~$ time sudo apt update
Hit:1 https://deb.debian.org/debian bullseye InRelease
Hit:2 https://deb.debian.org/debian bullseye-updates InRelease
Hit:3 https://deb.debian.org/debian-security bullseye-security InRelease
Hit:4 https://archive.raspberrypi.org/debian bullseye InRelease
Hit:5 https://deb.debian.org/debian bullseye-backports InRelease
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.

real	0m16.661s
user	0m15.410s
sys	0m1.051s
---------------------------------
# uncompress the indexes

Code: Select all

echo 'Acquire::GzipIndexes "false";' > /etc/apt/apt.conf.d/98dietpi-uncompressed
/boot/dietpi/func/dietpi-set_software apt-cache clean
---------------------------------

Code: Select all

dietpi@DietPi:~$ time sudo apt update
Get:1 https://deb.debian.org/debian bullseye InRelease [113 kB]
Get:2 https://deb.debian.org/debian bullseye-updates InRelease [36.8 kB]       
Get:3 https://deb.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Get:4 https://deb.debian.org/debian bullseye-backports InRelease [39.3 kB]
Get:5 https://archive.raspberrypi.org/debian bullseye InRelease [23.5 kB]
Get:6 https://deb.debian.org/debian bullseye/main arm64 Packages [8,066 kB]
Get:7 https://archive.raspberrypi.org/debian bullseye/main armhf Packages [115 kB]
Get:8 https://archive.raspberrypi.org/debian bullseye/main arm64 Packages [109 kB]
Get:9 https://deb.debian.org/debian bullseye/non-free armhf Packages [56.3 kB] 
Get:10 https://deb.debian.org/debian bullseye/non-free arm64 Packages [69.6 kB]
Get:11 https://deb.debian.org/debian bullseye/main armhf Packages [7,941 kB]   
Get:12 https://deb.debian.org/debian bullseye/contrib arm64 Packages [40.8 kB] 
Get:13 https://deb.debian.org/debian bullseye/contrib armhf Packages [40.2 kB] 
Get:14 https://deb.debian.org/debian-security bullseye-security/main armhf Packages [27.1 kB]
Get:15 https://deb.debian.org/debian-security bullseye-security/main arm64 Packages [27.5 kB]
Get:16 https://deb.debian.org/debian bullseye-backports/main armhf Packages [21.7 kB]
Get:17 https://deb.debian.org/debian bullseye-backports/main arm64 Packages [21.7 kB]
Fetched 16.8 MB in 20s (826 kB/s)                                              
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.

real	0m23.578s
user	0m10.287s
sys	0m2.466s
---------------------------------

Code: Select all

dietpi@DietPi:~$ time apt list -a libusb*
Listing... Done
libusb-0.1-4/stable 2:0.1.12-32 arm64

...blah blah blah...

libusbredirparser1/stable 0.8.0-1+b1 armhf

real	0m3.215s
user	0m2.972s
sys	0m0.256s
User avatar
Joulinar
Posts: 5619
Joined: Sat Nov 16, 2019 12:49 am

Re: apt and muon are slow

Post by Joulinar »

yes the 2nd run needs to download all index files again as we cleared the cache after setting the uncompressed value. That's as expected. running apt update should be faster now as no index files needs to be download.
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
srs4511351
Posts: 3
Joined: Wed Aug 18, 2021 2:44 am

Re: apt and muon are slow

Post by srs4511351 »

It must be that the other distributions are not using compression for the apt lists.

I think that your solution is working for me.

Thanks, Steve
User avatar
Joulinar
Posts: 5619
Joined: Sat Nov 16, 2019 12:49 am

Re: apt and muon are slow

Post by Joulinar »

yes compression is something DietPi specific as we aim to reduce r/w operation on SD cards as much as possible trying to enlarge lifetime. But it might not fit everybody's need like in your case. On a midterm we think on implementing an option inside our config menu to be able to switch it on/off.
Pls let us know if a solution is working. This could help others if they hit by similar situation. Your DietPi Team
Post Reply