Opening large files on raspi 3

Troubleshooting
1

First up a massive thanks for all of you, and I imagine especially Fourdee, for giving us dietpi.

I am a fairly novice linux /software user. I have been playing with raspberry pis (3 and zero W) for a couple of months now.
A pi 3 and and a zero W are running OSMC to provide TV replay. I have a pi 3 and a zero W to listen to music (mostly internet
radio) running dietpi with mpd/mopidy. All easy to set-up and keep running. I still have yet to get one pi to read an audio CD from
a cdrom drive, but that is off topic for this post.

My problem is with my fifth pi (3) that I use to synthesize piano sounds. I have a 25 yr old Roland piano, with a good feel
(proper weighted keys) and midi interface, but whose sound dates from the mid eighties. I have successfully installed dietpi,
added a midi-to-usb interface, a hifiberry amp hat, and some good speakers, and I run fluidsynth with various soundfonts.
I am hoping to get a true concert grand piano sound and so far its not too bad.

For the most part this works well. I have low latency (384khz kernel, I load fluidsynth on the four cores), and the sound
quality (dynamic range, distorsion, noise) is great. But I have yet to find the ideal soundfont, and I have tried a bunch.
The best ones so far are between 250 and 500MB. These load and play fine, I just have to add a wait to allow them to load
into memory. Running htop or using raspicheck.apk on my phone allows me to see that cpu usage is actually quite low, and
the memory used is essentially that required by dietpi plus the soundfont. I run headless so the default gpu memory (from
dietpi-config) is smallest (64M ?). I have disabled the swapfile.

But I cannot load a soundfont that is 600MB in size. I have checked that the sf2 file is good running a midi file in windows.
If i start fluidsynth with the soundfont, and immediately afterwards htop, with screen attached, I see memory usage going up as
the soundfont loads, fairly low cpu usage, and then it seems to hit a wall at around 70% memory and 99.9% cpu usage, and then
freezes. I have to pull the plug to regain control.

So first question, which could be linux related, should I do something particular to allow loading files bigger than half the total
system ram ? When serching through google I came across the question of disk caching. Can I tell the system that this font is
readonly when fluidsynth intially loads to avoid caching ? Should I create a temporary ram disk, load the file there, and then
run fluidsynth ? Are there other system tweaks needed in dietpi ? Is this a bug ? I have not tried any other distribution since
I have been totally won over by dietpi for lightweight ease-of-use and no sdcard corruption.

Thanks for any help and pointers to resolve this.

2

Having reread my post I tried a small change. I created a 200MB swapfile, so as to fake having 1.2GB of memory, and
then my 600MB soundfont will load. Nevertheless I am curious to understand why this bevaviour. Is this normal (dietpi
or just linux) ? Thanks for any comments.

I write as I debug. With 200MB swap or 300MB swap the soundfont file loads, and the pi no longer freezes. But total
memory usage is 934 out ot 970MB, and I cant get any sounds to play. So clearly memory usage is much greater than
the size of the soundfont. And I cannot play any music.

What about if I edit /etc/fstab and restrict the size of /tmp to say only 100MB (presently no limit), would that help /fix this ?
Tried this without success.

Another possibility I have just found would be to add the command “nocache” ahead of loading the soundfont. So first I
need to apt-get install nocache from the debian repository. Comments anyone ?

The nocache allows me to load the soundfont, without explosing memory, but I then hit a wall at 100% cpu load (shown by
htop) and then the pi freezes.

3

Hi,

You beat me to it, was going to suggest that reading your 1st post :slight_smile:

Its possible soundfonts use more memory than their actual filesize. Maybe due to compression/decompression on the soundfont file? Or the way it allocates memory for each sound, maybe it allocates more than it actually needs?

What about if I edit /etc/fstab and restrict the size of /tmp to say only 100MB (presently no limit), would that help /fix this ?

It appears the physical RAM size is the limitation here. Increasing the swapfile will resolve this but may cause delay, due to IO access needed if swap data is triggered.
DietPi is configured to only use swap when its “truley” needed (swappiness=1). So increasing the swapfile to 1GB, wont effect performance and only whats needed from the system will be used.

Another option (upgrade), is the Odroid C2, which features 2GB of RAM, 64bit and faster CPU/GPU. They do offer hi-end DAC’s that can compete with most RPi DACs:
http://www.hardkernel.com/main/products/prdt_info.php
http://www.hardkernel.com/main/products/prdt_info.php?g_code=G147589529288

I have one of the above running as my Kodi media/movie player. Highly rate it, although, 192KHz max.

Another possibility I have just found would be to add the command “nocache” ahead of loading the soundfont. So first I
need to apt-get install nocache from the debian repository. Comments anyone ?

I have no experience with nocache unfortunately. Unable to offer any assistance.

4

Thanks Fourdee for your remarks.

I spent some time yesterday with another SD card trying with raspbian lite.
It appears to be more receptive to loading large files.
A quick summary :
Memory size file load nocache raspbian-lite

htop initial 33/970 38
Salamander 238MB 300 301 306
Steinways 377MB 437 436 439
IowaPiano 506MB 522* 522* 525
SalaLarge 592MB 623 freeze 933 (56 swap/100)
City Piano 713MB 737*

  • = fluidsynth stops
    (despite my attempts with tabs and spaces I could not get the columns to line up nicely)

The initial memory usage reported by htop is respectively 33 and 38 out of 970 for dietpi and lite.
In the first column is the size of the file. The second and third columns are the total memory used (as shown by htop)
once loaded with fluidsynth, normally or with “nocache” - the latter seems to have no effect. The fourth column shows
fluidsynth loading the file in raspbian-lite. Beyond 500MB (IowaPiano soundfont) dietpi loads into memory but fluidsynth
stops, or with SalaLarge freezes (100% cpu, then need to pull the plug). Raspbian-lite allows to load both IowaPiano
and SalaLarge, and allows generating sounds. There is clearly something about the SalaLarge soundfont that
is different, since its memory needs are much greater than the initial file size, whereas all the other soundfonts have
an expected behaviour. So I think dietpi may have an issue loading large files as compared with raspbian-lite ? By default
the latter has 100MB swap. I tried dietpi with 100MB swap but this made no difference on my result.

One possibility is that I have always disabled swap during initial install; only later with dietpi-config added this back.
Could this have an effect during the initial installation in terms of setup that affects the swap behaviour if added later ?

5

Yes, thats it !

I have managed to reproduce the same behaviour as raspbian-lite with the SalaLarge font file if during initial dietpi setup
I accept default creation of a 100MB swap file. Then I can open this large soundfont file and play music. So conclusions :

  • an issue with this particular soundfont file that requires more memory than the other files
  • fluidsynth in dietpi allows opening files with size >500MB on pi 3 only if the swap is initially generated during install and
    not after, or fluidsynth has to be installed after swap is setup ?

Anyway I hope this experience helps anyone else coming across a similar issue.