4MB sectors - aka "Optimizing Linux with cheap flash drives"

Guides and tutorials for various stuff. Posted by DietPi users.
Post Reply
User avatar
WarHawk
Posts: 520
Joined: Thu Jul 20, 2017 7:55 am

4MB sectors - aka "Optimizing Linux with cheap flash drives"

Post by WarHawk » Mon Jul 09, 2018 7:10 am

Just came across this
Might be useful for developers
Improved Optimization/speed and wear leveling of software on SD Cards

https://lwn.net/Articles/428584/

It's an older report...but it might help since devices are getting bigger and the need to keep the cards alive longer is getting more and more important

My take on the document is that 4MB blocks or when writing the image
Write to SD card using Linux:
- Find the device (eg: /dev/sdb1) you need to write to by using blkid or mount.
- Double check that you have the correct dev path for your SD card (eg: /dev/sdb1).
- Unmount the SD card and all its partitions by using umount /dev/sdb?.
- Write the image using dd if=/path/to/DietPi_vXX.img of=/dev/sdb.

Code: Select all

sudo dd if=/path/to/DietPi_vXX.img of=/dev/sdb bs=4096
or

Code: Select all

sudo dd if=/path/to/DietPi_vXX.img of=/dev/sdb bs=4M

wdt
Posts: 17
Joined: Wed Aug 22, 2018 7:20 am

Re: 4MB sectors - aka "Optimizing Linux with cheap flash drives"

Post by wdt » Thu Oct 04, 2018 10:31 pm

Did you notice the date on that article? 7 years is a long time,
but yes, most flash is crap. You can find out much more, and more relevant at Armbian
And now there is class a1 and soon a2. It's hard to beat samsung evo or evo+ .
An iozone line from my history, if you want true and accurate numbers
iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
--edit--
the speed (for SD) is pretty irrelevant if the socket is running at 50 mHz,, not switched to 1v8
(as 95% are)

User avatar
WarHawk
Posts: 520
Joined: Thu Jul 20, 2017 7:55 am

Re: 4MB sectors - aka "Optimizing Linux with cheap flash drives"

Post by WarHawk » Fri Oct 05, 2018 7:11 am

Yeah...an old report...thought maybe it could help...but newer OS's and kernel builds more than likely make it a moot issue

Thanks!

User avatar
MichaIng
Legend
Posts: 1620
Joined: Sat Nov 18, 2017 5:21 pm

Re: 4MB sectors - aka "Optimizing Linux with cheap flash drives"

Post by MichaIng » Fri Oct 05, 2018 1:54 pm

wdt wrote:the speed (for SD) is pretty irrelevant if the socket is running at 50 mHz
Do you mean the default SD frequency on RPi, perhaps other SBCs as well?

I changed it to 100: dtparam=sd_overclock=100 => /DietPi/config.txt
Made my Pi quite a bid more responsive, I was impressed. However every source I found claims that this is dangerous for older SDcards, so I was afraid to add this to e.g. dietpi-config, if users might destroy their card and loose their data.

Would be nice if there was a reliable method to verify the card supports it :?.

Btw 4K block size seems to be default on Rufus and Etcher, Debian installer and such, or for ext4. At least all my systems are at 4096 / 4K.
@WarHawk
I just found above you say bs=4096 or bs=4M.
Note that bs takes the value as bytes. So bs=4096 means 4096 bytes = 4K, while bs=4M means 4*1024*1024 bytes, so factor 1024 between both results ;).

The 4M in the article is about the "erase sector" size, the size of a group of blocks, that can be removed at once. The alignment of the partitions (so start and end) should be according to those sector sizes, which are given, AFAIK, by the controller, so cant be changed (?).

A "block" size of 4M is most properly not what you want. Even if allocation of data should be faster, this means that every single file (even 0 size ones) take effective 4M on the disk. So a such installed system, with many many small files, will take much more space on the disk.

However I am no specialist, so if there is indeed something we can improve, e.g. optionally in case of weak SD, would be great to know. block size at least should be up to the flash program, we just could give advices there, but I guess they already do good, know better than us. If something depends on actual image creation, that affects alignment of final written image, then this would be important to know, but I am not aware of this.

User avatar
WarHawk
Posts: 520
Joined: Thu Jul 20, 2017 7:55 am

Re: 4MB sectors - aka "Optimizing Linux with cheap flash drives"

Post by WarHawk » Fri Oct 05, 2018 11:54 pm

MichaIng wrote:
Fri Oct 05, 2018 1:54 pm
wdt wrote:the speed (for SD) is pretty irrelevant if the socket is running at 50 mHz
Do you mean the default SD frequency on RPi, perhaps other SBCs as well?

I changed it to 100: dtparam=sd_overclock=100 => /DietPi/config.txt
Made my Pi quite a bid more responsive, I was impressed. However every source I found claims that this is dangerous for older SDcards, so I was afraid to add this to e.g. dietpi-config, if users might destroy their card and loose their data.
is there a SD card read/write test and variable overclock setting?

Found a benchmarking test/tool on RPi that might help
https://www.jeffgeerling.com/blog/2016/ ... berry-pi-3
Here is a script from that above site that might help, if you can pick thru it
http://www.nmacleod.com/public/sdbench.sh

User avatar
MichaIng
Legend
Posts: 1620
Joined: Sat Nov 18, 2017 5:21 pm

Re: 4MB sectors - aka "Optimizing Linux with cheap flash drives"

Post by MichaIng » Sat Oct 06, 2018 7:25 pm

dietpi-config > Tools > Benchmark :D!

With v6.17 those results can be uploaded too, to have a average/min/max comparison :): https://dietpi.com/survey/#benchmark

As mentioned, we did not yet implement the SDcard overclock setting, as we need to be very sure, that the end user SDcard supports it. If the SDcard does not work with 100MHz, then you will at first not see any errors or issues. The data will corrupt silently and when you recognize, it's too late. So we need to be very sure, when we want to implement this!

wdt
Posts: 17
Joined: Wed Aug 22, 2018 7:20 am

Re: 4MB sectors - aka "Optimizing Linux with cheap flash drives"

Post by wdt » Mon Oct 08, 2018 6:48 am

>Would be nice if there was a reliable method to verify the card supports it
Not just the card, but if the circuit board has good design,,
low capacitance in the D[0-3] lines, and the max mA drive for them
That is why 1v8 runs faster, less capacitance at lower voltage
Anyway, the iozone test will tell you how crappy the card is reading/writing 4K & 16K blocks
It's not a camera with single threaded large files

Post Reply