[SOLVED] Support for NanoPC-T3 (very similar to NanoPi-M2)

[awl29]

v134 is released. Good to go for creating your image on master branch

Great, thank you!
Will probably try and get this done tomorrow, so stay tuned for an update here and in your Dropbox folder on Tuesday morning...

EDIT: I've made various changes to the system prep file since we started this, so if your using your own script, you may need to update it: https://github.com/Fourdee/DietPi/blob/ ... _DIETPI.sh

Will definitely double-check for any changes and update my specific script for NanoPi3.

One more time, many thanks for your great support in getting DietPi ported to my device!

Best,
awl

[awl29]

Will probably try and get this done tomorrow, so stay tuned for an update here and in your Dropbox folder on Tuesday morning...

Phew - v134 image for NanoPi-M3/NanoPC-T3 successfully created and uploaded to your Dropbox. Wifi and Bluetooth working fine.

But still two small issues:
a) new timing issue with dietpi-banner on tty1 being called too early after each boot, i.e. before the eth0 link becomes ready (IP address not yet available) - have not seen this with the kernel from the FriendlyARM June image
b) I probably need to update the initramfs to include fsck.ext4 and force a fsck after resize, as resize2fs consistently results in corrupt size counts after expanding the root filesystem, which will cause bad corruption on the long run...

So unfortunately this still is not the final image. Will try to resolve b) myself. But what shall we do with regards to a)?

Thanks & best regards,
awl

[Fourdee]

So unfortunately this still is not the final image. Will try to resolve b) myself. But what shall we do with regards to a)?

Thanks & best regards,
awl

Hi,

Not much we can do with A. Unless we delay the boot (which i'am not keen on) to wait for DHCP ack to finish, before we launch the banner.
No IP wont occur all the time in banner, it really depends on the users network and how quickly DHCP offers are acknowledged by their router.
Personally, if the user is watching tty1, its most likely they are setting up the device locally, rather than over SSH. In which case, not knowing the IP at that time shouldn't be an issue.

as resize2fs consistently results in corrupt size counts after expanding the root filesystem

Is this when DietPi does the resize on 1st run?

[awl29]

Not much we can do with A.

Fine. If you are ok with this, I clearly am.

as resize2fs consistently results in corrupt size counts after expanding the root filesystem

Is this when DietPi does the resize on 1st run?

Yes, exactly. I will try to create a script that does the job of patching /boot/root.img.gz (initramdisk - have already located where it executes resize2fs), such that you can also apply it to the NanoPi-M2/T2 image (for which I am almost 100% certain it will also be needed).

Will report back later tonight.

Best,
awl

[awl29]

Ouch,

Will report back later tonight.

this took much more time than anticipated...

Findings:

• Debian Jessie comes with e2fsprogs 1.42.12-2 (https://packages.debian.org/de/jessie/e2fsprogs)
• resize2fs in "offline" mode (i.e. doing the resize while the partition is not mounted) is clearly buggy in 1.42.12, but possibly even in the most recent 1.43.3 (http://e2fsprogs.sourceforge.net/)
• fsck.ext4 as of 1.42.12 does NOT even uncover the "size count" inconsistency directly after resize2fs, but only after some write access to the SD card and a subsequent reboot.
• I then tried to update the initramfs with most recent 1.43.3 e2fsprog binaries. Seemed to work fine, but the build/initramfs update was complex, and also I did not want to include more recent e2fsprogs binaries into the root filesystem image than the Debian release it builds on...
• Only then, I found out about the "live"/"online" mode of resize2fs which was said to be much simpler (and less error-prone) in terms of resize algorithm (see the quote by Theodore Ts'o below: http://www.spinics.net/lists/linux-ext4/msg48774.html). Completely sufficient for our scenario, and much easier!
• I have now patched the initramdisk file to use online resizing and additionally force a file system check afterwards, while staying with Debian release version 1.42.12-2
• From my testing, I could not reproduce any more "size count" corruptions with this setup (but I've only tested with two different SD cards (8GB and 64GB)

I suspect it was caused by a bug in resize2fs 1.42.10. The problem is that off-line resize2fs is much more powerful; it can handle moving file system metadata blocks around, so it can grow file systems in cases which aren't supported by online resize --- and it can shrink file systems when online resize doesn't support any kind of file system shrink. As such, the code is a lot more complicated, whereas the online resize code is much simpler, and ultimately, much more robust.

So I have now uploaded my final v134 build of DietPi (including the patched initramfs) to your Dropbox Shared Folder (md5sum included). Feel free to release it any time you like (maybe in beta at first until we have some reports from other users that it also works fine for them?)...

I have also uploaded the script that I used to patch the initramfs to the Dropbox, as I think you should also apply it to your NanoPi-M2/NanoPC-T2 image, which most probably uses the same initramdisk file (/boot/root.img.gz). Call the script as follows:

Code: Select all

nanopi_fix_initramfs.sh <path-to-root-img.gz>

providing the compressed ramdisk image as the single parameter. The script will backup the original file (to end with ".orig") and patch the initramdisk file in place in the same directory.

Finally, I have also uploaded a terminfo file that you may want to install using "tic" which resolves the issue of an invisible softcursor in any NanoPi/NanoPC framebuffer text console ("tput cnorm" to activate, can be nicely put into a script in /etc/rc.local.d).

Thanks again for your support in making DietPi run on my device!

Best regards
awl

[awl29]

Sorry - bad news:

Please be warned that the NanoPC-T3 build still cannot be considered final:

Trying to move my DietPi install from SD card to the built-in emmc, one more time, I ran into resize2fs issues, this time with online resizing:

Code: Select all

resize2fs 1.42.12 (29-Aug-2014)                                                 
Filesystem at /dev/mmcblk0p2 is mounted on /root; on-line resizing required     
old_desc_blocks = 1, new_desc_blocks = 1                                        
Performing an on-line resize of /dev/mmcblk0p2 to 1892096 (4k) blocks.          
resize2fs: Invalid argument While trying to add group #25      
resize2fs exited with status code 1

Code: Select all

[  163.892000] EXT4-fs warning (device mmcblk0p2): reserve_backup_gdb:882: reserved block 128 not at offset 127
[  164.216000] EXT4-fs warning (device mmcblk0p2): reserve_backup_gdb:882: reserved block 128 not at offset 127
[  197.584000] EXT4-fs warning (device mmcblk0p2): reserve_backup_gdb:882: reserved block 128 not at offset 127
[  197.712000] EXT4-fs warning (device mmcblk0p2): reserve_backup_gdb:882: reserved block 128 not at offset 127

I will now try to update the initramfs with the most recent e2fsprogs - hoping that a current 1.43.3 dated Sept, 2016 fixes the issue (1.42.12 is dated Aug, 2014)...

Will report back shortly about my progress...
awl

[Fourdee]

this took much more time than anticipated...

Yep, it happens lol
Looks like you got one of the worst to implement so far, Rhkean did our Pine A64 which was also a nightmare, lots of little issues kept cropping up.

Wish I could help (let me know how I can), but you seem to have it covered, hopefully not far till the end now

[awl29]

OK, so I finally think I know what is going on here, and the root cause is FriendlyARM's imaging process:

Have a look at this thread: https://lists.96boards.org/pipermail/de ... 00715.html

we have indeed noticed issues when doing resize2fs on the release images. we are using the Android make_ext4fs utility to create the ext4 images, even for Linux, and it seems to be causing issues when we have 'large' filesystem (e.g. several GBs).
(...)
However we have noticed some slight differences in the sparse images produced by the 2 processes: while make_ext4fs produces image with 'DONTCARE' blocks, img2simg uses FILL blocks (see sparse_format for details). And that change was breaking the tools we use to create the SD image that flashes images on the DragonBoard.. Well i understand this is not a good excuse to not pursue.. and I think we should get back to this topic, and use mkfs.ext4fs instead of make_ext4fs tool to create our Linux images.
(...)
to follow up on that.. after a bit of discussion (on irc) and testing, it looks like we should be able to replace make_ext4fs by mkfs.ext4 + ext2simg. And we would get images built properly: the filesystem would be created with linux standard tools, and the resulting sparse image has DONTCARE blocks which is better too.
I did a quick test on my dragonboard, and I could flash the resulting image fine, and resize-helper.sh could do its job properly too.

So the resize issue is caused by the fact that FriendlyARM, in the same way like people in the thread cited above, seem to use Android's make_ext4fs tool to create ext4 sparse images, when they should rather use mkfs.ext4 plus ext2simg...

My take on this so far is: Due to FriendlyARM having used the "wrong" (i.e., buggy) tool to create the root file system, the root file system of the FriendlyARM Debian image (which I have based my DietPi image on) was already non standard compliant (aka corrupt) from the very beginning, even though fsck.ext4 does not report any issues, but later attempts to resize2fs an image after lots of write accesses clearly uncovered them... Ouch!

I will now try to validate this theory (using fsarchiver to backup and restore FriendlyARM's rootfs.img content into a new partition created by the "real" mkfs.ext4), and then make a sparse image from this, and see whether the resulting ext4 image can be used by resize2fs in both online and offline mode without issues.

Unfortunately, that would mean that most probably, the NanoPi-M2/NanoPC-T2 image is affected by the exact same issue. Have you seen any issues with it during file system resize (resize2fs) before?

Stay tuned - I hope to have a truly valid image by the end of tomorrow...

Have a nice Sunday!
awl

[Fourdee]

Have you seen any issues with it during file system resize (resize2fs) before?

Not on the NanoPi M2, works every-time for me. For other boards, its only when PSU/SD card isn't up to the job.

Just wondering if would be a "better" route to build the image from scratch, eg: http://wiki.friendlyarm.com/wiki/index. ... n_OS_Image?

[awl29]

Have you seen any issues with it during file system resize (resize2fs) before?

Not on the NanoPi M2, works every-time for me. For other boards, its only when PSU/SD card isn't up to the job.

Hard to believe, as I strongly doubt that they use mkfs.ext4/ext2img to create the S5P4418 image, and the buggy make_ext4fs to create the S5P6818 image...

And while I agree that read/write errors or kernel hangs/oops point to faulty SDs, any of the two types of errors that I have seen clearly points to software issues:

Code: Select all

resize2fs: Invalid argument While trying to add group #25

Code: Select all

[  197.712000] EXT4-fs warning (device mmcblk0p2): reserve_backup_gdb:882: reserved block 128 not at offset 127

Have you tried different size SD cards? The issue I uncovered gets more likely (1) the larger the target partition (SD/EMMC) is and (2) the more write accesses you made between image install and resize.

Unfortunately, fsck.ext4 even in the most recent release, is unable to detect the "non-conformance" (aka corruption) of the resulting file system - you will only notice on the attempt to resize (resize2fs) - or possibly fatal corruption only after quite some time when you happened to fill the filesystem close to its capacity...

Just wondering if would be a "better" route to build the image from scratch, eg: http://wiki.friendlyarm.com/wiki/index. ... n_OS_Image?

The wiki headline is misleading: They talk about compiling u-boot and kernel on your own, but not the rootfs image (with device tree and Debian arm base image). In order to make this a full image, I would exactly have to copy an existing Debian root file system infrastructure from "somewhere", so IMHO fsarchiver clearly is the way to go...

Best,
awl