First things first. In order to use any of the hard disks I have built into USB cases, I need to set up a Single Board Computer. Which SBC will I use? And what OS will be running. I will be running some Linux distro. Until now I have been running ArchLinuxARM on my ARM based SBCs. However, for this project I am considering Debian instead.

When creating an SD card to boot ArchLinuxARM on an SCB of my choice, the procedure would be to compile U-Boot for that SCB, partition an SD card into a boot and root partition, install the U-Boot to the appropriate location on the SD card (the boot partition or a specific sector), extract the rootfs tarball to the root partition, generate an appropriate U-boot config, and we are ready to boot.

Debian does not seem to provide rootfs tarballs. I might be able to generate a rootfs using debootstrap. However, Debian provides images for the Raspberry Pi at https://raspi.debian.net/. These images are generated by the Debian Cloud Images project. For another SBC then the Raspberry Pi, I suppose I would need the “nocloud-arm64” variant. As there are raw disk images, I will have to look into integrating a board specific bootloader into those images. That is something that I will investigate at a later point in time.

After writing the image to an SD card, I have attached a HDMI monitor and an USB keyboard to perform the initial configuration. When the image boots up, it prompts me for the time zone and root password. After that I can log in to the system. The image comes without an SSH server installed, so that is the first step. After creating my user account and setting up the hostname, the system is ready for headless operation.

Checking my disks

Disk /dev/sda: 232.89 GiB, 250059350016 bytes, 488397168 sectors
Disk /dev/sdb: 232.89 GiB, 250059350016 bytes, 488397168 sectors
Disk /dev/sdc: 232.89 GiB, 250059350016 bytes, 488397168 sectors
Disk /dev/sdd: 232.89 GiB, 250059350016 bytes, 488397168 sectors
Disk /dev/sde: 232.89 GiB, 250059350016 bytes, 488397168 sectors

Everything seems to be in order, they all have exactly the same size. For each disks, I will create RAID partitions

Welcome to fdisk (util-linux 2.41).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

And then we’re ready to go:

root@rpi-nas:/home/andre# mdadm --create /dev/md0 --level=5 --raid-devices=5 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
To optimalize recovery speed, it is recommended to enable write-indent bitmap, do you want to enable it now? [y/N]? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.

Checking the progress:

root@rpi-nas:/home/andre# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sde1[5] sdd1[3] sdc1[2] sdb1[1] sda1[0]
976261120 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/4] [UUUU_]
[>....................] recovery = 1.1% (2829212/244065280) finish=776.4min speed=5177K/sec
bitmap: 0/2 pages [0KB], 65536KB chunk

unused devices:

And a bit later

Personalities : [raid6] [raid5] [raid4]
md0 : broken (auto-read-only) raid5 sde1[5](S) sdd1[3] sdb1[1] sda1[0]
976261120 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/3] [UU_U_]
bitmap: 0/2 pages [0KB], 65536KB chunk

unused devices:

At this point, I assumed the fluctuations in speed are because of a mix of USB2 and USB3 disks.
But when I looked a little later:

root@rpi-nas:/home/andre# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : broken (auto-read-only) raid5 sde1[5](S) sdd1[3] sdb1[1] sda1[0]
976261120 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/3] [UU_U_]
bitmap: 0/2 pages [0KB], 65536KB chunk

unused devices:

One of the disks has disappeared!

In the kernel logs, I saw:

[ 7970.923738] I/O error, dev sdc, sector 97947496 op 0x0:(READ) flags 0x4000 phys_seg 1 prio class 0
[ 7970.923770] md/raid:md0: read error not correctable (sector 97945448 on sdc1).
[ 7976.005217] usb 2-2.1.4: reset SuperSpeed USB device number 9 using xhci_hcd
[ 7980.785190] usb 2-2.1.4: reset SuperSpeed USB device number 9 using xhci_hcd
[ 7985.281380] usb 2-2.1.4: reset SuperSpeed USB device number 9 using xhci_hcd

USB issues?! What might be the cause of this?

I have attempted to use a Raspberry Pi 3 in stead of a 4, but also here

[ 1848.496610] sd 4:0:0:0: [sde] tag#0 timing out command, waited 180s
[ 1848.504545] sd 4:0:0:0: [sde] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=180s
[ 1848.516929] sd 4:0:0:0: [sde] tag#0 Sense Key : Not Ready [current]
[ 1848.525071] sd 4:0:0:0: [sde] tag#0 Add. Sense: Logical unit is in process of becoming ready
[ 1848.536962] sd 4:0:0:0: [sde] tag#0 CDB: Write(10) 2a 00 00 05 37 18 00 00 f0 00
[ 1848.546309] I/O error, dev sde, sector 341784 op 0x1:(WRITE) flags 0x4000 phys_seg 30 prio class 0
[ 1934.311183] INFO: task md0_resync:798 blocked for more than 724 seconds.
[ 1934.319004] Tainted: G C 6.12.95+deb13-arm64 #1 Debian 6.12.95-1
[ 1934.328019] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1934.336974] task:md0_resync state:D stack:0 pid:798 tgid:798 ppid:2 flags:0x00000008
[ 1934.348429] Call trace:
[ 1934.351897] __switch_to+0xf4/0x168
[ 1934.356414] __schedule+0x398/0xf18
[ 1934.360888] schedule+0x30/0xf0
[ 1934.364984] raid5_get_active_stripe+0x1fc/0x650 [raid456]
[ 1934.371510] raid5_sync_request+0x3d0/0x418 [raid456]
[ 1934.377591] md_do_sync+0xa60/0xf08 [md_mod]
[ 1934.382902] md_thread+0xb4/0x1b8 [md_mod]
[ 1934.388048] kthread+0xe8/0xf8
[ 1934.392071] ret_from_fork+0x10/0x20
[ 5694.706515] usb 1-1.3.1.4: reset high-speed USB device number 10 using dwc2
[ 5725.282680] usb 1-1.3.1.4: reset high-speed USB device number 10 using dwc2
[ 7255.030746] usb 1-1.3.1.1: reset high-speed USB device number 7 using dwc2
[ 7255.214756] usb 1-1.3.1.2: reset high-speed USB device number 8 using dwc2
[ 7255.398762] usb 1-1.3.1.1: reset high-speed USB device number 7 using dwc2
[ 7257.490767] usb 1-1.3.1.2: reset high-speed USB device number 8 using dwc2
[ 7272.430858] usb 1-1.3.1.4: reset high-speed USB device number 10 using dwc2

I decided to switch to the Le Potato running ArchLinuxARM, but also here, USB issues.

In the end, I have traced the problem: power issues for the 2.5″ disks.
I have been mentioning the powered USB hubs having underpowered power supplies.
The supplies would not be able to handle full load on all ports, but these two
hard disks should be within the capabilities of the supply.

I have been trying different hub configurations, but in all previous attempts,
I did connect both disks to the same powered hub.

Note the SD disk is rated at 550 mA, and the Fujitsu disk at 600 mA.
These power draws are well within the 900 mA a USB3 port should be
able to deliver. With the 4 port USB hub having a 2.4 Amp supply,
this should not be a problem. The 7 port hub having a 12 Volt
supply, and thus an internal regulator, one might wonder, but
12 Volt at 2 Amp, thus 24 Watt, should not have an issue with that
kind of load.

Nevertheless… on the Le Potato, it seemed only one disk was dropping
out, and looking closely at the logs, it was one of the 2.5″ disks.

Now, I have connected one of the 2.5″ disks to the 7 port hub, and
the other to the 4 port hub, USB operation seems stable. It will still
take a while for the RAID array to be ready, so there is still time
for things to go wrong, but the results are promising.

Also… there are some more notes to make. Since I am creating the
RAID array on ArchLinuxARM now in stead of Debian, I noticed one
message in the kernel logs:

[ 361.833643] md0: array will not be assembled in old kernels that lack configurable LBS support (<= 6.18)

To use this RAID array, I will need at least Kernel version 6.19. Since Debian Trixie is at 6.12,
I will not be able to use this array in Debian. Not that it matters that much, it is just a
thing to keep in mind.

Looking at what this LBS support refers to, it is Logical Block Size. Now this Logical Block
Size support is in, I might as well configure it. Since modern disks use a Physical Block Size
of 4096, I might as well use this as the Logical Block Size.

When Looking at the mdadm options, I also noticed an option to fill the disks with zeroes,
allowing the skipping of the initial sync. I mean, there is nothing to sync initially, right?
So I will also pass this option. So, now, I will be creating the array with:

mdadm --create /dev/md0 --level=5 --raid-devices=5 --logical-block-size=4096 --write-zeroes /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1

Since mdadm will be writing these zeroes, this will not be a background process like the initial
sync, thus I am running this inside a screen.

[root@potato andre]# mdadm --create /dev/md0 --level=5 --raid-devices=5 --logical-block-size=4096 --write-zeroes /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
mdadm: Disk zeroing requested, setting --assume-clean to skip resync
To optimize recovery speed, it is recommended to enable write-intent bitmap, do you want to enable it now? [y/N]? y
mdadm: /dev/sda1 appears to be part of a raid array:
level=raid5 devices=5 ctime=Sat Jul 18 11:59:11 2026
mdadm: /dev/sdb1 appears to be part of a raid array:
level=raid5 devices=5 ctime=Sat Jul 18 11:59:11 2026
mdadm: /dev/sdc1 appears to be part of a raid array:
level=raid5 devices=5 ctime=Sat Jul 18 11:59:11 2026
mdadm: /dev/sdd1 appears to be part of a raid array:
level=raid5 devices=5 ctime=Sat Jul 18 11:59:11 2026
mdadm: /dev/sde1 appears to be part of a raid array:
level=raid5 devices=5 ctime=Sat Jul 18 11:59:11 2026
Continue creating array [y/N]? y
mdadm: Fail to create md0 when using /sys/module/md_mod/parameters/new_array, fallback to creation via node
mdadm: Fail to create md0 when using /sys/module/md_mod/parameters/new_array, fallback to creation via node
mdadm: Defaulting to version 1.2 metadata
mdadm: zeroing data from 135266304 to 249922846720 on: /dev/sda1
mdadm: zeroing data from 135266304 to 249922846720 on: /dev/sdb1
mdadm: zeroing data from 135266304 to 249922846720 on: /dev/sdc1
mdadm: zeroing data from 135266304 to 249922846720 on: /dev/sdd1
mdadm: zeroing data from 135266304 to 249922846720 on: /dev/sde1