The SD card is out — a day spent at the wrong end

The SD card in my Raspberry Pi in Merano is out. Home Assistant runs on an NVMe SSD, 234 GB, 210 of them free.

Getting there took: a day of work at the wrong end, two bugs in other people’s software, and two theories of my own that turned out wrong. Both belong in this text, because they cost the time.

Why at all

My long-term statistics were wiped twice. Both times every series was missing from the same moment onwards — total power, outside temperature, every valve sensor — and both times at exactly the same hour: two in the morning UTC, when the recorder’s nightly cleanup runs.

The second time it demonstrably was not an update. The version was unchanged. So it was something recurring, roughly monthly, and it held only about eight days of statistics.

On top of that, four corrupt database files totalling 4 GB. Suspicion fell on a tired SD card.

The lesson came before the hardware

Eight days of retention means: backing up weekly is mandatory, not optional.

My backup into the repository worked — it had saved everything up to its last run. It had simply last run sixteen days earlier. The 180 hours in between are gone for good; no second place holds them.

Among them the only data with a guest who ventilates differently from us. Exactly the kind of series you cannot repeat.

First: the ribbon cable was in backwards

After fitting the disk the kernel saw nothing at all. No nvme, no sda, just the SD card.

Here I was too quick. Two devices were listed in the PCI directory and I took one of them for the disk. It was the Pi 5’s own I/O chip. What was missing was the second PCIe port — the external one did not appear in the log at all.

The fix was mechanical: turn the ribbon cable around. After that:

nvme nvme0: pci function 0001:01:00.0
nvme0n1    250059096 blocks  (256.06 GB)

The configuration hint I had pre-emptively taken from a wiki was not needed. Just as well the cable came first.

Second: a negative disk size

The disk was running. The migration was not:

Cannot use ... as data disk as it is smaller than
the current one (new: -1065472, current: 125064691200)

A negative size. The kernel measures correctly, the system lists the disk properly — only the management layer reports size zero and then calculates with a zero.

And here is the observation that decides everything: this is not a problem with this disk. Three days earlier the same error had been reported with exactly the same number — different hardware, different disk size, the same -1065472.

A constant that does not change with the disk size does not come from the disk.

That is a test I have kept. If an error message quotes a number and the same number turns up on someone else’s hardware, the value is not measured but calculated — from something that is missing.

My second theory, and why it was wrong

Comparing the device properties revealed exactly one difference: the SD card had a partition table, the factory-new disk did not.

The assumption was obvious: the management layer derives the size from the partition table, and on a factory-new disk there is nothing to derive it from.

That was wrong. After writing a GPT the table was there — and the size stayed zero. After a reload too. And after a full restart of the host, where the responsible services start fresh and see the disk with a table from the outset.

The missing partition table was a symptom of the factory-new disk, not the cause.

Four dead ends just to reach the disk

Each one saves time next time:

1. The official terminal add-on cannot do it. Write access reports “Operation not permitted”. The reason: it does not request any special rights at all. That is why there is no protection-mode switch there either — there is nothing to unlock.

2. The advanced SSH add-on cannot do it either. It declares the necessary capabilities, but its device list does not contain the disk. A capability is useless against a list the device is missing from. With protection mode off: the same message.

3. The door is access to the host’s container service. A short-lived privileged container with the device directory passed through sees the disk in full.

4. Three hurdles on the way there:

• Host key changed. Two add-ons are two SSH services with their own keys. The warning was genuine and explicable; the old entry had to go.
• “Corrupted MAC on input”. The log named what was on offer: ETM variants only. The form without -etm did not exist there.
• Over SSH you are not root, while in the web terminal of the same add-on you are. Hence “Permission denied” on both the disk and the container service.

The thinking error that overturned the whole plan

My plan was to wait for a fix to the size bug. That was the wrong plan, for a reason I only saw when recalculating:

The built-in migration function moves only the data. The system would still have booted from the SD card.

So even a working migration would not have made the card redundant — and that was exactly the question: what do you have to do to remove the SD card entirely?

Waiting months for a fix that would not have solved the problem: that is the most expensive mistake of the whole day, and it was pure reasoning, not technology.

The route that worked

The Pi 5 can present itself as a card reader. A small boot utility loads a lean kernel into the Pi, which hands the attached drives — SD, eMMC, NVMe, USB — to the connected computer as an ordinary USB mass storage device.

The NVMe appears on the Windows machine as a perfectly normal drive. From there:

1. write Home Assistant OS straight onto it
2. boot the Pi normally
3. restore the backup
4. take the SD card out

Four steps. After half a day had gone into trying to do it from the inside.

Two stumbling blocks

The Windows driver. The device reported a failed driver installation and needed a manual assignment with elevated rights — so it was done by hand, not from the script.

A PowerShell gotcha. A variable named $gadget collided with the -Gadget switch: PowerShell does not distinguish case in variable names. The failure read “Cannot convert … to type SwitchParameter” and says precisely nothing about the cause. The variable has had a different name since.

What came out of it in passing

The operating system update that had failed over the air for weeks was simply included in the freshly written image.

The cause is curious: the service the update is fetched from has for some weeks been sending a security policy of over 9 KB in its headers. The network library in use accepts at most 8190 bytes and aborts.

The detour did not solve the problem, it bypassed it. That is a distinction worth being honest about when writing it down — it can come back.

State afterwards

Operating system	18.3, freshly written
Disk	234 GB total, 210 GB free
SD card	out
Four corrupt database files	gone with it — they were on the card

In Kapfenberg the same setup has long run on NVMe. The difference was never a decision but a sequence: there the disk was in from the start, here the system grew on a card until the card had had enough.

So what?

For you, if you are still on an SD card:

1. Back up your long-term statistics before you need them. Eight days of retention is eight days. What is gone is gone — there is no second copy.
2. Plan the migration before the card gets tired. Corrupt database files are the first sign, not the last.
3. The built-in migration function does not make the card redundant. It moves the data, not the boot. If your goal is “card out”, the route via a computer is the shorter one.
4. Check the ribbon cable first. It can be fitted the wrong way round and look right while doing it.

And one rule that reaches beyond this day: if an error message quotes the same number on someone else’s hardware, it is not your hardware. I spent half a day hunting a disk that was never the problem.