roro9stack/docs/adr/0007-own-copy-of-the-sd-driver.md

Our own copy of the SD driver, for one missing byte

Arduino-ESP32's SD library talks to the card over SPI through sd_diskio.cpp. That driver gives up on a write without saying why, and about once in 1,500 multi-block writes it gave up on one that had worked (issue #21). A 1.7 MB upload failed about three times in ten; before M3, the retry on top of it then filled the gap with zeros.

The cause, measured with a driver that records where it stops: after the "Stop Tran" token that ends a multi-block write, a card takes about a byte of clock to signal busy. The driver deselects, selects again, and reads one byte to see whether the card is ready. Read too early, that byte is 0xFF, "ready"; the status check (CMD13) then goes out while the card is still programming, and its answer (0xFF, 0x1F) is taken for an error. Every failure seen was this one: all blocks accepted, then a status that isn't one. ChaN's reference driver, which FatFs ships as its example, sends a dummy byte after selecting the card for this reason. Arduino's doesn't.

PlatformIO links the framework's library objects directly, so one file can't be replaced from src. A project library with the same name takes its place: lib/SD is Arduino-ESP32 3.3.12's SD library (Apache-2.0), with sd_diskio.cpp changed and the other files as they came. The changes are marked roro::

• A dummy byte after selecting the card, before the ready test, and one after Stop Tran.
• Each place a write gives up records the step and the card's answer (sd_fault.h): info shows the count, and the Debug Console's put prints the detail.

Halving the SPI clock to 10 MHz didn't change the failure rate, so the card stays at 20 MHz.

Consequences

• 30 uploads of 1.7 MB in a row, each read back and compared by SHA-256, ten of them with the LoRa radio listening on the same bus: no write fault. Before: 3 failures in 10.
• Every writer gains: Logs, Tracks, Gemini pages, Saved Pages, Captures and Update Files installed from the card all go through this driver, and none of them checked.
• The copy has to follow the framework. When the platform is updated, compare lib/SD with the new libraries/SD and carry the roro: changes over. If upstream fixes the ready test, drop the copy. Reported as arduino-esp32#12970; issue #39 follows it.
• One more defect was read in the code and left alone, because nothing here exercises it: the driver tests the card's answer to a data block against 0x0A and 0x0C, values it can't take (accepted is 0x05, CRC error 0x0B, write error 0x0D), so a block rejected for a CRC error is never resent. No such rejection was seen in any failure. If DataToken faults ever show in info, that's the next fix.
• A fault is now counted and explained instead of silent, so the next cause, if there is one, starts with evidence.