Storage: the SD card refuses a write about once in five 1.7 MB uploads #21

New Issue

2026-10-05T19:40:12Z

twisla commented

2026-10-05 19:40:12 +00:00

What happens

The SD card now and then refuses a write: about once in five 1.7 MB uploads over the Debug Console (rdbg.py put), with the LoRa radio listening or asleep. The file's write fails partway (put: write failed at <offset>, retry 1 on the console).

Since M3 the upload notices: a retry that finds the card lost earlier data gives up, and the finished file is read back and its SHA-256 compared before it's renamed. Before that fix, the retry extended the file with zeros and reported success (3,072 bytes of zeros, twice).

The refusal itself is not explained, and other writers don't check. Logs, Tracks, Gemini pages streamed to the cache, Saved Pages, Captures and Update Files installed from the card all write through FATFS the same way. A refused write there is either dropped silently or, for an Update File, caught only by its signature check.

Measured (M3, 2026-10-05)

10 uploads of about 1.7 MB: 3 refused writes (near offsets 307,200, 191,488 and 1,575,936). In the two that were measured, the 3,072 bytes written just before were lost with the file's write buffer.
4 of 4 uploads intact with the radio listening; 1 of 4 refused with it asleep. So it isn't the radio sharing the bus.
The card runs at 20 MHz on the SPI bus shared with the radio (src/services/storage_service.cpp, sdcard_init(..., 20000000)).
Uploads run at about 200 to 230 KB/s over Wi-Fi.

What's known

The retry code's comment already said "a card can fail one write and take the next": this was seen during the OTA work too.
Not tried: a lower SPI clock (10 or 16 MHz), another card, whether Wi-Fi traffic at the same moment matters, what error FATFS returns (the code only sees a short write).

Questions

What does the card answer when it refuses: a timeout, a CRC error, a busy token? Log the FATFS and SD driver error.
Does a lower SPI clock make it go away? At what cost in speed?
Is it this card, or any card?
Which other writers need a check after writing (read back, or at least the size), and which can afford it?
Should StorageService count write failures and show them (sd list, Settings > Storage, the System Monitor in #11)?

## What happens The SD card now and then refuses a write: about once in five 1.7 MB uploads over the Debug Console (`rdbg.py put`), with the LoRa radio listening or asleep. The file's write fails partway (`put: write failed at <offset>, retry 1` on the console). Since M3 the upload notices: a retry that finds the card lost earlier data gives up, and the finished file is read back and its SHA-256 compared before it's renamed. Before that fix, the retry extended the file with zeros and reported success (3,072 bytes of zeros, twice). **The refusal itself is not explained, and other writers don't check.** Logs, Tracks, Gemini pages streamed to the cache, Saved Pages, Captures and Update Files installed from the card all write through FATFS the same way. A refused write there is either dropped silently or, for an Update File, caught only by its signature check. ## Measured (M3, 2026-10-05) - 10 uploads of about 1.7 MB: 3 refused writes (near offsets 307,200, 191,488 and 1,575,936). In the two that were measured, the 3,072 bytes written just before were lost with the file's write buffer. - 4 of 4 uploads intact with the radio listening; 1 of 4 refused with it asleep. So it isn't the radio sharing the bus. - The card runs at 20 MHz on the SPI bus shared with the radio (`src/services/storage_service.cpp`, `sdcard_init(..., 20000000)`). - Uploads run at about 200 to 230 KB/s over Wi-Fi. ## What's known - The retry code's comment already said "a card can fail one write and take the next": this was seen during the OTA work too. - Not tried: a lower SPI clock (10 or 16 MHz), another card, whether Wi-Fi traffic at the same moment matters, what error FATFS returns (the code only sees a short write). ## Questions 1. What does the card answer when it refuses: a timeout, a CRC error, a busy token? Log the FATFS and SD driver error. 2. Does a lower SPI clock make it go away? At what cost in speed? 3. Is it this card, or any card? 4. Which other writers need a check after writing (read back, or at least the size), and which can afford it? 5. Should StorageService count write failures and show them (`sd list`, Settings > Storage, the System Monitor in #11)? ## Related `src/services/debug_console.cpp` (`put`), `lib/ota/src/file_receiver.cpp` (`cardChecked`), `src/services/storage_service.cpp`, `docs/milestones/M3.md` (step 3), #1 (USB drive), #3 (Files).

twisla added the

labels 2026-10-05 19:40:12 +00:00

twisla referenced this issue from a commit

2026-10-05 19:43:41 +00:00

M3 done: the listening hour heard nothing; a reference node is needed for M4

twisla added this to the S1 System basics milestone 2026-10-05 20:14:06 +00:00

twisla commented

2026-10-05 20:55:17 +00:00

Cause found and fixed on the s1 branch (3f2650c, ADR 0007). It was never the card, the bus speed or the radio: it's the SD driver that ships with Arduino-ESP32.

What was measured

Timing each write: the failing one comes back after 4 ms (7 ms at 10 MHz), while good ones take up to 107 ms. So it isn't the driver's 500 ms busy timeout.
10 MHz instead of 20 MHz: 1 failure in 10 uploads against 3 in 10. Not a cure.
A copy of the driver that records where it gives up (lib/SD, sd_fault.h): 4 failures in 12 uploads, all at the same step. Every data block was accepted, the write was ended with Stop Tran, and the status check (CMD13) then read 0xFF or 0x1F.

Why

After Stop Tran a card takes about a byte of clock to signal busy. The driver deselects, selects again and reads one byte to test for ready; read before the card has raised busy, that byte is 0xFF, "ready", and CMD13 goes out while the card is still programming. Its garbled answer is taken for an error, and FATFS is told the write failed. ChaN's reference driver sends a dummy byte after selecting the card for this reason.

The fix

A dummy byte after select, before the ready test, and one after Stop Tran. Since then: 30 uploads of 1.7 MB in a row, each read back by SHA-256, ten with the radio listening: 0 faults.

Also in the change

info shows the driver's write faults since boot; put prints the step and the card's answer if one happens.
The other writers named above (Logs, Tracks, Gemini pages, Captures, Update Files) go through the same driver, so they gain without changes. They still don't verify what they wrote; with the cause gone, that's question 4 above, to decide separately.

Left alone, on purpose

The driver also tests the card's answer to a data block against 0x0A and 0x0C, values it can't take (0x05 accepted, 0x0B CRC error, 0x0D write error), so a block rejected for a CRC error is never resent. None of the failures here was that, and nothing exercises that path, so it's documented in ADR 0007 rather than changed blind. Both defects are worth reporting to Arduino-ESP32.

To close when s1 is merged.

**Cause found and fixed on the `s1` branch** (`3f2650c`, ADR 0007). It was never the card, the bus speed or the radio: it's the SD driver that ships with Arduino-ESP32. **What was measured** - Timing each write: the failing one comes back after 4 ms (7 ms at 10 MHz), while good ones take up to 107 ms. So it isn't the driver's 500 ms busy timeout. - 10 MHz instead of 20 MHz: 1 failure in 10 uploads against 3 in 10. Not a cure. - A copy of the driver that records where it gives up (`lib/SD`, `sd_fault.h`): 4 failures in 12 uploads, **all at the same step**. Every data block was accepted, the write was ended with Stop Tran, and the status check (CMD13) then read 0xFF or 0x1F. **Why** After Stop Tran a card takes about a byte of clock to signal busy. The driver deselects, selects again and reads one byte to test for ready; read before the card has raised busy, that byte is 0xFF, "ready", and CMD13 goes out while the card is still programming. Its garbled answer is taken for an error, and FATFS is told the write failed. ChaN's reference driver sends a dummy byte after selecting the card for this reason. **The fix** A dummy byte after select, before the ready test, and one after Stop Tran. Since then: **30 uploads of 1.7 MB in a row, each read back by SHA-256, ten with the radio listening: 0 faults.** **Also in the change** - `info` shows the driver's write faults since boot; `put` prints the step and the card's answer if one happens. - The other writers named above (Logs, Tracks, Gemini pages, Captures, Update Files) go through the same driver, so they gain without changes. They still don't verify what they wrote; with the cause gone, that's question 4 above, to decide separately. **Left alone, on purpose** The driver also tests the card's answer to a data block against 0x0A and 0x0C, values it can't take (0x05 accepted, 0x0B CRC error, 0x0D write error), so a block rejected for a CRC error is never resent. None of the failures here was that, and nothing exercises that path, so it's documented in ADR 0007 rather than changed blind. Both defects are worth reporting to Arduino-ESP32. To close when `s1` is merged.

twisla added

and removed

labels 2026-10-05 20:55:18 +00:00

twisla referenced this issue from a commit

2026-10-05 21:03:43 +00:00

SD driver: a dummy byte before the ready test; say why a write failed (#21)

twisla referenced this issue from a commit

2026-10-05 21:15:31 +00:00

`sd card`: what the card says it is, from its CID register

twisla referenced this issue

2026-10-05 21:16:17 +00:00

SD driver: follow the upstream report, and drop our copy when it's fixed #39

twisla commented

2026-10-05 21:16:17 +00:00

Reported upstream: https://github.com/espressif/arduino-esp32/issues/12970. The follow-up (answering the maintainers, keeping lib/SD in step with the framework, dropping it once fixed) is #39. The card, read with the new sd card command: a Samsung 8 GB SDHC from June 2013.

Reported upstream: https://github.com/espressif/arduino-esp32/issues/12970. The follow-up (answering the maintainers, keeping `lib/SD` in step with the framework, dropping it once fixed) is #39. The card, read with the new `sd card` command: a Samsung 8 GB SDHC from June 2013.

twisla commented

2026-10-05 21:28:59 +00:00

Fixed in v0.6.1 (s1 merged into main). The upstream report and our copy of the driver are followed in #39.

Fixed in v0.6.1 (`s1` merged into `main`). The upstream report and our copy of the driver are followed in #39.

twisla closed this issue

2026-10-05 21:29:00 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: twisla/roro9stack#21