Audio: an App to record, visualise and play back sound #14

New Issue

2026-10-05T11:48:35Z

twisla commented

2026-10-05 11:48:35 +00:00

Idea

An audio App:

Record from the built-in microphone to the SD card.
See the sound live: level meter, waveform, spectrum and a scrolling spectrogram.
Play back recordings, and other WAV files on the card, through the speaker or headphones.

A pocket voice recorder and audio scope.

Why

Voice notes on the go, a quick sound level check, seeing a tone's frequency, or just watching a spectrogram of a room. The hardware is there and unused: today the firmware only beeps (M5Cardputer.Speaker.tone() for Toasts and the Wi-Fi Tools tick), and the microphone isn't used anywhere.

What's known

Hardware.
- Per M5Stack's specs, the Cardputer ADV has an ES8311 audio codec with a MEMS microphone, a speaker, and a 3.5 mm headphone jack. To be confirmed on the device, including how the jack switches the speaker off.
- M5Unified (through M5Cardputer.Mic and .Speaker) already drives it.
- The microphone and speaker share the codec. M5Unified uses them one at a time, so recording and playing at once (monitoring) may not be possible.
Recording.
- 16 kHz, 16-bit mono WAV is 32 KB/s, or about 1.9 MB a minute. That's easily handled by the card, written in pieces through StorageService's task, like Gemini's stream to the card.
- The bus is shared with the LoRa radio (platform/shared_spi.h). 32 KB/s is light, but needs checking with LoRa active (M3).
- I2S DMA buffers plus a double buffer for the card cost about 8-16 KB of RAM while recording, and nothing otherwise.
- Compression: IMA ADPCM (4:1, cheap to encode, still a standard WAV) is an easy option. Opus or MP3 encoding is too heavy for this RAM.
- Files go to /audio/YYYY-MM-DD-HHMMSS.wav, with the time from the Clock.
Visualisation.
- A level meter (RMS and peak, dBFS) and the waveform are cheap.
- Spectrum: an FFT of 512 or 1024 points with Espressif's esp-dsp, which has FFTs optimised for the S3. Fast enough for about 30 fps on 240 pixels.
- Spectrogram: a scrolling waterfall in the RGB332 palette, the same style as the Wi-Fi Tools charts.
- A rough frequency readout (peak bin) is a bonus: a tuner of sorts.
Playback.
- WAV (PCM and ADPCM) is straightforward. MP3 decoding (Helix) needs about 30 KB of RAM, which is possible but tight with IRC connected.
- Controls: play/pause, seek, volume. The output is the speaker, or headphones when plugged in.
Interactions with the rest.
- Toast beeps must not play over a recording, or they get recorded too: mute them, or queue them.
- Recording should go on with the screen off, or with another App in front, like a dictaphone. That means a service with its own task, not just the App.
- The file manager (#3) could open .wav files here. The Lua API (#13) could get an audio module later.
- Storage Clean-up gets a new category: recordings. Or are they kept, like Notes?

Questions for the design round

Which format and quality: 16 kHz PCM, ADPCM, or a choice in Settings?
Should it record in the background with another App in front, or only while the App is open?
Which views, and what layout: level and waveform while recording, then a spectrum/spectrogram tab?
Playback: WAV only, or MP3 too (RAM cost)?
Should recordings be named, with tags or a GNSS position, or just by date?
Should recordings be offered by Storage Clean-up, or kept until deleted, like Notes?
Any extras: a voice-activated recording start, or a simple tone generator for testing?
Should it share the codec handling with beeps (Notifier) through a small AudioService that owns the I2S bus?

## Idea An audio App: - **Record** from the built-in microphone to the SD card. - **See** the sound live: level meter, waveform, spectrum and a scrolling spectrogram. - **Play back** recordings, and other WAV files on the card, through the speaker or headphones. A pocket voice recorder and audio scope. ## Why Voice notes on the go, a quick sound level check, seeing a tone's frequency, or just watching a spectrogram of a room. The hardware is there and unused: today the firmware only beeps (`M5Cardputer.Speaker.tone()` for Toasts and the Wi-Fi Tools tick), and the microphone isn't used anywhere. ## What's known - **Hardware.** - Per M5Stack's specs, the Cardputer ADV has an ES8311 audio codec with a MEMS microphone, a speaker, and a 3.5 mm headphone jack. To be confirmed on the device, including how the jack switches the speaker off. - M5Unified (through `M5Cardputer.Mic` and `.Speaker`) already drives it. - The microphone and speaker share the codec. M5Unified uses them one at a time, so recording and playing at once (monitoring) may not be possible. - **Recording.** - 16 kHz, 16-bit mono WAV is 32 KB/s, or about 1.9 MB a minute. That's easily handled by the card, written in pieces through StorageService's task, like Gemini's stream to the card. - The bus is shared with the LoRa radio (`platform/shared_spi.h`). 32 KB/s is light, but needs checking with LoRa active (M3). - I2S DMA buffers plus a double buffer for the card cost about 8-16 KB of RAM while recording, and nothing otherwise. - Compression: IMA ADPCM (4:1, cheap to encode, still a standard WAV) is an easy option. Opus or MP3 encoding is too heavy for this RAM. - Files go to `/audio/YYYY-MM-DD-HHMMSS.wav`, with the time from the Clock. - **Visualisation.** - A level meter (RMS and peak, dBFS) and the waveform are cheap. - Spectrum: an FFT of 512 or 1024 points with Espressif's `esp-dsp`, which has FFTs optimised for the S3. Fast enough for about 30 fps on 240 pixels. - Spectrogram: a scrolling waterfall in the RGB332 palette, the same style as the Wi-Fi Tools charts. - A rough frequency readout (peak bin) is a bonus: a tuner of sorts. - **Playback.** - WAV (PCM and ADPCM) is straightforward. MP3 decoding (Helix) needs about 30 KB of RAM, which is possible but tight with IRC connected. - Controls: play/pause, seek, volume. The output is the speaker, or headphones when plugged in. - **Interactions with the rest.** - Toast beeps must not play over a recording, or they get recorded too: mute them, or queue them. - Recording should go on with the screen off, or with another App in front, like a dictaphone. That means a service with its own task, not just the App. - The file manager (#3) could open `.wav` files here. The Lua API (#13) could get an `audio` module later. - Storage Clean-up gets a new category: recordings. Or are they kept, like Notes? ## Questions for the design round 1. Which format and quality: 16 kHz PCM, ADPCM, or a choice in Settings? 2. Should it record in the background with another App in front, or only while the App is open? 3. Which views, and what layout: level and waveform while recording, then a spectrum/spectrogram tab? 4. Playback: WAV only, or MP3 too (RAM cost)? 5. Should recordings be named, with tags or a GNSS position, or just by date? 6. Should recordings be offered by Storage Clean-up, or kept until deleted, like Notes? 7. Any extras: a voice-activated recording start, or a simple tone generator for testing? 8. Should it share the codec handling with beeps (Notifier) through a small AudioService that owns the I2S bus? ## Related `src/ui/notifier.cpp` (beeps), `src/services/storage_service.cpp`, `src/platform/shared_spi.h`, `lib/storage_model` (Clean-up categories), #3, #13.

twisla added the

kind

feature

area/apps area/storage

concern/memory area/audio labels 2026-10-05 11:48:35 +00:00

twisla referenced this issue

2026-10-05 16:15:39 +00:00

Cluster: AtomS3 Lite co-processors on the Grove port, with our own link protocol #18