Notes: edit text files of any size #47

New Issue

2026-10-06T07:28:29Z

twisla commented

2026-10-06 07:28:29 +00:00

Idea

The editor opens a text file whatever its size and edits it in place, reading and writing only the part being worked on.

Why

The first version of Notes holds the whole note in memory, so it stops at 16 KB; anything bigger opens read-only in the Storage App's viewer (F1, Q144). That limit was accepted for the first version only: this must come in a later release. A long-running note, a log to annotate, a config file to fix shouldn't hit a wall.

What's known

Reading any size already works: TextPager (lib/files) reads about a kilobyte around the screen and wraps it the same in both directions.
The device has no PSRAM; with IRC connected about 60 KB of heap is free, and the steady floor is 40 KB (Q86). The file can't be loaded.
Editing in place needs a structure that keeps the changes apart from the file: a piece table (the original on the card, the additions in a small buffer or a second file) is the usual answer, and saving then streams old and new pieces into a temporary file that's renamed over the note, as saves do today (Q143).
Autosave after 5 s can't rewrite a megabyte each time: either the additions are journalled to the card and the full rewrite happens on leaving, or the save is incremental.
A copy runs at about 435 KB/s on this card, so rewriting 1 MB takes over two seconds, in 150 ms turns on the storage task.

Questions

Piece table with the additions in memory (bounded, flushed to a journal), or a different structure?
What does autosave write for a big file, and what is recovered after a power cut?
Is there still an upper limit (FAT's 4 GB aside), for instance from the line-start index needed to scroll backwards quickly?
Does the 16 KB in-memory path stay for small notes, or does everything go through the new one?

## Idea The editor opens a text file whatever its size and edits it in place, reading and writing only the part being worked on. ## Why The first version of Notes holds the whole note in memory, so it stops at 16 KB; anything bigger opens read-only in the Storage App's viewer (F1, Q144). That limit was accepted for the first version only: **this must come in a later release.** A long-running note, a log to annotate, a config file to fix shouldn't hit a wall. ## What's known - Reading any size already works: `TextPager` (`lib/files`) reads about a kilobyte around the screen and wraps it the same in both directions. - The device has no PSRAM; with IRC connected about 60 KB of heap is free, and the steady floor is 40 KB (Q86). The file can't be loaded. - Editing in place needs a structure that keeps the changes apart from the file: a piece table (the original on the card, the additions in a small buffer or a second file) is the usual answer, and saving then streams old and new pieces into a temporary file that's renamed over the note, as saves do today (Q143). - Autosave after 5 s can't rewrite a megabyte each time: either the additions are journalled to the card and the full rewrite happens on leaving, or the save is incremental. - A copy runs at about 435 KB/s on this card, so rewriting 1 MB takes over two seconds, in 150 ms turns on the storage task. ## Questions 1. Piece table with the additions in memory (bounded, flushed to a journal), or a different structure? 2. What does autosave write for a big file, and what is recovered after a power cut? 3. Is there still an upper limit (FAT's 4 GB aside), for instance from the line-start index needed to scroll backwards quickly? 4. Does the 16 KB in-memory path stay for small notes, or does everything go through the new one? ## Related `docs/milestones/F1.md` (Q143, Q144), #19, `lib/files/src/text_pager.h`.

twisla added this to the F1 Files and Notes milestone 2026-10-06 07:28:29 +00:00

twisla added the

kind

feature

area/apps area/ui area/storage

concern/memory labels 2026-10-06 07:28:29 +00:00

twisla referenced this issue from a commit

2026-10-06 08:18:20 +00:00