twisla/roro9stack
Public Access
1
0
Fork 0
Code Issues 47 Pull Requests Actions Packages Releases 18 Activity

Merge pull request 'Notes: edit a text file of any size (#47)' (#81) from notes-any-size into main

Browse Source 
Reviewed-on: #81
This commit is contained in:
twisla
2026-10-07 13:41:36 +00:00
parent 10c5291e15 de8af6ed92
commit f2eead0766
21 changed files with 1681 additions and 111 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+3 -3
View File
@@ -150,11 +150,11 @@ At the top of the card the last row, **Maintenance** (also `m`), holds the card'
The Notes App (docs/milestones/F1.md) keeps plain text notes in `/notes` on the SD card. The list shows each note's first line and its date, newest first; `s` switches to by file name. `n` starts a note, Enter opens one, `r` renames its file, `d` deletes it after asking.
In the editor, type. Enter starts a line, Del deletes backwards, Fn with the arrows moves the cursor through the wrapped text, Ctrl+A and Ctrl+E go to the start and the end of the line, Tab types two spaces, and the Compose Key gives accents as everywhere. **There's no save key:** the note is written five seconds after the last key, on Back, on leaving the App, when the screen turns off and before the device powers off. The top line says "typing" or "saved". Each save writes a temporary file and then puts it in the note's place, so a power cut costs a few seconds of typing and never the note; if a save was cut short, opening the note offers its copy back.
In the editor, type. Enter starts a line, Del deletes backwards, Fn with the arrows moves the cursor through the wrapped text, Ctrl+A and Ctrl+E go to the start and the end of the line, Ctrl with Fn+Up and Fn+Down to the start and the end of the note, Tab types two spaces, and the Compose Key gives accents as everywhere. **There's no save key:** the note is written five seconds after the last key, on Back, on leaving the App, when the screen turns off and before the device powers off. The top line says "typing" or "saved". Each save writes a temporary file and then puts it in the note's place, so a power cut costs a few seconds of typing and never the note; if a save was cut short, opening the note offers its copy back.
A new note has no file until something is typed; its file is then named after its first line (`shopping-list.txt`), or `note-<date>-<time>.txt`.
A note holds up to 16 KB while it's edited. A bigger text file opens read-only in the Storage App (editing any size is issue #47). The Storage App's text viewer has `e` to edit a file with the same editor, anywhere on the card, unless the file is read-only.
**A note can be any size** (issue #47): the editor keeps a window of about 8 KB around the cursor in memory and the rest on the card, so a megabyte opens as fast as a line and uses the same 17 KB. Up to 64 KB a save rewrites the file. Above, the five-second save writes only what changed to a side file, `<note>.edit`, and the file itself is rewritten when the note is left, with a progress bar (about 450 KB a second). After a power cut, opening the note picks the edit up where it was saved. Saving needs room on the card for a second copy. The Storage App's text viewer has `e` to edit a file with the same editor, anywhere on the card, unless the file is read-only.
## Shell
@@ -167,7 +167,7 @@ The Shell App (docs/milestones/S1.md) runs the commands below on the device's ow
| Command | Effect |
|---|---|
| `burst` | Publishes 5 Notifications at once |
| `key up\|down\|left\|right\|select\|back\|home\|del\|tab\|space\|help`, or `key <char>` | Injects a key press (`help` is Fn+h: the keys of the screen that is showing) |
| `key up\|down\|left\|right\|select\|back\|home\|del\|tab\|space\|help`, or `key <char>` | Injects a key press (`help` is Fn+h: the keys of the screen that is showing). `ctrl-`, `alt-` and `shift-` before it hold that key: `key ctrl-down`, `key alt-up`, `key ctrl-b` |
| `sound on` / `sound off` | Toggles the Sound setting (beep + LED) |
| `short` / `normal` | Screen timeouts 5 s / 10 s, or 30 s / 60 s |
| `wifi add <ssid><TAB><password>` | Adds a Saved Network (so credentials stay out of the repo) |
docs/milestones/F1.md
+58 -1
View File
@@ -98,7 +98,7 @@ Plain text notes on the SD card, written on the device. Q30 settled the base: `.
| Q141 | A **Notes** App in the Launcher. One row per note: its first line as the title, then the date. Newest first; `s` switches to by name. `n` new, Enter opens, `d` deletes after a confirmation, `r` renames the file. |
| Q142 | A new note's file name is never typed: it comes from the first line when the note is first saved (`shopping-list.txt`), or `note-20261006-0919.txt` if that line is empty. It doesn't change afterwards unless the note is renamed. |
| Q143 | **Autosave, no "discard changes?" prompt:** five seconds after the last key, on leaving the note or the App, and when the screen turns off. A save writes a temporary file and renames it over the note, so a power cut loses the last few seconds at most. A temporary file left behind is offered back at the next open. |
| Q144 | The whole note is in memory while it's edited, up to **16 KB**. A bigger text file opens read-only in the Storage App's viewer. The App refuses to open below the memory floors (Q86). **Editing files of any size must come in a later release: issue #47.** |
| Q144 | The whole note is in memory while it's edited, up to **16 KB**. A bigger text file opens read-only in the Storage App's viewer. The App refuses to open below the memory floors (Q86). *(Lifted by issue #47: see "Notes of any size" below.)* **Editing files of any size must come in a later release: issue #47.** |
| Q145 | The editor wraps at spaces, 38 columns by 8 rows, with a line for the name and the state. Enter is a new line, Del deletes backwards, Fn+arrows move (the Text Entry rule), Ctrl+A and Ctrl+E go to the start and the end of the line, Tab types two spaces, Back saves and returns. The Compose Key works as elsewhere. |
| Q146 | The Storage App's text viewer gets `e`: edit this file with the same editor, for a text file up to 16 KB that isn't read-only. That lifts Q135 without Apps opening each other (#43 stays). |
| Q147 | The list is flat: the files directly in `/notes`. Sub-folders are reached through the Storage App. |
@@ -159,3 +159,60 @@ Test notes were made in `/notes` and removed afterwards; the folder is left, emp
**Not checked:** accents through the Compose Key and Ctrl+A / Ctrl+E (the remote `key` command can't send them; the model's tests cover both), the power button's save (it needs a hand on the device), a missing card, and how typing feels on the keyboard itself.
**One slip during the checks:** a key sequence sent right after a restart opened IRC instead of Notes, and the test letters went into IRC's input line. Nothing was sent: the line was cleared and the App left. IRC connected to Libera as it does when opened.
## Notes of any size (issue #47)
Q144 held the whole note in memory and stopped at 16 KB, for the first version only. This lifts it: the editor opens a text file whatever its size.
### Decisions (design round 2026-10-07)
| # | Decision |
|---|---|
| Q223 | **The note is the file on the card plus one window in memory.** The window is the `NoteText` of before, up to 16 KB around the cursor; the rest is a list of pieces: runs of the file, and runs of a side file. The cursor leaving the window writes it to the side file if it was changed, and loads the next. Typing never fills a note: a full window is written away and loaded smaller. |
| Q224 | **The five-second save:** up to 64 KB it rewrites the file, as before (about 150 ms). Above, it appends the window and the list of pieces to `<note>.edit`: 8 KB or so, whatever the note's size. "saved" means "on the card" either way. |
| Q225 | **The file itself is rewritten on leaving the note** (Back, Home, another App), with a progress bar. The screen turning off and the device powering off write the side file only: powering off never waits. |
| Q226 | **After a power cut, opening the note picks the edit up** where it was last saved, without a question, and says so. Until then the file has the old text for anything else that reads it. |
| Q227 | If the file was changed elsewhere meanwhile, the side file no longer fits it: it is **kept as `<note>.edit.lost`** and the editor says so. Typed text is never deleted without a word. |
| Q228 | **No limit but the card:** a note over 16 KB needs room for a second copy to be opened for editing. No warning for a big file; the progress bar on leaving tells the cost. |
| Q229 | A side file over 1 MB, or a list of over 256 pieces, makes the next save a rewrite. |
| Q230 | **CRLF becomes LF** (Q148) for a long file too: in the window as it is read, and in the rest of the file as the rewrite streams it, so a saved file is never of both kinds. |
| Q231 | **One path.** A 16 KB note is the case with no pieces: there is no second editor for small notes. |
| Q232 | Notes, and `e` in the Storage App's viewer, which no longer says "Too big to edit". |
### As built
- **`NoteDocument`** (`lib/notes/src/note_document.h`, host-tested against a card in memory) is the list of pieces, the window's moves, the side file and the recovery. `NoteText` is unchanged but for being refilled.
- **The window moves** when the cursor comes within 2 KB of an end of it that isn't an end of the note: it is then 4 KB on each side of the cursor. It starts where a line starts on screen whenever that can be known (after a newline, or where the window before had a line start), so the same text wraps the same from one window to the next, and never in the middle of a character. The cursor keeps its row on screen.
- **Looking writes nothing:** a window that wasn't changed goes back as the pieces it was read from.
- **The side file** starts with a line of text, the note's size and checksums of its first and last kilobyte, which is how a file changed elsewhere is told. After that, text that left a window, and snapshots of the list of pieces, each with its checksum. The newest snapshot that checks out is the note as last saved; anything after it is ignored.
- **The rewrite** streams the pieces and the window into `<note>.tmp`, checks its size, then writes a mark at the end of the side file: from that mark on, the rewrite counts as done, and opening the note finishes it whatever was cut (remove the old file, rename, remove the side file). Before the mark, the note and its side file are still the truth and the temporary file is dropped.
- **On the device** the card is reached through an adapter that keeps the file being read and the file being appended to open between calls; every call runs on the storage task while the main loop waits. The rewrite runs in steps of 64 KB with the progress drawn between them.
- **The Notes list** doesn't show `.edit` and `.edit.lost` files, and a note's side file is deleted and renamed with it.
- **Ctrl with Fn+Up and Fn+Down** go to the start and the end of the note.
- **`key ctrl-down`**: the consoles' `key` command takes `ctrl-`, `alt-` and `shift-`, which these checks needed. It also lets the checks S1 couldn't make (Ctrl+b, the Alt scroll) be made.
- **Cost:** 15 KB of flash. Memory with a note open is what it was: 17.5 KB, for 62 bytes or for 1.2 MB.
### Host tests (15, `test/test_note_document`)
A walk down 3,000 lines and back up through the windows; start and end; an edit in the middle rewritten into the file; 48 KB typed into a new note; a journal picked up after a cut; **a cut at every 997th byte of a sequence of two saves and a rewrite**, after which the note is always one of the three texts it should be, what was reported saved is there, and no stray file is left; a file changed elsewhere; CRLF; windows on text with no space and no newline, made of 2, 3 and 4-byte characters; a full card; and **36,000 random keys** (typing, deleting, moving, jumping, saving, power cuts) on six notes of 30 to 130 KB, compared with a plain string after every key.
### Checks on the device (2026-10-07, driven over the Debug Console)
Test notes were copied to `/notes` and removed afterwards; the note that was already there was not touched.
| Check | Result |
|---|---|
| A 36 KB note | Opens (it was refused before). Two letters at the top, 400 lines down across the windows, four more: the file fetched back is exactly that, and no other file is left |
| A 1.2 MB note | Opens at once. Free memory 104.2 KB before, 86.7 KB with it open |
| Its five-second save | `zz-big.txt.edit`, 4 KB; the note's file untouched |
| Ctrl with Down, Ctrl with Up | The end and the start, as fast as any key |
| A restart with unsaved keys | "Your unsaved changes are back", the cursor where it was, the unsaved keys gone and nothing else |
| Leaving it | The progress bar, then one file: **1.2 MB rewritten in 2.6 s**. Fetched back: the original with what was typed at both ends, byte for byte |
| A restart in the middle of that rewrite | The note, its side file and an empty `.tmp` remain; opening picks the edit up, leaving rewrites it, the result is right |
| A new note | No file until typed in, then `zz-test-note.txt` from its first line |
| `e` in the Storage App on the 1.2 MB file | The same editor; edited and rewritten |
| The Notes list | Side files are not listed as notes |
**Not checked:** the power button's path (side file only), the screen turning off, a card pulled while editing, and memory with IRC connected, which wasn't connected for these checks: the editor's own use hasn't changed, and it still refuses to open without a free block of 24 KB. The real keyboard's Ctrl with Fn and the arrows. A file of tens of megabytes. Renaming or deleting a note from the Storage App leaves its side file behind.
**Measured against what was said:** the first build rewrote 1.2 MB in 3.5 to 4.5 s, with 2 KB blocks. With 4 KB blocks it is 2.6 s, about 450 KB a second, which is what the card gives a plain copy.
lib/core/src/app_keys.h
+2 -1
View File
@@ -255,7 +255,7 @@ inline constexpr KeyHelp kViewerText[] = {
{"; .", "a line up, down"},
{", /", "a page up, down"},
{"t b", "the top, the end"},
{"e", "edit it (up to 16 KB)"},
{"e", "edit it"},
{"Tab", "the file as hex, or back"},
};
@@ -310,6 +310,7 @@ inline constexpr KeyHelp kNotesEditor[] = {
{"Tab", "two spaces"},
{"Fn ; . , /", "move the cursor"},
{"Alt Fn ; .", "a page up, down"},
{"Ctrl Fn ; .", "start, end of the note"},
{"Ctrl a e", "start, end of the line"},
{"opt ' e", "an accent: \xC3\xA9"},
{"`", "done: it saves by itself"},
lib/notes/src/note_document.cpp
+622
View File
@@ -0,0 +1,622 @@
#include "note_document.h"
#include <algorithm>
#include <cstring>
namespace roro::notes {
namespace {
// The side file: this line, the note's size and two checksums of it (its first and last
// kilobyte), then, in any order, text that left a window and snapshots of the list of pieces.
// snapshot: "RSNP" cursor count { src at len }... crc32 length "PNSR" (numbers: 32 bits, low byte first)
// The newest snapshot that checks out is the note as it was last saved. kDone at the very end:
// the rewrite this file was for is complete in `<note>.tmp`, and only has to take the note's place.
const char kMagic[] = "roro9stack note edits 1\n";
constexpr size_t kMagicLen = sizeof(kMagic) - 1;
constexpr size_t kHeaderLen = kMagicLen + 12;
const char kSnap[] = "RSNP", kSnapEnd[] = "PNSR", kDone[] = "RDONE1\n\n";
constexpr size_t kDoneLen = 8;
constexpr size_t kCheck = 1024; // of each end of the note, in the header
constexpr uint32_t kStepBytes = 64 * 1024; // a rewrite's step
constexpr size_t kBlock = 4096;
constexpr uint32_t kSeekNewline = 1024;
constexpr size_t kMaxSnapshot = 12 + 9 * 4096 + 12;
bool continuation(int c) { return (c & 0xC0) == 0x80; }
uint32_t crc32(uint32_t crc, const uint8_t* data, size_t len) {
crc = ~crc;
for (size_t i = 0; i < len; i++) {
crc ^= data[i];
for (int k = 0; k < 8; k++) crc = (crc >> 1) ^ (0xEDB88320u & (0u - (crc & 1)));
}
return ~crc;
}
void put32(std::string& s, uint32_t v) {
for (int i = 0; i < 4; i++) s += static_cast<char>((v >> (8 * i)) & 0xFF);
}
uint32_t get32(const uint8_t* p) { return p[0] | (p[1] << 8) | (p[2] << 16) | (static_cast<uint32_t>(p[3]) << 24); }
const uint8_t* bytes(const std::string& s) { return reinterpret_cast<const uint8_t*>(s.data()); }
} // namespace
NoteDocument::NoteDocument(NoteCard& card, int cols, int rows) : card_(card), text_(cols, rows) { openNew(); }
void NoteDocument::reset() {
path_.clear();
sidePath_.clear();
pieces_.clear();
loaded_.clear();
win_ = 0;
windowLoaded_ = false;
before_ = after_ = 0;
droppedAtLoad_ = newlinesAtLoad_ = 0;
flushedSinceSave_ = sidePending_ = false;
sideSize_ = 0;
windowSaved_.valid = false;
rw_.active = false;
std::vector<uint8_t>().swap(rw_.block);
}
void NoteDocument::openNew() {
reset();
text_.buffer().clear();
text_.refilled(0, 0);
windowLoaded_ = true;
loadedRevision_ = savedRevision_ = text_.revision();
}
std::string NoteDocument::open(const std::string& path, std::string* told) {
openNew();
path_ = path;
sidePath_ = side();
uint32_t sideSize = 0, fileSize = 0, other = 0;
bool hasSide = card_.size(sidePath_, sideSize);
if (hasSide && sideIsDone(sideSize)) { // a rewrite was cut after its last write: finish it
if (card_.size(tmp(), other)) {
if (card_.size(path_, fileSize)) card_.remove(path_);
card_.rename(tmp(), path_);
}
card_.remove(sidePath_);
hasSide = false;
}
if (!card_.size(path_, fileSize)) {
card_.done();
openNew();
return "The card refused to open it";
}
if (fileSize > NoteText::kMaxBytes && card_.freeBytes() < static_cast<uint64_t>(fileSize) + 16 * 1024) {
card_.done();
openNew();
return "Not enough room on the card: saving it needs a second copy";
}
uint32_t cursor = 0;
bool resumed = false;
if (hasSide) {
if (card_.size(tmp(), other)) card_.remove(tmp()); // a rewrite that didn't get that far
sideSize_ = sideSize;
Resume r = resume(fileSize, cursor);
if (r == Resume::Ok) {
resumed = sidePending_ = true;
if (told) *told = "Your unsaved changes are back";
} else {
sideSize_ = 0;
pieces_.clear();
if (r == Resume::Mismatch) { // typed text is never thrown away without a word (Q227)
std::string lost = sidePath_ + ".lost";
card_.remove(lost);
card_.rename(sidePath_, lost);
if (told) *told = "The file changed: unsaved edits kept as .edit.lost";
} else {
card_.remove(sidePath_);
}
}
}
if (!resumed && fileSize) pieces_.push_back({0, 0, fileSize});
windowLoaded_ = false;
bool ok = load(cursor, cursor ? 1000 : 0, -1); // an edit picked up: its last lines above the cursor
card_.done();
if (!ok) {
openNew();
return "The card refused to read it";
}
savedRevision_ = text_.revision();
return "";
}
uint32_t NoteDocument::piecesBytes() const {
uint32_t n = 0;
for (const Piece& p : pieces_) n += p.len;
return n;
}
int NoteDocument::percent() const {
uint32_t all = size();
return all ? static_cast<int>(static_cast<uint64_t>(before_ + text_.top()) * 100 / all) : 0;
}
size_t NoteDocument::readDoc(uint32_t at, uint8_t* into, size_t len) {
size_t got = 0;
uint32_t pos = 0;
for (const Piece& p : pieces_) {
if (got == len) break;
if (at < pos + p.len) {
uint32_t skip = at - pos;
size_t n = std::min<size_t>(len - got, p.len - skip);
size_t r = card_.read(fileOf(p), p.at + skip, into + got, n);
got += r;
at += static_cast<uint32_t>(r);
if (r != n) break;
}
pos += p.len;
}
return got;
}
int NoteDocument::byteAt(uint32_t at) {
uint8_t b;
return readDoc(at, &b, 1) == 1 ? b : -1;
}
size_t NoteDocument::splitAt(uint32_t at) {
uint32_t pos = 0;
for (size_t i = 0; i < pieces_.size(); i++) {
if (at == pos) return i;
Piece& p = pieces_[i];
if (at < pos + p.len) {
uint32_t first = at - pos;
Piece rest{p.src, p.at + first, p.len - first};
p.len = first;
pieces_.insert(pieces_.begin() + static_cast<long>(i) + 1, rest);
return i + 1;
}
pos += p.len;
}
return pieces_.size();
}
void NoteDocument::merge() {
size_t kept = 0;
for (size_t i = 0; i < pieces_.size(); i++) {
const Piece p = pieces_[i];
if (!p.len) continue;
if (kept && pieces_[kept - 1].src == p.src && pieces_[kept - 1].at + pieces_[kept - 1].len == p.at) pieces_[kept - 1].len += p.len;
else pieces_[kept++] = p;
}
pieces_.resize(kept);
}
// The window dropped the CRs of the file's CRLFs when it was read. If it's put back untouched,
// the cursor is further along in the file than in the window: by one for each line before it.
uint32_t NoteDocument::noteCursor() const {
size_t c = text_.cursor();
if (windowLoaded_ && droppedAtLoad_ && text_.revision() == loadedRevision_) {
const std::string& t = text_.text();
if (droppedAtLoad_ == newlinesAtLoad_) c += static_cast<size_t>(std::count(t.begin(), t.begin() + static_cast<long>(c), '\n'));
else if (!t.empty()) c += droppedAtLoad_ * c / t.size(); // a file of both kinds of line: near enough
}
return before_ + static_cast<uint32_t>(c);
}
bool NoteDocument::headerFor(std::string& header) {
uint32_t fileSize = 0;
if (path_.empty() || !card_.size(path_, fileSize)) return false;
std::vector<uint8_t> buf(kCheck);
size_t n = std::min<size_t>(kCheck, fileSize);
if (card_.read(path_, 0, buf.data(), n) != n) return false;
uint32_t head = crc32(0, buf.data(), n);
if (card_.read(path_, fileSize - static_cast<uint32_t>(n), buf.data(), n) != n) return false;
uint32_t tail = crc32(0, buf.data(), n);
header.assign(kMagic, kMagicLen);
put32(header, fileSize);
put32(header, head);
put32(header, tail);
return true;
}
bool NoteDocument::ensureSide(std::string& why) {
if (sideSize_) return true;
std::string header;
if (!headerFor(header)) {
why = path_.empty() ? "the note has no file yet" : "the card refused to read the note";
return false;
}
if (!card_.create(sidePath_) || !card_.append(sidePath_, bytes(header), header.size())) {
card_.remove(sidePath_);
why = "the card refused a write";
return false;
}
sideSize_ = static_cast<uint32_t>(header.size());
return true;
}
bool NoteDocument::putBack(std::string& why) {
if (!windowLoaded_) return true;
if (text_.revision() == loadedRevision_) {
pieces_.insert(pieces_.begin() + static_cast<long>(win_), loaded_.begin(), loaded_.end());
} else {
Piece p{1, 0, static_cast<uint32_t>(text_.size())};
if (windowSaved_.valid && windowSaved_.revision == text_.revision()) {
p.at = windowSaved_.at;
} else if (p.len) {
if (!ensureSide(why)) return false;
if (!card_.append(sidePath_, bytes(text_.text()), p.len)) {
card_.done();
if (!card_.size(sidePath_, sideSize_)) sideSize_ = 0;
why = "the card refused a write";
return false;
}
p.at = sideSize_;
sideSize_ += p.len;
}
if (p.len) pieces_.insert(pieces_.begin() + static_cast<long>(win_), p);
flushedSinceSave_ = sidePending_ = true;
}
windowLoaded_ = false;
loaded_.clear();
windowSaved_.valid = false;
before_ = after_ = 0;
merge();
return true;
}
// The window's start is where a line starts on screen whenever that can be known: after a
// newline, or where the window before had a line start. Otherwise the same text could wrap
// differently from one window to the next.
bool NoteDocument::load(uint32_t cursor, int row, int64_t startHint) {
uint32_t total = piecesBytes();
cursor = std::min(cursor, total);
uint32_t s = 0, e = total;
if (total > NoteText::kMaxBytes - kEdge) {
uint32_t c = cursor > kHalf ? cursor - kHalf : 0;
if (c == 0) {
s = 0;
} else if (startHint >= 0 && startHint <= static_cast<int64_t>(c)) {
s = static_cast<uint32_t>(startHint);
} else {
uint8_t buf[128];
uint32_t at = c, limit = std::min(c + kSeekNewline, cursor);
bool found = false;
while (at < limit && !found) {
size_t n = readDoc(at, buf, std::min<size_t>(sizeof buf, limit - at));
if (!n) break;
for (size_t i = 0; i < n && !found; i++)
if (buf[i] == '\n') {
s = at + static_cast<uint32_t>(i) + 1;
found = true;
}
at += static_cast<uint32_t>(n);
}
if (!found) {
s = c;
for (int k = 0; k < 3 && s < cursor && continuation(byteAt(s)); k++) s++;
}
}
e = std::min(total, cursor + kHalf);
for (int k = 0; k < 3 && e < total && continuation(byteAt(e)); k++) e++;
if (e < total && e > 0 && byteAt(e) == '\n' && byteAt(e - 1) == '\r') e++;
e = std::min<uint32_t>(e, s + NoteText::kMaxBytes);
}
size_t i0 = splitAt(s), i1 = splitAt(e);
loaded_.assign(pieces_.begin() + static_cast<long>(i0), pieces_.begin() + static_cast<long>(i1));
pieces_.erase(pieces_.begin() + static_cast<long>(i0), pieces_.begin() + static_cast<long>(i1));
win_ = i0;
before_ = s;
after_ = total - e;
std::string& b = text_.buffer();
b.resize(e - s);
size_t got = 0;
bool ok = true;
for (const Piece& p : loaded_) {
size_t n = card_.read(fileOf(p), p.at, reinterpret_cast<uint8_t*>(&b[got]), p.len);
got += n;
if (n != p.len) {
ok = false;
break;
}
}
if (!ok) { // the note is whole in its pieces: stand on an empty window where the cursor was
pieces_.insert(pieces_.begin() + static_cast<long>(i0), loaded_.begin(), loaded_.end());
loaded_.clear();
win_ = splitAt(cursor);
before_ = cursor;
after_ = total - cursor;
s = cursor;
b.clear();
}
droppedAtLoad_ = text_.refilled(cursor - s, row);
newlinesAtLoad_ = static_cast<size_t>(std::count(b.begin(), b.end(), '\n'));
loadedRevision_ = text_.revision();
windowLoaded_ = true;
windowSaved_.valid = false;
return ok;
}
bool NoteDocument::wantsMove() const {
if (!windowLoaded_) return true;
size_t n = text_.size(), c = text_.cursor();
if (n + kSpare >= NoteText::kMaxBytes) return true;
if (before_ && c < kEdge) return true;
return after_ && n - c < kEdge;
}
bool NoteDocument::move(std::string& why) {
uint32_t cursor = noteCursor();
int row = text_.cursorRow();
int64_t hint = -1;
if (windowLoaded_ && !droppedAtLoad_ && cursor > kHalf) {
uint32_t c = cursor - kHalf;
if (c >= before_ && c < before_ + text_.size()) hint = static_cast<int64_t>(before_) + static_cast<int64_t>(text_.startOfLine(c - before_));
}
if (!putBack(why)) return false;
bool ok = load(cursor, row, hint);
card_.done();
if (!ok) why = "the card refused to read";
return ok;
}
bool NoteDocument::jump(uint32_t to, std::string& why) {
to = std::min(to, size());
if (windowLoaded_ && to == 0 && !before_) return text_.toStart(), true;
if (windowLoaded_ && to == size() && !after_) return text_.toEnd(), true;
if (!putBack(why)) return false;
bool ok = load(to, to ? 1000 : 0, -1);
card_.done();
if (!ok) why = "the card refused to read";
return ok;
}
bool NoteDocument::wantsRewrite() const { return size() <= kWholeLimit || sideSize_ > kSideLimit || pieces_.size() > kManyPieces; }
bool NoteDocument::journal(std::string& why) {
if (path_.empty()) {
why = "the note has no file yet";
return false;
}
bool modified = text_.revision() != loadedRevision_;
Piece w{1, 0, static_cast<uint32_t>(text_.size())};
uint32_t cursor = noteCursor();
if (!ensureSide(why)) return false;
bool wrote = true;
if (modified && w.len) {
if (windowSaved_.valid && windowSaved_.revision == text_.revision()) {
w.at = windowSaved_.at;
} else if ((wrote = card_.append(sidePath_, bytes(text_.text()), w.len))) {
w.at = sideSize_;
sideSize_ += w.len;
windowSaved_.valid = true;
windowSaved_.at = w.at;
windowSaved_.len = w.len;
windowSaved_.revision = text_.revision();
}
}
if (wrote) {
std::vector<Piece> all(pieces_.begin(), pieces_.begin() + static_cast<long>(win_));
if (!modified) all.insert(all.end(), loaded_.begin(), loaded_.end());
else if (w.len) all.push_back(w);
all.insert(all.end(), pieces_.begin() + static_cast<long>(win_), pieces_.end());
std::string rec(kSnap, 4);
put32(rec, cursor);
put32(rec, static_cast<uint32_t>(all.size()));
for (const Piece& p : all) {
rec += static_cast<char>(p.src);
put32(rec, p.at);
put32(rec, p.len);
}
put32(rec, crc32(0, bytes(rec), rec.size()));
put32(rec, static_cast<uint32_t>(rec.size()) + 8);
rec.append(kSnapEnd, 4);
wrote = card_.append(sidePath_, bytes(rec), rec.size());
if (wrote) sideSize_ += static_cast<uint32_t>(rec.size());
}
card_.done();
if (!wrote) {
windowSaved_.valid = false;
if (!card_.size(sidePath_, sideSize_)) sideSize_ = 0;
why = "the card refused a write";
return false;
}
savedRevision_ = text_.revision();
flushedSinceSave_ = false;
sidePending_ = true;
return true;
}
bool NoteDocument::rewriteStart(std::string& why) {
if (path_.empty()) {
why = "the note has no file yet";
return false;
}
if (card_.freeBytes() < static_cast<uint64_t>(size()) + 16 * 1024) {
why = "the card is full";
return false;
}
if (!card_.create(tmp())) {
why = "the card refused to open a file";
return false;
}
rw_.active = true;
rw_.stage = 0;
rw_.index = 0;
rw_.offset = rw_.done = rw_.wrote = rw_.wroteBefore = rw_.wroteAfter = 0;
rw_.total = size();
rw_.revision = text_.revision();
rw_.carry = false;
rw_.block.resize(kBlock);
return true;
}
int NoteDocument::rewriteStep(std::string& why) {
if (!rw_.active) return -1;
auto failed = [&](const char* what) {
card_.done();
card_.remove(tmp());
rw_.active = false;
std::vector<uint8_t>().swap(rw_.block);
why = what;
return -1;
};
auto out = [&](const uint8_t* data, size_t len) {
if (!len) return true;
if (!card_.append(tmp(), data, len)) return false;
rw_.wrote += static_cast<uint32_t>(len);
if (rw_.stage == 0) rw_.wroteBefore += static_cast<uint32_t>(len);
if (rw_.stage == 2) rw_.wroteAfter += static_cast<uint32_t>(len);
return true;
};
const uint8_t cr = '\r';
uint32_t budget = kStepBytes;
while (budget > 0 && rw_.stage < 3) {
if (rw_.stage == 1) {
uint32_t left = static_cast<uint32_t>(text_.size()) - rw_.offset;
if (!left) {
rw_.stage = 2;
rw_.index = win_;
rw_.offset = 0;
continue;
}
uint32_t n = std::min(left, budget);
if (!out(bytes(text_.text()) + rw_.offset, n)) return failed("the card refused a write");
rw_.offset += n;
rw_.done += n;
budget -= n;
continue;
}
size_t end = rw_.stage == 0 ? win_ : pieces_.size();
bool pieceOver = rw_.index < end && rw_.offset >= pieces_[rw_.index].len;
if (rw_.index >= end || pieceOver) {
if (rw_.carry && !out(&cr, 1)) return failed("the card refused a write"); // a CR that ended its piece stays
rw_.carry = false;
rw_.offset = 0;
if (pieceOver) rw_.index++;
else rw_.stage++;
continue;
}
const Piece& p = pieces_[rw_.index];
uint32_t n = std::min<uint32_t>(std::min<uint32_t>(p.len - rw_.offset, budget), static_cast<uint32_t>(rw_.block.size()));
uint8_t* b = rw_.block.data();
if (card_.read(fileOf(p), p.at + rw_.offset, b, n) != n) return failed("the card refused to read the note");
size_t m = n;
if (p.src == 0) { // CRLF becomes LF (Q148, Q230), in the file's own text: what was typed has none
if (rw_.carry && b[0] != '\n' && !out(&cr, 1)) return failed("the card refused a write");
rw_.carry = false;
m = 0;
for (uint32_t i = 0; i < n; i++) {
if (b[i] == '\r') {
if (i + 1 == n) {
rw_.carry = true;
continue;
}
if (b[i + 1] == '\n') continue;
}
b[m++] = b[i];
}
}
if (!out(b, m)) return failed("the card refused a write");
rw_.offset += n;
rw_.done += n;
budget -= n;
}
if (rw_.stage < 3) return std::min(99, static_cast<int>(static_cast<uint64_t>(rw_.done) * 100 / std::max<uint32_t>(1, rw_.total)));
// All of it is in the temporary file. From the mark in the side file on, the rewrite counts as
// done: whatever is cut after that, opening the note finishes it.
card_.done();
uint32_t written = 0, old = 0;
if (!card_.size(tmp(), written) || written != rw_.wrote) return failed("the card refused a write");
if (sideSize_) {
if (!card_.append(sidePath_, reinterpret_cast<const uint8_t*>(kDone), kDoneLen)) return failed("the card refused a write");
sideSize_ += kDoneLen;
card_.done();
}
rw_.active = false;
std::vector<uint8_t>().swap(rw_.block);
// FAT can't rename onto a file. Between these two lines only the temporary file exists: the
// Notes list puts such a file back under its name.
if ((card_.size(path_, old) && !card_.remove(path_)) || !card_.rename(tmp(), path_)) {
card_.done();
why = "the card refused to replace the note";
return -1;
}
if (sideSize_) card_.remove(sidePath_);
card_.done();
sideSize_ = 0;
uint32_t window = static_cast<uint32_t>(text_.size());
pieces_.clear();
loaded_.clear();
if (rw_.wroteBefore) pieces_.push_back({0, 0, rw_.wroteBefore});
win_ = pieces_.size();
if (rw_.wroteAfter) pieces_.push_back({0, rw_.wroteBefore + window, rw_.wroteAfter});
if (window) loaded_.push_back({0, rw_.wroteBefore, window});
before_ = rw_.wroteBefore;
after_ = rw_.wroteAfter;
loadedRevision_ = savedRevision_ = rw_.revision;
droppedAtLoad_ = 0;
windowLoaded_ = true;
flushedSinceSave_ = sidePending_ = false;
windowSaved_.valid = false;
return 100;
}
bool NoteDocument::sideIsDone(uint32_t sideSize) {
uint8_t tail[kDoneLen];
return sideSize >= kHeaderLen + kDoneLen && card_.read(sidePath_, sideSize - kDoneLen, tail, kDoneLen) == kDoneLen &&
std::memcmp(tail, kDone, kDoneLen) == 0;
}
NoteDocument::Resume NoteDocument::resume(uint32_t fileSize, uint32_t& cursor) {
uint8_t header[kHeaderLen];
if (sideSize_ < kHeaderLen || card_.read(sidePath_, 0, header, kHeaderLen) != kHeaderLen || std::memcmp(header, kMagic, kMagicLen) != 0)
return Resume::Nothing;
// The newest snapshot that checks out, looking back from the end: after it there may be text
// that left a window, or a write the power cut short.
auto snapshotEndingAt = [&](uint32_t end) {
uint8_t lenBytes[4];
if (end < kHeaderLen + 24 || card_.read(sidePath_, end - 8, lenBytes, 4) != 4) return false;
uint32_t len = get32(lenBytes);
if (len < 24 || len > kMaxSnapshot || len > end - kHeaderLen || (len - 24) % 9) return false;
std::string rec(len, '\0');
if (card_.read(sidePath_, end - len, reinterpret_cast<uint8_t*>(&rec[0]), len) != len) return false;
const uint8_t* r = bytes(rec);
if (std::memcmp(r, kSnap, 4) != 0 || get32(r + len - 12) != crc32(0, r, len - 12)) return false;
uint32_t count = get32(r + 8);
if (count != (len - 24) / 9) return false;
std::vector<Piece> list;
list.reserve(count);
for (uint32_t i = 0; i < count; i++) {
const uint8_t* q = r + 12 + 9 * i;
Piece p{q[0], get32(q + 1), get32(q + 5)};
uint64_t stop = static_cast<uint64_t>(p.at) + p.len;
if (p.src > 1 || !p.len) return false;
if (p.src == 1 && (p.at < kHeaderLen || stop > end - len)) return false;
list.push_back(p);
}
pieces_.swap(list);
cursor = get32(r + 4);
return true;
};
bool found = false;
std::vector<uint8_t> buf(1024 + 3);
for (uint32_t end = sideSize_; end > kHeaderLen && !found;) {
uint32_t a = end > 1024 + kHeaderLen ? end - 1024 : static_cast<uint32_t>(kHeaderLen);
size_t n = card_.read(sidePath_, a, buf.data(), std::min<size_t>(buf.size(), sideSize_ - a));
for (size_t i = n >= 4 ? n - 4 + 1 : 0; i-- > 0 && !found;)
if (std::memcmp(buf.data() + i, kSnapEnd, 4) == 0) found = snapshotEndingAt(a + static_cast<uint32_t>(i) + 4);
end = a;
}
if (!found) return Resume::Nothing;
std::string expect;
if (!headerFor(expect) || std::memcmp(header, expect.data(), kHeaderLen) != 0) {
pieces_.clear();
return Resume::Mismatch;
}
for (const Piece& p : pieces_)
if (p.src == 0 && static_cast<uint64_t>(p.at) + p.len > fileSize) return pieces_.clear(), Resume::Mismatch;
merge();
return Resume::Ok;
}
} // namespace roro::notes
lib/notes/src/note_document.h
+130
View File
@@ -0,0 +1,130 @@
#pragma once
#include <cstddef>
#include <cstdint>
#include <string>
#include <vector>
#include "note_text.h"
namespace roro::notes {
// The card, as a note needs it. On the device every call is made on the storage task.
class NoteCard {
public:
virtual ~NoteCard() = default;
virtual bool size(const std::string& path, uint32_t& size) = 0; // false: no such file
virtual size_t read(const std::string& path, uint32_t at, uint8_t* into, size_t len) = 0;
virtual bool create(const std::string& path) = 0; // an empty file, in place of what was there
virtual bool append(const std::string& path, const uint8_t* data, size_t len) = 0;
virtual bool remove(const std::string& path) = 0;
virtual bool rename(const std::string& from, const std::string& to) = 0;
virtual uint64_t freeBytes() = 0;
virtual void done() {} // what was appended is on the card now (files kept open are closed)
};
// A text file of any size, edited (issue #47, F1 Q223-Q232). The file stays on the card; what is
// in memory is one window of it, a NoteText of up to 16 KB around the cursor, and a list of pieces
// saying what the rest is made of: runs of bytes of the file, and runs of the side file
// `<note>.edit`, where a window that was changed is written when the cursor leaves it.
//
// the note = pieces before the window + the window + pieces after it
//
// Saving comes in two kinds. `journal` appends the window and the list of pieces to the side
// file: quick whatever the note's size, and enough to pick the edit up after a power cut.
// `rewrite` streams the whole note into `<note>.tmp` and puts it in the note's place: the file is
// then the note again, and the side file goes. A note of up to 64 KB is always rewritten.
//
// Every method marked [card] reads or writes the card.
class NoteDocument {
public:
static constexpr uint32_t kHalf = 4096; // loaded on each side of the cursor
static constexpr uint32_t kEdge = 2048; // this near an end of the window, it moves
static constexpr uint32_t kSpare = 256; // this near full, it moves
static constexpr uint32_t kWholeLimit = 64 * 1024; // up to here a save is a rewrite
static constexpr uint32_t kSideLimit = 1024 * 1024; // a side file this big asks for a rewrite
static constexpr size_t kManyPieces = 256; // and so does a list this long
NoteDocument(NoteCard& card, int cols, int rows);
// [card] "" or why not. `told`: something the user should read (an edit picked up, or set aside).
std::string open(const std::string& path, std::string* told = nullptr);
void openNew(); // nothing on the card until the first rewrite
const std::string& path() const { return path_; }
void setPath(const std::string& path) { // a new note's, before its first rewrite
path_ = path;
sidePath_ = path.empty() ? "" : side();
}
NoteText& text() { return text_; }
const NoteText& text() const { return text_; }
uint32_t size() const { return before_ + static_cast<uint32_t>(text_.size()) + after_; }
uint32_t cursor() const { return before_ + static_cast<uint32_t>(text_.cursor()); } // in the note
int percent() const;
bool windowed() const { return before_ || after_; } // the note is more than its window
// After each key: the cursor is near an end of the window that isn't an end of the note, or
// the window is nearly full.
bool wantsMove() const;
bool move(std::string& why); // [card] the window, around the cursor
bool jump(uint32_t to, std::string& why); // [card] the cursor, anywhere in the note
bool dirty() const { return text_.revision() != savedRevision_ || flushedSinceSave_; } // the card doesn't have it
bool filePending() const { return sidePending_; } // saved, but in the side file: a rewrite is owed
bool wantsRewrite() const; // the next save should be a rewrite
bool journal(std::string& why); // [card]
bool rewriteStart(std::string& why); // [card]
int rewriteStep(std::string& why); // [card] percent done; 100: the file is the note; -1: failed
bool rewriting() const { return rw_.active; }
private:
struct Piece {
uint8_t src; // 0: the note's file, 1: the side file
uint32_t at, len;
};
enum class Resume { Ok, Mismatch, Nothing };
std::string side() const { return path_ + ".edit"; }
std::string tmp() const { return path_ + ".tmp"; }
const std::string& fileOf(const Piece& p) const { return p.src ? sidePath_ : path_; }
uint32_t piecesBytes() const;
size_t readDoc(uint32_t at, uint8_t* into, size_t len); // from the pieces: the window is put back first
int byteAt(uint32_t at);
size_t splitAt(uint32_t at); // the index of the piece that starts there
void merge();
uint32_t noteCursor() const; // where the cursor is among the pieces once the window is put back
bool putBack(std::string& why);
bool load(uint32_t cursor, int row, int64_t startHint); // false: the card refused, and the window is empty
bool ensureSide(std::string& why);
bool headerFor(std::string& header);
Resume resume(uint32_t fileSize, uint32_t& cursor);
bool sideIsDone(uint32_t sideSize);
void reset();
NoteCard& card_;
NoteText text_;
std::string path_, sidePath_;
std::vector<Piece> pieces_; // without the window while it's loaded
std::vector<Piece> loaded_; // what the window was read from
size_t win_ = 0; // the window sits before pieces_[win_]
bool windowLoaded_ = false;
uint32_t before_ = 0, after_ = 0;
uint32_t loadedRevision_ = 0, savedRevision_ = 0;
size_t droppedAtLoad_ = 0, newlinesAtLoad_ = 0;
bool flushedSinceSave_ = false, sidePending_ = false;
uint32_t sideSize_ = 0; // 0: no side file
struct {
bool valid = false;
uint32_t at = 0, len = 0, revision = 0;
} windowSaved_; // the window as the side file already has it
struct {
bool active = false;
int stage = 0; // 0: pieces before, 1: the window, 2: pieces after
size_t index = 0;
uint32_t offset = 0, done = 0, total = 0, wrote = 0, wroteBefore = 0, wroteAfter = 0, revision = 0;
bool carry = false; // a CR at the end of a block, waiting to see what follows
std::vector<uint8_t> block;
} rw_;
};
} // namespace roro::notes
lib/notes/src/note_text.cpp
+17 -4
View File
@@ -16,11 +16,24 @@ NoteText::NoteText(int cols, int rows, std::string&& text) : cols_(std::max(1, c
text_.reserve(kMaxBytes);
}
void NoteText::dropCarriageReturns() {
size_t kept = 0;
for (size_t i = 0; i < text_.size(); i++)
if (!(text_[i] == '\r' && i + 1 < text_.size() && text_[i + 1] == '\n')) text_[kept++] = text_[i];
size_t NoteText::dropCarriageReturns(size_t* follow) {
size_t kept = 0, size = text_.size(), place = follow ? *follow : 0;
for (size_t i = 0; i < size; i++) {
if (follow && i == place) *follow = kept;
if (!(text_[i] == '\r' && i + 1 < size && text_[i + 1] == '\n')) text_[kept++] = text_[i];
}
if (follow && place >= size) *follow = kept;
text_.resize(kept);
return size - kept;
}
size_t NoteText::refilled(size_t cursor, int row) {
size_t dropped = dropCarriageReturns(&cursor);
cursor_ = std::min(cursor, text_.size());
while (cursor_ > 0 && cursor_ < text_.size() && continuation(text_[cursor_])) cursor_--;
top_ = lineOf(cursor_);
for (int i = 0; i < row && top_ > 0; i++) top_ = lineOf(top_ - 1);
return dropped;
}
bool NoteText::setText(const std::string& text) {
lib/notes/src/note_text.h
+13 -3
View File
@@ -7,8 +7,9 @@
namespace roro::notes {
// The text of a note while it's edited (F1, Q144, Q145): UTF-8 held whole in memory, a cursor, and
// the part of it on screen. Lines wrap at spaces, `cols` characters wide; a line owns the space or
// The text of a note while it's edited (F1, Q144, Q145): UTF-8 held in memory, a cursor, and the
// part of it on screen. Up to 16 KB: a longer note is edited through NoteDocument (note_document.h),
// which keeps this as its window on the file. Lines wrap at spaces, `cols` characters wide; a line owns the space or
// the newline it ends with, so every byte of the text belongs to exactly one line. No index of
// lines is kept (a note of newlines alone would need twice its size): where a line starts is
// worked out from the start of its paragraph, which is never far.
@@ -27,8 +28,17 @@ class NoteText {
bool setText(const std::string& text);
const std::string& text() const { return text_; }
size_t cursor() const { return cursor_; }
size_t size() const { return text_.size(); }
uint32_t revision() const { return revision_; } // changes with every edit: is it saved?
// For a window on a longer text (issue #47): the caller refills the buffer, then says where
// the cursor is in it and which row of the screen it should be on. CRLF becomes LF as in
// setText; returns how many CRs went. The revision doesn't change: nothing was edited.
std::string& buffer() { return text_; }
size_t refilled(size_t cursor, int row);
size_t startOfLine(size_t pos) const { return lineOf(pos); }
size_t top() const { return top_; }
bool insert(uint32_t codePoint); // false: the note is full
bool insertText(const std::string& s); // all of it or nothing
void backspace();
@@ -61,7 +71,7 @@ class NoteText {
bool hasLineAfter(size_t start) const;
void moved(bool keepGoal = false);
void follow(); // scrolls so the cursor is on screen
void dropCarriageReturns();
size_t dropCarriageReturns(size_t* follow = nullptr); // how many; `follow` is a place in the text, kept on its character
int cols_, rows_;
std::string text_;
site/content/dev/debug/commands.md
+2 -2
View File
@@ -29,7 +29,7 @@ gnss quiet on|off pause the GNSS receiver while the LoRa radio listens (it cos
gnss status | gnss restart | gnss track start|stop | gnss nmea on|off | gnss send <sentence without $ and checksum>
crash the last crash: firmware, reason, task, backtrace
coredump erase forget the core dump in flash
key <name|char> press a key: up down left right select back home del tab space help, or one character
key <name|char> press a key: up down left right select back home del tab space help, or one character; ctrl- alt- shift- before it (key ctrl-down)
wifi status | wifi add <ssid><TAB><password>
wifi ip <ssid> dhcp | wifi ip <ssid> <address>/<prefix> [gateway] a Saved Network's IP setting
wifi dns <a> [b] | wifi dns always on|off | wifi ntp <a> [b] DNS and NTP servers
@@ -62,7 +62,7 @@ In **Safe Mode** (see [Crashes and Safe Mode](/dev/debug/crashes/)) only a few r
| Command | Effect |
|---|---|
| `burst` | Publishes 5 Notifications at once |
| `key up\|down\|left\|right\|select\|back\|home\|del\|tab\|space\|help`, or `key <char>` | Injects a key press (`help` is Fn+h: the keys of the screen that is showing) |
| `key up\|down\|left\|right\|select\|back\|home\|del\|tab\|space\|help`, or `key <char>` | Injects a key press (`help` is Fn+h: the keys of the screen that is showing). `ctrl-`, `alt-` and `shift-` before it hold that key: `key ctrl-down`, `key alt-up`, `key ctrl-b` |
| `sound on` / `sound off` | Toggles the Sound setting (beep + LED) |
| `short` / `normal` | Screen timeouts 5 s / 10 s, or 30 s / 60 s |
| `wifi add <ssid><TAB><password>` | Adds a Saved Network (so credentials stay out of the repo) |
site/content/dev/debug/drive-the-ui.md
+1 -1
View File
@@ -20,7 +20,7 @@ Two things to know before you use them:
1. **A name `key` does not know is `select`.** `key sleect` presses Enter. A single character is typed as that character; anything else that is not a known name is treated as Enter. Check what you type.
2. **A key that wakes a dark screen only wakes it.** The device's power policy swallows the key press that turns the screen back on, as it does for the real keyboard: the first `key` after the screen went off does nothing else. Send `key back` (harmless) first, or keep the screen on with the `normal` and `short` commands below.
`Fn` combinations, modifiers and the compose key have no command: the arrows are `key up|down|left|right`, and `key back` is the back key (`` ` `` on the device). Text is typed one character at a time.
**Ctrl, Alt and Shift** go before the name: `key ctrl-down`, `key alt-up`, `key ctrl-b`, `key shift-alt-down`. `Fn` combinations and the compose key have no command: the arrows are `key up|down|left|right` (what `Fn` with `;` `.` `,` `/` gives on the device), and `key back` is the back key (`` ` `` on the device). Text is typed one character at a time.
**`key help` opens the help panel** (<kbd>Fn</kbd>+<kbd>h</kbd> on the device): the keys of the screen that is showing. A screenshot of it is the quickest way to learn what a screen accepts, and it is how every screen's list was checked. Any key but the arrows closes it.
site/content/dev/milestones/f1.md
+58 -1
View File
@@ -106,7 +106,7 @@ Plain text notes on the SD card, written on the device. Q30 settled the base: `.
| Q141 | A **Notes** App in the Launcher. One row per note: its first line as the title, then the date. Newest first; `s` switches to by name. `n` new, Enter opens, `d` deletes after a confirmation, `r` renames the file. |
| Q142 | A new note's file name is never typed: it comes from the first line when the note is first saved (`shopping-list.txt`), or `note-20261006-0919.txt` if that line is empty. It doesn't change afterwards unless the note is renamed. |
| Q143 | **Autosave, no "discard changes?" prompt:** five seconds after the last key, on leaving the note or the App, and when the screen turns off. A save writes a temporary file and renames it over the note, so a power cut loses the last few seconds at most. A temporary file left behind is offered back at the next open. |
| Q144 | The whole note is in memory while it's edited, up to **16 KB**. A bigger text file opens read-only in the Storage App's viewer. The App refuses to open below the memory floors (Q86). **Editing files of any size must come in a later release: issue #47.** |
| Q144 | The whole note is in memory while it's edited, up to **16 KB**. A bigger text file opens read-only in the Storage App's viewer. The App refuses to open below the memory floors (Q86). *(Lifted by issue #47: see "Notes of any size" below.)* **Editing files of any size must come in a later release: issue #47.** |
| Q145 | The editor wraps at spaces, 38 columns by 8 rows, with a line for the name and the state. Enter is a new line, Del deletes backwards, Fn+arrows move (the Text Entry rule), Ctrl+A and Ctrl+E go to the start and the end of the line, Tab types two spaces, Back saves and returns. The Compose Key works as elsewhere. |
| Q146 | The Storage App's text viewer gets `e`: edit this file with the same editor, for a text file up to 16 KB that isn't read-only. That lifts Q135 without Apps opening each other (#43 stays). |
| Q147 | The list is flat: the files directly in `/notes`. Sub-folders are reached through the Storage App. |
@@ -167,3 +167,60 @@ Test notes were made in `/notes` and removed afterwards; the folder is left, emp
**Not checked:** accents through the Compose Key and Ctrl+A / Ctrl+E (the remote `key` command can't send them; the model's tests cover both), the power button's save (it needs a hand on the device), a missing card, and how typing feels on the keyboard itself.
**One slip during the checks:** a key sequence sent right after a restart opened IRC instead of Notes, and the test letters went into IRC's input line. Nothing was sent: the line was cleared and the App left. IRC connected to Libera as it does when opened.
## Notes of any size (issue #47)
Q144 held the whole note in memory and stopped at 16 KB, for the first version only. This lifts it: the editor opens a text file whatever its size.
### Decisions (design round 2026-10-07)
| # | Decision |
|---|---|
| Q223 | **The note is the file on the card plus one window in memory.** The window is the `NoteText` of before, up to 16 KB around the cursor; the rest is a list of pieces: runs of the file, and runs of a side file. The cursor leaving the window writes it to the side file if it was changed, and loads the next. Typing never fills a note: a full window is written away and loaded smaller. |
| Q224 | **The five-second save:** up to 64 KB it rewrites the file, as before (about 150 ms). Above, it appends the window and the list of pieces to `<note>.edit`: 8 KB or so, whatever the note's size. "saved" means "on the card" either way. |
| Q225 | **The file itself is rewritten on leaving the note** (Back, Home, another App), with a progress bar. The screen turning off and the device powering off write the side file only: powering off never waits. |
| Q226 | **After a power cut, opening the note picks the edit up** where it was last saved, without a question, and says so. Until then the file has the old text for anything else that reads it. |
| Q227 | If the file was changed elsewhere meanwhile, the side file no longer fits it: it is **kept as `<note>.edit.lost`** and the editor says so. Typed text is never deleted without a word. |
| Q228 | **No limit but the card:** a note over 16 KB needs room for a second copy to be opened for editing. No warning for a big file; the progress bar on leaving tells the cost. |
| Q229 | A side file over 1 MB, or a list of over 256 pieces, makes the next save a rewrite. |
| Q230 | **CRLF becomes LF** (Q148) for a long file too: in the window as it is read, and in the rest of the file as the rewrite streams it, so a saved file is never of both kinds. |
| Q231 | **One path.** A 16 KB note is the case with no pieces: there is no second editor for small notes. |
| Q232 | Notes, and `e` in the Storage App's viewer, which no longer says "Too big to edit". |
### As built
- **`NoteDocument`** (`lib/notes/src/note_document.h`, host-tested against a card in memory) is the list of pieces, the window's moves, the side file and the recovery. `NoteText` is unchanged but for being refilled.
- **The window moves** when the cursor comes within 2 KB of an end of it that isn't an end of the note: it is then 4 KB on each side of the cursor. It starts where a line starts on screen whenever that can be known (after a newline, or where the window before had a line start), so the same text wraps the same from one window to the next, and never in the middle of a character. The cursor keeps its row on screen.
- **Looking writes nothing:** a window that wasn't changed goes back as the pieces it was read from.
- **The side file** starts with a line of text, the note's size and checksums of its first and last kilobyte, which is how a file changed elsewhere is told. After that, text that left a window, and snapshots of the list of pieces, each with its checksum. The newest snapshot that checks out is the note as last saved; anything after it is ignored.
- **The rewrite** streams the pieces and the window into `<note>.tmp`, checks its size, then writes a mark at the end of the side file: from that mark on, the rewrite counts as done, and opening the note finishes it whatever was cut (remove the old file, rename, remove the side file). Before the mark, the note and its side file are still the truth and the temporary file is dropped.
- **On the device** the card is reached through an adapter that keeps the file being read and the file being appended to open between calls; every call runs on the storage task while the main loop waits. The rewrite runs in steps of 64 KB with the progress drawn between them.
- **The Notes list** doesn't show `.edit` and `.edit.lost` files, and a note's side file is deleted and renamed with it.
- **Ctrl with Fn+Up and Fn+Down** go to the start and the end of the note.
- **`key ctrl-down`**: the consoles' `key` command takes `ctrl-`, `alt-` and `shift-`, which these checks needed. It also lets the checks S1 couldn't make (Ctrl+b, the Alt scroll) be made.
- **Cost:** 15 KB of flash. Memory with a note open is what it was: 17.5 KB, for 62 bytes or for 1.2 MB.
### Host tests (15, `test/test_note_document`)
A walk down 3,000 lines and back up through the windows; start and end; an edit in the middle rewritten into the file; 48 KB typed into a new note; a journal picked up after a cut; **a cut at every 997th byte of a sequence of two saves and a rewrite**, after which the note is always one of the three texts it should be, what was reported saved is there, and no stray file is left; a file changed elsewhere; CRLF; windows on text with no space and no newline, made of 2, 3 and 4-byte characters; a full card; and **36,000 random keys** (typing, deleting, moving, jumping, saving, power cuts) on six notes of 30 to 130 KB, compared with a plain string after every key.
### Checks on the device (2026-10-07, driven over the Debug Console)
Test notes were copied to `/notes` and removed afterwards; the note that was already there was not touched.
| Check | Result |
|---|---|
| A 36 KB note | Opens (it was refused before). Two letters at the top, 400 lines down across the windows, four more: the file fetched back is exactly that, and no other file is left |
| A 1.2 MB note | Opens at once. Free memory 104.2 KB before, 86.7 KB with it open |
| Its five-second save | `zz-big.txt.edit`, 4 KB; the note's file untouched |
| Ctrl with Down, Ctrl with Up | The end and the start, as fast as any key |
| A restart with unsaved keys | "Your unsaved changes are back", the cursor where it was, the unsaved keys gone and nothing else |
| Leaving it | The progress bar, then one file: **1.2 MB rewritten in 2.6 s**. Fetched back: the original with what was typed at both ends, byte for byte |
| A restart in the middle of that rewrite | The note, its side file and an empty `.tmp` remain; opening picks the edit up, leaving rewrites it, the result is right |
| A new note | No file until typed in, then `zz-test-note.txt` from its first line |
| `e` in the Storage App on the 1.2 MB file | The same editor; edited and rewritten |
| The Notes list | Side files are not listed as notes |
**Not checked:** the power button's path (side file only), the screen turning off, a card pulled while editing, and memory with IRC connected, which wasn't connected for these checks: the editor's own use hasn't changed, and it still refuses to open without a free block of 24 KB. The real keyboard's Ctrl with Fn and the arrows. A file of tens of megabytes. Renaming or deleting a note from the Storage App leaves its side file behind.
**Measured against what was said:** the first build rewrote 1.2 MB in 3.5 to 4.5 s, with 2 KB blocks. With 4 KB blocks it is 2.6 s, about 450 KB a second, which is what the card gives a plain copy.
site/content/faq.md
+1 -1
View File
@@ -65,7 +65,7 @@ For the radio, GNSS position, Wi-Fi tools, IRC chat and Gemini browsing, no. For
## How big can a note be?
Up to 16 KB while it is edited. A larger text file opens read-only in the [Storage App](/guide/storage/). Editing a text file of any size is planned.
Any size the card has room for. The editor only keeps the part around the cursor in memory, so a megabyte of text opens at once. A long note is rewritten when you leave it, which takes about a second for each 450 KB. See [Long notes](/guide/notes/#long-notes).
## Why can't I rename or delete some folders?
site/content/guide/notes.md
+14 -2
View File
@@ -22,7 +22,7 @@ Each note shows its first line and its date, newest first. <kbd>s</kbd> switches
## The editor
Type. <kbd>Enter</kbd> starts a line and <kbd>Del</kbd> deletes backwards. <kbd>Fn</kbd> with the arrow keys moves the cursor through the wrapped text, <kbd>Ctrl</kbd>+<kbd>A</kbd> and <kbd>Ctrl</kbd>+<kbd>E</kbd> go to the start and the end of the line, <kbd>Tab</kbd> types two spaces, and the compose key gives accents as everywhere.
Type. <kbd>Enter</kbd> starts a line and <kbd>Del</kbd> deletes backwards. <kbd>Fn</kbd> with the arrow keys moves the cursor through the wrapped text, <kbd>Ctrl</kbd>+<kbd>A</kbd> and <kbd>Ctrl</kbd>+<kbd>E</kbd> go to the start and the end of the line, <kbd>Ctrl</kbd> with <kbd>Fn</kbd> and up or down to the start and the end of the note, <kbd>Tab</kbd> types two spaces, and the compose key gives accents as everywhere.
**There is no save key.** The note is written five seconds after your last key, when you press Back, when you leave the App, when the screen turns off and before the device powers off. The top line says `typing` or `saved`.
@@ -32,9 +32,21 @@ Each save writes a temporary file and then puts it in the note's place, so a pow
A new note has no file until you type something. The file is then named after its first line (`shopping-list.txt`), or `note-<date>-<time>.txt` if that line gives no usable name.
## Long notes
**A note can be any size.** The editor keeps the part around the cursor in memory and the rest on the card, so a file of a megabyte opens as fast as a short one and uses no more memory.
What changes with size is how it is saved:
- **Up to 64 KB**, every save rewrites the file, as above.
- **Above**, the five-second save writes only what you changed, to a file next to the note (`<note>.edit`). The note itself is rewritten **when you leave it**, with a progress bar: about a second for each 450 KB.
- **After a power cut**, or if the device was switched off with the note open, opening the note again brings your saved changes back, and says so. Until then the file itself still has the old text, if you look at it from a computer.
Saving a long note needs room on the card for a second copy of it. If the file was replaced by something else while its changes were waiting, they can't be applied: they are kept as `<note>.edit.lost` and the editor tells you.
## Limits
A note holds up to **16 KB** while it is edited. A bigger text file opens read-only in the [Storage App](/guide/storage/); editing a file of any size is planned. In Storage, <kbd>e</kbd> on a text file opens it in the same editor, anywhere on the card, unless the file is read-only. Notes are never offered for deletion by the clean-up.
In Storage, <kbd>e</kbd> on a text file opens it in the same editor, anywhere on the card, unless the file is read-only. Notes are never offered for deletion by the clean-up.
## The keys, as the device lists them
site/data/keys.toml
+2 -1
View File
@@ -291,7 +291,7 @@ rows = [
["; .", "a line up, down"],
[", /", "a page up, down"],
["t b", "the top, the end"],
["e", "edit it (up to 16 KB)"],
["e", "edit it"],
["Tab", "the file as hex, or back"],
]
@@ -360,6 +360,7 @@ rows = [
["Tab", "two spaces"],
["Fn ; . , /", "move the cursor"],
["Alt Fn ; .", "a page up, down"],
["Ctrl Fn ; .", "start, end of the note"],
["Ctrl a e", "start, end of the line"],
["opt ' e", "an accent: é"],
["`", "done: it saves by itself"],
src/apps/note_editor.cpp
+183 -78
View File
@@ -9,24 +9,108 @@
#include "cleanup_plan.h"
#include "file_list.h"
#include "file_names.h"
#include "platform/console.h"
#include "ui/fonts.h"
#include "ui/theme.h"
#include "ui/widgets.h"
namespace roro {
using notes::NoteDocument;
using notes::NoteText;
std::function<void(const std::string&, int)> NoteEditor::onProgress;
// The SD card for a NoteDocument. Every call is made on the storage task. The file being read and
// the file being appended to stay open between calls (a window is read in a few pieces, a rewrite
// appends some five hundred blocks to the megabyte), until done().
class NoteEditor::Card : public notes::NoteCard {
public:
explicit Card(StorageService& storage) : storage_(storage) {}
bool size(const std::string& path, uint32_t& size) override {
close(path);
File f = SD.open(path.c_str(), FILE_READ);
if (!f || f.isDirectory()) return false;
size = static_cast<uint32_t>(f.size());
f.close();
return true;
}
size_t read(const std::string& path, uint32_t at, uint8_t* into, size_t len) override {
if (appendPath_ == path) closeAppend(); // what was appended has to be there to read
if (readPath_ != path || !read_) {
closeRead();
read_ = SD.open(path.c_str(), FILE_READ);
if (!read_) return 0;
readPath_ = path;
}
if (!read_.seek(at)) return 0;
int n = read_.read(into, len);
return n > 0 ? static_cast<size_t>(n) : 0;
}
bool create(const std::string& path) override {
close(path);
File f = SD.open(path.c_str(), FILE_WRITE);
if (!f) return false;
f.close();
return true;
}
bool append(const std::string& path, const uint8_t* data, size_t len) override {
if (readPath_ == path) closeRead();
if (appendPath_ != path || !append_) {
closeAppend();
append_ = SD.open(path.c_str(), FILE_APPEND);
if (!append_) return false;
appendPath_ = path;
}
return append_.write(data, len) == len;
}
bool remove(const std::string& path) override {
close(path);
return SD.remove(path.c_str());
}
bool rename(const std::string& from, const std::string& to) override {
done();
return SD.rename(from.c_str(), to.c_str());
}
uint64_t freeBytes() override {
StorageState s = storage_.state();
return s.totalBytes > s.usedBytes ? s.totalBytes - s.usedBytes : 0;
}
void done() override {
closeRead();
closeAppend();
}
private:
void close(const std::string& path) {
if (readPath_ == path) closeRead();
if (appendPath_ == path) closeAppend();
}
void closeRead() {
if (read_) read_.close();
readPath_.clear();
}
void closeAppend() {
if (append_) append_.close();
appendPath_.clear();
}
StorageService& storage_;
File read_, append_;
std::string readPath_, appendPath_;
};
namespace {
constexpr uint32_t kMessageMs = 4000;
// One block the size of a full note, and something left: without it, nothing is opened. Free
// One block the size of the window, and something left: without it, nothing is opened. Free
// memory in total isn't the measure: with IRC connected the largest free block is about 31 KB.
constexpr size_t kRoomWanted = NoteText::kMaxBytes + 8 * 1024;
const char* const kNoRoom = "Not enough memory to edit: close IRC or a Gemini page";
// On the storage task. Reads a file of up to `limit` bytes into a string that has that capacity
// already; false if it can't be read whole.
// On the storage task. Reads a file of up to `limit` bytes into a string; false if it can't be
// read whole.
bool readWhole(const std::string& path, std::string& into, size_t limit) {
File f = SD.open(path.c_str(), FILE_READ);
if (!f) return false;
@@ -51,42 +135,43 @@ void NoteEditor::say(const std::string& text) {
std::string NoteEditor::open(const std::string& path) {
close();
if (ESP.getMaxAllocHeap() < kRoomWanted) return kNoRoom;
std::string body, left, why;
body.reserve(NoteText::kMaxBytes); // the note's own buffer from here on: read into, then handed over
card_ = std::make_shared<Card>(storage_);
doc_.reset(new NoteDocument(*card_, kCols, kRows));
std::string left, why, told;
bool ran = storage_.runAndWait([&]() {
File f = SD.open(path.c_str(), FILE_READ);
if (!f) {
why = "The card refused to open it";
return;
}
size_t size = f.size();
f.close();
if (size > NoteText::kMaxBytes) why = "Too big to edit: 16 KB at most";
else if (!readWhole(path, body, NoteText::kMaxBytes)) why = "The card refused to read it";
if (!why.empty()) return;
// A save that never finished: its temporary file is offered back (Q143), if there's the
// memory to look at it now. If not, it stays for the next time.
// A save of a small note that never finished: its temporary file is offered back (Q143),
// if there's the memory to look at it now. A bigger note's unfinished saves are in its
// side file, and the document picks them up by itself.
std::string tmp = path + ".tmp";
File t = SD.open(tmp.c_str(), FILE_READ);
if (!t) return;
size_t tmpSize = t.size();
t.close();
if (tmpSize > 0 && tmpSize <= NoteText::kMaxBytes && ESP.getMaxAllocHeap() < tmpSize + 8 * 1024) return;
left.reserve(tmpSize <= NoteText::kMaxBytes ? tmpSize : 0);
if (!readWhole(tmp, left, NoteText::kMaxBytes) || left == body || left.empty()) {
size_t tmpSize = t ? t.size() : 0;
bool hasTmp = static_cast<bool>(t);
if (t) t.close();
bool hasSide = SD.exists((path + ".edit").c_str());
if (hasTmp && !hasSide && tmpSize > 0 && tmpSize <= NoteText::kMaxBytes && ESP.getMaxAllocHeap() >= kRoomWanted + tmpSize) {
left.reserve(tmpSize);
if (!readWhole(tmp, left, NoteText::kMaxBytes)) std::string().swap(left);
}
why = doc_->open(path, &told);
if (!why.empty()) return;
if (hasTmp && !hasSide && (left.empty() || doc_->windowed() || left == doc_->text().text())) {
std::string().swap(left);
SD.remove(tmp.c_str());
}
});
if (!ran) return "No SD card";
if (!why.empty()) return why;
text_.reset(new NoteText(kCols, kRows, std::move(body)));
if (!ran) why = "No SD card";
if (!why.empty()) {
doc_.reset();
card_.reset();
return why;
}
path_ = path;
folder_ = files::parentOf(path);
savedRevision_ = text_->revision();
problem_.clear();
message_.clear();
givenUp_ = false;
lastKeyMs_ = millis();
if (!told.empty()) say(told);
if (!left.empty()) {
recovered_ = std::move(left);
ask_ = Ask::Recover;
@@ -98,36 +183,38 @@ std::string NoteEditor::open(const std::string& path) {
std::string NoteEditor::openNew(const std::string& folder) {
close();
if (ESP.getMaxAllocHeap() < kRoomWanted) return kNoRoom;
text_.reset(new NoteText(kCols, kRows));
card_ = std::make_shared<Card>(storage_);
doc_.reset(new NoteDocument(*card_, kCols, kRows));
path_.clear();
folder_ = folder;
savedRevision_ = text_->revision();
problem_.clear();
message_.clear();
givenUp_ = false;
lastKeyMs_ = millis();
return "";
}
// Leaving rewrites the file, however long the note (Q225). Not when the device is powering off:
// then a long note's edits go to its side file, which is quick, and are picked up the next time.
void NoteEditor::close() {
if (dirty()) save();
text_.reset();
if (doc_ && !givenUp_ && owed()) save(!power_.poweringOff());
doc_.reset();
card_.reset();
dialog_.reset();
ask_ = Ask::None;
std::string().swap(recovered_);
}
// On the main loop, waiting for the storage task: no second copy of the note is made, and at
// 16 KB the wait is a fraction of a second, when nobody has typed for five.
bool NoteEditor::save() {
if (!text_) return true;
// On the main loop, waiting for the storage task: no second copy of the text is made. A note of
// up to 64 KB is rewritten in a fraction of a second, when nobody has typed for five; a longer
// one takes a second for each 400 KB or so, and shows how far it is.
bool NoteEditor::save(bool whole) {
if (!doc_) return true;
lastTryMs_ = millis();
const std::string& body = text_->text();
if (path_.empty() && body.empty()) { // a new note nothing was typed in: no file
savedRevision_ = text_->revision();
return true;
}
if (path_.empty() && doc_->size() == 0) return true; // a new note nothing was typed in: no file
std::string path = path_, why;
if (path.empty()) {
bool fresh = path_.empty();
if (fresh) {
char stamp[20] = "new";
int64_t now = clock_.utcNow();
if (now >= 0) {
@@ -137,10 +224,15 @@ bool NoteEditor::save() {
std::snprintf(stamp, sizeof stamp, "%04d%02d%02d-%02d%02d", local.tm_year + 1900, local.tm_mon + 1, local.tm_mday, local.tm_hour,
local.tm_min);
}
path = files::joinPath(folder_, notes::nameFromFirstLine(text_->firstLine(), stamp) + ".txt");
path = files::joinPath(folder_, notes::nameFromFirstLine(doc_->text().firstLine(), stamp) + ".txt");
}
bool fresh = path_.empty();
bool rewrite = whole || fresh || doc_->wantsRewrite();
int percent = 0;
bool ran = storage_.runAndWait([&]() {
if (!rewrite) {
doc_->journal(why);
return;
}
if (!SD.exists(folder_.c_str()) && !SD.mkdir(folder_.c_str())) {
why = "the card refused to make " + folder_;
return;
@@ -148,48 +240,59 @@ bool NoteEditor::save() {
if (fresh) { // a name nothing has yet: "list (2).txt"
std::string name = files::baseName(path);
for (int n = 2; n < 100 && SD.exists(path.c_str()); n++) path = files::joinPath(folder_, files::copyName(name, n));
doc_->setPath(path);
}
std::string tmp = path + ".tmp";
File f = SD.open(tmp.c_str(), FILE_WRITE);
if (!f) {
why = "the card refused to open a file";
return;
}
size_t wrote = body.empty() ? 0 : f.write(reinterpret_cast<const uint8_t*>(body.data()), body.size());
f.close();
File check = SD.open(tmp.c_str(), FILE_READ);
bool whole = wrote == body.size() && check && check.size() == body.size();
if (check) check.close();
if (!whole) {
SD.remove(tmp.c_str());
why = "the card refused a write";
return;
}
// FAT can't rename onto a file. Between these two lines only the temporary file exists:
// the Notes list puts such a file back under its name.
if (SD.exists(path.c_str())) SD.remove(path.c_str());
if (!SD.rename(tmp.c_str(), path.c_str())) why = "the card refused to rename the file";
if (doc_->rewriteStart(why)) percent = doc_->rewriteStep(why);
if (fresh && !why.empty()) doc_->setPath("");
});
bool show = doc_->size() > NoteDocument::kWholeLimit;
uint32_t started = millis();
while (ran && rewrite && why.empty() && percent >= 0 && percent < 100) {
if (show && onProgress) onProgress(files::baseName(path), percent);
ran = storage_.runAndWait([&]() { percent = doc_->rewriteStep(why); });
}
if (!ran) why = "no SD card";
if (!why.empty()) {
if (problem_ != why) say("Not saved: " + why);
problem_ = why;
redraw_ = true;
return false;
}
if (rewrite && show) console.printf("notes: rewrote %s, %u bytes in %.1f s\n", path.c_str(), (unsigned)doc_->size(), (millis() - started) / 1000.0);
path_ = path;
problem_.clear();
savedRevision_ = text_->revision();
redraw_ = true;
return true;
}
void NoteEditor::settle() {
if (!doc_ || !doc_->wantsMove()) return;
// A window that leaves memory needs a file to belong to: a new note is saved first.
if (path_.empty() && !save(true)) return;
std::string why;
bool ran = storage_.runAndWait([&]() { doc_->move(why); });
if (!ran) why = "no SD card";
if (!why.empty() && problem_ != why) say("The card: " + why);
if (!why.empty()) problem_ = why;
}
bool NoteEditor::jump(bool toEnd) {
std::string why;
uint32_t to = toEnd ? doc_->size() : 0;
bool ran = storage_.runAndWait([&]() { doc_->jump(to, why); });
if (!ran) why = "no SD card";
if (!why.empty()) say("The card: " + why);
return why.empty();
}
void NoteEditor::help(std::vector<KeyHelp>& out) const {
if (dialog_) return keys::add(out, keys::kDialog);
keys::add(out, keys::kNotesEditor);
}
bool NoteEditor::onKey(const KeyEvent& e) {
if (!text_) return false;
if (!doc_) return false;
NoteText* text_ = &doc_->text();
redraw_ = true;
if (dialog_) {
dialog_->onKey(e);
@@ -210,7 +313,7 @@ bool NoteEditor::onKey(const KeyEvent& e) {
return true;
}
if (asked == Ask::LeaveUnsaved && result == 1) {
savedRevision_ = text_->revision(); // given up on
givenUp_ = true;
return false;
}
return true;
@@ -225,37 +328,38 @@ bool NoteEditor::onKey(const KeyEvent& e) {
else if (lower == 'e') text_->lineEnd();
break;
}
if (!text_->insert(e.ch)) say("This note is full: 16 KB");
if (!text_->insert(e.ch)) say("Can't type: " + problem_);
break;
}
case Key::Select:
if (!text_->insert('\n')) say("This note is full: 16 KB");
if (!text_->insert('\n')) say("Can't type: " + problem_);
break;
case Key::Tab:
if (!text_->insertText(" ")) say("This note is full: 16 KB");
if (!text_->insertText(" ")) say("Can't type: " + problem_);
break;
case Key::Delete: text_->backspace(); break;
case Key::Left: text_->left(); break;
case Key::Right: text_->right(); break;
case Key::Up: page ? text_->pageUp() : text_->up(); break;
case Key::Down: page ? text_->pageDown() : text_->down(); break;
case Key::Up: e.ctrl ? void(jump(false)) : page ? text_->pageUp() : text_->up(); break;
case Key::Down: e.ctrl ? void(jump(true)) : page ? text_->pageDown() : text_->down(); break;
case Key::Back:
if (!dirty() || save()) return false;
if (!owed() || save(true)) return false;
ask_ = Ask::LeaveUnsaved;
dialog_.reset(new DialogModel({"Stay", "Leave"}));
break;
default: break;
}
settle();
return true;
}
bool NoteEditor::update(uint32_t) {
if (!text_) return false;
if (!doc_) return false;
uint32_t now = millis();
// A save that failed is tried again every five seconds, not at every pass.
if (dirty() && !dialog_ && now - lastTryMs_ >= kSaveAfterMs &&
(now - lastKeyMs_ >= kSaveAfterMs || power_.screen() == ScreenState::Off))
save();
save(false);
if (!message_.empty() && now - messageMs_ >= kMessageMs) {
message_.clear();
redraw_ = true;
@@ -266,7 +370,8 @@ bool NoteEditor::update(uint32_t) {
}
void NoteEditor::draw(Canvas& c) {
if (!text_) return;
if (!doc_) return;
NoteText* text_ = &doc_->text();
const auto& area = theme::kContent;
c.setTextDatum(top_left);
@@ -275,7 +380,7 @@ void NoteEditor::draw(Canvas& c) {
c.setTextColor(theme::kMuted);
std::string name = path_.empty() ? "New note" : files::fitName(files::baseName(path_), 26);
c.drawString(name.c_str(), 4, area.y + 1);
std::string state = formatBytes(text_->text().size()) + (dirty() ? (problem_.empty() ? ", typing" : ", NOT SAVED") : ", saved");
std::string state = formatBytes(doc_->size()) + (dirty() ? (problem_.empty() ? ", typing" : ", NOT SAVED") : ", saved");
if (path_.empty() && !dirty()) state = "empty";
c.setTextDatum(top_right);
c.setTextColor(dirty() && !problem_.empty() ? theme::kWarning : theme::kMuted);
@@ -288,9 +393,9 @@ void NoteEditor::draw(Canvas& c) {
c.setTextColor(theme::kText);
for (size_t i = 0; i < rows.size(); i++) c.drawString(rows[i].c_str(), 4, top + 1 + static_cast<int>(i) * theme::kLineHeight);
c.fillRect(3 + text_->cursorCol() * 6, top + text_->cursorRow() * theme::kLineHeight, 1, theme::kLineHeight, theme::kAccent);
if (text_->text().size() > static_cast<size_t>(kCols * kRows)) { // more than a screen: where we are in it
if (doc_->size() > static_cast<uint32_t>(kCols * kRows)) { // more than a screen: where we are in it
int h = kRows * theme::kLineHeight, barH = 12;
c.fillRect(area.w - 2, top + (h - barH) * text_->percent() / 100, 2, barH, theme::kMuted);
c.fillRect(area.w - 2, top + (h - barH) * doc_->percent() / 100, 2, barH, theme::kMuted);
}
c.setFont(&fonts::small);
src/apps/note_editor.h
+26 -10
View File
@@ -7,7 +7,9 @@
#include "dialog_model.h"
#include "key_help.h"
#include "key_event.h"
#include "note_text.h"
#include <functional>
#include "note_document.h"
#include "services/clock_service.h"
#include "services/power_service.h"
#include "services/storage_service.h"
@@ -15,10 +17,12 @@
namespace roro {
// Edits one text file of up to 16 KB (F1, Q143-Q145): the Notes App's editor, and the Storage
// App's for `e`. It saves by itself: five seconds after the last key, when the screen turns off,
// and on close. A save writes `<file>.tmp`, then puts it in the note's place, so the note on the
// card is always a whole one.
// Edits one text file of any size (F1, Q143-Q145; issue #47, Q223-Q232): the Notes App's editor,
// and the Storage App's for `e`. The text is a notes::NoteDocument: a window of the file in
// memory, the rest on the card. It saves by itself: five seconds after the last key, when the
// screen turns off, and on close. Up to 64 KB a save writes `<file>.tmp` and puts it in the
// note's place; a bigger note's saves go to `<file>.edit`, and the file is rewritten on leaving.
// Either way the note on the card is always a whole one.
class NoteEditor {
public:
static constexpr int kCols = 38, kRows = 8;
@@ -30,9 +34,13 @@ class NoteEditor {
std::string open(const std::string& path); // "" or why it can't be edited
std::string openNew(const std::string& folder); // the same; no file until there's something to save (Q142)
void close(); // saves what isn't yet
bool isOpen() const { return static_cast<bool>(text_); }
bool isOpen() const { return static_cast<bool>(doc_); }
const std::string& path() const { return path_; } // "" for a new note nothing was typed in
// A long rewrite shows how far it is: the editor is waiting for the card meanwhile, so the
// screen is drawn from here (set once, in main).
static std::function<void(const std::string& name, int percent)> onProgress;
bool onKey(const KeyEvent& e); // false: done, and saved
void help(std::vector<KeyHelp>& out) const;
bool update(uint32_t nowMs); // true: draw again
@@ -41,16 +49,24 @@ class NoteEditor {
private:
enum class Ask { None, Recover, LeaveUnsaved };
bool dirty() const { return text_ && text_->revision() != savedRevision_; }
bool save(); // true if the card has it now (or there was nothing to save)
class Card;
bool dirty() const { return doc_ && doc_->dirty(); }
bool owed() const { return doc_ && (doc_->dirty() || doc_->filePending()); } // the file isn't the note yet
// True if the card has it now (or there was nothing to save). `whole`: the file itself is
// rewritten; otherwise a note over 64 KB only gets its side file written, which is quick.
bool save(bool whole);
void settle(); // after a key: moves the window if the cursor is near an end of it
bool jump(bool toEnd);
void say(const std::string& text);
StorageService& storage_;
ClockService& clock_;
PowerService& power_;
std::unique_ptr<notes::NoteText> text_;
std::shared_ptr<Card> card_;
std::unique_ptr<notes::NoteDocument> doc_;
std::string path_, folder_;
uint32_t savedRevision_ = 0, lastKeyMs_ = 0, lastTryMs_ = 0;
uint32_t lastKeyMs_ = 0, lastTryMs_ = 0;
bool givenUp_ = false; // "Leave" after a save that failed
std::string recovered_; // what a temporary file left behind holds, until the user has chosen
Ask ask_ = Ask::None;
std::unique_ptr<DialogModel> dialog_;
src/apps/notes_app.cpp
+10 -1
View File
@@ -25,6 +25,10 @@ bool endsWith(const std::string& s, const char* tail) {
size_t n = std::strlen(tail);
return s.size() >= n && s.compare(s.size() - n, n, tail) == 0;
}
// Beside a note, and not one: a save cut short (.tmp), a long note's unsaved edits (.edit), and
// edits that no longer fit their file, kept for whoever wants to look (.edit.lost).
bool notANote(const std::string& name) { return endsWith(name, ".tmp") || endsWith(name, ".edit") || endsWith(name, ".edit.lost"); }
} // namespace
void NotesApp::onEnter() {
@@ -125,6 +129,7 @@ void NotesApp::onFinished(const FileOps::Status& s) {
if (list_.find(name.substr(0, name.size() - 4)) < 0) orphans.push_back(name);
continue;
}
if (notANote(name)) continue;
notes_.push_back(static_cast<uint16_t>(i));
}
if (!orphans.empty() && !mended_) {
@@ -202,6 +207,8 @@ bool NotesApp::onKey(const KeyEvent& e) {
selectIndex_ = rows_.selected();
selectAfter_.clear();
std::string why = ops_.remove(target_, false);
std::string side = target_ + ".edit"; // a long note's unsaved edits go with it (issue #47)
if (why.empty()) storage_.runJob([side]() { SD.remove(side.c_str()); });
if (why.empty()) wait_ = Wait::Work;
else say(why);
}
@@ -225,6 +232,8 @@ bool NotesApp::onNameKey(const KeyEvent& e) {
if (why.empty() && name != baseName(target_)) {
why = ops_.move(target_, false, joinPath(kFolder, name));
if (why.empty()) {
std::string side = target_ + ".edit", sideTo = joinPath(kFolder, name) + ".edit";
storage_.runJob([side, sideTo]() { SD.rename(side.c_str(), sideTo.c_str()); });
wait_ = Wait::Work;
selectAfter_ = name;
}
@@ -268,7 +277,7 @@ bool NotesApp::onListKey(const KeyEvent& e) {
list_.sort(sort_);
notes_.clear();
for (size_t i = 0; i < list_.count(); i++)
if (!list_.folder(i) && !endsWith(list_.name(i), ".tmp")) notes_.push_back(static_cast<uint16_t>(i));
if (!list_.folder(i) && !notANote(list_.name(i))) notes_.push_back(static_cast<uint16_t>(i));
for (size_t i = 0; i < notes_.size(); i++)
if (keep == list_.name(notes_[i])) rows_.select(static_cast<int>(i));
titlesFrom_ = -1;
src/apps/storage_app.cpp
-1
View File
@@ -322,7 +322,6 @@ bool StorageApp::onKey(const KeyEvent& e) {
if (view_ == View::Viewer && viewer_.showingText() && e.key == Key::Char && (e.ch == 'e' || e.ch == 'E')) {
// Edit it (Q146), if the rules and its size allow.
std::string path = viewer_.path(), why = ops_.whyReadOnly(path, false);
if (why.empty() && viewer_.size() > notes::NoteText::kMaxBytes) why = "Too big to edit: 16 KB at most";
if (why.empty()) {
viewer_.close();
why = noteEditor_.open(path);
src/main.cpp
+15 -1
View File
@@ -10,6 +10,7 @@
#include "app_manager.h"
#include "png_rgb332.h"
#include "apps/demo_app.h"
#include "apps/note_editor.h"
#include "apps/shell_app.h"
#include "apps/gemini_app.h"
#include "apps/gnss_app.h"
@@ -220,6 +221,8 @@ void setup() {
apps->registerApp({"shell", "Shell", false, new ShellApp(shellRun, shellProbe, shellList, shellCount, helpText(), *apps)});
// Leaving the foreground App makes it save: a note being typed, when the device is powered off.
power->beforePowerOff = []() { apps->home(); };
// A long note being rewritten (issue #47): the editor waits for the card, so it draws from there.
NoteEditor::onProgress = [](const std::string& name, int percent) { screen.renderUpdate("Saving", name, percent); };
apps->registerApp({"system", "System", false,
new SystemApp(*wifi, *battery, *storageService, *radioService, *gnssService, nvs)});
apps->registerApp({"settings", "Settings", false,
@@ -653,7 +656,7 @@ static const char* const kHelp =
"gnss status | gnss restart | gnss track start|stop | gnss nmea on|off | gnss send <sentence without $ and checksum>\n"
"crash the last crash: firmware, reason, task, backtrace\n"
"coredump erase forget the core dump in flash\n"
"key <name|char> press a key: up down left right select back home del tab space help, or one character\n"
"key <name|char> press a key: up down left right select back home del tab space help, or one character; ctrl- alt- shift- before it (key ctrl-down)\n"
"wifi status | wifi add <ssid><TAB><password>\n"
"wifi ip <ssid> dhcp | wifi ip <ssid> <address>/<prefix> [gateway] a Saved Network's IP setting\n"
"wifi dns <a> [b] | wifi dns always on|off | wifi ntp <a> [b] DNS and NTP servers\n"
@@ -1026,12 +1029,23 @@ static void runCommand(String line, bool fromSerial = false) {
bus.publish(Event::withText(EventType::Notification, ("Burst " + String(i)).c_str(), 0));
if (line.startsWith("key ")) { // key up|down|left|right|select|back|home|del|tab
String k = line.substring(4);
// ctrl- alt- shift- before the name, in any order: `key ctrl-down`, `key alt-up`, `key ctrl-b`.
bool ctrl = false, alt = false, shift = false;
for (bool more = true; more;) {
more = false;
if (k.startsWith("ctrl-") && k.length() > 5) ctrl = more = true, k = k.substring(5);
if (k.startsWith("alt-") && k.length() > 4) alt = more = true, k = k.substring(4);
if (k.startsWith("shift-") && k.length() > 6) shift = more = true, k = k.substring(6);
}
Key key = k == "up" ? Key::Up : k == "down" ? Key::Down : k == "left" ? Key::Left
: k == "right" ? Key::Right : k == "back" ? Key::Back : k == "home" ? Key::Home
: k == "del" ? Key::Delete : k == "tab" ? Key::Tab : k == "help" ? Key::Help : Key::Select;
KeyEvent ev = k == "space" ? KeyEvent::character(' ')
: k.length() == 1 && k[0] > ' ' ? KeyEvent::character((unsigned char)k[0])
: KeyEvent::of(key);
ev.ctrl = ctrl;
ev.alt = alt;
ev.shift = shift;
if (!power->onKey(millis())) apps->handleKey(ev);
}
if (line.startsWith("wifi add ")) { // wifi add <ssid>\t<password>: credentials never touch the repo
src/services/power_service.cpp
+1
View File
@@ -42,6 +42,7 @@ void PowerService::applyScreen(ScreenState state) {
}
void PowerService::powerOff() {
poweringOff_ = true;
if (beforePowerOff) beforePowerOff();
auto& display = M5Cardputer.Display;
display.wakeup();
src/services/power_service.h
+2
View File
@@ -28,6 +28,7 @@ class PowerService : public Service {
// Run before the device powers off: the last chance to put on the card what's only in memory.
std::function<void()> beforePowerOff;
bool poweringOff() const { return poweringOff_; } // from beforePowerOff on: no time for long work
private:
void applyScreen(ScreenState state);
@@ -38,6 +39,7 @@ class PowerService : public Service {
PowerButton button_{2000};
ScreenState applied_ = ScreenState::On;
int appliedBrightness_ = -1;
bool poweringOff_ = false;
};
} // namespace roro
test/test_note_document/test_note_document.cpp
+521
View File
@@ -0,0 +1,521 @@
#include <unity.h>
#include <cstdint>
#include <map>
#include <random>
#include <string>
#include "note_document.h"
using namespace roro::notes;
void setUp() {}
void tearDown() {}
namespace {
// A card in memory. `budget`: how many more bytes it takes before the power is cut: the write
// that crosses it is left half done, and nothing works after it.
struct FakeCard : NoteCard {
std::map<std::string, std::string> files;
int64_t budget = -1;
bool dead = false;
uint64_t free = 1ull << 30;
int reads = 0;
bool spend(size_t n) {
if (dead) return false;
if (budget < 0) return true;
if (static_cast<int64_t>(n) <= budget) return budget -= static_cast<int64_t>(n), true;
return false;
}
bool size(const std::string& path, uint32_t& size) override {
auto it = files.find(path);
if (dead || it == files.end()) return false;
size = static_cast<uint32_t>(it->second.size());
return true;
}
size_t read(const std::string& path, uint32_t at, uint8_t* into, size_t len) override {
auto it = files.find(path);
if (dead || it == files.end() || at > it->second.size()) return 0;
reads++;
size_t n = std::min(len, it->second.size() - at);
std::copy(it->second.begin() + at, it->second.begin() + at + static_cast<long>(n), into);
return n;
}
bool create(const std::string& path) override {
if (!spend(1)) return dead = true, false;
files[path].clear();
return true;
}
bool append(const std::string& path, const uint8_t* data, size_t len) override {
if (dead) return false;
if (!spend(len)) {
files[path].append(reinterpret_cast<const char*>(data), static_cast<size_t>(std::max<int64_t>(0, budget)));
return dead = true, false;
}
files[path].append(reinterpret_cast<const char*>(data), len);
return true;
}
bool remove(const std::string& path) override {
if (!spend(1)) return dead = true, false;
return files.erase(path) > 0;
}
bool rename(const std::string& from, const std::string& to) override {
if (!spend(1)) return dead = true, false;
if (!files.count(from) || files.count(to)) return false;
files[to] = files[from];
files.erase(from);
return true;
}
uint64_t freeBytes() override { return free; }
void powerBack() {
dead = false;
budget = -1;
}
};
constexpr int kCols = 38, kRows = 8;
const char* const kNote = "/notes/big.txt";
std::string lines(int from, int count) {
std::string s;
char b[32];
for (int i = 0; i < count; i++) {
std::snprintf(b, sizeof b, "line %06d\n", from + i);
s += b;
}
return s;
}
// Each line is 12 bytes: "line 000000\n".
constexpr uint32_t kLine = 12;
bool rewrite(NoteDocument& d) {
std::string why;
if (!d.rewriteStart(why)) return false;
for (int guard = 0; guard < 100000; guard++) {
int p = d.rewriteStep(why);
if (p == 100) return true;
if (p < 0) return false;
}
return false;
}
void settle(NoteDocument& d) {
std::string why;
if (d.wantsMove()) TEST_ASSERT_TRUE_MESSAGE(d.move(why), why.c_str());
}
void type(NoteDocument& d, const std::string& s) {
for (char c : s) {
TEST_ASSERT_TRUE(d.text().insert(static_cast<uint8_t>(c)));
settle(d);
}
}
std::string currentRow(NoteDocument& d) { return d.text().rows()[static_cast<size_t>(d.text().cursorRow())]; }
} // namespace
void test_a_small_note_is_all_window() {
FakeCard card;
card.files[kNote] = "Hello\nworld\n";
NoteDocument d(card, kCols, kRows);
TEST_ASSERT_EQUAL_STRING("", d.open(kNote).c_str());
TEST_ASSERT_FALSE(d.windowed());
TEST_ASSERT_FALSE(d.dirty());
TEST_ASSERT_TRUE(d.wantsRewrite());
type(d, "Oh! ");
TEST_ASSERT_TRUE(d.dirty());
TEST_ASSERT_TRUE(rewrite(d));
TEST_ASSERT_FALSE(d.dirty());
TEST_ASSERT_EQUAL_STRING("Oh! Hello\nworld\n", card.files[kNote].c_str());
TEST_ASSERT_EQUAL_size_t(1, card.files.size()); // no side file, no temporary file left
}
void test_a_new_note_has_no_file_until_it_is_rewritten() {
FakeCard card;
NoteDocument d(card, kCols, kRows);
type(d, "A list");
TEST_ASSERT_TRUE(d.dirty());
std::string why;
TEST_ASSERT_FALSE(d.rewriteStart(why));
TEST_ASSERT_FALSE(d.journal(why));
TEST_ASSERT_EQUAL_size_t(0, card.files.size());
d.setPath("/notes/a-list.txt");
TEST_ASSERT_TRUE(rewrite(d));
TEST_ASSERT_EQUAL_STRING("A list", card.files["/notes/a-list.txt"].c_str());
type(d, "!");
TEST_ASSERT_TRUE(rewrite(d));
TEST_ASSERT_EQUAL_STRING("A list!", card.files["/notes/a-list.txt"].c_str());
}
void test_a_big_file_opens_as_a_window() {
FakeCard card;
card.files[kNote] = lines(0, 20000); // 240 KB
NoteDocument d(card, kCols, kRows);
TEST_ASSERT_EQUAL_STRING("", d.open(kNote).c_str());
TEST_ASSERT_TRUE(d.windowed());
TEST_ASSERT_EQUAL_UINT32(240000, d.size());
TEST_ASSERT_TRUE(d.text().size() <= NoteText::kMaxBytes / 2 + 200);
TEST_ASSERT_EQUAL_STRING("line 000000", d.text().rows()[0].c_str());
TEST_ASSERT_FALSE(d.wantsRewrite());
TEST_ASSERT_FALSE(d.dirty());
}
void test_walking_down_and_back_up_through_windows() {
FakeCard card;
card.files[kNote] = lines(0, 5000);
NoteDocument d(card, kCols, kRows);
d.open(kNote);
for (int i = 1; i <= 3000; i++) {
d.text().down();
settle(d);
if (i % 500 == 0) {
TEST_ASSERT_EQUAL_UINT32(static_cast<uint32_t>(i) * kLine, d.cursor());
TEST_ASSERT_EQUAL_STRING(lines(i, 1).substr(0, 11).c_str(), currentRow(d).c_str());
TEST_ASSERT_EQUAL_INT(kRows - 1, d.text().cursorRow()); // the screen didn't jump when the window moved
}
}
for (int i = 2999; i >= 0; i--) {
d.text().up();
settle(d);
if (i % 500 == 0) {
TEST_ASSERT_EQUAL_UINT32(static_cast<uint32_t>(i) * kLine, d.cursor());
TEST_ASSERT_EQUAL_STRING(lines(i, 1).substr(0, 11).c_str(), currentRow(d).c_str());
}
}
TEST_ASSERT_FALSE(d.dirty()); // looking isn't editing
TEST_ASSERT_EQUAL_size_t(1, card.files.size()); // and writes nothing
}
void test_start_and_end() {
FakeCard card;
card.files[kNote] = lines(0, 5000) + "the end";
NoteDocument d(card, kCols, kRows);
d.open(kNote);
std::string why;
TEST_ASSERT_TRUE(d.jump(d.size(), why));
TEST_ASSERT_EQUAL_UINT32(d.size(), d.cursor());
TEST_ASSERT_EQUAL_STRING("the end", currentRow(d).c_str());
TEST_ASSERT_EQUAL_INT(kRows - 1, d.text().cursorRow());
TEST_ASSERT_EQUAL_INT(99, d.percent());
TEST_ASSERT_TRUE(d.jump(0, why));
TEST_ASSERT_EQUAL_UINT32(0, d.cursor());
TEST_ASSERT_EQUAL_STRING("line 000000", currentRow(d).c_str());
}
void test_an_edit_in_the_middle_is_rewritten_into_the_file() {
FakeCard card;
std::string was = lines(0, 20000);
card.files[kNote] = was;
NoteDocument d(card, kCols, kRows);
d.open(kNote);
std::string why;
TEST_ASSERT_TRUE(d.jump(120000, why));
type(d, "HERE ");
TEST_ASSERT_TRUE(d.jump(0, why)); // the changed window goes to the side file
TEST_ASSERT_TRUE(card.files.count(std::string(kNote) + ".edit"));
TEST_ASSERT_EQUAL_STRING(was.c_str(), card.files[kNote].c_str()); // the file isn't touched until the rewrite
type(d, "TOP ");
TEST_ASSERT_TRUE(d.filePending());
TEST_ASSERT_TRUE(rewrite(d));
std::string want = "TOP " + was.substr(0, 120000) + "HERE " + was.substr(120000);
TEST_ASSERT_TRUE(want == card.files[kNote]);
TEST_ASSERT_EQUAL_size_t(1, card.files.size());
TEST_ASSERT_FALSE(d.filePending());
TEST_ASSERT_FALSE(d.dirty());
// And it goes on from there.
TEST_ASSERT_TRUE(d.jump(d.size(), why));
type(d, "END");
TEST_ASSERT_TRUE(rewrite(d));
TEST_ASSERT_TRUE(want + "END" == card.files[kNote]);
}
void test_typing_never_fills_the_note() {
FakeCard card;
NoteDocument d(card, kCols, kRows);
d.setPath("/notes/long.txt");
std::string want;
for (int i = 0; i < 4000; i++) { // 48 KB, three windows' worth
std::string l = lines(i, 1);
for (char c : l) {
if (!d.text().insert(static_cast<uint8_t>(c))) TEST_FAIL_MESSAGE("the window was full");
if (d.wantsMove()) {
std::string why;
if (d.path().empty() || !d.windowed()) TEST_ASSERT_TRUE(rewrite(d)); // what the editor does: a file first
TEST_ASSERT_TRUE_MESSAGE(d.move(why), why.c_str());
}
}
want += l;
}
TEST_ASSERT_TRUE(d.windowed());
TEST_ASSERT_EQUAL_UINT32(48000, d.size());
TEST_ASSERT_TRUE(rewrite(d));
TEST_ASSERT_TRUE(want == card.files["/notes/long.txt"]);
}
void test_a_journal_is_picked_up_after_a_power_cut() {
FakeCard card;
std::string was = lines(0, 20000);
card.files[kNote] = was;
std::string why;
{
NoteDocument d(card, kCols, kRows);
d.open(kNote);
d.jump(60000, why);
type(d, "one ");
TEST_ASSERT_TRUE(d.journal(why));
TEST_ASSERT_FALSE(d.dirty());
TEST_ASSERT_TRUE(d.filePending());
d.jump(180000, why);
type(d, "two ");
TEST_ASSERT_TRUE(d.journal(why));
type(d, "never saved");
} // the power goes
TEST_ASSERT_EQUAL_STRING(was.c_str(), card.files[kNote].c_str());
NoteDocument d(card, kCols, kRows);
std::string told;
TEST_ASSERT_EQUAL_STRING("", d.open(kNote, &told).c_str());
TEST_ASSERT_EQUAL_STRING("Your unsaved changes are back", told.c_str());
TEST_ASSERT_EQUAL_UINT32(240008, d.size());
TEST_ASSERT_EQUAL_UINT32(180004, d.cursor()); // where it was
TEST_ASSERT_TRUE(d.filePending());
TEST_ASSERT_TRUE(rewrite(d));
TEST_ASSERT_TRUE(was.substr(0, 60000) + "one " + was.substr(60000, 119996) + "two " + was.substr(179996) == card.files[kNote]);
TEST_ASSERT_EQUAL_size_t(1, card.files.size());
}
void test_a_cut_at_any_byte_keeps_the_note_or_the_last_save() {
std::string was = lines(0, 9000); // 108 KB
std::string saved1 = was.substr(0, 50000) + "first " + was.substr(50000);
std::string saved2 = "second " + saved1;
int outcomes[3] = {0, 0, 0};
for (int64_t budget = 0; budget < 240000; budget += 997) {
FakeCard card;
card.files[kNote] = was;
std::string why;
bool journaled1 = false, journaled2 = false, rewritten = false;
{
NoteDocument d(card, kCols, kRows);
d.open(kNote);
card.budget = budget;
d.jump(50000, why);
for (char c : std::string("first ")) d.text().insert(static_cast<uint8_t>(c));
journaled1 = d.journal(why);
if (journaled1 && d.jump(0, why)) {
for (char c : std::string("second ")) d.text().insert(static_cast<uint8_t>(c));
journaled2 = d.journal(why);
if (journaled2) rewritten = rewrite(d);
}
}
card.powerBack();
// What the Notes list does with a temporary file that has lost its note.
if (!card.files.count(kNote) && card.files.count(std::string(kNote) + ".tmp")) card.rename(std::string(kNote) + ".tmp", kNote);
NoteDocument d(card, kCols, kRows);
TEST_ASSERT_EQUAL_STRING("", d.open(kNote).c_str());
TEST_ASSERT_TRUE(rewrite(d) || !d.dirty());
if (d.filePending()) TEST_ASSERT_TRUE(rewrite(d));
const std::string& now = card.files[kNote];
int which = now == was ? 0 : now == saved1 ? 1 : now == saved2 ? 2 : -1;
char msg[64];
std::snprintf(msg, sizeof msg, "budget %ld", static_cast<long>(budget));
TEST_ASSERT_TRUE_MESSAGE(which >= 0, msg); // never anything else
if (rewritten) TEST_ASSERT_EQUAL_INT_MESSAGE(2, which, msg); // what was said to be saved is there
else if (journaled2) TEST_ASSERT_EQUAL_INT_MESSAGE(2, which, msg);
else if (journaled1) TEST_ASSERT_TRUE_MESSAGE(which >= 1, msg);
TEST_ASSERT_EQUAL_size_t_MESSAGE(1, card.files.size(), msg); // and nothing is left lying about
outcomes[which]++;
}
TEST_ASSERT_TRUE(outcomes[0] > 0 && outcomes[1] > 0 && outcomes[2] > 0);
}
void test_a_file_changed_elsewhere_sets_the_edits_aside() {
FakeCard card;
card.files[kNote] = lines(0, 20000);
std::string why;
{
NoteDocument d(card, kCols, kRows);
d.open(kNote);
d.jump(60000, why);
type(d, "typed ");
TEST_ASSERT_TRUE(d.journal(why));
}
card.files[kNote] = lines(7, 100); // someone put another file there
NoteDocument d(card, kCols, kRows);
std::string told;
TEST_ASSERT_EQUAL_STRING("", d.open(kNote, &told).c_str());
TEST_ASSERT_TRUE(told.find(".edit.lost") != std::string::npos);
TEST_ASSERT_TRUE(card.files.count(std::string(kNote) + ".edit.lost"));
TEST_ASSERT_FALSE(card.files.count(std::string(kNote) + ".edit"));
TEST_ASSERT_TRUE(card.files[std::string(kNote) + ".edit.lost"].find("typed ") != std::string::npos); // it can be read
TEST_ASSERT_EQUAL_UINT32(1200, d.size());
TEST_ASSERT_FALSE(d.filePending());
}
void test_crlf_becomes_lf_everywhere() {
FakeCard card;
std::string crlf, lf;
for (int i = 0; i < 6000; i++) {
std::string l = lines(i, 1);
lf += l;
crlf += l.substr(0, 11) + "\r\n";
}
card.files[kNote] = crlf;
NoteDocument d(card, kCols, kRows);
d.open(kNote);
for (int i = 1; i <= 1500; i++) { // through several windows, looking only
d.text().down();
settle(d);
}
TEST_ASSERT_EQUAL_STRING("line 001500", currentRow(d).c_str());
TEST_ASSERT_EQUAL_INT(0, d.text().cursorCol());
type(d, "x");
TEST_ASSERT_TRUE(rewrite(d));
std::string want = lf;
want.insert(1500 * kLine, "x");
TEST_ASSERT_TRUE(want == card.files[kNote]);
}
void test_windows_never_cut_a_character() {
FakeCard card;
std::string text;
while (text.size() < 80000) text += "\xC3\xA9\xE2\x82\xAC\xF0\x9F\x98\x80"; // é, €, a face: no space, no newline
card.files[kNote] = text;
NoteDocument d(card, kCols, kRows);
d.open(kNote);
std::string why;
for (uint32_t to : {30001u, 12345u, 79990u, 40003u, 5u}) {
TEST_ASSERT_TRUE(d.jump(to, why));
const std::string& w = d.text().text();
TEST_ASSERT_TRUE((static_cast<uint8_t>(w[0]) & 0xC0) != 0x80);
for (size_t i = 0; i < w.size();) { // whole characters, to the last
uint8_t c = static_cast<uint8_t>(w[i]);
size_t n = c >= 0xF0 ? 4 : c >= 0xE0 ? 3 : c >= 0xC0 ? 2 : 1;
TEST_ASSERT_TRUE(c < 0x80 || c >= 0xC0);
TEST_ASSERT_TRUE(i + n <= w.size());
i += n;
}
TEST_ASSERT_TRUE((static_cast<uint8_t>(w[d.text().cursor()]) & 0xC0) != 0x80 || d.text().cursor() == w.size());
type(d, "a");
}
TEST_ASSERT_TRUE(rewrite(d));
TEST_ASSERT_EQUAL_size_t(text.size() + 5, card.files[kNote].size());
}
void test_random_edits_match_a_plain_string() {
std::mt19937 rng(47);
for (int round = 0; round < 6; round++) {
FakeCard card;
std::string model;
while (model.size() < 30000u + 20000u * static_cast<unsigned>(round)) {
int words = 1 + static_cast<int>(rng() % 30);
for (int w = 0; w < words; w++) model += std::string(1 + rng() % 9, static_cast<char>('a' + rng() % 26)) + " ";
model += rng() % 4 ? "\n" : "\n\n";
}
card.files[kNote] = model;
std::string saved = model; // what a power cut would leave
NoteDocument* d = new NoteDocument(card, kCols, kRows);
d->open(kNote);
std::string why;
for (int step = 0; step < 6000; step++) {
unsigned op = rng() % 100;
NoteText& t = d->text();
if (op < 40) {
char c = rng() % 8 ? static_cast<char>('A' + rng() % 26) : '\n';
uint32_t at = d->cursor();
TEST_ASSERT_TRUE(t.insert(static_cast<uint8_t>(c)));
model.insert(at, 1, c);
} else if (op < 50) {
uint32_t at = d->cursor();
t.backspace();
if (at > 0 && d->cursor() == at - 1) model.erase(at - 1, 1);
else TEST_ASSERT_EQUAL_UINT32(at, d->cursor());
} else if (op < 60) t.left();
else if (op < 70) t.right();
else if (op < 78) t.up();
else if (op < 86) t.down();
else if (op < 90) t.pageUp();
else if (op < 94) t.pageDown();
else if (op < 96) TEST_ASSERT_TRUE(d->jump(rng() % (d->size() + 1), why));
else if (op < 98) {
if (d->dirty()) {
TEST_ASSERT_TRUE(d->wantsRewrite() ? rewrite(*d) : d->journal(why));
saved = model;
}
} else if (op == 98) {
TEST_ASSERT_TRUE(rewrite(*d));
saved = model;
TEST_ASSERT_TRUE(model == card.files[kNote]);
} else { // the power goes, and comes back
delete d;
d = new NoteDocument(card, kCols, kRows);
TEST_ASSERT_EQUAL_STRING("", d->open(kNote).c_str());
model = saved;
}
settle(*d);
TEST_ASSERT_EQUAL_UINT32(static_cast<uint32_t>(model.size()), d->size());
// The window is the note's text at its place.
TEST_ASSERT_TRUE(d->cursor() <= model.size());
uint32_t start = d->cursor() - static_cast<uint32_t>(d->text().cursor());
TEST_ASSERT_TRUE(model.compare(start, d->text().size(), d->text().text()) == 0);
}
TEST_ASSERT_TRUE(rewrite(*d));
TEST_ASSERT_TRUE(model == card.files[kNote]);
TEST_ASSERT_EQUAL_size_t(1, card.files.size());
delete d;
}
}
void test_a_full_card_and_a_missing_file() {
FakeCard card;
card.files[kNote] = lines(0, 20000);
card.free = 1000;
NoteDocument d(card, kCols, kRows);
TEST_ASSERT_TRUE(d.open(kNote).find("Not enough room") != std::string::npos);
TEST_ASSERT_TRUE(d.open("/notes/none.txt").find("refused") != std::string::npos);
card.files["/notes/small.txt"] = "small";
TEST_ASSERT_EQUAL_STRING("", d.open("/notes/small.txt").c_str()); // a small one still opens
type(d, "x");
std::string why;
TEST_ASSERT_FALSE(d.rewriteStart(why));
TEST_ASSERT_EQUAL_STRING("the card is full", why.c_str());
TEST_ASSERT_TRUE(d.dirty());
}
void test_the_side_file_and_the_piece_list_ask_for_a_rewrite() {
FakeCard card;
card.files[kNote] = lines(0, 20000);
NoteDocument d(card, kCols, kRows);
d.open(kNote);
std::string why;
int saves = 0;
while (!d.wantsRewrite() && saves < 1000) {
type(d, "x");
TEST_ASSERT_TRUE(d.journal(why));
saves++;
}
TEST_ASSERT_TRUE(saves > 50 && saves < 300); // about 1 MB of 8 KB windows
TEST_ASSERT_TRUE(rewrite(d));
TEST_ASSERT_FALSE(d.wantsRewrite());
TEST_ASSERT_EQUAL_size_t(240000 + static_cast<size_t>(saves), card.files[kNote].size());
}
int main() {
UNITY_BEGIN();
RUN_TEST(test_a_small_note_is_all_window);
RUN_TEST(test_a_new_note_has_no_file_until_it_is_rewritten);
RUN_TEST(test_a_big_file_opens_as_a_window);
RUN_TEST(test_walking_down_and_back_up_through_windows);
RUN_TEST(test_start_and_end);
RUN_TEST(test_an_edit_in_the_middle_is_rewritten_into_the_file);
RUN_TEST(test_typing_never_fills_the_note);
RUN_TEST(test_a_journal_is_picked_up_after_a_power_cut);
RUN_TEST(test_a_cut_at_any_byte_keeps_the_note_or_the_last_save);
RUN_TEST(test_a_file_changed_elsewhere_sets_the_edits_aside);
RUN_TEST(test_crlf_becomes_lf_everywhere);
RUN_TEST(test_windows_never_cut_a_character);
RUN_TEST(test_random_edits_match_a_plain_string);
RUN_TEST(test_a_full_card_and_a_missing_file);
RUN_TEST(test_the_side_file_and_the_piece_list_ask_for_a_rewrite);
return UNITY_END();
}