CI: stop rebuilding the framework at every run; a build cache and ccache (#74) #76

2026-10-07T00:10:36Z

twisla commented

2026-10-07 00:10:36 +00:00

For #74: why CI was slow, and the fix. Details and every measurement are in docs/milestones/R1.md.

The cause: the firmware step took 358 s, of which 260 s rebuilt the framework that was already in the volume. The platform decides whether the rebuilt framework matches by reading sdkconfig.defaults in the project folder; that file is generated and not in git, so no fresh checkout had it, and every run reinstalled and rebuilt the framework.

What changes

That file is kept in the volume, inside the libraries it describes, and copied into the checkout before a build. The platform still checks its hash against platformio.ini, so changed settings rebuild.
The version is out of the compiler flags: scripts/version.py writes one generated header, read by one file. Before, each commit recompiled everything (locally too), and no cache could have helped.
PlatformIO's build cache in the volume for a pull request's firmware; ccache for the host tests (they are built for coverage, and the build cache would lose the .gcno files).
A tag builds its firmware once (it was built twice), and a release compiles its own sources from nothing: it reuses the rebuilt framework, not the build cache.
PlatformIO and gcovr in a venv in the volume.

Measured (development machine, fresh copies of the tree, the same volume): the firmware step 358 s to 27 s with a new version and one changed file (82 s when it has to fill the cache); tests and coverage 49 s to 33 s; a local rebuild with nothing changed 77 s to 13 s. 476 host tests pass.

Not measured yet: the runner itself. This pull request's first run is a slow one by design (the runner has no mark yet, so it rebuilds the framework once more and keeps it). I'll push a second commit to get a warm run and report both here.

To decide: releases don't use the build cache, so a tag's build stays around 90 s plus tests and signing. Using the cache there too would bring it under 30 s, at the price of published files containing objects built by earlier runs.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EhqxQ49eCju4CzKYNjZzwT

For #74: why CI was slow, and the fix. Details and every measurement are in `docs/milestones/R1.md`. **The cause:** the firmware step took 358 s, of which 260 s rebuilt the framework that was already in the volume. The platform decides whether the rebuilt framework matches by reading `sdkconfig.defaults` in the project folder; that file is generated and not in git, so no fresh checkout had it, and every run reinstalled and rebuilt the framework. **What changes** - That file is kept **in the volume, inside the libraries it describes**, and copied into the checkout before a build. The platform still checks its hash against `platformio.ini`, so changed settings rebuild. - **The version is out of the compiler flags:** `scripts/version.py` writes one generated header, read by one file. Before, each commit recompiled everything (locally too), and no cache could have helped. - **PlatformIO's build cache** in the volume for a pull request's firmware; **ccache** for the host tests (they are built for coverage, and the build cache would lose the `.gcno` files). - **A tag builds its firmware once** (it was built twice), and **a release compiles its own sources from nothing**: it reuses the rebuilt framework, not the build cache. - PlatformIO and gcovr in a venv in the volume. **Measured** (development machine, fresh copies of the tree, the same volume): the firmware step **358 s to 27 s** with a new version and one changed file (82 s when it has to fill the cache); tests and coverage 49 s to 33 s; a local rebuild with nothing changed 77 s to 13 s. 476 host tests pass. **Not measured yet: the runner itself.** This pull request's first run is a slow one by design (the runner has no mark yet, so it rebuilds the framework once more and keeps it). I'll push a second commit to get a warm run and report both here. **To decide:** releases don't use the build cache, so a tag's build stays around 90 s plus tests and signing. Using the cache there too would bring it under 30 s, at the price of published files containing objects built by earlier runs. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01EhqxQ49eCju4CzKYNjZzwT

twisla added 1 commit 2026-10-07 00:10:36 +00:00

CI: stop rebuilding the framework at every run; a build cache, ccache, and the version out of the compiler flags (#74 )

Site / build (pull_request) Successful in 9s

Details

CI / build (pull_request) Successful in 7m11s

Details

07ac7ba457

A pull request's firmware step took 358 s: 260 of them rebuilding the
framework that was already in the volume, because the platform decides by
sdkconfig.defaults in the project folder, which is generated and not in git,
so no fresh checkout had it. It is now kept in the volume, inside the
libraries it describes, and copied into the checkout; the platform still
checks its hash against platformio.ini.

The version was a -D on every command line: each commit recompiled
everything, and no cache could help. scripts/version.py now writes one
generated header, read by one file.

PlatformIO's build cache in the volume for pull requests' firmware, ccache
for the host tests (built for coverage, which the build cache can't keep),
the tools in a venv in the volume, and a tag builds its firmware once.
A release still compiles its own sources from nothing.

Measured on fresh copies of the tree: the firmware step 358 s to 27 s (a new
version and one changed file), tests and coverage 49 s to 33 s, a local
rebuild with nothing changed 77 s to 13 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EhqxQ49eCju4CzKYNjZzwT

twisla added 1 commit 2026-10-07 00:18:08 +00:00

CI: say how to start the caches again from nothing

CI / build (pull_request) Successful in 1m6s

Details

Site / build (pull_request) Successful in 9s

Details

ca5874fe70

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EhqxQ49eCju4CzKYNjZzwT

twisla added 1 commit 2026-10-07 00:22:19 +00:00

R1: the CI timings measured on the runner

Site / build (pull_request) Successful in 8s

Details

CI / build (pull_request) Successful in 1m17s

Details

828f7ce821

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EhqxQ49eCju4CzKYNjZzwT

twisla merged commit c278a06ca1 into main

2026-10-07 02:08:50 +00:00

twisla referenced this pull request

2026-10-07 02:08:52 +00:00

CI: runs are too slow; investigate the build cache #74

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: twisla/roro9stack#76