The Ur Project

Documenting the craft of stacking things so they remain stacked. For a very long time.

We're not idle, on the contrary

It's been a few days since the last post. I didn't have a good topic to write about - but the opposite is actually true, there's probably too much to write about.

A Buildroot cycle happened: 7.x

I had delayed an LLVM update. The reason is that an LLVM update changes the binary contract in Rust. And a Rust change means every compiled rustlib becomes incompatible with existing ones. And that means every Rust-based package needs recompiling. So I generally wait until there's an opportune time to do this change, land LLVM+Rust in Sandcastle-Base, and then roll a new Buildroot.

The new Buildroot repository works as follows: Because this Buildroot repo is a full bootstrap repository, we compile Buildroot-7.1 against the last generation of the previous Buildroot, which was Buildroot-6.4. Then we keep rebuilding over and over until the binary output of each next generation no longer changes. We verify this by doing one extra rebuild, and so we check that Buildroot-7.1 == Buildroot-7.2 == Buildroot-7.3. If that holds, we promote Buildroot-7.3 and the cycle ends. Usually it takes 4 or 5 cycles. For 5.x we had to go to Buildroot-5.5, because there was a GCC update in there. For 7.x, we completed it with Buildroot-7.4.

That means that there's now a pkg-repo-buildroot-7-4 that will allow you to verify the reproducibility of Sandcastle-Base packages, since Sandcastle-Base is now floored on Buildroot-7.4. All new compiles in Sandcastle-Base will use this new floor from this week onwards.

This cycle will happen about every 2-4 weeks depending on upstream changes and need. After a cycle like this, all Rust-based software needs a full recompile. The total time needed for this process is about 2 days, the bulk of which is taken by the Buildroot waves (3-4 hours each), and the Rust world rebuild is another 3-4 hours, with the current hardware I have.

More work to tighten up pkg signature handling

I've been going over many of the changes that impact how pkg handles signed assets and tightening down things that weren't tight enough. Mostly these are edge cases with newer code. We added proxy support to pkg recently and that exposed a gap where a rogue proxy could cause pkg to wrongly accept a signing key it shouldn't have trusted. That's a big bug, of course. So that's fixed in 0.0.104. As a follow up to that, I'm auditing the producer side of the same issue and this will need some extra handling when the clerk keys get rotated. That side fortunately is just minor work and nothing complex - it just means we may need to re-sign existing package content at rotation time.

CVE handling overhaul in dispatcher

Admittedly this was a huge gap in dispatcher at launch, and I wrote so explicitly. The new scheme does away with how dispatcher previously handled CVEs and now has an identity mechanism that allows the operator to connect packages to an upstream CVE identity (e.g. a CPE, or distro-related). This means CVE streams with correctly set up identities will arrive at the correct package, blocking publication of vulnerable changes in the right place, and weeding out false positives, or badly identified CVE sources. For instance, the npm ecosystem publishes CVEs for tar, which show up in the GNU Tar package as a vulnerability. Which is wrong, of course.

This new system means a lot of extra work up front to route the CVE data to the right package, but then there are zero false positives. And then we can possibly start treating them like actionable items: We could decide to patch the issue, or look for an upstream patch. Or we could disable vulnerable code with a configure flag. And we can still decide to publish with a vulnerability, and let users know they are vulnerable.

Fun stuff

Yes, there's a ton of fun stuff this week already. I'm going down the list of "hell yeah" and so I'm adding things I'd love to see as native packages and there's really no reason not to. This week's batch includes:

Also, the upcoming live USB images will have an option to install Plasma as the desktop (with Plasma Login Manager) instead of the GNOME desktop, as an alternative.

Still in progress

DKMS is coming. For those stuck with required proprietary kernel drivers and those who want or need others, we're patching DKMS so that it will work with our kernel-in-erofs style kernels (see boot-manager(8)). This should make it really easy to work with them. A full guide still needs writing, though, to explain and demonstrate this.

I'm working on the test framework for image-based testing. It already was working well enough to get good data on the quality of all the changes and integrates well into the builder system, but I wasn't happy with the organization of the code and while I don't want to have shell code salad anymore, I do need some scriptable way of writing test cases. So the focus is on that.

The test system is very simple from dispatcher's view - dispatcher just exposes images to test runners, and test runners can POST test results back for each of the images available. How the test runners work could be up to whoever writes these tests. It could be a person doing a manual thing and using curl to post a test result. Or an LLM agent doing a manual test. In my case, the test runners look like a harness that runs Lua in a sandbox to execute tests through a system probe, so it can run either in a container, or on a VM image, and have full access to everything in the image in a secure way.

--sofar (aka Auke)