The Ur Project

Documenting the craft of stacking things so they remain stacked. For a very long time.

Architecture - Compositions - part 9

Our dispatcher can spawn any sort of job. So we can make it build images automatically. And because it knows how to resolve dependencies in closures, it can detect whether an image requires rebuilding or not. This makes it super simple to have dispatcher manage compositions and do automatic image builds, as soon as the image could be updated.

This works so well that the images could end up being built several times per minute, if packages are constantly streaming into the repositories. So I've put in a 1 hour backoff - just to make sure I'm not spinning endless image generations. The image generation is fast, too. The smaller images take 10-20 seconds to generate. Even the live USB image takes under a minute to build.

Everyone can make images

The pkg tool has the subcommand compose which is exactly what produces post-package compositions. Compositions are nothing more than describing the mechanism of taking inputs from the repositories and modifying them into a new product. Logically we'd want to make these compositions:

All you'd need to do is write a composition.toml file to describe the sort of composition you want, and what the inputs (repositories) are.

Here's a simple start (not complete) of a composition example:

[composition]
name        = "generic-iso"
description = "Hybrid live ISO for providers that take nothing but an uploaded ISO"
type        = "disk"
arch        = "x86_64"
version     = "2026.09"

[[repos]]
# TODO: insert repo URLs here

[packages]
needed = [
    "+sys",
    "linux-kernel-stable/data", "live-init/bin-static",
    "syslinux/data", "systemd/boot", "instance-setup",
    "pkg", "live-install",
    "shell-defaults", "procps-ng", "grep", "sed", "gawk",
    "findutils", "which", "less", "vim/minimal", "sudo",
    "openssh/server",
]

[disk]
format          = "iso"
filesystem_root = "erofs"
firmware        = "hybrid"
bootloader      = "systemd-boot"
root_label      = "generic-iso"

[guest]
machine_id        = "0ec1a5d4b7f24c0e9a3b6d8f10245c37"
dhcp              = true
seed_boot_entries = true
boot_timeout      = 5
boot_extra        = 0

That's about it. For simplicity I've omitted the repo URLs here since the public repo server isn't online yet, so I can't show yet how that will look.

Dispatcher will manage compositions

Here's the overview screen of what this looks like in dispatch. It's a simple list of what compositions exist and what their status is:

The composition list in dispatch-web

I'm currently working on automated testing and this consists of a runner that will pull new images and subject them to various testable items. The runner reports back so that the dispatcher can show you the status of the tests:

The runners show that there was a test failure

future compositions

The biggest missing composition output right now is immutable system images of some sort. The distribution is entirely mutable right now - it's a classic FHS install where binaries and software can be modified by a root user.

The immutable distro projects a significant hurdle. If you make an immutable distribution but it can't install end user software packages, then it's just a desktop environment without things that make users excited. So the content of the repositories must grow some more first. The basics must really be covered well. Not everything can be done through flatpaks either.

Whether we want just a single immutable image, or multiple slices is another determination that I'm just not yet sure of. Maybe we want hundreds of slices, or maybe just a few. For instance, maybe we just make slices for Desktop Environments and call it good from there on.

What's next for compositions?

I've got the basics covered and we're producing and testing images continuously now and this has already helped a ton to detect issues and getting bugs fixed, as well as detecting regressions.

The long work now is to expand the number of tests so that we get far better coverage over the actual functionality, and not just quick touch testing of the basics. This means just writing hundreds of test cases. A long and tedious task. But worth every penny in the long run.

--sofar (aka Auke)