It's not going to be perfect

Time to put a realistic perspective on. I'm prepping to fully make all the code public and fully operational, with posted images and a breathing distribution that can handle any user request, and turn it around quickly.

But, such a thing is complex and I've been building it now for about 5 months, full time. Literally putting in 60-80 hours a week to make it work well. It can't be perfect. There's just things that need a lot more work to even be decent. Today's post is my shortlist of things that still suck. Not that it matters a whole lot to all of you who are going to be future users and even developers, but you might end up running into these flaws and wonder why they exist.

Rust packaging

I want to get this right. It's important. It's even more so an ecosystem that is going to be dramatically intruding into the distro over time.

Here's what's the issue. In the rust world, there's crates. These are small components, usually libraries, that rust programs use to build upon. Now, any rust program can be compiled by cargo and cargo will just go on the internet and download all the required crates for you.

But the build environment is a sandbox without networking. On purpose. Not negotiable. So rust packages can't just cargo build. Either every single dependency needs to be known beforehand and fetched (causing the recipe.toml to have to list these hundreds of source components), or the rust package must be built from a vendored tarball. And those are generally very large and cumbersome to work with. They also would explode the cost of doing license scanning, malware scanning, CVE scanning and all that work.

Neither of these solutions are decent. The only remaining option is that we package each individual rust crate as a separate rustlib package. It doesn't provide anything, except a static library that only rust programs can link against. Then when the actual end programs compile, they link against these rustlib packages. Only the build system will ever use them.

Sounds simple enough, yes? Well it's not that easy. Because soon enough you'll find out that rustix was built without the hooks feature enabled and serde built at version X doesn't work for a second program because that second program can't use version X but instead must have Y.

Then there's the fact that a Rust base version change literally requires a full rebuild of every rust crate or else every rust based package is now incompatible because it was compiled against an older Rust version.

Even with all the tooling I've thrown at the problem to unmap and plan these rust changes I'm spending a significant amount of time rolling these upgrades into the actual distro. The time to actually build all of it is nothing - most rust crates build in 10 seconds or so. The time spent to match them all? Days of time lost.

No cross or native non-x86

Sometimes you just don't consciously make the connection yet. But this morning I did. I was just thinking about when a good point in time would be to start scaffolding Aarch64 builds, or even others. The build system is more than capable, I have the necessary compute to build it, but it's not even there. I have no hardware to do native non-x86 builds, so for now it'll be cross compiled. Which saves me from having to implement a distributed build architecture.

No immutable distro

This seems awfully popular. But it's just something that is low on the list of problems to solve, but highly desired by many. I'm honestly not even sure about whether I want to make this a completely tailored thing, or an a-la carte sort of thing. The difference matters.

The package manager would work just fine in A/B mode. So making a piecemeal immutable OS would actually be doable. You'd declare a local composition on your system, and the A/B updates would reflect the natural updates to the composition closure. If you'd make a composition with Plasma, you'd have your own A/B atomic Plasma desktop OS.

Or we make it bespoke. No user choices. Whatever composition we decide, you get it. If we decide it's just GNOME, sorry Plasma fans. The composition lives on the server - you don't control it. And with that model, who decides when updates are available? It could be that the server doesn't create the A/B compositions at all unless someone pushes a button. Or when a certain minimum amount of time passes. Or just once a month? Any of these has valid use cases, but the choice needs to be made. Exactly why I haven't made them yet - I would be the last person to even use the atomic release builds anyway.

Note that the build system itself could make all of these functional, as I've mentioned before, composition is just a layer in the build process, and interchangeable, or extensible. So we could do all of these if we wanted to. Until there's an actual audience, nothing will get decided.

No central point for failures

There's several watchers that can trigger a re-packaging or version upgrade of components. But these changes are automated release bumps, and not something super smart. So, they can and they will fail. For lots of reasons. For instance, a GNOME version bump is a 70+ component change, not a single package version update. GNOME 51 requires a full rust chain package just for a single library, and it's a doozy. I've backed it out for now until I can sink more time into it. Perhaps the rust package tooling has evolved enough that it should now work, but, until I try, we won't know.

This is just one of the things that can fail. There's not yet a nice dashboard for non-build failures. There should be. The journal is probably too noisy. So the operation of the build system currently requires a significant amount of vigilance. That's something that must get easier.

Docs, docs, docs

There's nowhere near enough documentation for any of the components, how they work, what their interfaces are, why they even exist, and how you should or could use them. I've got a lot of manual pages to write. Possibly a full documentation web site needs to be built. From scratch, or using something that is simple and makes it accessible.

The unknowns

The last thing to mention here is that I might just be blind to some of the problems that still exist. There could be bugs or gaping security holes. All that stuff. I'm only a person. I might need your help ;)

--sofar (aka Auke)