Technical debt

In the development of software, one will always accumulate technical debt.

This is a given. No matter how small a project is, there will always have to be trade-offs. You may remember the triangle of software engineering: "fast", "good", "cheap". You can only ever have 2 out of three. You can't choose fast and cheap and still expect it to be good.

I've been hacking, testing, redesigning, reviewing the code base for just about 5 months now. I started back in May, believe it or not. At first I thought I'd just do an experiment to see what I could make work and what shape it would take. Green field design. No constraints other than the fact that I have the experience to call out when it goes off the rails, guided by rigorous testing. No code can remain untested.

I think it took me about 6 weeks to get a fully booting OS. It'd have been somewhere mid June. At that point things accelerated and I went from a simple build system with some basic packages to a full system with all the features I'd want from it, in just a few weeks. Conceptually.

The rest of the time spent, a whopping 3+ months, has been for a very large part nothing more than mopping up technical debt. Stuff that just didn't quite yet work the way it was intended. Things that were just really still rough edges. Interfaces that turned out to be too lax, and needed tightening up.

It didn't reproduce

One of these things that needed to be shored up, but I didn't get to yet has been reproducible builds. The build server already builds everything twice, but that doesn't guarantee that anyone else gets the exact output if they try. Short explanation for that: I didn't factor in how much the build code would change over time, and subtle changes to how files are sorted into subpackages already break it. There are a few more little surprises like it.

As I'm spending these days mostly doing validation for things that were made quite some time ago already, I'm basically just working on technical debt. Stuff like this has to work. It is the whole promise of the build system - generate content that users can 100% validate so that trust is founded on truth, and not a promise.

These things hit fast and hard

A broken installer isn't going to do anyone any good. It would prevent every first user from even starting to use the OS. So it has to work well. A big uncertainty is how it behaves in a situation I hadn't thought of. A broken security/trust promise hits hard on the second wave. The build system itself also can't be a beast to deploy. People will give up if they can't get it to work in 20 minutes. It has to be up and running in 5.

So, back to technical debt. There'll be a lot of that in the next few weeks/months/who knows. You'll bump into these issues. There's no way I can make something "good" and "fast" and "cheap". I'm going for good. And it's just me and my free time, so it won't be fast.

--sofar (aka Auke)