The N-V-R and what it isn't

What is the classical Linux packaging nightmare? Everyone will tell you the very basic concept of the issue. You can't just easily upgrade a system where dependencies are complex and potentially circular. The first generation of package managers would often just strand you with no clue on how to proceed or even unbreak a broken setup.

But there's a second layer of madness that's waiting to pounce the unattentive user. Even with perfect DAG resolvers, who decides which package in the repository is the newer, and which the older? In comes the epoch. Which is just arguably a terrible way to solve this problem. Once a package declares an epoch value > 0, it can never get rid of it.

Now, we've designed a content-addressed-store already. But we do want to continuously put more and more packages into it. And we can't put more things into it if new content would erase or overwrite old content. So uniqueness of packages is a hard stop. N-V-R is still king. N is determined by the recipe. V (the version) is determined by the upstream project. And where V conflicts between two different versions or builds, we can fix the uniqueness with the R - release version. It's a number we control, and short lived. It resets to 1 every time V changes.

No epoch

But we can't determine in a repository whether a downgrade happened with just N-V-R, or can we? The answer is: We don't need the epoch. Essentially: The repo contains dozens of copies of the same package, all different (and unique) N-V-R's. But, there is only one "current" list of packages in the repo. This "current" list is an exact list of N-V-R's. It not only solves the upgrade ambiguity problem, allowing us to downgrade, it makes upgrades easier because we just need to reference the "current" pins of N-V-R's, and we get erasing packages for free. We just drop the N-V-R from the pins without replacing it with another.

This pins list is updated by the repo publisher on every successful publish, and erasure of a recipe in the corresponding registry lands as a pin erasure. We've now solved downgrades and removals. The cost was publishing an index of pins. This published pins file makes doing upgrades easier for the client. It's all a win.

I accidentally upgraded openssl from 3.x to 4.0, and then tried my first recipe downgrade. That blew up my entire build system in a terribly bad way. This solution provided the recovery, and the simplicity is its beauty. Of course, I think I spent 2 whole evenings to unwedge it somehow before I realized I needed a structural solution, not knowing that I'd end up with such a simple method.

What about downgrades?

These aren't yet implemented, but the shape is similar: They're just pins at a different version. You don't necessarily need to have a list of old pins around, but you could just make one. Of course, I think we'll end up publishing a signed list of older pins, so that clients can actually reverse-walk through updates, and do actual (real) bisects. Maybe even programmatically! pkg bisect good <ref>, etc.

The concept of "latest" also makes transformations to another (compatible) base possible. Ideally you'd do this as an A/B update or a swap during reboot. There's no real reason that this would even need to download all the full binary content again if a large portion will have the same checksum and binary content. Yet another efficiency win for our CAS data model.

Streams

It does put an interesting angle on streams. Fundamentally everyone understands their need and from a business perspective stable streams are a very good thing, but stable streams need rooting on an older base than a rolling release model stream, which is the default we'll start out with. That has some compatibility issues and means that developers who want to base their streams on a stable stream might not get their software working on rolling release streams - mixing streams is after all entirely without guarantees. Future work on streams will have to iron out whether we just want to let this happen naturally or draw a hard boundary to prevent it. Time will tell.

--sofar (aka Auke)