pkg-state(5)

NAME

pkg-state - on-disk client state of an Ur Project system

SYNOPSIS

/var/lib/pkg/

DESCRIPTION

pkg has to remember what it installed, why it installed it, and what it did last time. And it has to keep remembering correctly when the power goes out halfway through a commit. All of that lives under one directory, the state root: /var/lib/pkg by default, or wherever pkg --state-root points.

Why not a database?

Because a directory of manifests is all it needs to be. There's no database here in the usual sense: just the manifests of what's installed, plus a little bookkeeping. You can read all of it with ls and cat, and it survives a crash without a repair pass, because every change lands with a rename.

Everything under it is owned by root. Reading it needs no privileges, and the read-only commands that touch installed state degrade when they can't read it anyway: pkg-search(1) and pkg-set(1) warn and carry on with repository data only.

LAYOUT

installed/

The installed set. One directory per installed subpackage:

installed/<name>/<subpkg>/manifest.toml

A multi-installable package gets its version-release as one more component. That's what lets several kernels sit side by side:

installed/<name>/<subpkg>/<version>-<release>/manifest.toml

A subpackage name with a / in it becomes nested directories: lang/en@boldquot lives at installed/<name>/lang/en@boldquot/.

The manifest is the full package manifest as it was published: the file list with modes, sizes, checksums and symlink targets, the capabilities, and the payload digest. It's what pkg-verify(1), pkg-repair(1) and pkg-query(1) go by.

installed.staging/

Where transactions are built up. Each one in flight gets its own directory, txn-<pid>-<n>, holding:

plan.toml: what the transaction means to add and remove.
add/: manifests on their way into the installed set, in the same <name>/<subpkg>[/<variant>] shape.
backup/: manifests the transaction displaces, in the same shape.
roots.toml and history/: the roots, record and snapshot the transaction will leave behind.
applied: a marker written once the transaction has committed. It's what tells recovery whether to roll forward or back.

installed.staging/ must be on the same filesystem as installed/, because the commit is a rename. That's checked every time the state root is opened for writing.

roots.toml

The selection roots: what you asked for, as opposed to what got pulled in. Two arrays:

packages = ["vim", "htop"] sets = ["console"]

Package roots are recorded the way you typed them, with no version: a bare name (vim) or a name/subpkg coordinate (vim/enhanced). Converging follows the pin. Set roots are bare set names without the + sigil; the sigil is only worn where a set is referenced. The same transaction that changes the installed set writes this file, so a history record shows the roots that transaction left. See pkg-sets(7).

history/

One record per committed transaction, <seq>.toml, and the installed set and roots right after it, <seq>.snap.toml, with <seq> zero-padded to eight digits. Those are the numbers pkg-history(1) lists and /var/log/pkg.log lines start with. The commit itself writes them, so a transaction is in history exactly when it's in installed/. Pruned to what pkg.toml(5) [history] keeps.

trust/

The newest root, timestamp, snapshot and targets versions this machine has verified, one JSON file per repository. A TUF signature only proves the producer meant those bytes once. It can't tell you they're still the current bytes, so without this an attacker could replay an old, validly signed package set forever. Each file is named by a digest of the repository's URL and its anchor's content, so the same URL under a different root starts over. It lives here and not in a cache because deleting it would quietly switch the protection off, and because the operator running pkg check-update and root running pkg upgrade must share one floor. A missing or unreadable file just means first contact. pkg repo remove drops the repository's file.

cache/blobs/

Payloads that pkg-stage-update(1) downloaded ahead of time, stored by digest. A payload's staged copy is dropped once it's installed, unless you ask to keep it. It follows the state root, so whoever stages and whoever applies land on the same directory, and a second state root gets its own blobs.

.lock

The store-wide advisory lock, held with flock(2). Two things must never run at the same time against one state root: orphan recovery, and a live transaction. Opening the state root for writing holds the lock for the recovery scan only. A transaction holds it from begin to commit or rollback. The kernel releases it when the holder dies, which is exactly the case recovery exists for.

Read-only commands never take it. Asking what's installed isn't a privileged question, and a writer that was killed mid-commit holds no lock worth waiting for anyway. See READING A TORN STATE.

repo/

The default single local repository, used when nothing is configured in pkg-repos.d(5) and no --repo-url is given. Its structure is described in pkg-repository-layout(7).

index/

Older versions of pkg created this as an empty directory. Nothing reads or writes it now. The map of which subpackage owns which path is built in memory from the installed manifests every time the state root is opened.

CRASH RECOVERY

What happens when pkg dies mid-commit?

Every time the state root is opened for writing, pkg looks in installed.staging/ for txn-\* directories left behind by a process that died. Each one is resolved to a coherent outcome by one question: is the applied marker there? If it is, the transaction committed and recovery finishes it. If it isn't, recovery rolls it back. This runs unattended, under the store lock, on every such open. A read-only open never repairs anything.

A transaction's roots, record and snapshot are staged beside its manifests and land with them. Rolling forward finishes moving them into place. Rolling back restores the old roots.toml and drops the record.

READING A TORN STATE

What does a read see in the meantime?

A commit renames every removed subpackage out of installed/ before it renames any added one in. For that short window, the tree holds neither the old nor the new side. If the writer is killed right there, it stays that way until some command that writes runs recovery.

Read-only commands don't read the tree naively, and they don't wait for a lock. When installed.staging/ holds a txn-\* directory with a plan.toml and no applied marker, they work out the state that transaction started from -- dropping its planned adds and reading its backup/ manifests -- entirely in memory. Nothing is written, and no directory is created. So pkg-list(1), pkg-query(1), pkg-verify(1) and the rest report the last coherent committed state, both while a transaction is running and after one was interrupted. The repair is left to the next command that writes.

Two things worth knowing. A read-only command against a store whose staging area can't be read, or whose plan.toml is corrupt, fails rather than quietly reporting the torn tree. And a commit that lands while a read is in progress may be reported as the state just before it. Stale, but coherent.

COMPOSED ARTIFACTS

pkg compose puts a composed artifact's own state database inside the artifact, at <output>/var/lib/pkg by default, so the result is self-contained. Chroot it, tar it or boot it, and it carries its own package database. --state-dir puts it somewhere else.

It's the same path a live system uses, and that's the point. A composed tree becomes a live system the moment someone boots it, and its pkg shouldn't have to be told where its own records are.

Every backend that materializes files records them, not just the FHS one. A disk image composed with pkg compose --composition carries the same database an install would have written. Both use the same transaction, so an image's first pkg upgrade diffs against records in the shape it expects. An image whose files are there and whose records aren't isn't a lighter image. It's a system that believes nothing is installed.

Selection roots are the exception: they aren't recorded at compose time. By the time a resolved closure reaches a composer, "asked for" and "pulled in" are the same set, and nothing tells them apart. The damage is limited: pkg-autoremove(1) refuses to act on an empty roots file, rather than treating every package as an orphan.

EXAMPLES

What is installed, straight from the filesystem

ls /var/lib/pkg/installed/ ls /var/lib/pkg/installed/gzip/

The supported way to ask the same question:

pkg list pkg list --collapse=main

Reading one installed manifest

pkg query gzip --files --provides --requires

Or directly:

head -40 /var/lib/pkg/installed/gzip/bin/manifest.toml

Which selection roots this machine has

cat /var/lib/pkg/roots.toml pkg list-sets

Confirming no transaction is stuck

ls /var/lib/pkg/installed.staging/

An empty directory is the healthy state. A leftover txn-\* directory is harmless: every read-only command reads around it, and the next command that writes clears it under the lock:

pkg autoremove --dry-run

Operating on a state root that is not the live one

Handy against a mounted image, a staged tree, or a test fixture:

pkg --state-root /mnt/image/var/lib/pkg list pkg --state-root /mnt/image/var/lib/pkg verify coreutils \ --root /mnt/image

--state-root picks which database to read, and --root picks which filesystem to check against. Verifying an image needs both.

Backing the state up before a risky change

tar -C /var/lib/pkg -caf /root/pkgstate.tar.zst \ installed roots.toml

Composing a self-contained root and inspecting its database

pkg compose root --output /srv/roots/minimal \ --closure +console --trust-key "$PUBKEY" pkg --state-root /srv/roots/minimal/var/lib/pkg list