The Ur Project

Documenting the craft of stacking things so they remain stacked. For a very long time.

pkg-stage-update(1)

NAME

pkg-stage-update - pre-download the payloads an upgrade would install

SYNOPSIS

pkg stage-update [--dry-run] [--verify] [--provisional] [name...]

pkg stage-update --prune [--dry-run] [--provisional]

DESCRIPTION

Downloading is the slow part of an upgrade, and the part that needs the network. So do it ahead of time. This command fetches the payloads pkg-upgrade(1) would install and leaves them in the local cache. Nothing is unpacked, and nothing about the running system changes. The upgrade later on applies from disk: quickly, and without the network.

Why isn't this part of check-update?

Because asking and pulling are very different costs. Asking whether updates exist is cheap, read-only and unprivileged; see pkg-check-update(1). Pulling them is none of those. If one command did both, a timer would quietly eat disk and bandwidth on a metered link. So you opt into the expensive half yourself.

Without names, the whole upgrade plan is staged. Naming packages limits staging to them; a +set name stands for its installed members, as in pkg-upgrade(1).

Where does it go?

Payloads land in the shared content-addressed cache under <state-root>/cache/blobs, in the same layout a repository uses. The payload store checks there before it goes to the network. Content-addressed means the file name is the SHA-256 of its contents, so the same bytes are never stored twice.

Staged bytes still get verified against the manifest digest when they're extracted, like any other payload. Nothing here is trusted just because it was downloaded earlier.

WHAT RUNNING IT TWICE COSTS

The downloads are incremental

Staging is keyed on the payload digest, so a second run only fetches what actually changed. A repository that rolled one package out of fifty costs you one payload, not fifty. Payloads are fetched four at a time, so one big one doesn't hold up the rest.

The plan is not

Payload digests live in the manifests, so every run pays for a full index build. Against a remote chain that's one request per manifest, which dwarfs the download savings whenever only a few packages moved. Run pkg-check-update(1) first, which is cheap, to decide whether this is worth running at all.

There's no resume inside a payload

A partial download can't be trusted. The digest is a property of the whole payload, not of a prefix. So each payload is streamed to a temporary file and moved into place with a single rename: complete, or not at all. An interrupted run restarts the one payload it was on, and keeps every whole payload it already staged. A retry costs at most one payload, not the whole plan.

WHEN IS A STAGED PAYLOAD GOOD ENOUGH?

Just being there isn't enough. A staged file can get damaged after the fact: truncated by a full disk, corrupted on the media, edited by hand. If presence counted, this command would report success and the failure would show up much later, as a digest mismatch while an upgrade is extracting. That's the worst place to find out, because making that step reliable was the whole point.

So there are two checks:

  • Every payload is hashed while it's written, and a mismatch lands nothing. The cache never holds bytes that were wrong on arrival. That includes staging from a local repository directory, which does no verification of its own.
  • A payload that's already on disk is checked for later damage before it's reused. By default that's its size, which catches truncation for free. Under --verify, it's a full re-hash.

A manifest that declares no size leaves nothing cheap to compare, so for that one, presence is all there is unless you pass --verify.

KEEPING THE CACHE BOUNDED

Staging holds on to bytes for a plan that may never be applied. The repository moves on, and what you staged is suddenly for an N-V-R nothing points at. --prune is the fix. It drops every staged payload the current plan doesn't name, then exits without fetching anything.

Without a periodic prune, the cache grows without limit. That's why it's a flag and not an automatic eviction policy: a policy would have to guess at your disk budget, and I'd rather it didn't.

A successful pkg-upgrade(1) already drops the staged copies of whatever is now installed, since those are the same bytes stored twice. Its --keep-staged keeps them instead. Prune covers what an upgrade never used.

Only 64-character hexadecimal payload names under cache/blobs/data are considered, so a stray file in the cache is never swept up.

OPTIONS

--dry-run
Report what would be downloaded, and how much, without fetching anything. With --prune, report what would be deleted, and delete nothing.
--prune
Delete staged payloads the current plan doesn't need, then exit. Stages nothing. It's always measured against the whole upgrade, so it takes no package names. It also removes the <sha>.tmp.<pid> file an interrupted download left behind, once that's an hour old.
--verify
Re-hash every payload that's already staged instead of trusting its size. That catches a same-size corruption a size check can't, at the cost of reading everything staged.
--provisional
Stage the plan that converges onto the staged provisional pins (metadata/provisional_pins.json) instead of the blessed ones (metadata/pins.json). Only a repository read from a local directory can have both files; a remote repository serves the staged pins either way. See pkg-chain(7).

All global options apply; see pkg(1).

OUTPUT

staged 9 of 94 package(s), 152.4 MiB

The second number is the whole plan. The first is what actually got fetched, so a second run that re-stages one payload out of fifty counts to one.

Nothing to do:

nothing to stage (94 package(s) already present)

Under --dry-run:

would stage 9 of 94 package(s), 152.4 MiB

Under --prune, and --prune --dry-run:

pruned 12 staged package(s), freed 384.1 MiB would prune 12 staged package(s), would free 384.1 MiB

nothing to prune

JSON

Staging, and --dry-run:

{ "ok": true, "dry_run": false, "blobs_total": 94, "blobs": 9, "bytes": 159771259 }

--prune:

{ "ok": true, "dry_run": false, "pruned": 12, "bytes_freed": 402692506 }

bytes only counts what was fetched, or would be. JSON always carries the exact byte count; the human line prints a magnitude.

CONFIRMATION

This command never prompts, at any confirm level. It installs nothing and removes nothing the system is using. What it writes is cache, and what --prune deletes is cache the current plan doesn't name, which you can fetch again at any time. The global --yes is accepted and does nothing here. See pkg.toml(5).

EXAMPLES

Pull now, apply later

pkg stage-update pkg upgrade

Only when there's something to pull

pkg check-update && exit 0 pkg stage-update

check-update exits 0 when there's nothing to do, so the && stops the script right there, and the expensive half never runs.

How much would this cost?

pkg stage-update --dry-run

Staging one package

pkg stage-update openssl pkg stage-update kernel kernel/mod

Overnight, applied in a window

pkg --yes stage-update pkg --yes upgrade

The staging half needs the network. The upgrade half doesn't.

Reclaiming what the repository moved past

pkg stage-update --prune pkg --json stage-update --prune | jq '.bytes_freed'

Are the staged bytes still good?

pkg stage-update --verify --dry-run

Anything whose digest no longer matches is reported as needing a fetch again, without fetching it.

Testing a staged promotion

pkg stage-update --provisional --dry-run pkg stage-update --provisional pkg upgrade --provisional

Keeping the staged copies after applying

pkg stage-update pkg upgrade --keep-staged

Scripting

pkg --json stage-update --dry-run | jq '.bytes' pkg --json --yes stage-update | jq '.blobs'

Watching it work

pkg -v stage-update pkg -vv stage-update 2>&1 | grep restaging

The -v line accounts for the plan:

stage: 94 package(s) in plan, 85 already staged, 9 to fetch

-vv names each payload it decided to stage again, and why:

restaging 9c8084eb4024: staged 4096 bytes, manifest declares 92815360 restaging 543df49455f6: digest mismatch

FILES

<state-root>/cache/blobs/data/<xx>/<yy>/<sha256>
Staged payloads, in the canonical content-addressed repository layout. See pkg-state(5) and pkg-repository-layout(7).

EXIT STATUS

0 on success, including the nothing-to-stage and nothing-to-prune cases. 1 on any failure: a plan that doesn't resolve, an unreachable repository, a payload whose digest doesn't match on arrival, or a write error.

This command does not use the exit-code convention of pkg-check-update(1). Nothing to stage is success, not a separate code.

SEE ALSO

pkg(1), pkg-check-update(1), pkg-upgrade(1), pkg-offline(1), pkg-verify(1), pkg.toml(5), pkg-state(5), pkg-chain(7), pkg-repository-layout(7)

docs/design/repo.md, docs/design/staged-promotion.md, pkg-cli/JSON.md, pkg-cli/VERBOSITY.md