pkg-compose(1)

NAME

pkg-compose - compose a closure into a deployable artifact

SYNOPSIS

pkg compose --composition FILE [--output PATH | --target DEVICE] [--trust-key HEX]... [--force] [--sbom FORMAT] [--licenses] [--dry-run]

pkg compose root --output DIR --closure SPEC... --trust-key HEX... [--state-dir DIR] [--sbom FORMAT] [--licenses] [--dry-run]

pkg compose slice --output PATH --closure SPEC... --trust-key HEX... [--sbom FORMAT] [--licenses] [--dry-run]

pkg compose oci --output PATH --closure SPEC... --trust-key HEX... [--layout MODE] [--sbom FORMAT] [--licenses] [--dry-run]

pkg compose image (--output PATH | --target DEVICE) --closure SPEC... --trust-key HEX... [--force] [--sbom FORMAT] [--licenses] [--dry-run]

DESCRIPTION

Packages are the raw material. What you actually run is something built out of them: a container image, a VM disk, a live ISO, a plain directory tree you can chroot into. That's a composition, and pkg compose is the tool that makes one.

The recipe is always the same. Take a list of what you want, resolve it into a closure (everything those packages need, transitively), verify every manifest's signature against the keys you trust, and lay the result down in some shape. All the backends share that front half. They only differ in what they write at the end.

Composing is not installing

Nothing on the running system changes. The local package database isn't touched either. The one exception is compose root, which writes a fresh database inside its output, because that tree is a system of its own.

Two ways to say what you want

You can describe the artifact in a file, or you can spell it out on the command line. Both build the same artifacts.

A descriptor: pkg compose --composition FILE reads a composition.toml. The file names its own backend, its own repositories, its own trust keys and its own selection, so there's no subcommand. This is the one you want for anything you'll build twice. The descriptor gets reviewed and kept in git, and the same file produces the same artifact. Nobody has to retype a long command line correctly. See COMPOSITION DESCRIPTORS, below.
A backend subcommand: pkg compose root, slice, oci and image take the selection and the keys as arguments. Good for a one-off. It's also what the descriptor form turns into internally.

You pick one or the other. Give neither and you're told so:

error: compose needs either --composition=<file.toml> or a backend subcommand (root | slice | oci | image); see pkg compose --help

BACKENDS

root

Lays the closure down in a directory as an ordinary FHS tree. That tree carries its own package database at <output>/var/lib/pkg by default, the same path a live system uses. So the result stands on its own: chroot into it, tar it up, or boot it.

slice

Writes an immutable, signed erofs slice.

oci

Writes an OCI image layout. --layout picks the layer layout: merged (the default) or per-slice. The tree is staged next to the output, at the output path with its extension replaced by .staging.

image

Writes a bootable disk image: GPT with an ESP, systemd-boot, the default partition layout. The closure is staged in a tree beside the image, at the output path with its extension replaced by .staging. The image is built raw first, and that raw intermediate stays beside the staging tree. The file at --output is converted from it: qcow2 from the subcommand, or whatever a descriptor's [disk].format asks for (qcow2, raw or iso).

The staging tree is left in place afterwards. The next compose into the same output removes it first, so a smaller closure never inherits the files and installed records of the previous one. If anything is still mounted under it, that compose refuses instead of deleting through the mount. The same goes for the oci staging tree.

With --target instead of --output, the image is written straight onto a block device, and no image file is published at all. See --target, below.

COMPOSITION DESCRIPTORS

A composition.toml carries everything the subcommands take as arguments, plus the chain it resolves against. The full schema is in docs/design/images.md. Here's the shape:

[composition] name = "generic-cloud" description = "Headless cloud disk image" type = "disk" arch = "x86_64"

[[repos]] name = "Base" path = "/data/repo/Base" trust_key = "9c8084eb...f704"

[[repos]] name = "Common" path = "/data/repo/Common" trust_key = "543df494...9bb4"

[packages] needed = ["+sys", "openssh/server"]

What does type do?

[composition].type names the backend, which is why there's no subcommand. It is one of disk, oci, wsl, slice, fhs, pins or pkgtar:

disk: the image backend, shaped by the [disk], [guest] and [seed] tables.
oci: the oci backend, shaped by [oci].
wsl: a WSL distribution, which is the merged OCI layer as one gzip tar.
slice: the slice backend.
fhs: the root backend, with the database inside the tree.
pins: no artifact at all; see below.
pkgtar: refused. A bundle is made by pkg-bundle(1), which needs a --tuf-dir a descriptor has no field for.

The backend table in the file has to match the type. A type = "disk" file that only carries [oci] is refused. The alternative is composing it with every disk default while the author's [oci] gets silently ignored, and nobody wants that.

Where do the packages come from?

[[repos]] replaces whatever the command line configured. The composition resolves against the chain the file names, not against the invoking shell's --repo. File order is chain order, lowest precedence first. An empty or absent [[repos]] leaves the command line's configuration alone.

Relative path and trust_key_file values resolve against the descriptor's own directory. A composition, its repositories and its keys move as one unit.

You can also name a repository instead of describing it. repos = ["Dev"] takes the repository called Dev from --repos-dir, plus every repository it consumes, transitively. A name that isn't configured is refused. Strings and tables can be mixed in one inline array. The dispatcher uses this: it composes descriptors that name repositories, against a repos.d it writes from its own topology.

Which keys does it trust?

A trust_key is one hex key. A trust_key_file holds one hex key per line; blank lines and anything after a # are skipped, and a file with no key in it is refused. A repository whose signing key has rotated still carries packages signed by every key that built something it still publishes. So its file lists them all, and a rotation is one appended line.

A repository that names neither trusts every key its own verified root chain ever enrolled for the targets role. For a path repository that walk starts at metadata/1.root.json. For a url repository it starts at the first published root when that chain passes through the tuf_root anchor, and at the anchor otherwise. That way a clerk rotation needs no edit, and packages a retired clerk signed stay composable. The catch: a key revoked for compromise is trusted too. If that matters to you, name the keys explicitly.

With no [[repos]] at all, the command line's chain is walked the same way. --trust-key on the command line adds to all of this. If the whole lot comes up empty, the compose is refused and told where a key could come from.

What goes in?

[packages].needed takes the same grammar as --closure, +set references included. It takes no version pins. An artifact is composed from the repository's current set, so a pin could only repeat what's already current or name something the current set doesn't carry.

[packages].excluded is parsed, but not implemented. A descriptor that lists exclusions is refused, rather than quietly composed without them.

Seeding an account

[seed] points path (default seed, relative to the descriptor) at a directory of NoCloud documents. user-data is required; meta-data and network-config are optional. A disk provisions its first account from them.

layout = "nocloud", the default, writes them onto a FAT partition labelled cidata, and needs a nonzero [disk].seed_mib.

layout = "baked" partitions nothing. After the closure is installed, the target's own instance-setup --offline writes the account into the installed root. That needs --target, a seed_mib of 0 or absent, and no label. The compose fails if the target has no /usr/bin/instance-setup, or if the run exits nonzero. An image file with a baked account in it would be a shared credential in a published artifact, which is why this only works onto a device. See docs/design/bootable-images.md.

Where does it write?

Without --output, the artifact lands beside the descriptor as <name>.<ext>. The extension follows the type: .iso for a disk whose [disk].format is iso and .qcow2 for any other disk, then .oci, .wsl, .erofs, .root and .pins.toml.

A composition that builds nothing

type = "pins" materializes nothing. It resolves the closure, verifies every manifest's signature against the trust keys, and writes the closure as a pins file, in the format pkg offline export-pins writes (see pkg-offline(1)). A host that updates in place can compare its own export against it.

HELD MEMBERS ARE FATAL HERE

An interactive pkg-install(1) carries on when a set member is held: it reports the member and installs the rest. A composition won't:

error: compose: set member(s) no repo in the chain pins: bzip2, tar

An image built from a set that didn't fully resolve isn't the image anyone asked for, and it isn't reproducible either. Fix the chain, or name the members you want explicitly.

OPTIONS

--composition FILE

Compose from a composition.toml instead of a backend subcommand. You can't use both.

--output PATH

Where to write. A directory for root, a file for everything else. Required with a backend subcommand, except that image takes --target in its place. Optional with --composition, which otherwise writes beside the descriptor.

--target DEVICE

(--composition with type = "disk", or image.) Compose straight onto this block device, in place. The device's own size is used, nothing is created or truncated, and no image file is published. The device has to be an empty whole disk: unmounted, not held by anything, with no partition table and no filesystem signature on it. Can't be combined with --output, --sbom or --licenses.

A writable root is mounted under /run and installed onto directly. A read-only root is staged first, under /var/tmp, not under /tmp. That staging tree is the whole closure uncompressed, and /tmp is usually tmpfs.

--force

(With --target.) Write the device even though it isn't an empty whole disk. Everything it overrides is named on stderr. It never implies --yes: under the default confirm setting, passing it is exactly what makes the command ask first.

--closure SPEC

Can be repeated. The specs the closure is built from: a bare name, name/subpkg, or +set, the same grammar as pkg-install(1), minus version pins. Required with a backend subcommand. With --composition the selection comes from [packages].needed instead.

--trust-key HEX

Can be repeated. A hex-encoded Ed25519 public key to trust. Every manifest in the closure has to verify against one of them. Required with a backend subcommand. With --composition it adds to the keys the descriptor's repositories name or their root chains enrolled (see above), and it's never required.

--state-dir DIR

(root only.) Where to put the composed tree's package database. Defaults to <output>/var/lib/pkg.

--layout MODE

(oci only.) merged or per-slice. Default merged.

--sbom FORMAT

Also write a software bill of materials beside the artifact: spdx, cyclonedx, or both, comma-separated. The document lands at <artifact>.spdx.json or <artifact>.cdx.json: the artifact's whole name plus an extension, so the pair stays together under a glob.

The component list is the closure that was just resolved, not a scan of the produced tree. The document is only written after the backend produced something. Its timestamp is pinned rather than read from the clock, so it reproduces along with the artifact. See pkg-sbom(1) for what the projection guarantees.

Refused with --target, like --licenses. A device publishes no file, so there's nowhere beside it for a sidecar to go. Compose to --output and write the device from that.

--licenses

Also write <artifact>.licenses.json beside the artifact: each package's declared expression, the per-file evidence behind it, and how much of the closure couldn't be identified.

Why not inside the SBOM? The SBOM already states every declared expression exactly, in a format an auditor's tooling reads. Bending it to also carry per-file scanner evidence means bending a standard to hold something it doesn't model.

License text isn't carried. It's already inside the artifact, at the paths the report names.

--dry-run

Print the resolved closure and write nothing. No reports either, since there's no artifact for them to describe. With --target, the device isn't even looked at. That's what lets an installer size a disk before you've picked one.

All global options apply; see pkg(1).

OUTPUT

compose closure (94 package(s)) -> /srv/roots/console

bash 5.3-4 [/bin, /data]
coreutils 9.8-2 [/bin]

... composed target root: /srv/roots/console pkg-state: /srv/roots/console/var/lib/pkg

The header counts subpackages, because that's what the closure is and what gets fetched. The rows below it are collapsed to one per package. Pass --collapse=none for one row per subpackage; see pkg(1).

JSON

The plan, printed before anything is fetched:

{ "ok": true, "kind": "compose-plan", "output": "/srv/roots/console", "install": [ { "name":..., "subpkg":..., "version":..., "release":..., "digest":... } ] }

--dry-run stops there. Otherwise, once the artifact and its sidecars are written, a second document follows. It has the same shape, with kind compose-result, and it's the closure the artifact was composed from. A caller can record what went in without resolving the descriptor again. output is the artifact, or the device under --target. A failed compose emits the error envelope instead, and no result.

The release and digest fields are there for a caller deciding whether anything moved. A rebuild at the same version, which is what a release bump produces, would otherwise look identical to no change at all.

EXAMPLES

From a descriptor

pkg compose --composition generic-cloud/composition.toml

This writes generic-cloud/generic-cloud.qcow2, and says what it's building first:

composition generic-cloud (disk) -> .../generic-cloud.qcow2

Somewhere else

pkg compose --composition generic-cloud/composition.toml \ --output /srv/images/cloud.qcow2

Checking a descriptor before building it

pkg compose --composition generic-cloud/composition.toml --dry-run

A descriptor that names its repositories

The repos line has to come before any [table], or TOML puts it inside that table:

printf 'repos = ["Sandcastle-Common"]\n' > min.toml printf '[composition]\nname = "min"\ntype = "fhs"\n' >> min.toml printf '[packages]\nneeded = ["+console"]\n' >> min.toml pkg compose --composition min.toml --dry-run

A minimal filesystem tree

PUBKEY=$(cat /etc/pkg/signing.pub) pkg compose root --output /srv/roots/console \ --closure +console --trust-key "$PUBKEY"

Looking before writing

pkg compose root --output /srv/roots/console \ --closure +console --trust-key "$PUBKEY" --dry-run

Building a closure from explicit packages

pkg compose root --output /srv/roots/tiny \ --closure bash --closure coreutils --closure util-linux \ --trust-key "$PUBKEY"

Mixing sets and packages

pkg compose root --output /srv/roots/dev \ --closure +console --closure git --closure vim \ --trust-key "$PUBKEY"

Including development content on purpose

Compose doesn't read [install] include_subpkgs, so name what you want:

pkg compose root --output /srv/roots/build \ --closure +console --closure gcc --closure glibc/dev \ --trust-key "$PUBKEY"

Looking at what came out

pkg --state-root /srv/roots/console/var/lib/pkg list --collapse main pkg --state-root /srv/roots/console/var/lib/pkg \ verify coreutils --root /srv/roots/console

Going in

chroot /srv/roots/console /bin/bash

Keeping the database outside the tree

pkg compose root --output /srv/roots/console \ --state-dir /var/lib/composed/console \ --closure +console --trust-key "$PUBKEY"

An immutable slice

pkg compose slice --output /srv/slices/base.erofs \ --closure +console --trust-key "$PUBKEY"

An OCI image

pkg compose oci --output /srv/oci/console \ --closure +console --trust-key "$PUBKEY" pkg compose oci --output /srv/oci/console \ --layout per-slice \ --closure +console --trust-key "$PUBKEY"

A bootable disk image

pkg compose image --output /srv/images/ur.qcow2 \ --closure +console --trust-key "$PUBKEY" ls -d /srv/images/ur.staging

Straight onto a disk

pkg compose --composition server/composition.toml --target /dev/sdb

Trusting more than one key

During a key rollover, accept both:

pkg compose root --output /srv/roots/console \ --closure +console \ --trust-key "$OLD_PUBKEY" --trust-key "$NEW_PUBKEY"

Composing from an unconfigured repository

pkg --repo /srv/mirror/Common --repos-dir /nonexistent \ compose root --output /srv/roots/mirror-test \ --closure +console --trust-key "$PUBKEY"

Pulling out the closure list for review

pkg --json compose root --output /tmp/x \ --closure +console --trust-key "$PUBKEY" --dry-run \ | jq -r '.install[] | "(.name)/(.subpkg) (.version)"'

Watching it work

pkg -v compose root --output /srv/roots/console \ --closure +console --trust-key "$PUBKEY"

NOTES

Every backend resolves against the configured chain, the same one pkg-install(1) uses, and falls back to the single --repo directory only when no chain is configured. A descriptor that declares [[repos]] brings its own chain, for every backend.

Triggers are prebaked into composed artifacts, rather than run against the live system; see pkg-triggers(1) and docs/design/triggers.md.

ENVIRONMENT

PKG_COMPOSE_SCRATCH: Where a read-only root is staged before it's written onto a --target device. Defaults to /var/tmp.

EXIT STATUS

0 on success. 1 on any failure: a closure that doesn't resolve, a held set member, a manifest that doesn't verify against the trusted keys, a refused device, or a write error.