worker-compose.yaml, starts
each container in dependency order, waits for the engine to register it, and supervises it
afterwards.
The daemon is itself a worker. It registers under the name compose in a namespace of its own and
exposes the compose::* functions there, so every project operation is a normal
trigger, addressed to one daemon with --namespace.
Bare iii compose is the daemon command. It reads worker-compose.yaml when the file exists so
the namespace and engine URL can configure the daemon. The --up flag also loads the file as a
project and starts its containers. iii compose logs is a read-only client for a daemon that is
already running. iii compose build is a local action that prepares registry packages without
starting the daemon.
Build registry packages
build reads and validates the compose file, then downloads every package:// container into the
same cache used by compose::up. The file defaults to ./worker-compose.yaml. The command does not
connect to an engine, start a worker, or run lifecycle hooks. Local path:// workers and built-in
engine workers need no registry download and are skipped.
~/.iii/compose/packages, or under III_COMPOSE_STATE_DIR when that
variable is set. A later compose::up still resolves package metadata, but it reuses a valid cached
artefact and does not download its bytes again. If the cache is missing, up keeps its existing
download fallback.
The daemon
A daemon holds any number of projects, and any number of daemons share one engine. What tells them
apart is the namespace: the worker name is always
compose, so the engine leases (namespace, compose) to one connection. Two daemons with different namespaces coexist; a second claiming one
that is taken is refused at registration with DAEMON_ALREADY_SERVING.
The daemon inherits namespace: from worker-compose.yaml when --namespace is absent. If the
file has no namespace, or if the daemon starts outside a project, it uses default. An explicit
CLI namespace also overrides namespace: for every project loaded by that daemon. The engine
refuses a second daemon claiming a namespace that is already served.
SIGINT and SIGTERM both stop the daemon and take every project down with it. compose::stop
does the same over the engine. When --up owns the engine, every project and worker stops before the
engine process is stopped.
Starting a project with the daemon
iii compose --up validates the initial file, selects engine ownership, and brings the project up
without waiting for a call to name it. The daemon stays in the foreground to supervise its
containers and, in managed mode, the engine process.
--file defaults to ./worker-compose.yaml, the same fallback a compose::* call gets when it
names no file. The daemon then stays in the foreground and serves, so every other operation is a
compose::* call as usual.
If --namespace is absent, this initial --up also uses the file’s namespace: for the compose
daemon. An explicit CLI namespace has priority over the file.
When the file contains engine: and no explicit --engine is present, Compose materializes an engine-only config under
~/.iii/compose/<daemon-namespace>/engine-config.yaml, starts the engine, and removes the generated
file after a clean shutdown. An explicit --engine takes priority and connects to that existing
engine instead. A process-wide III_URL does not override a managed file engine.
engine.log.1 through engine.log.3, so one
namespace uses at most about 40 MiB for engine logs. The printed follow command keeps reading the
active engine.log across rotations. Compose strips terminal control sequences before persisting
the engine output.
Reading worker output
Compose captures each worker’s stdout and stderr until the worker exits, including output written after it becomes ready. Retained history is bounded: the active file rotates at 10 MiB and keeps three archives, so one worker uses at most about 40 MiB. Older output can be truncated or deleted after rotation. Terminal control sequences are removed before output is stored or returned. Use the logs client for a recent snapshot or a live view:logs. --file names a compose file on the daemon host;
when it is omitted, the daemon uses worker-compose.yaml in its own working directory. Each line
is prefixed with the worker name, and stderr uses a bold prefix on a terminal.
A project that does not start ends the command with PROJECT_DID_NOT_START. Rollback has already
stopped whatever came up, so there is nothing left to supervise.
Without engine:, Compose never starts or stops an engine. It uses --engine, then III_URL, then
the local default ws://127.0.0.1:49134.
Running it in the background
Compose never backgrounds itself. Even when--up runs its managed engine in the background, the
compose daemon serves in the foreground and writes its own output to stdout. This is the shape every
process supervisor already expects, and it leaves log rotation and restart-on-failure to something
that already does both.
For a quick session, a shell redirect is enough:
The compose::* functions
All functions accept the same payload.
A project is its compose file, and nothing else names one. The same file reached twice is the same
project however it was spelled, so there is no second identity to keep in sync and no way to point
one at the wrong file.
file is not required. Left out, it falls back to a worker-compose.yaml in the daemon’s own
working directory. worker and workers have no fallback.
compose::schema is read-only. With no function_id, it returns every compose::* contract.
Pass a function id to return one contract. The pseudo-id worker-compose.yaml returns the file’s
JSON Schema as request and a complete small example as response. The same function schemas,
descriptions, and metadata are also published through engine::functions::info.
Adding workers
compose::add declares one or more workers in the compose file and reconciles the project once.
On the CLI, repeat worker= for each worker. Each value takes a registry name (state), a name
with a version (state@0.21.4), or a directory (./workers/api); a leading . or / is what
makes it a path, since a registry reference may carry a host of its own. The JSON function contract
uses one list: { "workers": ["database", "web"] }. The old singular JSON field remains accepted
for one worker.
An unpinned name is resolved once and written out as an exact version, so a
later up cannot quietly get a different build. The same worker at the same
version changes nothing and says so; at a different version it is replaced,
which is how an upgrade or a rollback is asked for.
Workers whose declarations did not change remain running. Existing workers whose resolved
versions changed restart in place, and newly declared workers start through the normal dependency
plan. This lets a worker call compose::add for its own project without stopping the caller before
the result can return.
A worker is rarely alone. Its manifest names what it calls, and the registry
answers with that whole graph already pinned to versions that satisfy each
other, so those are declared too, as containers with their own start_after.
Nothing starts behind the file: what runs is still what the file says, and an
operator can read it, pin it differently, or take a container out.
Two rules shape the expansion. Requesting a root package whose registry kind is engine fails with
ENGINE_WORKER_IS_BUILTIN, because the engine already supplies it. Engine-kind dependencies of a
normal project worker are filtered from the graph; an edge to one is dropped rather than written,
since start_after may only name a container the file declares. And a worker two others need is
declared once, named by both.
The file is edited, not rewritten: comments, blank lines and quoting survive,
entries are appended, and the result is parsed before it is written, so a bad
edit never reaches disk. What is written is worker, version and
start_after. It writes no scripts, so a path:// worker needs
scripts.start in its own iii.worker.yaml to start. A path:// worker is added alone: its
dependencies are declared in a manifest on disk rather than in the registry’s
answer.
file out when the daemon’s own working directory holds a
worker-compose.yaml; without one, it fails with NO_COMPOSE_FILE. A relative file resolves
against the daemon’s directory, not the caller’s, so pass an absolute path when they differ.
compose::restart is a down followed by an up, with the compose file read again between them,
so an edit made by hand takes effect without restarting the daemon. It accepts container like the
two halves it is made of. Nothing cleverer yet: no rolling restart, and no keeping a container that
did not change.
compose::stop stops a compose project but returns before the daemon exits.
compose::validate answers for a file and holds nothing: a CI job that only ever validates leaves
the daemon owning nothing. Validation is offline, so package:// containers are reported under
deferred_packages instead of being resolved.
Removing a worker
compose::remove worker=state removes the named container and every surviving start_after
reference to it. The complete edited declaration is validated before the file or any process
changes. Compose then stops only that container and runs an idempotent up, so healthy surviving
workers stay running and anything already missing can start.
Removal does not resolve the registry graph or remove other workers that were added with this
worker. Those remain declared until they are removed explicitly.
Restarting one worker
compose::restart worker=state stops that container and starts it again. Nothing else moves: not
what it depends on, and not what depends on it.
That is a decision, not a shortcut. A dependent holding a connection to the worker sees it drop, and
compose does not restart the dependents to hide it. Which of them tolerate a drop is the operator’s
knowledge, not compose’s. Left out, worker restarts the whole project instead: down, then up, with the file
read again between them.
Updating a worker
compose::update worker=state moves a declared container to another version of the same package.
changed is false.
The container has to be declared already, and has to be a package:// one: this edits a version
line, it does not add a container, and a path:// worker has no version to move. Use
compose::add to declare something new.
Only the version line changes. A start_after written by hand comes through as it was, since
rewriting the graph is what compose::add is for.
Unlike
compose::restart worker=, an update restarts the whole project. A project is held as its
file was read, so a new version is only picked up once the project is dropped and read again.
Dropping it while its other containers run would leave them supervised by nothing.Operation results
compose::up and compose::down return the same fields.
Each entry in
containers contains container, state, changed. If a container failed it will
also contain an error object with code and message.
A failed up tears down any running containers, in reverse startup order, and reports those
containers as stopped with changed: false. Containers that were already running before the
operation are left alone.
Container states
compose::status reports each declared container with its state, its pid, an owned flag, its
rotating log_path, and last_error when there is one. owned is false for a container this
daemon can see but did not start.
Validation reports
worker-compose.yaml
This documentation reflects version 1 of a worker compose file. Unknown keys and duplicate keys
are errors. Durations can specify a unit:
500ms, 30s, 2m.Top-level fields
The namespace must already be
[a-z0-9_-]. A value outside that set is refused with
INVALID_NAMESPACE rather than rewritten to fit, so what the file declares is what an operator
types into --namespace. Nothing about the file’s path enters it, so two copies of one project
collide instead of running side by side.
Engine fields
Allowed worker keys are
configuration, iii-worker-manager, iii-http-functions, iii-stream,
and iii-sandbox. Use #instance for another instance of an allowed type, for example
iii-worker-manager#rbac. Internal iii-engine-functions, iii-telemetry, and
iii-observability are injected and must not be declared.
The engine section belongs to the file that started the managed daemon. A runtime change returns
ENGINE_RESTART_REQUIRED; restart the Compose invocation to apply it. A different file cannot
merge another section into that daemon and fails with ENGINE_ALREADY_OWNED.
Container fields
Each key undercontainers is the worker name the container registers under.
path:// directories resolve against the compose file’s directory. A missing directory fails with
MISSING_WORKER_DIRECTORY, and a missing env_file fails with MISSING_ENV_FILE during
validation.
A path worker normally runs as a host process. A non-empty runtime.base_image in its
iii.worker.yaml selects a local VM instead. The worker’s scripts.install and start command then
run inside that image, and compose keeps the VM state inside the project. An invalid image reference
fails the start instead of falling back to the host or to another image.
Worker kinds
The registry answers with a kind, and it decides how the container runs. A kind compose cannot run fails withUNSUPPORTED_PACKAGE_KIND.
A bundle is published code that compose did not build, so it is started behind the same boundary
iii add puts it behind, rather than as a host process. Its configuration is published into the
guest, so III_CONFIG names a path inside the VM; a worker reads it the same way either way.
Bundle support can be refused machine-wide with III_BUNDLE_WORKERS_DISABLED=1, which compose
honours.
A bundle’s VM is booted by iii-worker, which the installer ships beside iii and which needs
glibc on Linux. Compose runs it as a process rather than linking it, so the engine stays portable;
a VM container on a machine without iii-worker fails saying so, and every host worker is
unaffected.
Bundles need a VM, and windows has none: a bundle container there fails with
BUNDLE_NEEDS_A_VM before anything is downloaded. Run compose under WSL, where the VM has KVM to
run on. Every other worker kind runs on windows as it always has.
Scripts
Both hooks run with the container’s environment, working directory, and their own process group.
The start command for a
path:// container is run, then scripts.start from the worker’s
iii.worker.yaml. A container with neither fails with MISSING_START_COMMAND.
When the manifest declares runtime.base_image, the same precedence applies inside the VM: run
replaces scripts.start, but it never moves the worker back to the host. Host source changes do not
automatically restart this VM. Use compose::restart worker=<container> after a source change.
Where the two files describe the same thing, worker-compose.yaml wins and the manifest is the
default. run overrides scripts.start, and the container key overrides the manifest’s name: the
key is what reaches the child as III_WORKER_NAME, so a worker honouring the reserved contract
registers under it whatever its manifest declares. A worker that hardcodes its name instead is
caught at readiness by WORKER_NAME_MISMATCH, which reports the name it took.
Configuration precedence
Lowest to highest: the configuration a package ships, the entry the configuration worker holds, thenconfig_override. The merged result is written to an owner-only file and its path is passed to the
container as III_CONFIG. A container that declares config_name does not start when the fetch
fails; the error is CONFIG_FETCH_FAILED.
The container environment
A container’s environment is built, and the daemon’s own environment is not inherited wholesale. Three layers apply, lowest to highest.- A host baseline. On Unix:
PATH,HOME,USER,LOGNAME,SHELL,TERM,TMPDIR,TZ,LANG,LC_ALL. Windows adds the variables the platform needs, such asSystemRoot,COMSPEC, andPATHEXT. - The container’s
env_fileentries, then itsenvironmentmap. - The reserved variables, which the daemon owns.
Declaring a reserved variable in
environment or an env_file fails with RESERVED_ENV_OVERRIDE,
in both cases at compose::validate time.
For why the daemon owns these eight rather than treating them as defaults a container can replace,
see Understanding iii / Compose.
Reading a value from the host
The baseline is short on purpose, so nothing an operator exported reaches a worker by accident. A file names the values it wants instead, with${VAR}.
environment: a worker path, a version, an env_file
entry. The file on disk is never rewritten.
What config_override keeps
config_override is never expanded, at any depth. That block is not compose’s to read: it is
carried to the configuration worker, which resolves ${VAR} references of its own at read time.
That is how a secret is stored as a reference rather than as a value.
Readiness
A container is considered up when the engine reports a worker of that name in the project’s namespace. Compose pollsengine::workers::list every 200 ms until the container’s
startup_timeout runs out.
After a container is ready, the daemon checks it every 250 ms. A container that exits takes its
transitive dependents down with it and is recorded as
failed. Nothing is restarted: v1 has no
restart policy, so a container that dies stays down until the next up. When the engine connection
drops and comes back, every running container gets its startup_timeout to register again.
What a failure takes with it depends on when it happens, and v1 has no way to declare otherwise.
So a container nothing depends on ends the whole
up if it fails at start, and is contained if it
fails a minute later. See Understanding iii / Compose.Where compose keeps state
Everything sits under~/.iii/compose, or under $III_COMPOSE_STATE_DIR when that is set.
<ns> is the daemon’s namespace. <project> is derived from the compose file’s canonical path:
readable enough to recognise, hashed enough that two projects in directories of the same name stay
apart. Because it is derived, it cannot be guessed. compose::status reports it as state_dir
and reports the exact log_path for each worker:
For why this is one place per machine rather than a directory beside each compose file, see
Understanding iii / Compose.
print,
console.log, println!, stdout, and stderr output available even when the worker does not use the
structured logger. Use engine::logs::list when structured OpenTelemetry fields and trace
correlation are required.
A clean shutdown clears the state file. After an unclean exit, the daemon compares each record
against the live process: a match is adopted, a dead process is recorded failed, and a live pid it
cannot verify is left running and reported for manual cleanup.
Error codes
Compose errors cross the wire with a stable code and a message.Related
For why compose is a worker and why a project is its file, see Understanding iii /
Compose. For the namespace dimension itself, see Understanding iii
/ Namespaces. For the
iii compose entry in the command tree,
see the CLI reference.