worker-compose.yaml, is able to
resolve, start, and stop each worker in dependency order, handles engine registration tasks, and
supervises running workers.
The daemon
The daemon is itself a worker. It registers under the namecompose and exposes the compose::*
functions, so every project operation is a standard trigger.
Bare iii compose is the command to start the compose worker/daemon. It reads worker-compose.yaml
in the working directory and starts only the compose worker.
iii compose --up is provided as a convenience. It starts the compose worker, the iii engine, and
the workers specified in worker-compose.yaml. It is approximately the equivalent of running iii,
iii compose, and iii trigger compose::up as separate commands.
iii compose --up is recommended for ease of development while more granular control and
independent operation of iii and iii compose daemons is suggested for production.
Starting a project with the daemon
Ctrl^C, SIGINT and SIGTERM all gracefully stop the daemon, every worker run by the daemon, and
the iii engine if compose was started with --up. When --up has started the engine, every project
and worker stops before the engine process is stopped. compose::stop is the function equivalent of
this operation.
compose::* functions as documented below are the intended way to manage a running compose daemon.
Compose logs
Compose logs stdout and stderr output from started workers to$HOME/.iii/compose/namespace/.
Logs are rotated every 10 MiB. Compose keeps up to 40 MiB of logs. Compose strips terminal control
sequences before persisting the engine output.
You can use the logs client for a recent snapshot or a live view:
Each line is prefixed with the worker name, and stderr uses a bold prefix on a terminal.
Running it in the background
Compose does not background or daemonize itself. To accomplish this please use standard tooling For example on most terminals (bash, zsh) you can run:The compose::* functions
These are the functions that control the compose worker and are the canonical way of interacting
with it and making basic changes to the worker-compose.yaml file.
file is not required. Left out, it falls back to a worker-compose.yaml in the daemon’s own
working directory; without one, the call fails with NO_COMPOSE_FILE. A relative file path is
considered relative to the daemon’s directory, not the caller’s working directory. You may pass an
absolute path as well.
Starting a project
compose::up starts every worker in the compose file, in dependency order. Workers that are already
ready stay as they are.
compose::up failures
A project that fails to start due to a project-related issue ends the command withPROJECT_DID_NOT_START. Partial starts are rolled back in reverse dependency order.
Stopping a project
compose::down stops the project in reverse dependency order.
Adding workers
compose::add worker=state worker=./workers/api declares one or more workers in the compose file
and reconciles the project once. On the CLI, repeat worker= for each worker. Each value takes a
registry package name (state), a package name with a version (state@0.21.4), or a directory
(./workers/api).
A JSON payload for this function can use a single list: { "workers": ["database", "web"] }.
An package without a version specified will pin to the latest available version (ex.
0.23.1).
Workers whose declarations did not change remain running. Existing workers whose resolved versions
changed restart in place, and newly declared workers start through the normal dependency plan.
Removing a worker
compose::remove worker=state removes the named worker and every reference to it. The changes are
validated before a worker is removed. Compose then stops only that worker and runs an idempotent
up.
Removal does not resolve the registry graph or remove other workers that were added with this
worker. Those remain declared until they are removed explicitly.
Restarting one worker
compose::restart worker=state stops that specified worker and starts it again. All other workers,
including dependencies, are left unchanged.
compose::restart without a worker argument restarts the entire compose project. It is
approximately the equivalent of compose::down followed by compose::up.
Updating a worker
compose::update worker=state updates a worker to either the current latest version when no
version is specified or to the target version when it is.
If the requested version is already installed this operation is treated as a NOOP.
package://. Use
compose::add to add new workers.
Workers specified with path:// are not versioned, any updates to these workers will be reflected
the next time the worker is restarted.
Unlike
compose::restart worker=state, an update restarts the whole project.Checking status
compose::status reports each declared worker with its state, its pid, an owned flag, its
rotating log_path, and last_error when there is one. owned is false for a worker this daemon
has knowledge of but does not manage (ie. was not started by the compose daemon).
Worker states
Viewing logs
compose::logs returns recent stdout and stderr for the workers of one project.
The response holds one entry per worker with
container, entries, cursor, and truncated. Each
entry in entries has stream and message. A cursor has generation and offset. truncated
is true when the cursor sent is older than the retained archives.
Listing projects
compose::list returns the daemon name, its namespace, its pid, and other project information.
Validating a file
compose::validate validates a compose file and is intended for package develpment work. It takes
file. Validation is offline, so package:// workers are reported under deferred_packages and
not resolved.
Validation reports
Stopping the daemon
compose::stop stops the compose project, all associated workers, and optionally the engine if
started with (--up). The compose daemon will also exit. Before exiting the daemon will return its
name, pid, and the projects it is about to stop.
Viewing schema
compose::schema takes a function_id argument. With no function_id, it returns every
compose::* schema. Pass a function id to return the schema for a given function_id. The pseudo-id
worker-compose.yaml returns the file’s JSON Schema as request and a complete small example as
response.
Each entry holds function_id, description, request, response, default_timeout_ms, and
idempotent. The same schemas, descriptions, and metadata are also published through
engine::functions::info.
Namespaces
Namespaces are used to allow advanced architectures that require more than one running copy of a given worker, multi-tenancy, some multi-agent workflows, and various isolation schemes between different parts of a iii application. Namespaces are arbitrary and their usage depends largely on the given usecase. They do not prescribe a specific way of constructing your iii application. The two primary points where Namespaces are used are during Worker registration via Compose and during Function and Trigger interactions. All have ways of declaring which namespace to use.Precedence
namespace is commonly defined in worker-compose.yaml but can be overriden on compose daemon
startup with the --namespace flag.
Likewise, compose’s own compose::* functions will exist within the same declared namespace.
What a namespace may hold
A valid namespace is made up of the lowercase charactersa-z, 0-9, - and _. All other
characters are not permitted and will result in an INVALID_NAMESPACE error; including uppercase
letters. To prevent naming conflicts there is no coercion of invalid namespaces to valid namespaces.
One daemon to a namespace
Two compose daemons with different namespaces can share an engine. However, a second daemon claiming a namespace that is already served is refused at registration withDAEMON_ALREADY_SERVING.
worker-compose.yaml
Below is an example of version 1 of a worker compose file. Unknown keys and duplicate keys are
errors. Durations can specify a unit such as: 500ms, 30s, 2m.
Top-level fields
Engine fields
Allowed worker keys are
configuration, iii-worker-manager, iii-http-functions, iii-stream,
and iii-sandbox. Use #instance for another instance of an allowed type, for example
iii-worker-manager#rbac. Internal iii-engine-functions, iii-telemetry, and iii-observability
are injected and must not be declared.
These workers are always engine managed and started so this method of operating these workers is
an exception to the typical way for other workers. Changes to these workers requires an engine
restart.
Worker fields
Each key undercontainers is the name the worker registers under.
path:// directories resolve against the compose file’s directory. A missing directory fails with
MISSING_WORKER_DIRECTORY, and a missing env_file fails with MISSING_ENV_FILE during
validation.
A path worker normally runs as a host process. A non-empty runtime.base_image in its
iii.worker.yaml selects a local VM instead. The worker’s scripts.install and start command then
run inside that image, and compose keeps the VM state inside the project. An invalid image reference
fails the start instead of falling back to the host or to another image.
Worker kinds
Worker packages can be released in multiple different “kinds”. A kind compose cannot run fails withUNSUPPORTED_PACKAGE_KIND.
A bundle’s VM is booted by
iii-worker, which the installer ships beside iii and which needs
glibc on Linux. Compose runs it as a process rather than linking it, so the engine stays portable; a
bundle worker on a machine without iii-worker fails saying so, and every other worker kind is
unaffected.
Bundles need a VM, and windows has none: a bundle worker there fails with BUNDLE_NEEDS_A_VM before
anything is downloaded. Run compose under WSL, where the VM has KVM to run on. Every other worker
kind runs on windows as it always has.
Bundle support can be refused machine-wide with III_BUNDLE_WORKERS_DISABLED=1, which compose
honours.
Scripts
Both hooks run with the worker’s environment, working directory, and their own process group. A
post_run runs after the worker stops but before the compose daemon exits. The top-level
stop_timeout argument is a global timer for a compose daemon to stop. If this time is exceeded all
scripts will be exited along with the compose daemon.
A path worker’s start command is scripts.run in worker-compose.yaml, or scripts.start in the
worker’s own iii.worker.yaml when run is absent. A worker with neither fails with
MISSING_START_COMMAND.
Configuration precedence
Lowest to highest: the configuration a package ships, the entry in the configuration worker, thenconfig_override. The merged result is written to an owner-only file and its path is passed to the
worker as III_CONFIG. A worker that declares config_name does not start when the fetch fails;
the error is CONFIG_FETCH_FAILED.
The worker environment
A worker’s environment is defined by the following sources:- A host baseline. On Unix:
PATH,HOME,USER,LOGNAME,SHELL,TERM,TMPDIR,TZ,LANG,LC_ALL. Windows adds the variables the platform needs, such asSystemRoot,COMSPEC, andPATHEXT. - The worker’s
env_fileentries, then itsenvironmentmap. - The reserved variables, which the daemon owns.
Declaring a reserved variable in
environment or an env_file fails with RESERVED_ENV_OVERRIDE.
Expanding values from environment variables
Variables can be expanded with${VAR}. References expand in any value, not only in environment.
For example:
Build registry packages
build reads and validates the compose file, then downloads every package:// worker into the same
cache used by compose::up. The file defaults to ./worker-compose.yaml. The command does not
connect to an engine, start a worker, or run lifecycle hooks. Local path:// workers and
engine-managed workers need no registry download and are skipped.
~/.iii/compose/packages, or under III_COMPOSE_STATE_DIR when that
variable is set. A later compose::up reuses a valid cached artifact if it exists.
Readiness
A worker is considered up when the engine reports a worker of that name in the project’s namespace. Compose pollsengine::workers::list every 200 ms until the worker’s startup_timeout runs out.
After a worker is ready, the daemon checks it every 250 ms. A worker that exits takes its transitive
dependents down with it and is recorded as
failed. Automatic restarts are not performed. When the
engine connection drops and comes back, every running worker gets its startup_timeout to register
again.
Dependency shutdown is dependent upon when a shutdown happens:
Where compose keeps state
Compose state is stored under~/.iii/compose, or under $III_COMPOSE_STATE_DIR when that is set.
<ns> is the daemon’s namespace. <project> is derived from the compose file’s canonical path + a
short hash to prevent project name collisions.
Error codes
Compose can output the following error codes:Related
For why compose is a worker see Understanding iii / Compose. For
understanding namespaces see Understanding iii / Namespaces.
For the
iii compose entry in the command tree, see the CLI
reference.