Runtimes
A runtime is any program that runs one keyed process at a place and tells
Fleet about it.
Fleet reaches every session through it and links no runtime's code.
runtime-mux (crates/runtime-mux) is the default runtime. How the runtime fits between thurbox
and Fleet is the overview.
How runtime-mux reaches hosts, where psmux differs from tmux, and its
status channel are runtime-mux.md.
Two roles #
- A runtime is a role. Any program that answers
the runtime protocol at a place is a runtime.
runtime-muxis the runtime that Fleet ships. It runs sessions in tmux or psmux.examples/runtimes/tmux.shis a minimal runtime in a shell script. Write your own for containers, VMs or a remote service.fleet runtime checkruns the conformance suite against it. - A workspace provider is a role inside a runtime. It is a program that
a runtime runs to turn a workspace URL into a directory.
workspace-gitis the provider that Fleet ships, for git URLs. Fleet does not see the provider: Fleet sees only the runtime'sworkspaceverb.runtime-muxruns the provider that itsworkspace.programsetting names. Any program that answers the same four calls can replaceworkspace-git(the calls). Another runtime can make workspaces without a provider.
This page owns two contracts and one verb:
- the runtime protocol, how Fleet drives a runtime;
- the terminal stream, how a view attaches to a session on any runtime;
- workspaces, how a list of URLs becomes the directories a session works in.
The runtime protocol #
A runtime is any program. Fleet runs it through fleet-exec, one bounded
process per call: <program> <verb>, a JSON request on stdin, and on stdout
{"known": …} or {"unknown": {"why": …}}. Unknown never concludes.
| Verb | Required | Answer |
|---|---|---|
manifest |
yes | protocol version, the verbs supported and, from version 2, each place's readiness |
start |
yes | adopts the one live process holding the key, else starts argv with env in the workspace made under that key |
inspect |
yes | None, Running{id}, Exited{id, code}, and Paused with the pause verb |
stop |
yes | a process already gone counts as stopped; release: true also removes the key's workspace |
pause |
no | stops the key's process and keeps the key: inspect answers Paused until the next start or stop (pause) |
list |
no | every key the runtime holds a process, a pause or a workspace for, for the collector |
input |
no | types text, optionally submits |
capture |
no | screen text, cursor, foreground process, title |
exec |
no | a bounded command, in: host or in: workspace (inside the place), deadline and output cap |
workspace |
no | {key, urls} → one directory per URL, as the process sees it (workspaces) |
attach |
no | {stream: {argv}} or {tty: {argv}}: how to open a terminal stream (the terminal stream) |
shell |
no | {stream: {argv}} or {tty: {argv}}: how to open a shell beside the key's process, on its place (the shell) |
browse |
no | a directory's subdirectories on the place, each marked when it is a git checkout (browse and branches) |
branches |
no | a repository's branches on the place and its default (browse and branches) |
Every verb is idempotent by key. The runtime knows no harness, Task or
journal: pause is pause, resume and revive are start with the harness's
resume argv, idle comes from agent hooks, delivery is proven by the agent's
prompt hook. Fleet reads an agent's state off the capture screen with its
own rules, below the hooks; the runtime only returns the screen.
Fleet's config names each runtime in [runtimes.<name>], a program, and
its places. A Task's spec.host names a place,
<runtime>/<place>, so it chooses the program
(configuration).
Protocol version 1 #
The reference script answers it, runtime-mux answers it as well as
version 2, Fleet's exec runtime kind speaks
both, and fleet runtime check holds both runtimes to it in CI. Fleet calls
workspace before a Task's first start, bounded by the runtime's
workspaceTimeoutMs. It reads the manifest's workspace.schemes first and
refuses a URL whose scheme is not listed, without a call. fleet attach asks
attach once the runtime's manifest declares it.
Every request is a JSON object carrying "protocol": 1; manifest answers
whatever the request says. A runtime that does not speak the version asked
answers Unknown. An answer is {"known": …} or {"unknown": {"why": …}}
on stdout, with exit status 0; anything else (a program that is not there, a
non-zero exit, a timeout, output that does not parse) is Unknown to Fleet.
A key is 1 to 128 ASCII letters, digits, ., - or _; any other answers
Unknown.
| Verb | Request (besides protocol) |
Known answer |
|---|---|---|
manifest |
— | {"protocol": 1, "verbs": [...]}, and workspace: {schemes, survivesStop} with the workspace verb |
start |
key, argv, env (object), dir (absolute, optional) |
{"started": {"id", "adopted"}}, or {"failed": {"why"}} when nothing can have started, such as a missing multiplexer |
inspect |
key |
"none", {"running": {"id"}}, {"exited": {"id", "code"}}, code null for a signal, or "paused" |
stop |
key, release (optional) |
"stopped", whether or not a process was there |
pause |
key |
"paused", or "none" when no process holds the key and the key is not paused |
list |
— | {"keys": [...]} |
input |
key, text, submit |
"sent", or "none" when no process holds the key |
capture |
key, lines (optional) |
{"screen": {"text", "cursorRow", "cursorCol", "foreground", "title"}}, or "none"; title is the one the program set (OSC 0 or 2), and a runtime that cannot read it omits it |
exec |
argv, in (host), dir, timeoutMs, outputCap |
{"exited": {"code", "stdout", "stderr"}}, {"timedOut": {"stdout", "stderr"}} or {"failed": {"why"}} |
attach |
key |
{"stream": {"argv"}}, {"tty": {"argv"}}, or "none" when no process holds the key |
shell |
key |
{"stream": {"argv"}}, {"tty": {"argv"}}, or "none" when no process holds the key (the shell) |
workspace |
key, urls |
{"made": {"dirs", "adopted"}}, {"unsupported": {"url"}} or {"failed": {"why"}} (Workspaces) |
runtime-mux answers every verb above. It runs its windows on a tmux
server of its own, socket runtime-mux (RUNTIME_MUX_SOCKET names
another), never on thurbox's interface socket: an inherited
THURBOX_SOCKET is ignored. On Linux, when a systemd user manager answers
at $XDG_RUNTIME_DIR/systemd/private, runtime-mux starts that server in
a scope of its own (systemd-run --user --scope --collect). The server is
then outside the cgroup of the process that called runtime-mux, so a
stop of the Fleet unit does not stop it. runtime-mux asks once per
process whether it can have a scope (systemd-run --user --scope -- true).
When that probe fails, it starts the server without one. A process is a
window named after its key, and a pane kept after its program exits is
how inspect reports the code; a start replaces it. A host reboot
takes the server and its windows, so runtime-mux also keeps a start
record for each key it starts:
- A
startwrites the record, a file that holds the key's pane id, under${XDG_DATA_HOME:-~/.local/share}/runtime-mux/started/, which a reboot keeps. The record is<digest>/<server>/key.<key>.<digest>names the private directory that holds the server's socket.<server>is the server's name,<socket>or<socket>@<place>, in hexadecimal, split into directories of at most 128 characters. So no key or place name can make two servers share a directory, and a long name still fits the filesystem's limit on a file name. - A
stopremoves the record. inspectanswers{"exited": {"id", "code": null}}for a key with a record and no window. Fleet then revives the session on its conversation.listnames a key with a record.
Starts and stops on one server run one at a time, so
two starts of one key make one window: they hold a kernel lock on a file in
the private directory tmux keeps the socket in (<TMUX_TMPDIR or /tmp>/tmux-<uid>, made 0700 when missing and refused when another user
can write to it), opened without truncating or following a link. A remote
place's server has locks of its own, named after the place, so a slow place
never holds up a start on this machine. exec runs in: host only, on the
request's place, and kills the command's process group at the deadline. It reads
each stream to its end and keeps outputCap bytes, so a long output does
not change how the command ends; a child that left the group and holds a
stream open does not hold the answer past the deadline. The runtime-mux
program serves this machine and remote places over ssh and WSL
(runtime-mux's places). It makes workspaces on this
machine by running workspace-git there
(its workspaces). A workspace call takes a lock
of its key only, never the server's lock: a first clone or a long setup step
does not hold up a start or a stop of another key. A stop that releases a
workspace takes the key's lock first, then the server's, and lets go of the
server's lock once the process is stopped: the teardown and the release hold
the key only.
thurbox's stream pane (stream_panes in its settings.toml) is a view: it
opens a local tmux pane through runtime-mux view --scrollback <lines> --pane <id> on its own socket, with its own scrollback_lines, and decodes the
frames into the chunks its control-mode pane gives.
Size requests still go to the server through thurbox's own backend.
runtime-mux answers attach with a stream: its own view on the key,
naming its socket and TMUX_TMPDIR. runtime-mux view adopts the key's pane
through its control-mode client, sized by the view's first resize, and
encodes what the server sends as frames; a detach lets go of the pane and
leaves the process running. On a multiplexer that snapshots a pane in step
with its output (tmux, rmux), the first screen is the first snapshot or
resync, and a state frame follows it with the title and modes set before
the view came. On one that cannot (psmux), hello names no snapshot and
the history and that state arrive as output. The reference script answers attach with a raw
tmux client on the key's pane, a tty answer. An attach argv runs where the view runs, so it names
everything it needs, such as a socket by its path.
What Fleet's exec kind does with the answers:
startruns the harness profile's launch argv (harness profiles), withFLEET_SIGNAL_FILEinenv;- a session's id is its key, so it outlives each process started under it;
- readiness, workspace trust and an idle agent come from the agent's signals:
an agent that reported anything but
blockedis ready and past any trust dialog, and a running agent is at rest once it last reportedidle. An agent that reports nothing before its first turn is read off itscapturescreen as its harness profile says (harness profiles). Before its first report, a trust dialog on itscaptureis Untrusted, and anything else Unknown, so a brief waits; - delivery is proven by the agent's prompt hook, not by
input; - the agent state of a session whose hooks do not say it now is read off its
capturetext and title with the harness's screen rules (screen rules), at most one capture per session every 2 s. A screen state is for a view: readiness, rest and delivery still wait for the hooks; - an
exitedprocess is lost, since astopleavesnone: revive isstartwith the profile's resume argv on the conversation the hooks reported. Fleet first marks the restart in the signal file, so readiness and trust wait for the new process's own report; pausedis a paused session, and resume isstartwith the resume argv, as for a revive. Fleet pauses a session only on a runtime whosemanifestdeclarespause. On any other runtime, whether a session is paused stays Unknown, and Unknown never pauses.
Agent hooks append to the file Fleet passes as FLEET_SIGNAL_FILE in start
(signals on a remote place). Fleet reads it
directly on local, so a local runtime needs nothing for signals.
fleet runtime check -- <program> [args] is the conformance suite. It prints
one line per case, ok or FAIL with why, and exits non-zero when any fails:
manifestnames protocol 1 or 2 and the four required verbs;- a protocol mismatch answers Unknown;
- an error (a request that is not JSON, a bad key, an unknown verb) answers Unknown, never None;
inspectof an absent key answers None, and a stop of it succeeds;- concurrent starts with one key leave one process: four at once answer one id, and only one of them made it;
- two starts with one key leave one process: the same id, the second
adopted, and the key once in
listwhen the runtime has it; - a stop ends the process;
- an exited process answers Exited with its code, or None;
- with
workspacedeclared: two calls with one key answer the same directories, the second adopted, and a release leaves afile://directory in place; - with protocol 2:
manifestreportslocal, the place it was asked about, and a request without a place answers Unknown. The other cases namelocal; - with
browsedeclared: a directory with a plain subdirectory, a checkout and a file lists the two subdirectories, the checkout marked, and a missing directory answersfailed; - with
branchesdeclared: a repository with the branchesmainandfeatlists both, in any order, withmainas its default, and a directory inside a checkout answersfailed. The check makes the repository with this machine'sgit; - with
pausedeclared: a pause of a running key answerspaused, and theninspectanswerspausedandlistnames the key; a pause again answerspaused; astartruns the key again; a pause of a key nothing holds answersnone; and astopof a paused key leavesnone; - with
shelldeclared: a key no process holds answersnone, a running key answers astreamor attyargv on two calls in a row, and after a stop the key answersnone. The check does not open the shell, so it does not prove that the second call reaches the same one.
The cases start sleep and sh in the runtime's place. CI runs it, through
crates/fleet-cli/tests/e2e_runtime_exec.rs, against runtime-mux and against
examples/runtimes/tmux.sh, a reference
runtime in one POSIX sh script over tmux and jq. It serializes starts and
stops with a kernel lock in tmux's private socket directory, taken with
flock(1) or else perl, which a killed holder releases.
Pause #
pause stops the key's process and keeps the key, so a later start
resumes the session in the same workspace. Fleet uses it to free the memory
and the processor time of an idle agent
(pausing an idle session).
- After a pause,
inspectanswers"paused"andlistnames the key.input,capture,attachandshellanswer"none": no process holds the key. - The workspace stays as it is.
- A
startof a paused key starts itsargv, and the key is no longer paused. Astopof a paused key leavesnone, andrelease: trueremoves its workspace as for any key. - A pause of a paused key answers
"paused". A pause of a key that no process holds answers"none". - The runtime knows no harness and no conversation: the argv that resumes
the conversation is Fleet's, in the
startthat follows.
runtime-mux pauses a key in three steps:
- It closes the key's shell.
- It writes a mark for the key, an empty file
${XDG_DATA_HOME:-~/.local/share}/runtime-mux/paused/<digest>/<server>/key.<key>.<digest>and<server>are the start record's. - It stops the agent's window.
A pause takes the server's lock, as a start and a stop do. While the
agent's window is there, it answers for the key, so a pause cut short
before step 3 reads as running. inspect answers paused for a key with
a mark, no agent window and no shell. A mark with a shell left answers
Unknown, and the next pause closes the shell. A start removes the mark after it makes or adopts the
window, and a stop removes it after it stops the process. A reboot keeps
the mark, as it keeps the start record, so a paused key reads as paused
after a reboot and never as exited.
An earlier runtime-mux wrote the mark as
runtime-mux-<socket>.paused-<key> in the private directory tmux keeps the
socket in, which a reboot empties. For a rollback to that version:
- A pause writes the mark in both places.
- A
start,stop,pauseorinspectof a key copies its earlier mark to the data directory, under the server's lock. Alistcopies every earlier mark. The copy leaves the earlier mark in place. - A
startand astopremove both marks. - An earlier
runtime-muxremoves only its own mark. A key that it starts or stops after a rollback keeps its mark in the data directory, and reads as paused again after the next update.
Places (protocol version 2) #
The decision is ADR-42.
A place is where a runtime runs a process: this machine, an ssh host, a WSL
distro, a VM. A place's id is <runtime>/<place>, with local as the short
form of <runtime>/local on the default runtime
(configuration).
Fleet config Fleet runtime
[runtimes.mux] │ │
program = "runtime-mux" │ manifest {places} │
[runtimes.mux.places.devbox] ────────▶├─────────────────────▶│ checks each place
kind = "ssh" │◀─────────────────────┤ each place: ready, why
destination = "me@devbox" │ │
│ start {key, place} │
Task spec.host = "mux/devbox" ───────▶├─────────────────────▶│ runs it on devbox
│ place = {name,config}│
Changes from version 1:
-
placeon every request. Every request butmanifestcarriesplace: {name, config}, solistanswers for one place.configis the place's table from Fleet's config, as JSON. Fleet does not read it. The runtime trusts it: it comes only from the operator's Fleet config. A runtime that gets a place or a config it cannot use answers Unknown with why, and so does a version 2 request without a place. The runtime keeps no place config of its own. -
manifestreports places. The request carriesplaces, every configured place with its config. The answer addsplaces: [{name, platform, dynamic, ready, why}]:platformisposixorwindows;dynamicis true when each key gets a new machine, such as a VM;readyis false when the place lacks something, andwhynames it and, where the runtime knows it, the command that fixes it;- the runtime may add places it discovers, such as WSL distros, with an empty config.
A runtime with no
[runtimes.<name>.places]table has one place,local, with an empty config. An unreachable runtime makes its places Unknown, never absent.
Fleet asks manifest with "protocol": 2, which a runtime of either version
answers. Each later request carries the version that manifest named: a
version 1 runtime gets version 1 requests, with no place. Fleet keeps the
version for the life of its process and asks manifest again after any
Unknown answer, so a runtime replaced by one of another version is followed.
Each request names the Task's place, with its table from
Fleet's config. A version 1 runtime serves only local, with no config: Fleet
sends it nothing for another place, and that place stays Unknown.
runtime-mux's places #
runtime-mux answers both versions. A version 1 request runs at local,
this machine, which takes no config. Any other place is a remote place: a
machine over ssh (kind = "ssh") or a WSL distro (kind = "wsl"), with the
keys of its table.
request.place ──▶ local ─────────▶ this machine's multiplexer
└─▶ kind = "ssh" ──▶ ssh <destination> tmux -L runtime-mux …
└─▶ kind = "wsl" ──▶ wsl.exe -d <distro> tmux -L runtime-mux …
runtime-muxrefuses a place with an unknown key, a missingkind, ansshplace without adestination, or a config forlocal. It answers Unknown and names what is wrong, andmanifestreports the place not ready.localis ready when the multiplexer answers. A remote place is ready when its launcher reaches it without a prompt,fleetis on itsPATH, and its multiplexer answers at tmux 3.2 or newer, or psmux 3.3.8 or newer. A WSL place also needsflock.whynames what is missing and the command that shows or fixes it.manifestprobes each place on a thread of its own, and waits 8 s for all of them together. A place that has not answered by then is not ready: its probe timed out. Fleet asks a Task's runtime about the Task's own place only, so a place that does not answer never holds up a Task on another.- A remote place's commands get its
PATH:path_prepend, then the place's loginPATH, then the launcher's (runtime-mux.md). The loginPATHis read once and cached for 10 minutes beside the locks.execreads it inside its own deadline, and runs withpath_prependalone when it cannot.startlooks for the agent on thatPATHfirst, and answersfailedwhen it is not there. - A session's
env,FLEET_SIGNAL_FILEamong it, and the place'sPATHreach its window in thenew-windowline itself:-efor each variable, and the login wrap'sPATHassignment. The line travels on the stdin of the control connection that ssh orwsl.exelaunches, never in its argv. So nothing depends on which of this machine's variableswsl.exepasses (WSLENV). - A session's
envcan name aPATH. Then thatPATHis the window'sPATH, in its own order. Onlocal,runtime-muxputs the directory of itsthurbox-cliin front when thePATHdoes not name it. On a remote place, the login wrap puts eachpath_prependdirectory that thePATHdoes not name in front. tmux ignores-e PATH, so thePATHgoes in the window command: anenv PATH=…prefix onlocal, and the login wrap's assignment on a remote place. - On
local,startlooks for the agent on the session'sPATHfirst, then on its own. On a remote place,startlooks for the agent on thePATHthat the window gets. - On the
localplace, tmux and RMUX make the window with a one-shotnew-window, and every user of the machine can read a client's argv. So the client's argv holds at mosthas-sessionand the session's options, thensource-file -. Thenew-windowwords,-efor each variable among them, go on the client's stdin, as one line of tmux's command syntax. The server sets each variable in the window: no shell assigns one, so a name that a shell keeps read-only, such asUID, starts the window as any other. - psmux takes the environment in its window's PowerShell script, as
-EncodedCommand. That argument holds every value, base64-encoded: on a psmux place,envis in an argv. - A
startwhoseargvorenvnames a file inFLEET_ASSET_DIR, a directory on this machine, copies that directory to~/.local/state/fleet/assets/<key>on the place. Every word ofargvand every value ofenvthat names the directory names the copy instead, so a harness's hook file, such as pi's extension, is the place's own. Astopremoves the copy. A launch that names none of its files copies nothing. - A
startwhoseenvnamesFLEET_HARNESS_DIR, the user's harness profiles on this machine, copies it to~/.local/state/fleet/harnesses/<key>on a POSIX place and names the copy, sofleet signalthere reads the profile's own payload fields. - A
start'sdiris absolute by the place's rules, not by this machine's:/…on a POSIX place, a drive or UNC path on a Windows one. start,inspect,stop,pause,list,input,captureandexecrun on the place. A pause mark and a start record of a remote key stay on this machine.workspaceand a releasingstoprunworkspace-giton the place, from the place'sPATH, over the same launcher. A place without it takesfile://URLs only: directories already there, which a release keeps, with no setup steps. A release there of a key that has a directory under the root is Unknown untilworkspace-gitruns again. Agit+URL there answersfailedand namesworkspace-git. The workspaces are under~/.local/share/runtime-mux/workspaceson the place. Agit+URL gets a worktree there, on the key's own branch, with the same#ref, preserved refs and retention as onlocal(workspace-git). The config'spreserve.daysand its setup and teardown steps apply on the place too; itsrootandprogramare this machine's only.browseandbranchesread the place's folders and repositories with oneshscript over the launcher. Abrowsewithoutdirlists the place's home.attachandshellanswer withruntime-mux view --place <place>. The view builds that place's backend and reads the pane through its control connection over ssh orwsl.exe, as the other verbs do.- A call whose place does not answer is Unknown, and names the place.
- A native-Windows place (
multiplexer = "psmux"orplatform = "windows") runs its commands through an encoded PowerShell script, withpath_prependfirst on itsPath.- Its readiness looks for
fleetwithGet-Command. execandworkspacestart the program aslaunch_on_windowsdoes.- A copy of Fleet's files goes through
tar.exe, which Windows has had since Windows 10 1803. browse,branchesand a workspace's setup and teardown steps read the place withsh, so they answerfailedthere and say why.- A window's environment goes, over the stdin of one PowerShell command,
to a new file under
%LOCALAPPDATA%\runtime-mux. The file is created with an ACL of its own that lets only the user open it, whatever the directory allows. The window's script names only that file: it loads it, deletes it, then starts the program. So no value is in any process's arguments. A failed launch removes the file. A psmux server on this machine does the same. execfinds its program as PowerShell finds a command, so an npm.cmdshim or a.ps1script runs undercmd.exeorpowershell. A shim's arguments are quoted so thatcmd.exereads no&,|,<,>,^,%NAME%or!NAME!in them: Rust std's rules for a batch file, undercmd /v:off.
- Its readiness looks for
- WSL stops a distro, and every process in it, soon after its last
wsl.execlient exits. Eachruntime-muxcall is such a client. So astartor ashellon a WSL place also starts one waiter: a detachedwsl.exethat runsflock -nandtmux wait-foron the place's server. The waiter keeps the distro running while that server runs, and exits with it. A second waiter for the same server exits at once. The distro needsflock(util-linux), and readiness says so when it is missing.
Signals on a remote place #
An agent's hooks run on its place, and they cannot reach a file on Fleet's
machine. On a place other than local:
place: hook ─fleet signal─▶ ~/.local/state/fleet/signals/<key>.jsonl
Fleet: runtime exec: fleet signal-read <that file> ──▶ fold ──▶ session state
- Fleet passes
FLEET_SIGNAL_FILE=~/.local/state/fleet/signals/<key>.jsonl. Thefleeton the place expands~to its home and makes the directory. - Fleet reads the file through the runtime's
exec, over the connection the runtime has to the place:execrunsfleet signal-read <file> [<length>], so the place's ownfleetreads it, on POSIX and on Windows alike. A file not written yet is a session that reported nothing. Anexecthat fails is Unknown, and so is a path that is not a regular file, such as a FIFO or a symlink: the read neither blocks nor follows it. - The session's event outbox is beside the signal file, at
<that file>.events. Fleet reads it throughexectoo:fleet signal-read --from <offset> <file>prints the offset its read starts at on a line, then at most 512 KiB from there. It starts at 0 when the file is shorter than the offset. Each byte that is not part of valid UTF-8 is printed as NUL, so every offset stays as it is in the file, and the line it is on fails to parse and is skipped. A path that is not a regular file, such as a FIFO or a symlink, is refused, and Fleet reads it as Unknown. Exit 3 is a file not written yet. - Before a revive or a resume, Fleet reads the file's length through
exec(fleet signal-read --length <file>) and keeps it in its own data,<data dir>/signals/<key>.started. It writes nothing on the place. A state in the file before that length is the previous process's: readiness and trust wait for the new process's own report. No time is compared, so the place's clock need not agree with Fleet's. A file shorter than the kept length was made again, and is read whole. - One read takes the last 256 KiB before that length, for the prompts and
the conversation, and the last 512 KiB after it, for the state. Both stay
under
exec's 1 MiB cap, so the newest report is never cut. Each range is computed from the file's length first and read by seeking to it, so a read never scans the whole file. - A restart on a remote place is three runtime calls: the manifest, the
length through
exec, thenstart. The manager's lease covers three calls per operation. - When the length read does not answer, Fleet starts nothing and asks
again on the next pass. When the runtime cannot read the file at all (no
exec, or a place whereexecfails, such as native Windows), Fleet starts the session and marks the boundary unknown: its state stays Unknown until the next restart, and no state the previous process reported is taken for the new one's. - The runtime writes no signal line and knows nothing of their format.
The shell #
shell answers how to open an interactive shell beside the key's process:
on its place, in the directory the process is in. The answer has the shapes
of attach, and its argv runs where the view runs.
fleet shell <task> asks it, and bridges a tty answer as fleet attach
does (contracts). thurbox's companion shell opens
with it, through the session's Fleet, so thurbox needs no route of its own
to the place.
"none"when no process holds the key, asattachanswers.- The shell belongs to the key. A second call opens the same shell, so a
view that leaves and comes back finds what it typed. A
stopof the key closes it.
runtime-mux opens the shell in a window of its own, beside the key's, on
the same server. Its program is the machine's shell ($SHELL, else
/bin/sh; %COMSPEC% on Windows). Its directory is the one the key's
process is in now, as the multiplexer reads it. When that directory is
gone, the multiplexer starts the shell in its own home. runtime-mux reads
nothing about the directory from its own disk, which need not be the
place's. The answer is a stream: runtime-mux view on the shell's pane.
The window closes when the shell exits, and the next call opens a new one.
list does not name the shell. A stop closes every window named as the
key's shell, so two of them never keep a stop from concluding.
Browse and branches #
Two optional read-only verbs let a creation flow pick a repository on a
place, and the branch a worktree there starts from. Each request carries
protocol and, from version 2, place.
| Verb | Request | Known answer |
|---|---|---|
browse |
dir (absolute, optional: the place's home) |
{"listing": {"dir", "entries": [{"name", "repo"}]}}, or {"failed": {"why"}} |
branches |
repo (absolute) |
{"branches": {"names", "default"}}, or {"failed": {"why"}} |
browselists the subdirectories ofdir, sorted by name. It follows a link to a directory and leaves files out.repois true when the entry is the top of a git work tree: a checkout or a linked worktree. A directory that cannot be read answersfailed, with why. A relativediranswers Unknown.brancheslists each local branch of the repository atrepoand each of itsorigin/*branches, as they are: the runtime does not fetch. The order is the runtime's.runtime-muxlists the local branches first, and each group by name.defaultis the branch a new one starts from:origin's default (origin/HEAD, elseorigin/main, elseorigin/master), else the branch checked out, else the first, ornullwith no branch.origin's branches are told from local ones by their refs: a local branch namedorigin/mainis local.repomust be the top of the work tree: a directory inside a checkout answersfailed.- A dedicated verb is simpler than
exechere. Anexecanswer is the text of the place's own command, so the reader would need one parser per platform. The runtime knows its place, and a VM runtime can answer from its own API.
runtime-mux answers both at local, from this machine's filesystem and
its git. The new-session modal still takes a clone URL, whose branches
git ls-remote lists on the machine thurbox runs on.
What a runtime is #
A runtime answers one question: run this command, under this key, in this directory, at this place, and tell me about it. Where and how is its own business.
runtime-mux (crates/runtime-mux) runs its processes on this machine, in
tmux, psmux or rmux (ADR-31). A new multiplexer of the tmux family is a
module in runtime-mux, not a runtime. A new kind of place is a runtime. No
runtime-* crate depends on another; shared code is a crates.io dependency.
Fleet must run a Task on the four required verbs alone, so a runtime without a terminal is a full runtime:
- the brief reaches the agent at
start, through the harness profile's argv or a file in the workspace, not only by typing; - state reaches Fleet through
FLEET_SIGNAL_FILE, not through the screen; - the optional verbs add features and nothing more: without
input,fleet sendwaits for the next start; withoutcapture, no screen checks; withoutattach, no live view, and thurbox shows the Task without a terminal; withoutpause, an idle session keeps running.
The default install configures runtime-mux and nothing else. With no
multiplexer on the machine, Fleet does not fall back to another runtime: the
first start answers the missing dependency by name, and other runtimes are opt-in in [runtimes.<name>].
The terminal stream #
A tmux server keeps the screen, accepts several clients and arbitrates their
sizes; a raw interactive command such as docker attach does none of that.
"Run this command in a pty" is therefore not an interface: what a view gets
would depend on the runtime.
What a view needs from a live session, whatever runs it:
- the screen as it is now, on connect and after a dropped stretch;
- the output that follows, in order, with size changes at their place in it;
- a way to type, paste and resize;
- to leave without stopping the process, and to come back;
- other viewers at the same time, with one rule for who sizes the screen.
The protocol #
A stream is a process. The view starts it and speaks over its stdin and
stdout; it ends when either side closes. A frame is a type byte, the
payload's length as a big-endian 32-bit number, then the payload: output,
state and input are raw bytes, paste is UTF-8 text, and every other
payload is a JSON object. The vocabulary is PaneChunk (crates/runtime-mux/src/contract.rs),
which carries every item in the output's order on one channel:
| Direction | Type | Frame | Payload |
|---|---|---|---|
| down | 0x01 |
hello |
{version, capabilities}, capabilities from snapshot, scrollback, viewers, resize |
| down | 0x02 |
snapshot |
{cols, rows, cursor: [col, row], normal, alternate}: normal-screen history and rows, alternate rows or null, styled |
| down | 0x03 |
output |
bytes |
| down | 0x04 |
resized |
{rows, cols} |
| down | 0x05 |
sized_elsewhere |
{elsewhere}: whether another viewer sizes the screen |
| down | 0x06 |
resync |
a snapshot replacing the view's grid after dropped output |
| down | 0x07 |
state |
title and input modes as terminal bytes, to parse but not count as output |
| down | 0x08 |
exit |
{"known": {code}}, code null for a signal, or {"unknown": {why}} |
| up | 0x81 |
input |
bytes, sent as typed |
| up | 0x82 |
paste |
text, bracketed when the program asked for it |
| up | 0x83 |
resize |
{rows, cols}: a request the holder of the screen arbitrates |
| up | 0x84 |
detach |
empty: leave; the process keeps running |
A frame of a type the reader does not know, or a payload over 64 MiB, ends the stream.
A view's first frame is a resize: the end that keeps the screen starts the
stream once it knows the size. snapshot comes next when the capability is
there, so a view never starts on an empty grid. Sizing follows ADR-27 everywhere: the viewer
being typed into sizes the screen, the others get sized_elsewhere.
term-stream (crates/term-stream) is a crate with only the frame types and
their codec: no process, no runtime, no transport. thurbox, Fleet and every
runtime-* may depend on it, as they may on serde.
The runtime's attach #
attach answers how to open a stream, in one of two shapes:
{"stream": {"argv": […]}}: the command speaks the protocol. This is the full answer.{"tty": {"argv": […]}}: a raw interactive command, for a runtime that has nothing better (docker attach,virsh console,kubectl attach -it).
fleet attach always hands a view a stream. It passes a stream answer
through, and wraps a tty answer in fleet term-bridge -- <argv>, which runs
the command on a pty of its own and relays it as frames. A bridge keeps no
screen: it has nothing from before it started, so its hello names no
snapshot, and the view's first resize sizes the pty the command starts on,
which is what makes a program draw. That is the price of a runtime without a
screen keeper. A paste is bracketed when the command's output last asked
for bracketed paste. A detach ends the bridged command, such as a
docker attach client, and not the process it reached, even after the
command closed its terminal.
Who keeps the screen #
| Runtime | Screen keeper | attach answers |
|---|---|---|
runtime-mux |
tmux, psmux, rmux | stream: runtime-mux view <key>, control mode encoded as frames |
| any runtime, minimal | none | tty: bridged by Fleet, no history; the reference script answers it |
Workspaces #
Fleet hands a runtime a list of URLs and gets one directory per URL back (the Fleet side). Everything else is the runtime's: where the files live, how they are made, what is installed, how work is kept when they go. That is where a workspace is customized, by configuring a runtime or by writing one.
The verb #
workspace {key, urls} answers one of:
| Answer | Meaning |
|---|---|
Made{dirs, adopted} |
one directory per URL, in order, as the process sees it; adopted when the key's workspace already existed |
Unsupported{url} |
this runtime does not handle that URL |
Failed{why} |
nothing was made, or what was made was rolled back |
stop {key, release: true} removes the key's workspace with the process.
A runtime that implements workspace holds to these rules, and
fleet runtime check tests each:
- Idempotent by key. A second call with the same key answers the same directories, adopted. A partial workspace is never answered: the next call rolls it back and makes it again.
- Never removes what it did not make. A
file://directory is used as it is and survives every release. - Never loses work on release. Anything the workspace holds that exists nowhere else (uncommitted changes, unpushed commits) is preserved before the files go, in a durable place outside the workspace. How is the runtime's choice. Files the repository ignores are not work: it declares them disposable.
- Never waits for a person. Release always finishes or answers Unknown.
The manifest declares what Fleet may rely on:
workspace.schemes: the URL schemes the runtime handles, such asgit+https,git+ssh,git+file,file.workspace.survivesStop: the files outlive the process.
A runtime without workspace runs only Tasks with an empty list.
URLs #
The URL is the whole request: Fleet adds nothing to it.
| URL | What workspace-git makes |
|---|---|
git+https://host/org/repo.git |
a worktree on a new branch from the default branch |
git+https://host/org/repo.git#v2 |
the same, from v2 |
git+ssh://git@host/org/repo.git?branch=feat-x#main |
a worktree on feat-x, made from main when it does not exist |
git+file:///home/me/app?branch=feat-x#main |
a worktree of that checkout itself, on its own feat-x |
git+file:///home/me/app#refs/fleet/preserved/<key>/app |
a worktree on work an earlier release preserved in that checkout |
git+file:///srv/mirrors/repo.git#refs/fleet/preserved/<key>/repo |
a worktree on work an earlier release preserved in a bare repository |
file:///home/me/notes |
nothing: the directory, as it is |
Restoring preserved work is therefore a new Task whose URL names the preserved ref; Fleet needs no verb for it.
workspace-git, the shipped workspace provider #
workspace-git is a small program a runtime runs where its files belong. No
runtime links it: each runs the program, so it runs where the runtime's files
are. Its layout:
<root>/<key>/<name>/ one real worktree per git URL
<root>/<key>/.fleet-workspace.json written last; without it the workspace is partial
<root>/<key>/.fleet-sources the checkouts the key's worktrees belong to
<root>/.sources/<digest>.git a bare source per remote repository, fetched before each make
<root>/.sources/<digest>.lock one lock per repository a worktree is made from
<root>/.checkouts the local checkouts worktrees were made from
- A
git+fileURL that names a local checkout, a repository with a working tree, gets a worktree of that checkout itself. Nothing is copied, and the checkout'soriginis fetched first, so itsrefs/remotes/origin/*andorigin/HEADare the upstream's as they are now. A fetch that fails or stalls for 30 seconds leaves the refs as they were:- an existing branch is the checkout's own branch;
- a new branch is made in the checkout;
- a commit made in the worktree is in the checkout at once;
git -C <checkout> worktree listlists the worktree.
- Any other git URL, a remote or a bare repository, gets a bare source of its
own. A source fetches its remote's branches into
refs/remotes/origin/*, and its tags. A fetch therefore never moves a branch that a worktree has checked out. Every branch in a source'srefs/heads/is one thatworkspace-gitmade. - Each worktree belongs to its repository, the checkout or the source, so its branch and its preserved ref are there.
- One lock per repository orders every make, release, rollback and retention on it, across runtimes that share a root.
- With one URL the session starts in
<root>/<key>/<name>; with several the first URL's worktree is still the start, and every worktree is a real directory. Two URLs with one name get<name>-2. - With no
branch, the branch is named after the key. An existing branch is checked out, and one checked out elsewhere gets<branch>-<n>, from 2. - A fragment names the base: a remote branch, a tag, a branch of the source,
or a ref or commit as given, in that order. A checkout reads its own
branches before its remote's, except that
origin/<branch>always names the remote's. Arefs/…fragment that a fetch of branches does not bring, such as preserved work, is fetched by name. With no fragment, the base is the remote's default branch (origin/HEAD, elseorigin/main, elseorigin/master); a checkout with no remote starts from the branch it has checked out. - A
file://URL takes no query and no fragment, and names an existing absolute directory. A%XXin a path is decoded. - A key holds one workspace: the same key with other URLs fails.
- On release, a dirty worktree is committed, untracked files included,
through a temporary index onto
refs/fleet/preserved/<key>/<name>in its repository; then the worktree goes. The temporary index lives in the key's directory, never in a checkout's working tree. A read-only directory in the worktree does not stop its removal: once the work is preserved, each directory removed gets its owner's write permission, and the removal is tried again. Branches are never deleted on release. The release of a key that holds no workspace succeeds. - On each make it prunes its own retention: preserved refs older than
preserve.days(default 30), in sources and in checkouts, and a source's branches not checked out whose head the remote already has. A checkout's branches are the user's and are never deleted. A repository another call holds is left for the next make. - Git never waits for a person:
GIT_TERMINAL_PROMPT=0, and ssh runs in batch mode unlessGIT_SSH_COMMANDorGIT_SSHnames another command.
The program takes one JSON request on stdin and prints one answer:
| Call | Request | Answer |
|---|---|---|
workspace-git make |
{root, key, urls, preserveDays?} |
{"made": {dirs, adopted}}, {"unsupported": {url}} or {"failed": {why}} |
workspace-git release |
{root, key} |
"released" |
workspace-git discard |
{root, key} |
"discarded": the worktrees go, nothing is preserved, for a workspace no session worked in |
workspace-git inspect |
{root, key} |
{"workspace": {urls, dirs}}, or "none" |
Any other answer is {"unknown": {why}}, such as a rollback that did not
finish, and the caller concludes nothing from it. root is an absolute
path.
Customizing a workspace #
Each runtime offers what fits its places: runtime-mux runs setup and
teardown steps (its workspaces). A need no
configuration covers is a runtime of one's own: a script that answers
the protocol, held to the same rules by fleet runtime check.
runtime-mux's workspaces #
runtime-mux runs the workspace-git program on the session's place, so a
session sees the same paths. On a remote place it is the place's own
workspace-git, and the root is ~/.local/share/runtime-mux/workspaces
there (runtime-mux's places). Its own config file,
on this machine, holds the settings: the file at
RUNTIME_MUX_CONFIG, else ${XDG_CONFIG_HOME:-~/.config}/runtime-mux/config.toml.
A missing file gives the defaults. An unknown key is refused.
[workspace]
root = "/home/me/.local/share/runtime-mux/workspaces"
program = "workspace-git"
preserve.days = 30
[[workspace.setup]]
match = "git+https://github.com/acme/app.git"
run = ["npm ci"]
| Key | Default | Meaning |
|---|---|---|
workspace.root |
${XDG_DATA_HOME:-~/.local/share}/runtime-mux/workspaces |
where workspace-git makes workspaces; an absolute path |
workspace.program |
workspace-git beside the runtime-mux program, else on PATH |
the program it runs |
workspace.preserve.days |
30 | how long preserved work is kept |
workspace.setup, workspace.teardown |
none | steps, as above |
- A step matches a URL as written, or the URL without its query and fragment.
Its commands run in order, through
sh -c, in the directory made for that URL. - Setup runs in a workspace the call made, and in one it adopted only when the mark of a finished setup is proven absent. When the mark cannot be read, as on a place that does not answer, the call is Unknown and setup does not run.
- A failing command fails the call, naming the command. The workspace is released, which preserves any work in it first, as a stop does.
- A place's
browse,branchesandworkspace-gitanswers may be up to 8 MiB on a stream. A longer one is an error, never a cut answer. stop {release: true}stops the process first, then runs the teardown steps, then releases the workspace. A failing teardown command does not stop the release.- A
workspace-gitthat does not start answersfailed: nothing was made. One that starts and gives no answer is Unknown.