ChangeRequest controller
A ChangeRequest mirrors one change on a forge. Its controller decides, from the stored snapshot and one forge observation, whether anything changed and when to look again. It performs no effect: the forge is read, never written, from here.
| Read | Answers |
|---|---|
| ChangeRequest | What do I apply, and what does the snapshot contain? |
| State machine | How does a change move between Watching, Closed and Settled? |
| Interface | What are the probes, the Forge port and the write rules? |
| Forge controller | Who consumes the snapshot, and how? |
| Native config | How are forge hosts and adapters configured? |
| Overall architecture | Where does this controller fit in Fleet? |
One run #
- A ChangeRequest is applied, by a person or by
fleet declarefor the change a worker opened. The adapter for its host parses the URL at apply, and again before the first poll, when the controller records it asstatus.ref. - On each due pass the controller asks one probe,
Change { change }. The manager answers it through the Forge port with one complete observation or Unknown. - A Known answer is compared with the stored snapshot. If anything differs,
revisionis incremented, every new or changed item gets thatseq, and the snapshot is written. If nothing differs, only the bookkeeping fields,observedAtandnotBefore, are written. - An Unknown answer sets
Reachable=Unknown, leaves the snapshot untouched and waits one poll interval, or theretryAfterthe forge gave. Unknown never means closed, merged or empty. - A merged change settles. A closed change is watched more slowly for a grace period, then settles; reopened within it, it is watched again.
- A settled ChangeRequest is retained for
settledTtlMs, then the controller deletes it. A person may delete one at any time. There is no finalizer and no owner: no Task's deletion reaches it.
Polling cadence is status.notBefore, which the manager already honours for
Tasks. One ChangeRequest costs one bounded observation per interval, a few
forge calls when the change has many notes, and one small bookkeeping write.
Why a resource of its own #
- One poller per change, however many consumers care. The Forge controller, a reviewer and a conductor read the same snapshot instead of each calling the forge.
- The forge's vocabulary is translated once, in the adapter. No consumer sees a GitHub or GitLab field.
- An unreachable forge is this controller's condition. No consumer can mistake it for a state of the change.
- It outlives its followers. A Task that declared the change can be deleted and another can take over on a different host; the mirror and its sequence numbers stay, so the newcomer's cursor means the same thing.
Implementation and evidence #
| Responsibility | Source | Tests |
|---|---|---|
| Pure decisions, numbering, cadence and retention | ChangeRequestController | Controller transitions |
| Resource, status and port types | ChangeRequest types, Forge port, plans | Forge E2E |
| Sweep by kind, host-ordered deferral, fenced delete | Manager | Forge E2E, Store contract |
| Apply by URL, the derived name, ref verification | Verbs | Forge E2E |
GitHub through gh, GitLab through glab, Azure DevOps Services through az, Bitbucket Cloud through authenticated curl, the fake |
fleet-forge: github, gitlab, azure-devops, bitbucket-cloud, fake | Truth tables in each adapter's unit tests; Forge E2E against the stub CLIs |
[forge] configuration |
CLI config | Forge E2E |
| Crate edges and the field manager | architecture.toml | Architecture suite |
The forge E2E suite drives the built CLI through apply by URL, idempotent
re-apply, the aliasing refusal, bookkeeping-only polls, numbering of notes,
edits, unresolves, head and state transitions, the 100-note cap, Unknown
answers and retryAfter, host-wide deferral, the closed grace and reopen,
merged settling, retention and the fenced delete, against the fake forge; and
pagination, truth tables, rate limits, missing logins, a hung CLI's killed
process tree and proven absence against the stub gh, glab, az and
authenticated HTTP client. The
Forge controller guide
lists the steps that follow this one.
ChangeRequest state machine #
status.phase records how closely the change is watched. The snapshot fields
record what the forge last said. The interface defines the one
probe used here.
Transition rules #
The manager only reconciles a ChangeRequest whose notBefore is absent or
due. Deletion by a person removes the record from any phase. Every poll
writes the bookkeeping fields, observedAt and notBefore; only a changed
snapshot moves revision.
| Current phase | Observation | Next phase | Decision |
|---|---|---|---|
| Watching | Known, nothing differs from the snapshot | Watching | Bookkeeping only; notBefore = now + pollMs. |
| Watching | Known, something differs; state draft or open | Watching | revision += 1; assign seq to what changed; set everReady if the state is not draft; write; wait pollMs. |
| Watching | Known, state merged | Settled | Write the final snapshot; settledAt = now; Settled=True/Merged; no further polls. |
| Watching | Known, state closed | Closed | Write the snapshot; closedAt = now; wait 10 × pollMs. |
| Watching or Closed | Known, the change is proven absent | Settled | Settled=True/Gone; no further polls. |
| Watching or Closed | Unknown | unchanged | Reachable=Unknown with the reason; snapshot untouched; wait the phase's interval, or the answer's retryAfter when it carries one. |
Watching, no status.ref yet |
The URL could not be parsed by the configured adapter | Watching | Reachable=Unknown/UrlNotParsed; wait pollMs. |
| Closed | Known, state draft or open | Watching | revision += 1 through state.seq even if nothing else differs; clear closedAt; wait pollMs. |
| Closed | Known, state merged | Settled | As from Watching. |
| Closed | Known, still closed, within closedGraceMs of closedAt |
Closed | Write only if something differs; wait 10 × pollMs. |
| Closed | Known, still closed, past closedGraceMs |
Settled | settledAt = now; Settled=True/ClosedGraceExpired; no further polls. |
| Settled | none asked; settledTtlMs not yet passed since settledAt |
Settled | Wait until it has. |
| Settled | none asked; settledTtlMs passed |
Record removed | Request deletion of this object. A change reopened after the grace is a new ChangeRequest. |
Proven absence is the port's Known(None): the forge answered and said the
change does not exist. A refused login, a missing permission, a partial
answer or an ambiguous not-found is Unknown, never gone.
Sequence numbers #
revision only moves when the snapshot changes, so a quiet change costs no
snapshot writes. Each note carries the revision that first showed it, moved
again by an edit or an unresolve; state, reviewDecision and head carry
the revision at which their value last changed. The newest 100 notes are
kept; droppedBefore rises to the seq of any note rotated out, so a
consumer whose cursor is below it knows it has a gap rather than silently
missing a note. The ChangeRequest resource
says why the counter is Fleet's and not the forge's clock, and what makes an
observation complete.
Conditions #
| Type | True means / relevant reason |
|---|---|
| Reachable | The last poll read the change completely (Observed). Unknown with the reason after a timeout (Timeout), an incomplete read (Incomplete), an unparseable answer (Unreadable), a throttled answer (Throttled), a missing login (NotLoggedIn), a refusal that is not proven absence (Refused), a CLI that could not start (CliUnavailable) or a URL the adapter could not place (UrlNotParsed). |
| Mergeable | The forge's own answer, as status.mergeable: True (Mergeable) when the forge says the change can be merged now, False with the forge's reason as a CamelCase word (Conflicts, Blocked, CiMustPass, Draft, ...; NotMergeable when the forge gives none) and the raw reason in the message, Unknown (Computing) while the forge is still computing it or has no such answer. Fleet reports it and never acts on it. |
| Settled | Merged, ClosedGraceExpired or Gone. |
Conditions keep their lastTransitionTime unless their status changes, so a
pass that sees nothing new writes nothing to them.
Retention #
A ChangeRequest is tied to no Task. Settled ones would otherwise accumulate
one per change ever opened, so the controller deletes its own object once
settledTtlMs has passed since settledAt, thirty days by default. This is
the pattern of a Kubernetes Job with a TTL after finishing: the object's own
controller removes it a configured time after it is done, and nothing else
has to know. The delete is fenced like every controller write: it names the
object's uid and resourceVersion and runs under the lease, so a stale
plan cannot remove a recreated mirror. A follower that still names a deleted
ChangeRequest sees readable absence and raises ChangeGone.
An open ChangeRequest nobody follows, such as one left by a crash between
the two steps of fleet declare, is polled until it settles like any other,
then retained and removed. Nothing collects open orphans; they cost one
bounded call per interval and are visible in fleet list changerequest.
Deletion #
fleet delete changerequest NAME removes the record on the next pass, at any
phase. There is no finalizer because nothing on the forge is held, and no
owner, so no other object's deletion reaches a ChangeRequest.
ChangeRequest controller interface #
The controller is a pure decision function, like TaskController. The manager owns I/O. The state machine defines how the values below change the ChangeRequest; the resource defines the external JSON fields.
Controller API #
Source: ChangeRequestController and plan types.
pub struct ChangeRequestInput<'a> {
pub object: &'a Object,
pub observations: ChangeRequestObservations,
pub now: Millis,
pub defaults: ChangeRequestDefaults,
}
pub fn probes(&self, input: &ChangeRequestInput<'_>) -> Result<Vec<Probe>, StoreError>;
pub fn reconcile(&self, input: &ChangeRequestInput<'_>) -> Result<Plan, StoreError>;
object is the stored ChangeRequest. now comes from the manager's Clock.
defaults carries forge.closedGraceMs and forge.settledTtlMs from the
native config, as values, for a spec that names neither. Missing status
decodes to the default Watching state with revision: 0; malformed typed
spec or status returns StoreError::Invalid. Neither method reads the
clock, disk, environment, configuration, forge or runtime itself.
Requested probes #
| State | Probes |
|---|---|
| Watching or Closed, due | Change { change }, carrying the ChangeRef. |
| Settled | None. |
| Deleting | None. |
ChangeRequestObservations holds two optional answers. change, a
ChangeObservation: Known(Some)
carries the adapter's complete, forge-neutral reading of the change;
Known(None) says the forge answered that the change does not exist;
Unknown { kind, why, retryAfter } carries the reason, as the
Uncertainty the Reachable condition is named from, and, when the forge
said so, how long to wait. None means the probe was not asked. parsed,
an Observation<ChangeRef>, is the manager's parse of spec.url, supplied
only while status.ref is absent: the probe names the ref, so the first
poll needs it before probes is asked, and the controller records it as
status.ref in the same write as that poll's result.
Returned plan #
The plan carries status, and once a Settled object's retention has passed,
delete: true with no status, which asks the manager to delete this object
by uid and resourceVersion under the lease. effects is always empty
and remove_finalizer is always None: this controller writes its own
object and touches nothing else.
Forge port #
Source: fleet_api::forge::Forge. Every answer is Known or Unknown;
a refused write is Known(EffectOutcome::Failed { why }). A host is always
a forge hostname, never a runtime host, and every call runs on the machine
the manager runs on, with that machine's CLI logins.
| Method | Input | Answer |
|---|---|---|
operation_bound |
None | Worst-case duration of one call, in milliseconds. The lease is sized to outlast it. |
parse |
A URL | The ChangeRef for it, normalised, using the adapter [forge.hosts] names for the host. Unknown for a host not configured. |
whoami |
A host | Fleet's login on that host, or Unknown. |
change |
A ChangeRef |
Some(ChangeSnapshot), complete; None for proven absence; Unknown on timeout, an incomplete read, a throttled answer or an unreadable answer, with retryAfter when the forge gave one. |
mark_ready |
A ChangeRef, the expected head.id and state, and an intent key |
MarkedReady { already } once the change is out of draft, after a fresh read proved it still had the expected head and state; Moved { head, state } when either differed, nothing mutated; Failed when the forge answered no; Unknown when the answer could not be read, since the mutation may have landed. The Forge controller's one effect, and Fleet's one forge write. |
The port has no merge method.
One bounded observation, not one process. change may make several
calls, paginating notes, threads and reviews, within operation_bound. It
answers Known only when it read everything the forge holds for the change;
if the bound runs out first it answers Unknown with Incomplete. The
controller never numbers a partial read.
Adapters live in the fleet-forge
crate, which depends on fleet-api and fleet-exec. The crate also holds
ForgeHosts, the adapters of a configuration's [forge.hosts], which answers
the port by the host a URL or a ref names. Each observation spends at most
six bounded calls, each of forge.callTimeoutMs; the port's
operation_bound is six worst-case calls.
- github: bounded
gh api graphql --hostname <host> --includeprocesses read the pull request, its reviews, comments and review threads with pagination, each author's__typenameto tell an app from a person,mergeableandmergeStateStatus, and the check runs and statuses of the head. ARetry-Afteror rate-limit reset header becomesretryAfter. A head with more than 100 checks or a thread with more than 100 comments is Unknown, never a partial read.mark_readyreadsisDraft,stateandheadRefOidafresh in one call, then runsmarkPullRequestReadyForReviewwith the intent key asclientMutationId; the forge answering that the pull request is no longer a draft isMarkedReady { already: true }. - gitlab: bounded
glab api --hostname <host> --includeprocesses read the merge request, its discussions with pagination, its approvals,detailed_merge_statusand the head pipeline. A 404 of the merge request is proven absence only when the project itself answers.mark_readyreads the merge request afresh, then updates its title with the draft marker removed (Draft:,[Draft],(Draft),Draft -, or the legacyWIPforms, however cased), which is how GitLab takes a merge request out of draft; a draft whose title carries no such marker is a Known refusal. - azure-devops: bounded
az devops invokeprocesses read the pull request, its iterations, latest-iteration statuses, blocking policy evaluations and threads from Azure DevOps Services REST 7.1 (policy evaluations use the documented7.1-preview.1). The current head is the newest iteration'ssourceRefCommit, neverlastMergeSourceCommit. Collection counts must match the returned values; a partial or malformed collection is Unknown. Reviewer votes set the aggregate review decision, but produce noReviewentries because a vote does not identify the head on which it was cast. Policy evaluation reads use the documented CodeReview artifact identity and an explicit 1,000-record page; a full page is Unknown rather than silently truncated.mark_readyfails closed: Azure DevOps can publish a draft, but exposes no atomic head-and-state precondition for that update, so Fleet cannot provide the port's optimistic-concurrency guarantee. Azure DevOps Server is not served by this adapter. The mappings follow Microsoft's REST 7.1 references for pull requests, iterations, statuses, threads, policies and the documentedaz repos pr update --draft false. - bitbucket-cloud: bounded
curlprocesses read hostedbitbucket.orgREST 2.0. Fleet sends only HTTPS URLs and Accept headers and uses--netrc-optional; curl's config, netrc or credential helper owns authentication. The pull request is read before and after its paginated comments, commit statuses and mergeability checks; each collection follows the API's opaquenextlinks. A moved head or repository UUID makes the whole observation Unknown. Every pull-request commit status must link to that exact head. Pipelines are not read separately because they have a separate permission and duplicate pipeline-produced commit statuses. Participant votes set the aggregate decision but create no head-boundReview. Inline root comments must carry the API'sresolutionfield; missing resolution evidence is Unknown, never zero unresolved threads.account_id, falling back to UUID, is the user identity.mark_readyfails closed: the documented draft update has no atomic head-and-state precondition. Bitbucket Server/Data Center is not served. These mappings follow Atlassian's references for pull requests, commit statuses, users, pagination, API request limits and draft semantics. - fake: a file-backed forge for tests and trials, with injectable faults
(lost answers, timeouts, throttling with
retryAfter, incomplete reads, malformed answers, proven absence; and formark_ready,moved,failed,lostandlost-applied, the last two Unknown with the mutation not landed and landed). It runs no process, and records the intent key of every mark-ready that mutated a change. Its URLs take any shape a real forge uses (/pull/N,/-/merge_requests/N,/changes/N).
Credentials stay with each CLI or authenticated client's own configuration. Fleet passes none and stores none.
Adapter truth tables #
Each adapter maps the forge's answer onto the vocabulary by these rules; an answer the rule cannot place is Unknown for the whole observation, never a guess.
A terminal forge state wins over the draft flag. Only an open change can be
draft. A forge can close a draft without marking it ready, and the draft
flag then stays set. The GitHub and GitLab mark_ready fresh reads use the
same rule. A closed draft is therefore Moved with state closed, and Fleet
never marks it ready.
| Field | GitHub | GitLab | Azure DevOps Services | Bitbucket Cloud |
|---|---|---|---|---|
state |
state OPEN, CLOSED, MERGED map to open, closed, merged; OPEN with isDraft is draft. |
state opened, closed, merged map to open, closed, merged; opened with draft or work_in_progress is draft. |
status active, abandoned, completed map to open, closed, merged; active with isDraft is draft. |
state OPEN, DECLINED/SUPERSEDED, MERGED map to open, closed, merged; OPEN with draft is draft. |
head.checks |
Over check runs and commit statuses of the head: any failed or errored is failed; else any queued, in progress or pending is pending; else any success is passed; none at all is none. Skipped and neutral count as absent. |
Over the head pipeline: failed or canceled is failed; running, pending, created or manual is pending; success is passed; no pipeline is none. Skipped is absent. |
Over latest-iteration statuses and enabled blocking policy evaluations: failed, error, rejected or broken is failed; pending, notSet, queued or running is pending; succeeded or approved is passed; notApplicable and nonblocking policies are absent. |
Over paginated pull-request commit statuses bracketed by, and linked to, the same source.commit.hash: FAILED or STOPPED is failed; INPROGRESS is pending; SUCCESSFUL is passed; none is none. Pipelines are not read separately. |
reviewDecision |
reviewDecision APPROVED, CHANGES_REQUESTED, else none. |
Approved when the approval rules are satisfied; changes-requested is not a GitLab concept and is never produced; else none. |
Any reject (-10) or wait-for-author (-5) vote is changes-requested; otherwise any approve (10) or approve-with-suggestions (5) vote is approved; else none. |
Any participant changes_requested wins; otherwise any approved participant is approved; else none. |
mergeable |
mergeable MERGEABLE is true; CONFLICTING is false with conflicts; UNKNOWN is null. mergeStateStatus BLOCKED, BEHIND, UNSTABLE, DRAFT give false with that word. |
detailed_merge_status mergeable is true; checking and unchecked are null; every other value is false with that value. |
Draft is false; mergeStatus succeeded is true; conflicts, rejectedByPolicy and failure are false; queued, notSet or absent is null. |
Draft is false; a blocking FAILED mergeability check is false with its type; any UNKNOWN or a blocking PENDING check is null; PASSED, SKIPPED, and other non-blocking checks do not block merging. |
threads.unresolved |
Review threads with isResolved false, over all pages. |
Resolvable discussions with resolved false, over all pages. |
Inline threads in active or pending state. Fixed, won't-fix, closed and by-design are resolved; an unknown state makes the observation Unknown. | Inline root comments whose resolution is null, over every comments page. Deleted roots remain structural for visible replies; a missing root or resolution field makes the observation Unknown. |
reviews |
Submitted reviews on the head commit, state APPROVED, CHANGES_REQUESTED, COMMENTED; dismissed reviews are omitted. | Approvals as approved reviews; there are no others. |
Empty: reviewer votes carry no head identity. | Empty: participant votes carry no head identity. |
bot |
__typename is Bot. |
The user is a bot or a project access token user. | false: the thread identity does not make a reliable bot assertion. |
User is false; the documented app-user type is true; any unknown type makes the observation Unknown. |
Azure DevOps URL identity is
azure-devops/{organization}.visualstudio.com/{project}/{repository}/{id}.
Both https://dev.azure.com/{organization}/{project}/_git/{repository}/pullrequest/{id}
and the legacy {organization}.visualstudio.com form parse to that identity.
The organization-specific canonical host also scopes whoami; equal project,
repository and pull-request numbers in different organizations cannot collide.
Bitbucket Cloud URL identity is
bitbucket-cloud/bitbucket.org/{workspace}/{repository}/{id}. Workspace and
repository slugs are lowercased; each observation verifies the destination's
full_name, immutable repository UUID, and source and destination commit hashes
before and after collecting evidence.
Manager execution protocol #
Source: Manager.
- For each due ChangeRequest, renew the lease. When
status.refis absent, parsespec.urlthrough the Forge port and hand the answer in asparsed. Askprobes, perform theChangeprobe through the Forge port, renewing before it. - Call
reconcilewith the answers, the time and the configured defaults. - CAS-write the planned status against the resourceVersion read. A conflict drops the plan; the next pass rereads.
- When the plan says
delete, delete the object byuidandresourceVersionunder the lease (Store::delete_fenced). A conflict means it was rewritten or recreated since: nothing is removed.
There are no effects and no receipts. fleet up sweeps ChangeRequests in the
same loop as Tasks, after them; a ChangeRequest is due when notBefore is
absent or past, like a Task. ChangeRequests of one host are swept in a stable
order, by host then name, so that a retryAfter from the forge defers all of
them together: the ones after it in the order are not asked and take the same
Unknown answer with the same wait. A sweep spends no more than its bounded
run on them, so forge polling cannot starve Task delivery, deletion or lease
renewal. A store without a [forge] section reports each ChangeRequest as
an error and polls nothing.
Store port and write ownership #
| Writer | May change | Required fencing |
|---|---|---|
apply |
New ChangeRequest and labels; request deletion | Store validation, exclusive creation. |
apply, through fleet declare |
A new ChangeRequest named from the canonical ref, on a Task's behalf | Exclusive creation; an existing object is reused unchanged after its status.ref is verified against the URL. |
change-request-controller |
All of status; deletion of its own object after retention |
Current resourceVersion, the object's uid and the live manager lease. |
The spec is immutable: any change to it is refused. ownerReferences is
refused, as on every kind: a ChangeRequest belongs to nobody.