Skip to content

fix(daemon): answer repo_loading for configured repos during startup - #956

Open
zzet wants to merge 2 commits into
mainfrom
fix/startup-repo-loading
Open

zzet wants to merge 2 commits into
mainfrom
fix/startup-repo-loading

Conversation

@zzet

@zzet zzet commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

After every daemon restart, a configured repository read as untracked until its own warmup job registered it. That takes about a minute cold, about 10 s warm with a couple of hundred worktrees, and minutes on large stores. In that window:

  • MCP answered repo_not_tracked with "Run gortex track <cwd>".
  • initialize handed out the INACTIVE "track, then reconnect" instructions.
  • The CLI printed "add it with gortex track", or "run gortex repos reconcile" from a worktree.
  • The PreToolUse hook could still deny native Read from the persisted rows.

Agents re-tracked, stopped, or deadlocked. This is the most likely shared cause of reports that the MCP catalog was unavailable for an hour after an upgrade.

With this PR, configured repositories read as loading and get a retryable answer. Every load ends in a terminal state, and sessions start working in place when their repository registers.

Changes

Per-repository load state (internal/indexer/repo_load_state.go)

  • A copy-on-write, lock-free table on MultiIndexer: queued → running → ready | failed{reason} | missing, keyed by path identity (lexical and symlink-resolved).
  • Warmup announces its job set (Global().Repos) before checkout seeding, so the seed window no longer reads as untracked. Each job records start and finish. A batch that fails before draining settles the leftovers, so no repository reads as loading after the work that would have loaded it is gone.
  • failed covers a failed cold track or reconcile, a recovered panic, and a job that ended without registering (including prefix_taken). missing covers a configured path that does not exist.
  • PendingRepoForPath matches a cwd inside a configured root, above one, or in a linked worktree of a loading primary (by git common dir, without spawning git). It reads no store and takes no mi.mu.

MCP admission (cmd/gortex/daemon_mcp*.go, internal/mcp/repo_loading.go)

  • The loading check runs after the registry and scope arms and before the checkout and router arms. A loading cwd is never admitted through the router roster, and a worktree of a loading primary is never told to reconcile.
  • While loading:
    • tools/call returns repo_loading with retryable:true, phase, percent, repo and repo_path. The suggestion never mentions tracking.
    • The allow-list is checked after facade resolution, so workspace(operation:"index") counts as index_health. It admits:
      • index_health, answered from the load record and listing only the caller's repo;
      • capabilities and tools_search;
      • subscribe_* / unsubscribe_*;
      • track and index.
    • resources/read, prompts/get, graph reads, graph_stats and mutations return repo_loading.
  • failed / missing return repo_unavailable (retryable:false, with reason). Capabilities and track still answer, because tracking again is the repair.
  • initialize / server/discover: the server picks the instructions, so they stay in step with the gate.
    • A loading session gets a LOADING preamble ("tools start working in place, no reconnect is needed") ahead of the ordinary instructions.
    • An unavailable session gets repair instructions.
  • track_repository / index_repository / workspace_admin(track) for a still-loading configured root return {status:"already_configured", state:"loading"} instead of queueing behind warmup.
  • A path outside every configured root still gets repo_not_tracked.

Refresh on registration

  • The two first-registration sites (multi.go cold track and warm reconcile) mark the repository ready and fire a registration event. Re-index and file-save rewrites of an existing entry do not fire it, so edits don't clear scopes.
  • The event does four things:
    • calls InvalidateSessionScopes();
    • publishes workspace_readiness {event:"repo_state", repo, state:"ready"} without replacing the warmup phase;
    • sends one notifications/message to each session whose initialize reported the repository as loading;
    • calls CheckoutLifecycle.PrimaryRegistered(prefix).
  • sessionScope no longer caches the blind sentinel for a loading cwd.
  • PrimaryRegistered starts the coordinators that were skipped because their primary was unregistered.
    • On restart that is every admitted coordinator: seeding applies them before warmup registers anything, and until now they waited for the janitor (1 h by default).
    • The skips are recorded in ensureCoordinator, so the order of the family reconcile and the registration doesn't matter.
    • Retries run as activations on the lifecycle context and are joined by Close.
  • ProbeRepo gains state (loading|ready|failed|missing) and state_reason, read from the lock-free table, so the no-lock, no-store contract holds.

CLI

  • repo_loading → "the gortex daemon is still loading (phase, N%)", never "track" or "reconcile". It is a typed error with exit code 75 (EX_TEMPFAIL).
  • On a TTY the call retries for up to --loading-wait (default 30 s). Off a TTY (hooks, scripts) it fails fast. When the wait runs out it reports "still loading", never a failure of the work.
    • The flag is --loading-wait because gortex track already defines a boolean --wait.
    • The wait polls the call every second rather than subscribing to readiness.
  • The worktree pre-flight reads the probe's per-repo state, so a worktree of a loading family goes to the daemon (which answers repo_loading) instead of printing "run gortex repos reconcile".

Hooks

Docs and comments

  • New "Startup states" section in docs/mcp.md; docs/multi-repo.md updated.
  • Fixed the daemon.go comment that promised serving from the persisted store, the stale tools/list comments in daemon_mcp.go, and the probeCWDReach comment.

Not in this PR (deliberately)

  • Untracking a repository that is still loading stays refused, now with repo_loading rather than repo_not_tracked. Admitting it before warmup can be cancelled per repository would let warmup re-register the repository and re-add its config entry after an accepted untrack.
  • Agent-facing wording (the MCP-required line, profile bodies, the facade outcome text) is separate work. JSON-RPC errors never reach the facade outcome classifier today, so that change needs more than wording. This PR only supplies the state.
  • Shortening the wait itself is separate work:
    • taking checkout seeding off the critical path;
    • serving warm repositories from persisted rows while they reconcile;
    • the bulk-window query plan.
  • Dead-code removal (IndexAll / IndexScoped / indexMultiRepo): tests still use IndexScoped, so it's left for a follow-up.

Testing

  • All tests pass (go test -race ./...), run locally (macOS):
    • go test -race ./...: every other package passes. internal/graph/store_sqlite, internal/indexer and internal/mcp hit the 10-minute default timeout, the reason CI shards them.
    • internal/indexer: all six CI shards pass under -race, using the patterns from scripts/ci-test-partition.py --dry-run (678–1344 s each).
    • internal/mcp passes under -race with a 55-minute timeout (1387 s).
    • cmd/gortex, internal/hooks, internal/daemon and internal/pathkey pass under -race.
    • internal/graph/store_sqlite is untouched and was not rerun past its timeout.
    • gofmt -s and go vet are clean. golangci-lint (local v2.15.0) is clean on the changed files; it flags 3 errcheck hits on case <-errChan: in untouched tests (internal/daemon/session_reconnect_test.go, internal/indexer/checkout_deferred_retirement_ready_restart_test.go), a check newer than CI's pinned v2.14.0.
  • New tests added for new functionality
  • Benchmarks run if performance-relevant (n/a: the admission path adds one lock-free map lookup and, only for unregistered cwds, an EvalSymlinks)

New tests:

  • internal/indexer/repo_load_state_test.go:
    • announcement, duplicate entries and missing paths;
    • re-announcement drops unconfigured records;
    • terminal states with reasons;
    • registration announces exactly once, and a re-index does not announce again;
    • PendingRepoForPath for a root, a subdirectory, a parent and a linked worktree;
    • reads don't block while mi.mu is held;
    • PrimaryRegistered starts a coordinator skipped on restart, while another repo's registration (or registration alone) starts nothing.
  • cmd/gortex/daemon_mcp_repo_loading_test.go:
    • state × method table: graph read, facade graph/read, edit, resources/read, prompts/get, graph_stats, index_health legacy and facade (scoped to the caller), capabilities, track → already_configured, initialize, tools/list;
    • a worktree of a loading primary;
    • loading wins over the router arm;
    • failed → repo_unavailable;
    • an unconfigured path still gets repo_not_tracked;
    • the same session is served after registration, with a notifications/message and no reconnect.
  • internal/mcp/repo_loading_test.go: the sentinel scope is not cached while loading; loading and unavailable instructions; facade → legacy resolution.
  • internal/hooks/repo_loading_test.go: loading coverage is never indexed; Read, Edit and Bash cat are allowed; the Grep scope is allowed; the deny resumes when ready.
  • cmd/gortex/daemon_repo_loading_test.go:
    • probe per-repo state: ready, loading, failed with reason, missing; project-only stays empty;
    • coverage repo_state;
    • CLI typed error and exit 75; wait, retry and timeout; non-TTY fails fast.

Checklist

  • Code follows existing patterns in the codebase
  • No unnecessary abstractions added

The registry starts empty on every boot, so until its own warmup job
registered it a configured repository read as untracked: MCP answered
repo_not_tracked with "run gortex track", initialize handed out the
INACTIVE track-then-reconnect instructions, the CLI said "add it with
gortex track", and the hook could deny native reads from persisted rows.

Record each configured repository's load state (queued, running, ready,
failed, missing) in a lock-free table on MultiIndexer, announced before
checkout seeding and settled by every warmup job. The admission gate,
the server instructions, the control probe, file coverage and the CLI
read it: a loading repository gets a retryable repo_loading with
progress, a failed or missing one gets repo_unavailable with a reason,
and a path outside every configured root is still repo_not_tracked.

When a repository first registers, cached session scopes are dropped,
a repo_state readiness event and a per-session message go out, and the
checkout coordinators that were skipped for an unregistered primary
start, so sessions work in place with no reconnect.
@zzet
zzet force-pushed the fix/startup-repo-loading branch from 8782e59 to 2211f27 Compare October 24, 2026 08:44
The track subtest pasted the configured root into the JSON-RPC frame as
is. On Windows the path holds backslashes (D:\a\_temp\...), which are
invalid JSON escapes, so the gate could not parse the call and answered
repo_loading instead of already_configured. Encode the path with
json.Marshal.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant