Repository navigation
fix(kanban): report a fully held ready queue instead of an idle board - #123964
yoyodine-industries wants to merge 34 commits into
Conversation
The diagnosis is right and the guard-vocabulary unification (guard 4 honouring 1. Blocker:
Drove the real
Both failure modes from one cause: the strip shows on healthy tenants while the genuinely starved one stays silent — the inverse of the PR's goal. The CLI and Minor: Unverified / please confirm: the table is executed, not inferred — real |
|
Thanks — confirmed, this is a real defect and I reproduced it independently at head Cause (verified read-only at head): Independent probe — the real
On your unverified note: tenant-scoped dashboards are supported — The fix (thread |
board_health scanned every ready row on the board, so its starved-vs-idle verdict lied for a ?tenant= view in BOTH directions: a tenant with one held ready row read "idle", and a tenant with no ready rows read "starved" because another tenant was stalled. Thread tenant through board_health (exact match, the same semantics as kanban_db.list_tasks), the embedded /board dispatch_health tuple and /health, and the CLI health read; add --tenant to the health verb. The scoped receipt names the tenant it measured, so it cannot be mistaken for the unscoped one. The per-profile-cap running count stays board-wide on purpose (the cap is a physical per-profile limit, not a tenant one). Pin the cap-branch-before-guard invariant the reviewer flagged. Reviewer: Enough1122 (PR NousResearch#123964).
|
Fixed — New head: What changed — 4 source files, 2 test files:
Both directions, RED without the fix:
Tests. |
…s event spam The ready-lane respawn guards could hold an entire board's queue while every surface still read as a correctly idle board, and one of them wrote an event per tick, so the only evidence was both unfindable and unreadable. - active_pr now lifts on a deliberate re-queue (status / promoted / unblocked) in addition to a handoff (assigned / changes_requested / review_reopened). reclaimed stays out of it: crash recovery is not a decision. - respawn_guarded is written once per hold episode instead of every tick. - describe_suppression names every bucket that can hold a ready row. - board_health reports (ready_total, spawnable, suppressed_by_reason) plus a state verdict, surfaced as `hermes kanban health`, on the board payload and as a dashboard strip, so starved != idle at any row count. - six consecutive stalled ticks file one routed escalation card with repeat spacing, instead of one more line in a log nobody is tailing.
board_health scanned every ready row on the board, so its starved-vs-idle verdict lied for a ?tenant= view in BOTH directions: a tenant with one held ready row read "idle", and a tenant with no ready rows read "starved" because another tenant was stalled. Thread tenant through board_health (exact match, the same semantics as kanban_db.list_tasks), the embedded /board dispatch_health tuple and /health, and the CLI health read; add --tenant to the health verb. The scoped receipt names the tenant it measured, so it cannot be mistaken for the unscoped one. The per-profile-cap running count stays board-wide on purpose (the cap is a physical per-profile limit, not a tenant one). Pin the cap-branch-before-guard invariant the reviewer flagged. Reviewer: Enough1122 (PR NousResearch#123964).
591e7b8 to
0d43a06
Compare
…resolution) Conflict: hermes_cli/kanban_db_dispatch.py `check_respawn_guard` (guards 3 and 4). This one is a REAL semantic overlap — two independent reworks of the same region — so it is resolved as a union, not by taking a side: * Upstream (after this branch's base) routed both guards through a new `_rows_since()` helper and hardened every guard timestamp with `_kb._to_epoch()` (NousResearch#97455: a TEXT stamp sorts above every INTEGER, so the raw `int()` reads this branch still carried would raise on one). * This branch introduced the named kind vocabulary (`_RESPAWN_GUARD_REQUEUE_KINDS` / `_RECOVERY_KINDS` / `_HANDOFF_KINDS`) with the `_kind_clause()` placeholder helper, made guard 4 honour a deliberate re-queue, and added the `respawn_guarded` episode throttle. Resolution: keep upstream's `_rows_since()` structure (so the `_to_epoch` hardening applies to every guard read), and re-express this branch's named kind tuples through it — guard 3 keeps its requeue+recovery vocabulary and guard 4 keeps its requeue-lifts / `reclaimed`-does-not semantics verbatim, including the payload-aware `_is_handoff_event` loop. `_rows_since()` gains one keyword, `params`, so a caller-supplied `where` may carry its own placeholders, bound ahead of `since`. Validation: the six suites that exercise the guard pass (150 passed, 1 skipped); the one failure, test_infrastructure_spawn_refusal_never_charges_the_card, also fails at this branch's pre-merge tip and at pristine upstream/main, so it is a pre-existing environment failure and not a resolution artefact. Fork policy: the resolved branch is pushed to the FORK branch only; nothing is merged to fork main, which is an upstream mirror. Agent: platform-coder
A ready lane whose every spawnable row is held by a respawn guard spawns nothing,
tick after tick, and no surface outside
gateway.logsays so.describe_suppression()summarises the tick from skip buckets it only partially reads, and there is no
board-level read that separates "all of this work is held" from "there is no work".
This PR makes the hold visible, bounds the event it writes, and lets a deliberate
re-queue release it.
One concern:
0 spawnedmust not be the only number available, and a board that isfully held must not read as idle.
Related Issue
Fixes #123963. The PRs that share this symptom are cross-linked under "Related work"
below; none of them carries the board-level read, and no covering issue existed, which
is the half that issue is filed against.
Type of Change
What changes
hermes_cli/kanban_db_dispatch.pyactive_prlifts onstatus,promotedorunblockedas well as on the handoffkinds (
assigned,changes_requested,review_reopened).reclaimedisdeliberately excluded, unlike guard 3: a crash or reclaim is not a decision, and
the PR is still open and unattended.
_RESPAWN_GUARD_PR_WINDOWis untouched at 86400, and a test asserts it.respawn_guardedis written once per hold episode: re-emitted when the reasonchanges, when a new episode starts, or after an hour.
describe_suppression()countsskipped_per_profile_capped,skipped_nonspawnableand
skipped_unassignedin addition to the buckets it already counted.board_health()returns(ready_total, spawnable, suppressed_by_reason)with astate verdict, and
observe_dispatch_tick()files one routed card after sixconsecutive stalled ticks with repeat spacing.
hermes_cli/kanban.py,hermes_cli/kanban_parser.pyhermes kanban healthrenders the verdict and the per-reason counts.plugins/kanban/dashboard/plugin_api.py,dist/index.js,dist/style.cssdispatch_health, and a strip renders it, so a starvedboard does not look like a quiet one.
No config key, no env override, no GitHub API call from the tick.
Root cause
check_respawn_guardmatches any GitHub PR URL in a card's thread(
hermes_cli/kanban_db_dispatch.py:1784-1802, regex at :85) inside a 24 hour window(:83). The URL does not have to belong to this card and its state is never consulted:
a card naming a parent's PR, or a PR that has since merged, is held exactly like one
with its own open PR. The lift set at :1797 is stricter than the sibling guard's at
:1766-1776 over a window 24 times longer.
Verification
Base is
d0288be5b3, the same commit asupstream/mainat branch time.d0288be5b3= 21 failed / 4 passed(the 4 passing are the controls: handoff vocabulary, the
reclaimednegative, theempty tick, the window assertion), tip = 25 passed.
scripts/run_tests.shover the nine kanban test files: 129 passed, 1 failed,1 skipped, with
test_kanban_dispatch_health.pyat 25 passed. The single failure(
test_kanban_db.py::test_infrastructure_spawn_refusal_never_charges_the_card) ispre-existing: the same file, run the same way at
d0288be5b3, fails identically.How to Test
scripts/run_tests.sh tests/hermes_cli/test_kanban_dispatch_health.py— 25 passed.hermes kanban healthreports
state=starvedand names the guard insuppressed_by_reason; an emptyboard reports
state=idle. The state line does not depend on how many rows are held.hermes kanban unblock <id>(orpromote) on a held row, thenrun a dispatch tick — the row starts, and the guard event is not written again.
Related work
Searched for existing PRs before writing this one. Each of these is open and covers a
part of the same symptom:
respawn_guardedevents.The root cause here is complementary rather than the same: those change when a hold is
released or how the identical events are collapsed, while this one adds the board-level
read and the stalled-board escalation, so a fully held queue is reported as its own
condition instead of as an idle board. If a maintainer would rather take the visibility
half alone, it splits cleanly at
board_health()plus the dashboard and CLI renderers.Checklist
Code
fix(kanban):, one commit, seven paths)tests/hermes_cli/test_kanban_dispatch_health.py, 25 cases, 21 of which fail at the base commitscripts/run_tests.shover the nine kanban test files → 129 passed, 1 failed, 1 skipped, withtests/hermes_cli/test_kanban_dispatch_health.pyalone → 25 passed. The one failure,tests/hermes_cli/test_kanban_db.py::test_infrastructure_spawn_refusal_never_charges_the_card, reproduces identically at the base commitd0288be5b3, so it is pre-existing rather than introduced here. The fullpytest tests/ -qwas not run on this host.Documentation & Housekeeping
docs/, docstrings) — or N/A: this tree has nodocs/directory and no README content covers these reads. The new subcommand's help and description ship inhermes_cli/kanban_parser.py, and_cmd_healthcarries its docstring, both in the diff.cli-config.yaml.exampleif I added/changed config keys — or N/A: no config key is added, on purpose, so the behaviour cannot be configured away.CONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/A: no architecture or workflow changed. The new read is additive, and the escalation only fires after six stalled ticks.For New Skills
Not applicable — this PR adds no skill.
Screenshots / Logs
scripts/run_tests.sh tests/hermes_cli/test_kanban_dispatch_health.pyon the branch tip:Authored with AI assistance (Hermes Agent); diff verified manually, conventional commits
per the contributing guide.