Skip to content

feat(stovepipe): Add open issue lookup storage - #984

Merged
prathshenoy merged 1 commit into
prath.shenoy/feat/stovepipe-issue-materializerfrom
prath.shenoy/feat/stovepipe-open-issue-store
Oct 30, 2026
Merged

feat(stovepipe): Add open issue lookup storage#984
prathshenoy merged 1 commit into
prath.shenoy/feat/stovepipe-issue-materializerfrom
prath.shenoy/feat/stovepipe-open-issue-store

Conversation

@prathshenoy

@prathshenoy prathshenoy commented Oct 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

What:

  • Add queue-scoped storage for open-issue references with bounded listing and conditional replacement or deletion.

Why:

  • Avoid scanning retained closed summaries to find open issues. Summaries intentionally keep closed records for ordering, so a separate lookup provides the access path for the upcoming open-issues view. Populating that lookup and exposing it through an API remain follow-up work.
  • Protect a newer opening from an old close. The same issue key can close and reopen, so replacement and deletion must check which opening they are changing. Reuse the existing table without adding columns or changing operational issue tracking.

Test Plan

  • Add unit tests.
  • Verify conditional writes and bounded listing against a real MySQL database.

Revert Plan

  • Revert this PR. No runtime writer uses the lookup, and the existing table layout is unchanged.

Issue Links

Stack

  1. feat(stovepipe): Add issue summary storage #982
  2. feat(stovepipe): Materialize issue summaries #983
  3. feat(stovepipe): Add open issue lookup storage #984
  4. feat(stovepipe): Maintain open issue lookups #986
  5. feat(stovepipe): Add open issue listing #988
  6. feat(stovepipe): Expose open issue listing #990
  7. feat(stovepipe-web): Show open issues #993

This was referenced Oct 29, 2026
Merged
Merged
Merged
@prathshenoy
prathshenoy added this pull request to stack #987 October 29, 2026 04:50
@prathshenoy prathshenoy mentioned this pull request Oct 29, 2026
Merged
1 task done
@prathshenoy
prathshenoy force-pushed the prath.shenoy/feat/stovepipe-open-issue-store branch 2 times, most recently from 8118adb to e0c724d Compare October 29, 2026 20:05
This was referenced Oct 29, 2026
Merged
Merged
@prathshenoy
prathshenoy added this pull request to the merge queue Oct 30, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue because a pull request earlier in the stack was removed Oct 30, 2026
@prathshenoy
prathshenoy force-pushed the prath.shenoy/feat/stovepipe-open-issue-store branch from e0c724d to 79dc99f Compare October 30, 2026 18:09
@prathshenoy
prathshenoy added this pull request to the merge queue Oct 30, 2026
Merged via the queue into main with commit ea8c271 Oct 30, 2026
19 checks passed
@prathshenoy
prathshenoy deleted the prath.shenoy/feat/stovepipe-open-issue-store branch October 30, 2026 18:51
DivyanshGarg380 pushed a commit to DivyanshGarg380/submitqueue that referenced this pull request Oct 31, 2026
@prathshenoy
feat(stovepipe): Add issue summary storage (uber#982)
c6c164c 
## Summary
**What**:
- Add queue-scoped storage for open and closed issue summaries with
version-checked writes.

**Why**:
- Establish a separate read model for the upcoming open-issues view. The
current issue records support operational tracking; this adds the
storage foundation for a user-facing projection without changing issue
processing or enabling a new writer.
- Retain closed summaries so a later materializer can distinguish an old
opening from a newer close. Deleting a closed summary would lose that
ordering information and let a replay look like a newly discovered
issue.

## Test Plan
- [x] Add unit tests.
- [x] Verify persistence, closure, reopening, and conditional updates
against a real MySQL database.

## Revert Plan
- Revert this PR. No runtime writer uses the new table, which can remain
unused after rollback.

## Issue Links
- [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis)

## Stack
1. uber#982
1. uber#983
1. uber#984
1. uber#986
1. uber#988
1. uber#990
1. uber#993
DivyanshGarg380 pushed a commit to DivyanshGarg380/submitqueue that referenced this pull request Oct 31, 2026
@prathshenoy
feat(stovepipe): Materialize issue summaries (uber#983)
1f378bf 
## Summary
**What**:
- Update issue summaries when normal processing or recovery records an
opening or closing.
- Preserve newer summaries when older transitions are replayed.

**Why**:
- Populate the read model introduced by the preceding storage PR.
Recording issue history alone leaves those summaries empty; connecting
accepted changes to the summary prepares the user-facing view without
adding open-only mappings or an API yet.
- Repair partial writes without letting delayed workers undo newer
changes. History can be retained before a summary write fails, so
retries must update the summary even when the history already exists.
Source request order and conditional writes keep an older opening from
replacing a newer close or reopening.

## Test Plan
- [x] Add unit tests.

## Revert Plan
- Revert this PR before its storage dependency. Operational issue state
and history remain intact; summaries stop updating.

## Issue Links
- [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis)

## Stack
1. uber#982
1. uber#983
1. uber#984
1. uber#986
1. uber#988
1. uber#990
1. uber#993
DivyanshGarg380 pushed a commit to DivyanshGarg380/submitqueue that referenced this pull request Oct 31, 2026
@prathshenoy
feat(stovepipe): Maintain open issue lookups (uber#986)
0712b9d 
## Summary
**What**:
- Maintain open-issue lookup entries during normal issue tracking and
recovery.

**Why**:
- Populate the lookup needed to list open issues without scanning closed
summaries. Storage support alone does not maintain those rows as issues
open, close, and reopen.
- Repair partial writes without restoring older openings. A summary can
be saved while its lookup write fails, so redelivery must repair the
lookup even when history already exists. Use the latest persisted
summary so an older delivery cannot make the lookup follow an outdated
transition.

## Test Plan
- [x] Add unit tests.
- [x] Verify issue lifecycle and lookup repair against a real MySQL
database.

## Revert Plan
- Revert this PR. Issue state, history, and summaries continue updating;
lookup maintenance stops, and no public API depends on it yet.

## Issue Links
- [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis)

## Stack
1. uber#982
1. uber#983
1. uber#984
1. uber#986
1. uber#988
1. uber#990
1. uber#993
DivyanshGarg380 pushed a commit to DivyanshGarg380/submitqueue that referenced this pull request Oct 31, 2026
@prathshenoy
feat(stovepipe): Add open issue listing (uber#988)
480cb68 
## Summary
**What**:
- Add bounded listing of current open issues within a queue.

**Why**:
- Establish the read behavior needed by the upcoming open-issues API.
Storage already lists open-issue rows, but those rows can temporarily
refer to closed issues or an earlier opening; callers must check the
summaries before returning results.
- Fill pages with valid issues instead of stopping at stale lookup
entries. Cap the work done by each request so a long stretch of stale
references cannot cause unbounded reads, and let callers continue from
the last inspected entry.

## Test Plan
- [x] Add unit tests.

## Revert Plan
- Revert this PR. Listing is not exposed through the service yet, and no
persisted data changes.

## Issue Links
- [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis)

## Stack
1. uber#982
1. uber#983
1. uber#984
1. uber#986
1. uber#988
1. uber#990
1. uber#993
DivyanshGarg380 pushed a commit to DivyanshGarg380/submitqueue that referenced this pull request Oct 31, 2026
@prathshenoy
feat(stovepipe): Expose open issue listing (uber#990)
f5ca767 
## Summary
**What**:
- Expose current open issues through the service API with bounded
pagination.

**Why**:
- Make recorded issues available to clients. The listing logic already
reads and validates current issue summaries, but clients cannot use it
until the service exposes the endpoint. This connects that behavior to
the upcoming open-issues view.

## Test Plan
- [x] Add unit tests.
- [x] Verify pagination and stale-reference filtering through the
running reference service with MySQL.

## Revert Plan
- Revert this PR. The endpoint is read-only, with no schema changes or
persisted state changes.

## Issue Links
- [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis)

## Stack
1. uber#982
1. uber#983
1. uber#984
1. uber#986
1. uber#988
1. uber#990
1. uber#993
DivyanshGarg380 pushed a commit to DivyanshGarg380/submitqueue that referenced this pull request Oct 31, 2026
@prathshenoy
feat(stovepipe-web): Show open issues (uber#993)
8930876 
## Summary
**What**:
- Show current queue issues grouped by task, with task-wide failures
first within each task.
- Add continuation loading, opening-request links, and refresh behavior
that preserves the last successful display when reads fail.

**Why**:
- Make recorded failures visible to users. The backend already maintains
and lists open issues, but the web UI currently exposes only requests
and project results.
- Keep long issue lists readable while new results arrive. Polling must
not discard the pages a user has loaded, and a failed refresh must not
erase the last successful display.
<img width="1509" height="860" alt="Screenshot 2026-10-30 at 11 04 53"
src="https://github.com/user-attachments/assets/c5a82164-26df-4568-977b-142420c32604"
/>



## Test Plan
- [x] Add unit tests.
- [x] Verify grouped listing, pagination, refresh, and accessibility in
a browser against the running service and MySQL.

## Revert Plan
- Revert this PR. The view is read-only, and the backend APIs and stored
issue data are unchanged.

## Issue Links
- [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis)

## Stack
1. uber#982
1. uber#983
1. uber#984
1. uber#986
1. uber#988
1. uber#990
1. uber#993
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mnoah1 mnoah1 mnoah1 approved these changes
@behinddwalls behinddwalls Awaiting requested review from behinddwalls behinddwalls is a code owner
@sbalabanov sbalabanov Awaiting requested review from sbalabanov sbalabanov is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@prathshenoy @mnoah1