docs: define model information management architecture by lpcox · Pull Request #10033 · github/gh-aw-firewall

lpcox · 2026-10-15T17:39:31Z

Summary

add ADR 0004 defining a provider-qualified canonical model information subsystem
specify pricing tariffs, reasoning-effort semantics, feature-level API compatibility, provenance, freshness, and precedence
define generated projections, runtime/offline exports, updater governance, CI requirements, and staged migration gates
incorporate the concrete gaps identified by Add curated AI-credit prices for current Copilot models #10029 and Complete routing effort metadata for current OpenAI and Anthropic models #10030 and connect the rollout baseline to test: characterize model metadata behavior and drift #10032

Validation

npx markdownlint --config .markdownlint.json docs/adr/0004-model-information-management.md
git diff --check -- docs/adr/0004-model-information-management.md

Notes

This PR contains only the ADR. The implementation remains intentionally split into incremental, parity-gated follow-up changes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

# Conflicts: # scripts/ci/smoke-claude-workflow.test.ts # scripts/ci/smoke-cloud-hypervisor-workflow.test.ts # scripts/ci/smoke-codex-workflow.test.ts

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 05030fb2-8413-41a3-9790-349049a61b19

github-actions · 2026-10-15T17:40:11Z

Documentation Preview

Documentation has been built for this PR.

Download preview artifact

To view locally:

Download the docs-preview-pr-10033 artifact from the workflow run
Unzip and open index.html in your browser

Built from commit c2e7545

Copilot

🟡 Changes recommended

The proposal needs explicit accounting safeguards and compatible, reproducible migration requirements.

4 open findings

What changed in this PR

Defines a unified model-information subsystem for AWF’s host and API proxy, with implementation deferred to incremental follow-ups.

Changes:

Specifies provider-qualified identity, pricing, reasoning, compatibility, and provenance.
Defines generated projections, exports, and updater governance.
Establishes characterization tests and parity-gated migration.

File	Description
docs/adr/0004-model-information-management.md	Adds the model-information architecture and rollout plan.

🧠 Review effort: Max

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+Promotional pricing is time-bounded data. It MUST include an expiry or effective
+end and, where known, the successor tariff. A source-code reminder is not an
+acceptable expiry mechanism.


+- public documentation mapping;
+- static catalogue export;
+- generated documentation tables; and
+- manifests containing schema version, catalogue revision, and generation time.


+Overlays are validated against a versioned overlay schema at startup. Unknown
+fields and invalid prices fail before requests are accepted. Operator overlays
+have highest configured authority within their declared provider, channel, and
+deployment scope.


+Add deterministic compilation, migrate maintained facts, and represent the
+current pricing and reasoning gaps as canonical records. Generate existing
+legacy formats without requiring consumers to change.


rabeckett · 2026-10-15T17:59:57Z

Looks good. I'd like gh-aw-router-training to eventually sync its pricing and effort data from
this. Two suggestions:

Add a drift check. Live /models data correctly outranks maintained records, but that's
also how the Add curated AI-credit prices for current Copilot models #10029 gaps went unnoticed. I'd have the updater regularly compare live data with
the registry. It would flag missing models, price differences beyond rounding, and changed
efforts or limits.
Merge prices one tier at a time. Field-level merging can pair a live input price with an
older cache-read price, giving a tariff nobody published. I'd only fill components that the
higher source omits, and label the tier as assembled.

SivaKesava1 · 2026-10-15T18:06:19Z

+1 to Ryan's points. Adding the gh-aw side, since gh-aw is the other big consumer of model facts and the ADR doesn't mention it yet:

Treat gh-aw as a planned consumer. gh-aw keeps its own models.dev-derived models.json (two copies kept in sync by hand) for AIC estimates in audit and logs. That's the same kind of independent table this ADR wants to remove, and it already drifted: Fix direct Anthropic Claude Sonnet 5.5 cache-read price gh-aw#69071 had Sonnet 5.5's cache-read price at $0.20 instead of $0.10. Could the ADR name gh-aw's models.json as a downstream projection (from awf models export or the npm package) and say at which gate it can switch? Until then, gh-aw and AWF will keep disagreeing on costs for the same run.
Make /reflect/v2 easy for older consumers to detect. gh-aw's harness and SDK mode read the model list from /reflect, and gh-aw runs against whatever AWF version a workflow pins. A schema version field and a documented fallback to v1 would let gh-aw adopt v2 without breaking older AWF versions.
Export where each value came from. Audit and AIC numbers should be able to say "estimated" or "stale" instead of showing a guess as a confirmed price. If the export carries each value's source and freshness, gh-aw and the router can show that label instead of rebuilding the logic themselves.

lpcox · 2026-10-15T22:11:57Z

@copilot address review and comment feedback

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot · 2026-10-15T22:16:59Z

@copilot address review and comment feedback

Updated ADR 0004 to address the review and actionable feedback; committed as 16a6d72.

Define workflow-configured model overlays, configurable central pricing refresh, and model-to-harness version constraints. Clarify tariff component scope and directed protocol compatibility, and wire the requirements into rollout and validation gates. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-10-15T22:27:39Z

✅ Copilot review passed with no inline comments.

@lpcox Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

Assume the default pricing feed is blocked. Define workflow and AWF config source replacement, internal mirrors and local snapshots, no implicit network access, and approved fallback behavior. Extend validation and rollout requirements. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Use governed provider-qualified models.dev snapshots for inference providers without native AWF pricing support. Preserve precedence, provenance, and offline operation, and cover the fallback in validation and rollout requirements. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

lpcox and others added 4 commits October 8, 2026 16:08

test: fix main test suite failures

65a60ba

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Merge remote-tracking branch 'origin/main' into fix-failing-tests

4b5643f

# Conflicts: # scripts/ci/smoke-claude-workflow.test.ts # scripts/ci/smoke-cloud-hypervisor-workflow.test.ts # scripts/ci/smoke-codex-workflow.test.ts

docs: add model information management ADR

171b94a

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 05030fb2-8413-41a3-9790-349049a61b19

chore: align reused branch with main

5aaa9ac

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 05030fb2-8413-41a3-9790-349049a61b19

Copilot AI max review requested due to automatic review settings October 15, 2026 17:39

Copilot started reviewing on behalf of lpcox October 15, 2026 17:40

View session

Copilot AI reviewed Oct 15, 2026

View reviewed changes

Copilot started work on behalf of lpcox October 15, 2026 22:12

View session

docs: address model catalog ADR feedback

16a6d72

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot finished work on behalf of lpcox October 15, 2026 22:17

pelikhan mentioned this pull request Oct 15, 2026

docs: define configurable, offline-first model information architecture #10041

Merged

lpcox merged commit 4e0104a into main Oct 16, 2026
39 checks passed

lpcox deleted the fix-failing-tests branch October 16, 2026 04:30

Navigation Menu

Repository navigation

docs: define model information management architecture - #10033

docs: define model information management architecture#10033
lpcox merged 8 commits into
mainfrom
fix-failing-tests

lpcox commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

rabeckett commented Oct 15, 2026

Uh oh!

SivaKesava1 commented Oct 15, 2026 •

edited

Loading

Uh oh!

lpcox commented Oct 15, 2026

Uh oh!

Copilot AI commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

lpcox commented Oct 15, 2026

Summary

Validation

Notes

Uh oh!

github-actions Bot commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation Preview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

rabeckett commented Oct 15, 2026

Uh oh!

SivaKesava1 commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lpcox commented Oct 15, 2026

Uh oh!

Copilot AI commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

github-actions Bot commented Oct 15, 2026 •

edited

Loading

SivaKesava1 commented Oct 15, 2026 •

edited

Loading