fix(memory/hindsight): make HINDSIGHT_* env vars overlay config.json (#51166)

[mkslzk]

Summary

_load_config() in the Hindsight memory plugin short-circuited with return json.loads(...) whenever a ~/.hermes/hindsight/config.json existed, so the HINDSIGHT_BANK_ID (and HINDSIGHT_MODE, HINDSIGHT_API_KEY, HINDSIGHT_BUDGET, ...) env vars in a profile's .env were silently ignored.

The user reported setting HINDSIGHT_BANK_ID=nihai-tcm in the profile .env but the plugin still loaded bank hermes (the value from config.json) and routed every auto retain/recall to the wrong bank.

Fix

HINDSIGHT_* env vars are now treated as a per-process overlay on top of the loaded config. JSON remains the source of truth when no env var is set; unsetting the env var is the operator-facing way to let the JSON value win.

HINDSIGHT_BANK_ID is surfaced at both config.bank_id (the read site in HindsightMemoryProvider.__init__ at line 1287) and the legacy config.banks.hermes.bankId location so both lookup shapes work regardless of which path seeded the config.

Tests

• 5 new regression tests in tests/plugins/memory/test_hindsight_load_config.py
• All 128 existing Hindsight tests still green
• Manual repro confirmed via real _load_config() call

Related

• Issue: [Bug]: Hindsight plugin ignores HINDSIGHT_BANK_ID env var when config.json exists #51166

[alt-glitch]

This was generated by AI during triage.

Competing fix for #51166 alongside #51173 (@westkite1201). #51173 takes a narrow config.get("bank_id") or HINDSIGHT_BANK_ID fallback; this PR is the broader per-process overlay of all HINDSIGHT_* env vars on top of config.json. Same goal, different scope/mechanism — flagging the cluster so a maintainer can pick.

[tonydwb]

Code Review Summary

Verdict: Approved

Important fix for the Hindsight memory plugin: HINDSIGHT_* env vars now overlay matching keys on the loaded config, so a per-profile .env like HINDSIGHT_BANK_ID=nihai-tcm is honored regardless of whether a config.json exists on disk.

Looks Good

• The env-var overlay pattern is clean: load JSON first, then overlay env vars on top
• HINDSIGHT_BANK_ID is surfaced at both top-level bank_id and nested banks.hermes.bankId for backward compatibility
• 153 lines of regression tests cover: env var wins over JSON, JSON wins when no env var, bank-id reaches the read site
• Well-documented with clear comments explaining the #51166 bug and the fix

---

Reviewed by Hermes Agent

[yanjiaxi0123-collab]

When can we go live?

[teknium1]

Thanks for addressing a real current-main configuration bug: _load_config() still returns existing JSON before environment fallback (plugins/memory/hindsight/__init__.py:360-363), and bank resolution defaults to hermes when JSON has no bank ID (plugins/memory/hindsight/__init__.py:1286-1299).

Problems

• The claimed general overlay omits documented HINDSIGHT_API_URL (plugins/memory/hindsight/README.md:140); persisted api_url still wins at plugins/memory/hindsight/__init__.py:1283.
• tests/plugins/memory/test_hindsight_load_config.py:72-73 is tautological because line 70 already proves the second or condition.
• The new tests do not initialize HindsightMemoryProvider through the production resolution path at plugins/memory/hindsight/__init__.py:1243-1299.

Suggested changes

• Either narrow the patch to HINDSIGHT_BANK_ID, or include and test every documented override claimed by the new precedence rule.
• Add an initialized-provider regression using an existing profile config that lacks bank_id, and replace the tautological assertion with direct representation checks.

Automated hermes-sweeper review.

[teknium1]

This map claims to overlay matching HINDSIGHT_* values, but it omits documented HINDSIGHT_API_URL (README.md:140). With persisted api_url, current initialization still ignores that override at initialize() line 1283. Please either add and test this mapping or narrow the patch's stated scope.

[teknium1]

This assertion is vacuous: line 70 has already established cfg.get("bank_id") == "nihai-tcm", making the second or branch true regardless of the nested value. Assert the intended nested value directly, or remove this check and test the initialized provider's resolved bank.

[teknium1]

Closing: the bundled Hindsight provider this PR patches has moved out of this repo.

Thanks @mkslzk for this contribution. In #119888 (merge 9d799e0531c; removal commit 4cbf862abe4) the in-tree plugins/memory/hindsight/ provider was removed — Hindsight now installs from the plugin catalog and its code lives in vectorize-io/hindsight hindsight-integrations/hermes (maintained by @nicoloboschi). There is no longer any code in this repo for the Hindsight half of this PR to patch, so we are closing every open PR against the bundled provider rather than leaving them stranded.

Triage notes:

• Touches only plugins/memory/hindsight/init.py plus a new hindsight test file. Not superseded: competing fix(memory): respect HINDSIGHT_BANK_ID when config exists #51173 is still open and unmerged, and no env-overlay landed on main before removal (the only -S HINDSIGHT_BANK_ID hits are the original provider commit, a docstring refactor, and the removal). At the pin, init.py:298-301 still return data for an existing config.json before any env lookup, so [Bug]: Hindsight plugin ignores HINDSIGHT_BANK_ID env var when config.json exists #51166 is still present upstream. Sweeper caveats: the overlay omitted HINDSIGHT_API_URL and one test assertion was tautological.
• Still relevant at the catalog pin (dc750388)? Yes — the same code is at hindsight-integrations/hermes/__init__.py:301 in the upstream tree. It is listed with your credit in Fixes from Hermes-side PRs worth carrying into hindsight-integrations/hermes vectorize-io/hindsight#4662 so it is not lost; if you want to carry the fix yourself, please open it against vectorize-io/hindsight — it would be welcome there.

If you believe this was closed in error, comment and we will reopen.

(Bulk-closed in the hindsight-move close pass.)