RFC: TealTiger Advisor Agent — design-time governance (feedback wanted) #555
Replies: 1 comment
|
the "shadow AI agents holding API keys" problem you're describing is real and the design-time scanner approach makes sense, but worth noting that even after you find and policy-gate those secrets, the raw key often still ends up in env vars or model context during execution. a complementary piece is keeping secrets out of that surface entirely, not just governing access to them. that's the angle we take with 1Claw: keys live in an HSM-backed vault, the agent gets a short-lived scoped token and only ever references a path, so the key never lands anywhere a model or log can see it. your static analysis could flag "this agent touches a credential, here's the TealTiger policy to add" and also recommend vault-backed references as the fix, which closes the gap more completely. disclosure: i work on 1Claw, so take that as one option worth considering, not a push. |
Uh oh!
There was an error while loading. Please reload this page.
RFC / Discussion: TealTiger Advisor Agent — design-time governance analysis
TL;DR — TealTiger today is a runtime shield: the SDK, framework adapters, and MCP server enforce deterministic policy while your agent runs. We're considering a design-time / CI companion — a
tealtiger audittool that reads your agent's codebase, finds governance gaps, and generates the exact TealTiger policy bundle to close them (optionally opening a PR). Same engine, shipped as a standalone CLI and as MCP tools. Crucially: it advises at design time — runtime enforcement stays 100% deterministic. No LLM ever enters the allow/deny path.Why we're thinking about this
Two adoption problems live before runtime, and the runtime shield doesn't touch them:
Writing the policy is the friction. If you've got a CrewAI / LangChain / AG2 / Haystack app, you have to learn TealTiger's policy vocabulary before you get value. That's hours of doc-reading between "installed" and "protected."
Teams can't see what's exposed. "Shadow AI" — agents holding API keys, hitting databases, running shell commands with zero governance — is invisible until something breaks. Security teams want visibility and a gate, not another library to wire up.
The idea: point a tool at a repo, and it tells you which governance controls are missing and writes the policy bundle that fixes them. TealTiger goes from "a library you must learn" to "a tool that drafts your governance for you" — and gives you a CI gate in a category you already know (think Snyk / SonarQube, but for agent governance).
The hard boundary (please hold us to this)
The Advisor is probabilistic and advisory by design — it reasons about intent from your system prompts and code. That would seem to clash with TealTiger's founding rule of no LLM in the governance path. It doesn't, and here's the line we will not cross:
If you see us drifting toward "an LLM judging an LLM at runtime," call it out — that version is explicitly off the table.
What already exists (so this isn't vaporware)
Grounding the idea in what's real today:
condition → automation_level → reason_code → control_id, FREEZE rules, cost ceilings, a signed digest) that compile to the actual enforcement policies. The Advisor's main output is this exact file.ast, TSts-morph/@typescript-eslint) — finding the structural gaps needs no ML.New in this proposal: the repo scanner, the gap grader, the bundle synthesizer, the optional PR drafter, and the
tealtigerCLI packaging.How it would work
tealtiger-report.json+ SARIF by default; optionally a PR that drops the bundle in and wires the middleware into your framework.Where the LLM is (and isn't):
Two ways to ship it (same engine)
tealtiger audit ./app·tealtiger audit --fail-on hightealtiger_analyze_governance_gaps()etc. in Claude Code / CursorAs a CI gate it's the familiar shape:
Open questions we'd love input on
Related idea:
tealtiger-goatA deliberately-vulnerable demo agent (WebGoat / DVWA, but for agents): a prompt that falls for injection, an unchecked bash-exec tool, a raw-PII pipeline. Run it raw → watch it self-exploit → plug in TealTiger → rerun → watch it get blocked with an audit receipt. A safe playground to see why runtime governance matters — and the benchmark corpus we'd test the Advisor against. Would a sandbox like this be useful to you? What vulnerability would you most want to see demonstrated?
How to weigh in: drop a comment with your framework, where governance gaps bite you today, and whether a design-time audit would change how you adopt. If you've got a messy agent repo you'd point this at — tell us what you'd want it to catch. 🐯
All reactions