Skip to content

Let the extension follow PI_TYPESAFE_BACKEND and state the host it actually uses - #33

Merged
DevMortimer merged 6 commits into
DevMortimer:mainfrom
UriZafrir:feat/extension-backend-selection
Oct 24, 2026
Merged

DevMortimer merged 6 commits into
DevMortimer:mainfrom
UriZafrir:feat/extension-backend-selection

Conversation

@UriZafrir

Copy link
Copy Markdown
Contributor

What and why

Closes #32.

The client already supports five backends; the extension could not select one, because src/extension.ts called createTypeSafe() bare. This adds the selection and makes the text the user consents to match where the data actually goes.

  • PI_TYPESAFE_BACKEND picks the backend (default typesafe). An unknown name is a configuration error at session start, not a silent fallback to TypeSafe.
  • PI_TYPESAFE_MODEL overrides the backend's default model, written in the backend's own id form (jev-1.14; mapped to typesafe/jev-1.14 on OpenRouter).
  • The consent line and /typesafe status name the host, key variable, and model actually used. Previously the disclosure was the fixed api.typesafe.ai sentence, which would have stated a false destination once routing changed.
  • /typesafe login and /typesafe logout decline on a backend that reads its key from the environment instead of the TypeSafe store, and say which variable to set.
  • Four exports added to src/index.ts (BACKEND_ENV, MODEL_ENV, backendFromEnvironment, modelFromEnvironment), each documented in docs/api.md as CONTRIBUTING requires.

Verification

  • npm ci and npm run check pass (build, typecheck, 175 offline tests; 169 before, 6 added).
  • Changed behavior has offline tests, per CONTRIBUTING. New cases in tests/backends.test.ts cover the readers, including blank, unknown, and whitespace-padded values. New cases in tests/extension.test.ts load the extension through Pi's native loader under an override and assert the request URL goes to openrouter.ai and never api.typesafe.ai, that an invalid name rejects before any request, and that login/logout do not prompt.
  • Documentation updated: the README sentence that stated the tool always uses TypeSafe, plus docs/api.md.

Live testing: not applicable to the transport or response validation, which are untouched. The new tests assert the destination host at the fetch boundary instead. I did also confirm the routing by hand in a Pi session with PI_TYPESAFE_BACKEND=openrouter, where the tool returned a real judgment from typesafe/jev-1.13-20260917.

Compatibility and release notes

Additive. With PI_TYPESAFE_BACKEND unset the extension behaves exactly as before — same host, same key store, same model — so existing installs are unaffected. The one behavioral difference is that /typesafe login and /typesafe logout now decline on a non-TypeSafe backend instead of storing a key the session would not read.

This changes a documented guarantee (the README sentence above), so if single-backend-only was deliberate, close it and I will rework the disclosure instead. No version bump, per CONTRIBUTING.

The client supports five backends, but the extension called createTypeSafe()
bare, so the tool and /typesafe commands could only use TypeSafe.

Add PI_TYPESAFE_BACKEND to select the backend, defaulting to TypeSafe, and
reject an unknown name as a configuration error at session start rather than
falling back silently. Add PI_TYPESAFE_MODEL to override the backend default
model in the backend's own id form.

Derive the consent line and status text from the backend actually used. The
disclosure was the fixed api.typesafe.ai sentence, which would have stated a
false destination once routing changed.

Decline /typesafe login and logout on a backend that reads its key from the
environment instead of the TypeSafe store.

@DevMortimer DevMortimer left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — the destination side of this is right. With PI_TYPESAFE_BACKEND=openrouter a request goes to openrouter.ai/api/alpha/decisions, an unknown name raises a startup callout and a configuration error on the tool, and login/logout decline naming the right variable. Three things need to change before this merges; I reproduced each against bc4d1e2.

1. The consent line and status name a model that is not sent. disclosureFor and backendLine (src/extension.ts:25, :32) and the status line (:177) print model ?? defaultModelId(backend) — the caller's bare id — while the request carries mapModel(requested) (src/client.ts:227). With PI_TYPESAFE_BACKEND=openrouter PI_TYPESAFE_MODEL=jev-1.14:

  • the dialog says … as model "jev-1.14" …
  • the request body carries "model": "typesafe/jev-1.14"

tests/extension.test.ts:362 already asserts the body carries typesafe/jev-1.14, and line 365 asserts the status says Model: jev-1.14, so the test pins the mismatch rather than catching it. Mapping the id in those three places and asserting printed model equals body model closes it.

2. README.md:117 still promises one destination. “Only the state and questions you … submit, to https://api.typesafe.ai only.” Under the override the request went to openrouter.ai on the wire. This is the sentence the privacy section rests on, so it needs the same treatment as the disclosure: name the host /typesafe status reports, or name the override explicitly.

3. An inherited property name passes the backend check and breaks the whole extension. backendConfig reads DECISIONS_BACKENDS[name] (src/backends.ts:112), so any Object.prototype key satisfies the “unknown backend” check. With PI_TYPESAFE_BACKEND=toString — and the same for constructor, __proto__ and hasOwnProperty — backendFromEnvironment returns the name and registration throws TypeError: model.includes is not a function. Pi then reports Failed to load extension, so there is no tool, no /typesafe command and no callout at all. Object.hasOwn(DECISIONS_BACKENDS, name) before the return, plus those four names in the existing throw test, fixes it.

Smaller, not blocking:

  • With the default backend and PI_TYPESAFE_MODEL set, the enable dialog still prints the fixed sentence with no model named.
  • The status line names the model twice: model "jev-1.14" and Model: jev-1.14.
  • With an invalid name, login still prompts and logout still deletes the stored TypeSafe key, although no judgment can be sent.

Everything else in the description holds: with both variables unset the host, key store, model and dialog text are unchanged, the four exports are documented in docs/api.md, the per-backend variables and decline messages are right, and npm ci && npm run check passes with 175 tests.

`PI_TYPESAFE_BACKEND=toString` — and `constructor`, `__proto__`,
`hasOwnProperty` — passed the unknown-name check, because a bare registry
lookup finds those on `Object.prototype`. Registration then threw
`TypeError: model.includes is not a function`, and Pi loaded no tool and
no command at all.

The registry lookup now requires an own entry, so the name is refused as
unknown with a `configuration` error and the extension still loads.
`disclosureFor`, `backendLine`, and the status `Model:` line printed
`model ?? defaultModelId(backend)` — the caller's bare id — while the
request went out with the backend's own id form. With
`PI_TYPESAFE_BACKEND=openrouter PI_TYPESAFE_MODEL=jev-1.14` the dialog and
the status said `jev-1.14` while OpenRouter received `typesafe/jev-1.14`.

The sent id is now derived in one place, `sentModelId`, which the client
sends and the extension prints, so the two cannot drift. A test captures
the request body at `fetch` and asserts the dialog and status strings
name the same id.

With that value in hand:

- The consent sentence names the host, the backend, and the model
  whenever either variable is set, and is the unchanged fixed sentence
  when both are unset, so `PI_TYPESAFE_MODEL` on the default backend is
  also named instead of silently changing what is sent.
- The status line names the model once: the backend line gives the host
  and key variable, `Model:` appears only when the consent sentence does
  not name the model, the stray double space is gone, and a status taken
  with an unusable backend name claims no model.
- The README's "What is sent?" sentence no longer promises one
  destination; it names the default host and the override that replaces
  it.
`PI_TYPESAFE_BACKEND` that names no known backend leaves nothing able to
send a judgment, yet `/typesafe login` still prompted for a key nothing
would read and `/typesafe logout` still deleted the stored TypeSafe key.

Both now decline with the reason and the variable to fix, so a typo cannot
cost the operator a stored key. Enabling and disabling are untouched.
Backend selection and the model readers are additive API, so the minor
version moves to 0.11.0 per CONTRIBUTING. The changelog gains the backend
selection, the consent and status text that names what is sent, the
login/logout decline, and the registry guard; `## Unreleased` stays in
place, empty.
`sentModelId` is a public export, and CONTRIBUTING requires a line for
every export in docs/api.md. Its sentence sat at the end of the backend
registry paragraph, away from the mapping it names; move it into the
mapping paragraph so a reader who looks for how an id is mapped finds it.
@DevMortimer
DevMortimer merged commit 0e0297f into DevMortimer:main Oct 24, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

typesafe_evaluate and /typesafe cannot use a backend other than TypeSafe

2 participants