diff --git a/CHANGELOG.md b/CHANGELOG.md index eeb2c668..bd077a01 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to this repository will be documented in this file. ## Unreleased +- Add `agentcontrol/legacy-sdk-migrate` 0.1.0: migrate an app from the legacy AI SDKs (`launchdarkly-server-sdk-ai`, `@launchdarkly/server-sdk-ai`) to the current handler-based SDKs, following [Migrate from the legacy AI SDKs](https://launchdarkly.com/docs/sdk/ai/migration). Inventories legacy calls, maps each to a convenience function, `config().invoke()`, or `graph()`, moves tools into `tool_handlers` / `toolHandlers` or a `Registry`, replaces trackers with automatic metrics (and `track_feedback` with `result.track_data`), ports `enabled=False` fallbacks to error handling and `inspect_config()` / `inspectConfig()`, and covers streaming, history, async judges, and graphs. `references/api-mapping.md` has before/after code in Python and Node.js using the handler factory names the packages export. - `feature-flags/flag-release` 0.2.0 and `feature-flags/flag-and-release-change` 0.2.0: release with environment release plans instead of release policies, which the plans replace at GA. Each environment's default release plan decides how a flag rolls out there, and a stated release intent (now, hold, `notBefore`, segment, prerequisite) still overrides it per environment. Precedence is intent, then an explicit override, then a flag release plan, then the environment's default release plan. The preview reads each environment's default plan with `get-environment-release-plans` (it does not depend on the flag, so the orchestrator's plan phase uses it before the flag exists), and reads a flag release plan through `launchdarkly-release-plans` where one applies. An environment with no default release plan is held, with a pointer to the onboarding skill or the environment's release plan settings, and there is no fallback. A flag release plan that names a plan other than the default is recorded through the REST API (`environmentReleasePlanKey`, `LD-API-Version: beta`): the skill shows the curl command, and the user runs it in their own terminal with their LaunchDarkly API access token. The agent never runs it or reads a token. The skills no longer call `match-release-policies` or `get-release-policies`. Evals: both suites put SKILL.md in the agent's instructions (`inline_skill`, since the skill text does not load with builtins off), get the release plan tool definition and a guarded default plan mock, hide the release policy tools, assert the plan read, and add "default plan exists" and "no default plan" cases. A `flag-release` case with built-in tools and a planted `.env` checks that, for a non-default plan, the agent shows the REST command and does not run curl or read a token. - Add `release-autopilot/launchdarkly-release-plans` 1.0.0 (Claude Code plugin 1.0.8, Cursor plugin 1.0.9): one skill for both kinds of release plan. Environment release plans: read, choose (the environment's default first, other complete plans as alternatives), create from starter plans (Progressive, guarded with rollback metrics whose `analysisUnits` match the unit, at most 50% per step, or Custom) after checking the randomization unit with `search-contexts`, set the default, and edit through the MCP tools; delete, and clear the default, through the REST API. Flag release plans: list, get, create, update in place, make default, and delete, and start a flag's release from one, through the REST API (`/api/v2/projects/{projectKey}/flag-release-plans`, `/flag-release-plans/{planKey}`, `/flags/{flagKey}/flag-releases`). An environment with no complete release plan gets one first (or is left out when the user declines), and the skill never invents or points at an incomplete plan. A flag release plan stores each environment's release plan key, so a later change to that environment's default does not update it. For each REST operation the skill shows the exact curl command (with `LD-API-Version: beta`), the user runs it in their own terminal with their own API access token and pastes the output back, and the skill reads that output to continue; the agent never runs a REST command or uses a token, and never asks for a token in the chat. The commands use curl because the MCP server has no flag release plan tools and `ldcli` 3.16.0 has no command for these endpoints. Before any create, the skill checks for an existing default (environment or project), names it with a short summary, and asks whether to use it, replace it, or create another; it also names incomplete plans. Modes: Apply (confirm every write), Read-only, and Record only for skills that confirm and write later. Starter plan definitions and the custom plan questions move here from onboarding's `references/release-plans.md`. - `release-autopilot/launchdarkly-release-autopilot-onboarding` 1.1.0: release setup follows `launchdarkly-release-plans` in Record only mode for environment release plans and the project's flag release plan; onboarding keeps the wizard steps, the confirmation, the write order, and verification, and its own `references/release-plans.md` is gone. The flag release plan is a new "Default release" plan when the project has no default, no change when the default matches, or the default updated in place ("Update " recommended, or "Keep "; never a second plan). Step 7 shows what an update changes and notes that the plan keeps its release plan keys when an environment's default changes later. Step 8 writes it after the environment release plans and their defaults and before the repo writes, and Step 9 reads it back. The flag release plan is a step the user runs after "Looks good", like the browser steps: the agent shows the command, waits for the pasted output, and verifies from it. If the user skips it or it fails, it is a follow-up, auto-releasing stays on as picked, and the summary says Release Autopilot proposes no release on pull requests until the plan exists. A starter offer can be declined ("Not now"). Evals: a scripted `{ command, text }` answer stands for output the user pastes after running a shown command, and the provider's `scrub_ld_credentials` option removes LaunchDarkly credentials from the agent's shell, so cases check that the agent shows each command with the beta header, never runs one or reads a token, and reads the pasted output; the starter, guarded, no-contexts, and custom plan cases move to the new skill's suite, and onboarding adds flag release plan cases (with a shell, skipped as a follow-up with auto-releasing on, Preview, and an in-place update). diff --git a/README.md b/README.md index 259fb4ea..b642eece 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,8 @@ Agent Skills are modular, text-based playbooks that teach an agent how to perfor | Skill | Description | |-------|-------------| | `agentcontrol/configs-create` | Create configs with variations for agent or completion mode | -| `agentcontrol/migrate` | Migrate an app with hardcoded LLM prompts to AgentControl in five stages (extract, wrap, tools, tracking, evals) | +| `agentcontrol/migrate` | Migrate an app with hardcoded LLM prompts (not on AgentControl yet) to the current AI SDKs in five stages | +| `agentcontrol/legacy-sdk-migrate` | Migrate between LaunchDarkly AI SDKs: from the legacy Python and Node.js AI SDKs to the current handler-based SDKs (packages, init, `config()`, tools, registries, judges, graphs, fallbacks) | | `agentcontrol/configs-update` | Update and delete configs, manage lifecycle | | `agentcontrol/configs-variations` | Manage config variations for A/B testing | | `agentcontrol/tools` | Create and attach tools for function calling | diff --git a/skills.json b/skills.json index cf0745b2..d25fabf5 100644 --- a/skills.json +++ b/skills.json @@ -519,6 +519,14 @@ "rest-api" ] }, + { + "name": "legacy-sdk-migrate", + "description": "Migrate an app from the legacy LaunchDarkly AI SDKs (Python `launchdarkly-server-sdk-ai` / `ldai`, Node.js `@launchdarkly/server-sdk-ai`) to the current handler-based AI SDKs (`launchdarkly-ai-server`, `@launchdarkly/ai-node` / `@launchdarkly/ai-server` plus handler packages). Use when code calls `LDAIClient`, `initAi`, `completion_config` / `completionConfig`, `agent_config` / `agentConfig`, `agent_configs`, `create_tracker` / `trackMetricsOf`, `create_judge`, `create_model`, or `agent_graph`, or when the user asks to upgrade between LaunchDarkly AI SDKs. For apps with hardcoded LLM prompts that aren't on LaunchDarkly yet, use `migrate` instead.", + "path": "skills/agentcontrol/legacy-sdk-migrate", + "version": "0.1.0", + "license": "Apache-2.0", + "compatibility": "Python 3.12+ (`launchdarkly-ai-server` 1.0.0+ with `launchdarkly-server-sdk`) or Node.js (`@launchdarkly/ai-node` 1.0.0+). No LaunchDarkly MCP tools needed; the AgentControl configs already exist." + }, { "name": "mcp-configure", "description": "Configure the LaunchDarkly hosted MCP server during onboarding. Use when the parent LaunchDarkly onboarding skill reaches the MCP offer, after the first flag works. Supports Cursor, Claude Code, Windsurf, GitHub Copilot, and other MCP-compatible agents. OAuth authentication; no API keys for the hosted server.", diff --git a/skills/agentcontrol/legacy-sdk-migrate/SKILL.md b/skills/agentcontrol/legacy-sdk-migrate/SKILL.md new file mode 100644 index 00000000..d8810c36 --- /dev/null +++ b/skills/agentcontrol/legacy-sdk-migrate/SKILL.md @@ -0,0 +1,143 @@ +--- +name: legacy-sdk-migrate +description: "Migrate an app from the legacy LaunchDarkly AI SDKs (Python `launchdarkly-server-sdk-ai` / `ldai`, Node.js `@launchdarkly/server-sdk-ai`) to the current handler-based AI SDKs (`launchdarkly-ai-server`, `@launchdarkly/ai-node` / `@launchdarkly/ai-server` plus handler packages). Use when code calls `LDAIClient`, `initAi`, `completion_config` / `completionConfig`, `agent_config` / `agentConfig`, `agent_configs`, `create_tracker` / `trackMetricsOf`, `create_judge`, `create_model`, or `agent_graph`, or when the user asks to upgrade between LaunchDarkly AI SDKs. For apps with hardcoded LLM prompts that aren't on LaunchDarkly yet, use `migrate` instead." +license: Apache-2.0 +compatibility: Python 3.12+ (`launchdarkly-ai-server` 1.0.0+ with `launchdarkly-server-sdk`) or Node.js (`@launchdarkly/ai-node` 1.0.0+). No LaunchDarkly MCP tools needed; the AgentControl configs already exist. +metadata: + author: launchdarkly + version: "0.1.0" +--- + +# Migrate from the legacy AI SDKs + +You're using a skill that will guide you through moving an application from the legacy LaunchDarkly AI SDKs to the current handler-based AI SDKs. Your job is to inventory every legacy call site, map each one to its current equivalent, rewrite the code, and verify that the app still behaves the same, including when LaunchDarkly is unreachable. + +The current SDKs are **not drop-in replacements**. In the legacy SDKs the app evaluates a config, builds a provider client, merges messages, calls the provider, and wraps the call in a tracker. In the current SDKs the app registers handlers and tools once, then calls `config()`, `graph()`, or a handler package's convenience function. The SDK evaluates the config, routes to the handler that matches the variation's provider and mode, calls the provider, and records metrics and OpenTelemetry spans. + +Source guide: [Migrate from the legacy AI SDKs](https://launchdarkly.com/docs/sdk/ai/migration). Full before/after code for every pattern: [references/api-mapping.md](references/api-mapping.md). + +## Prerequisites + +- The app already uses a legacy AI SDK, and its AgentControl configs (formerly AI Configs) already exist in LaunchDarkly. The config keys, variations, tools, and judges carry over unchanged; this is a code migration only. (If the app calls an LLM provider directly with hardcoded prompts and does not use LaunchDarkly yet, use the `migrate` skill instead.) +- `LD_SDK_KEY` (server-side SDK key) and the provider API keys (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, ...) available as environment variables. The current SDKs read them from the environment. +- **Python:** the current packages require **Python 3.12 or later** (`launchdarkly-ai-server` 1.0.0+), and the API is **async** (`await config(...).invoke(...)`). `launchdarkly-server-sdk` is required at runtime for standard initialization and `ldclient.Context` (installed via `"launchdarkly-ai-server[ld]"` / `"launchdarkly-ai-python[ld]"` or by keeping `launchdarkly-server-sdk`). Check Python 3.12+ before you start. +- **Node.js:** `@launchdarkly/ai-node` 1.0.0+ for Node (bundles `@launchdarkly/node-server-sdk`); `@launchdarkly/ai-server` plus a pre-initialized client for edge or custom runtimes. + +## Core Principles + +1. **Inventory before you edit.** Find every legacy import and call site first, and confirm the list with the user. +2. **Use the real export names.** Handler factories are `create___handler()` in Python and `createHandler()` in Node (`createOpenAIMessagesHandler`, `createOpenAIAgentsHandler`, `create_openai_agents_handler`). Check the installed package's exports before writing an import; don't guess. +3. **Keep the fallback.** The current SDKs take no fallback argument. `invoke()` raises when the variation is disabled (targeting off) or LaunchDarkly is unreachable, so every call site that relied on `enabled=False` defaults needs an explicit `try`/`except` (or `try`/`catch`) that preserves today's degraded behavior. +4. **Delete, don't wrap.** Remove trackers, `track_*` calls, provider-client construction, message merging, and `model=config.model.name` plumbing. Leaving them in double-counts metrics. +5. **Register once.** Put handlers and tool implementations in one `Registry` (or the global registry) instead of rebuilding them at every call site. +6. **Ask before rewriting business logic.** If the app post-processes provider-specific response objects (for example `completion.choices[0].message.content`), switch to `result.response`, and ask before changing anything downstream that depends on the raw provider object. + +## Workflow + +### Step 1: Explore (read-only) + +Find every legacy usage. Search for these patterns and record file and line for each hit: + +| Language | Imports and packages | Calls | +|----------|----------------------|-------| +| Python | `launchdarkly-server-sdk-ai`, `launchdarkly-server-sdk-ai-openai`, `launchdarkly-server-sdk-ai-langchain`; `from ldai`, `from ldai_openai`, `from ldai_langchain` | `LDAIClient(`, `completion_config(`, `agent_config(`, `agent_configs(`, `judge_config(`, `create_judge(`, `create_model(`, `create_agent(`, `agent_graph(`, `create_agent_graph(`, `create_tracker(`, `track_metrics_of`, `track_duration`, `track_tokens`, `track_success`, `track_error`, `track_feedback`, `track_tool_call`, `track_judge_result`, `FeedbackKind`, `AICompletionConfigDefault`, `AIAgentConfigDefault`, `get_ai_metrics_from_response` | +| Node.js | `@launchdarkly/server-sdk-ai`, `@launchdarkly/server-sdk-ai-openai`, `@launchdarkly/server-sdk-ai-langchain`, `@launchdarkly/server-sdk-ai-vercel` | `initAi(`, `completionConfig(`, `agentConfig(`, `agentConfigs(`, `judgeConfig(`, `createJudge(`, `createModel(`, `createAgent(`, `agentGraph(`, `createTracker(`, `.tracker`, `trackMetricsOf`, `trackStreamMetricsOf`, `trackDuration`, `trackTokens`, `trackSuccess`, `trackError`, `trackFeedback`, `trackToolCall`, `trackToolCalls`, `trackJudgeResult`, `LDFeedbackKind`, `getAIMetricsFromResponse`, `getAIMetricsFromStream`, `createLangChainModel` | + +For each call site, also record: + +- **Provider and SDK** used for the actual model call (OpenAI, Anthropic, LangChain/LangGraph, Vercel AI SDK, LiteLLM, Bedrock, Gemini, other). +- **Mode**: completion (`completion_config`) or agent (`agent_config`), and whether it streams. +- **Tools**: any function-calling loop, and the tool names it dispatches. +- **Fallback behavior**: what the code does when `enabled` is false. +- **Extras**: template variables, conversation history, judges, agent graphs, custom tracker calls such as `track_feedback`. + +Present the inventory and **stop** until the user confirms it. + +### Step 2: Assess + +Pick a target for each call site: + +| Legacy pattern | Current replacement | +|----------------|---------------------| +| One provider, one mode, manual completion call | That handler package's convenience function, for example `openai_messages()` / `openaiMessages()` | +| Config can serve several providers or modes | `config(key=..., handler=[...])` then `.invoke(user_input, context, variables)` | +| Agent mode with a tool loop | `config()` with an agent handler and `tool_handlers` / `toolHandlers`; mode comes from the variation | +| Streaming (`trackStreamMetricsOf`, manual TTFT) | `config(...).stream(...)`; iterate events (`chunk` events carry `text`; the final `done` event carries `response` and `usage`) | +| Multi-turn chat (`createChat`, message arrays you maintain) | Pass prior turns as the `history` argument (fourth positional argument) to `invoke()` / `stream()` | +| `agent_configs()` / `agentConfigs()`, `agent_graph()`, `create_agent_graph()`, managed graph runners | `graph(key, handlers=[...]).invoke(...)`, or `resolve_graph()` / `resolveGraph()` plus a native runner (`to_lang_graph` / `toLangGraph`, `to_openai_agents` / `toOpenAIAgents`, `to_claude_agents` / `toClaudeAgents`) | +| `create_judge()` / `createJudge()` / `judge_config()` | Judges attached to the config variation run automatically. For background scoring, pass `skip_judges=True` / `skipJudges: true`, hand each `judge_tasks` / `judgeTasks` item to a durable worker, run `run_judge()` / `runJudge()`, and explicitly track the returned score(s) | +| `create_model().run()` / `createModel().run()`, `create_agent()` | Same convenience function or `config().invoke()` path | +| Same handlers and tools at many call sites | One shared `Registry` passed as `registry=` | + +Handler packages (each `launchdarkly-ai-` in Python, `@launchdarkly/ai-` in Node): + +| Package suffix | Provider | Mode | +|----------------|----------|------| +| `openai-messages` | OpenAI | completion (`messages`) | +| `openai-agents` | OpenAI | agent | +| `claude-messages` | Anthropic | completion (`messages`) | +| `claude-agents` | Anthropic (Claude Agent SDK) | agent | +| `langchain-messages` / `langchain-agents` | Any LangChain chat model / LangGraph | completion / agent | +| `litellm-messages` / `litellm-agents` | Any LiteLLM provider (use for Bedrock, Gemini, and others without a dedicated package) | completion / agent | +| `vercel-messages` / `vercel-agents` | Any Vercel AI SDK provider | completion / agent | + +A variation whose provider has no matching registered handler raises `Handler for provider not found`. Register a handler for every provider and mode your variations can serve, or write one with `createHandler` / `create_handler` for an internal endpoint. + +Confirm the plan with the user: target per call site, packages to add and remove, and how each fallback will be preserved. + +### Step 3: Execute + +Work one call site at a time. Full code for each step is in [references/api-mapping.md](references/api-mapping.md). + +1. **Swap packages.** Remove the legacy AI packages and install the current core package plus one handler package per provider and mode. + - Python: `pip install "launchdarkly-ai-server[ld]" launchdarkly-ai-openai-messages` (or `"launchdarkly-ai-python[ld]"`). Remove `launchdarkly-server-sdk-ai` and `launchdarkly-server-sdk-ai-*`. Keep `launchdarkly-server-sdk` installed (either directly or via the `[ld]` extra) unless the app runs on a custom runtime and passes its own client to `init_client(client=...)`. + - Node: `npm install @launchdarkly/ai-node @launchdarkly/ai-openai-messages`. Remove `@launchdarkly/server-sdk-ai` and `@launchdarkly/server-sdk-ai-*`. `@launchdarkly/ai-node` bundles `@launchdarkly/node-server-sdk`, so keep a direct `@launchdarkly/node-server-sdk` dependency only if the app still imports it for ordinary feature flags. Edge runtimes install `@launchdarkly/ai-server` and pass their client to `initClient(client)`. +2. **Replace initialization and check content capture.** Delete `LDAIClient(...)` / `initAi(...)`. The SDK initializes lazily from `LD_SDK_KEY` on the first AI call. Call `init_client()` / `initClient()` only to pass options or pre-warm, and call `shutdown()` on exit to flush events and spans. In 1.0+, OpenTelemetry spans capture prompts, model output, tool payloads, and judge reasoning **by default** (`capture_content=True`). Ask if the app handles sensitive data and needs to opt out via `configure(options=SDKOptions(capture_content=False))` (Python) or `configure({ options: { captureContent: false } })` (Node.js). +3. **Rewrite each call and check parameter forwarding.** Replace evaluate → build messages → call provider → track with one convenience function or `config(...).invoke(user_input, context, variables)`. Read the output from `result.response`. Python contexts passed to `invoke()` / `stream()` / `graph()` are plain dicts (`{"kind": "user", "key": "user-123"}`). Handler packages forward a fixed per-handler allowlist of `model.parameters` (mapping `max_tokens` to `max_output_tokens` on OpenAI) and drop unknown keys; move app-scoped knobs to `model.custom`. If writing a custom handler (`create_handler` / `createHandler`), forward `model.parameters` through an explicit allowlist (never a raw `**params` spread or denylist alone, and never forward `tools`). +4. **Move template variables** from the `completion_config` / `agent_config` argument to the `variables` keyword (convenience functions, Python), the `{ variables }` option (convenience functions, Node), or the third argument to `invoke()`. Note that the current SDKs inject the evaluation context as `{{ldContext.*}}` (replacing legacy `{{ldctx.*}}`) and require Mustache placeholders without spaces (`{{name}}`, not `{{ name }}`). +5. **Move tools.** Register implementations by name in `tool_handlers` / `toolHandlers` or a `Registry`. Keys must match the tool names in the config variation exactly, including case. Each tool function is called with a single `args` dict/object (not unpacked keyword arguments), so wrap multi-argument functions (`lambda args: fn(**args)`) or LangChain `@tool` objects (`lambda args: tool.invoke(args)`). Delete the hand-written tool loop. +6. **Preserve fallbacks.** Wrap `invoke()` and return the app's previous fallback response when the config is disabled or LaunchDarkly is unreachable. In the `except` / `catch` block, check `(await inspect_config(key, context))["enabled"]` / `(await inspectConfig(key, context)).enabled` and re-raise when `True` so mid-run provider or tool-loop errors aren't mistaken for a disabled config. Use `inspect_config()` / `inspectConfig()` (never raises; returns `enabled`, `config`, `meta`) for health checks or to branch before invoking. +7. **Replace tracking code and convert Python `track()` contexts.** Remove legacy `create_tracker`, generation/token/duration tracking, and `get_ai_metrics_from_response` / `getAIMetricsFromResponse` helpers; `invoke()` and `stream()` record those automatically. Preserve explicit app events: replace user feedback calls with `get_client().track()` / `getClient().track()` using `result.track_data` / `result.trackData` (see [references/api-mapping.md](references/api-mapping.md#user-feedback)). **In Python, `get_client().track()` requires an `ldclient.Context` (`Context.from_dict(context)`), not the plain dict passed to `invoke()`**—passing a raw dict raises `AttributeError: 'dict' object has no attribute 'valid'`. **In Node.js, pass a plain object literal as the context;** the `LDContext` type from `@launchdarkly/node-server-sdk` doesn't type-check as the context argument to `config().invoke()`. **Background judges are the exception:** `run_judge()` / `runJudge()` returns score data but deliberately does not call the LaunchDarkly client's `track()` method. Track every returned metric from the worker as shown in the reference. +8. **Centralize.** If two or more call sites share handlers or tools, create one `Registry` (or register on `global_registry` / `globalRegistry` at startup) and pass `registry=`. +9. **Remove dead imports and packages** once nothing references them. + +### Step 4: Verify + +1. **No legacy references remain:** re-run the Step 1 search. Expect zero hits for `ldai`, `server-sdk-ai`, `create_tracker`, `createTracker`, `track_metrics_of`, and `trackMetricsOf`. +2. **Build and tests pass:** run the project's type check, linter, and test suite. Update mocks that patched `LDAIClient` / `initAi` to patch `config` or the convenience function instead. +3. **Happy path:** with `LD_SDK_KEY` set, run each migrated call and confirm it returns a response from the provider the variation names. +4. **Fallback path:** with targeting off for the config (or an invalid `LD_SDK_KEY`), confirm the app returns its fallback instead of crashing. +5. **Metrics:** after a few calls, the config's Monitoring tab in LaunchDarkly shows generations, tokens, and duration for the new SDK version. If judges run in a worker, also confirm their evaluation metrics appear. +6. **Report:** list each migrated call site (file and line), packages added and removed, and anything left unmigrated with the reason. + +## Edge Cases + +| Situation | Action | +|-----------|--------| +| Python below 3.12 | Stop and tell the user. The current Python packages need 3.12+. | +| Synchronous Python code path (Flask, Django views, scripts) | Run the async call with `asyncio.run(...)` at the boundary, or move the handler to an async framework. Ask which. | +| App reads provider-native fields (`choices`, `tool_calls`, `stop_reason`) | Use `result.response` and `result.usage`; ask before changing downstream logic that needs the raw object. | +| Provider has no handler package (Bedrock, Gemini, Azure, internal gateway) | Use `litellm-*`, `langchain-*`, or `vercel-*`, or build a custom handler with `createHandler` / `create_handler` (match the config's provider name case-sensitively, such as `"OpenAI"`, `"Anthropic"`, `"Bedrock"`, `"Gemini"`, call `parse_template` / `parseTemplate` on raw prompt strings, and forward `model.parameters` through an explicit allowlist rather than spreading `**params`). | +| App stores custom app/tool knobs in `model.parameters` | Handler packages forward only an allowlist of model parameters and drop unknown keys. Move app-scoped knobs to `model.custom` and read them via `inspect_config()` / `inspectConfig()` or inside a custom handler (`ld_config["model"].get("custom")`). | +| App walks an agent graph by hand (`agent_graph().traverse()` / `get_edges()`) | Prefer `graph().invoke()` or a native runner (`to_lang_graph`, `to_openai_agents`, `to_claude_agents`). If keeping a custom walk with `resolve_graph()` / `resolveGraph()`, note that it returns topology only (`enabled`, `root`, `node.config`, `node.edges`, `edge.target_key` / `edge.targetKey`) and emits **no** telemetry unless you execute hops via `graph.run_node()` / `graph.runNode()` or `graph.route()` (passing handlers to `resolve_graph()` / `resolveGraph()`) or emit the `$ld:ai:graph:*` events yourself. Graph event keys are mostly unchanged; only `$ld:ai:graph:path` became `$ld:ai:graph:node`, and `$ld:ai:graph:redirect` is gone. Tell the user to update only metrics built on those two keys. | +| App handles sensitive/regulated prompts or outputs (PII, PHI) | Content capture on OpenTelemetry spans is **on by default** in 1.0+ (`capture_content=True`). Disable it process-wide with `configure(options=SDKOptions(capture_content=False))` (Python) or `configure({ options: { captureContent: false } })` (Node.js), or pass `capture_content=False` / `{ captureContent: false }` to handler factories. | +| App uses `claude-agents` and relies on Claude Code built-ins or `~/.claude/settings.json` | In 1.0+, `create_claude_agents_handler()` / `createClaudeAgentsHandler()` disables built-in tools unless a `NativeTool` sentinel is attached to the variation, and loads no host settings (`setting_sources=[]` / `settingSources: []`). Pass `setting_sources=["user"]` / `settingSources: ['user']` or set env vars if the app routes to Bedrock/Vertex/a gateway through `~/.claude/settings.json`. | +| App calls `track_feedback` / `trackFeedback` for thumbs up or down | Replace `tracker.track_feedback` with `get_client().track("$ld:ai:feedback:user:positive" or ":negative", Context.from_dict(context), result.track_data, 1)` in Python or `getClient().track(event, context, result.trackData, 1)` in Node.js (note: `stream()` does not currently return `track_data`). | +| Large app, gradual rollout | Migrate one call site at a time, verify it, and remove the legacy packages only after the last call site moves. | +| Judges must not add latency | `skip_judges=True` / `skipJudges: true`, then put `judge_tasks` / `judgeTasks` on a durable worker or queue. Run `run_judge()` / `runJudge()` there and track the returned score(s) with their supplied track data. | +| Agent Skills from AgentControl are needed | Agent Skills reach apps only through the current SDKs; the legacy SDKs never deliver them. | + +## What NOT to Do + +- Don't keep the legacy tracker around the new call. The SDK already records metrics; you'll double-count. +- Do not fire and forget `run_judge()` / `runJudge()`. A request-scoped task can be cancelled before it finishes, and the runner returns metrics for the worker to track rather than tracking them itself. +- Don't pass a default or fallback config object to `config()`; it has no such argument. Handle errors instead. +- Don't construct an OpenAI or Anthropic client to pass in. Handlers build their own clients from environment variables. +- Don't rename config keys, tools, or variations in LaunchDarkly as part of this migration. +- Don't invent export names. If an import fails, list the package's exports and use the real name. + +## References + +- [references/api-mapping.md](references/api-mapping.md): before/after code for initialization, completion calls, agent mode and tools, registries, streaming, history, judges, and graphs, in Python and Node.js +- [Migrate from the legacy AI SDKs](https://launchdarkly.com/docs/sdk/ai/migration) +- [Python AI SDK reference](https://launchdarkly.com/docs/sdk/ai/python) and [Node.js (server-side) AI SDK reference](https://launchdarkly.com/docs/sdk/ai/node-js) diff --git a/skills/agentcontrol/legacy-sdk-migrate/references/api-mapping.md b/skills/agentcontrol/legacy-sdk-migrate/references/api-mapping.md new file mode 100644 index 00000000..85d76018 --- /dev/null +++ b/skills/agentcontrol/legacy-sdk-migrate/references/api-mapping.md @@ -0,0 +1,572 @@ +# Legacy to current AI SDK: before and after + +Each section shows the legacy pattern and its replacement in Python and Node.js. All current examples assume `LD_SDK_KEY` and the provider API keys are set in the environment. + +Handler factory names follow `create___handler` (Python) and `createHandler` (Node.js), for example `create_openai_messages_handler`, `create_openai_agents_handler`, `createOpenAIMessagesHandler`, `createOpenAIAgentsHandler`, `createClaudeMessagesHandler`, `createClaudeAgentsHandler`. Some published examples use other names, such as `createOpenAIHandler` or `create_openai_agent_handler`; these are not exported by the current packages. + +## Contents + +1. [API mapping](#api-mapping) +2. [Packages](#packages) +3. [Initialization](#initialization) +4. [Completion call: convenience function](#completion-call-convenience-function) +5. [Completion call: multi-provider handlers](#completion-call-multi-provider-handlers) +6. [Fallbacks and inspection](#fallbacks-and-inspection) +7. [Agent mode and tools](#agent-mode-and-tools) +8. [Registries](#registries) +9. [Streaming and history](#streaming-and-history) +10. [User feedback](#user-feedback) +11. [Judges](#judges) +12. [Agent graphs](#agent-graphs) + +## API mapping + +| Concern | Legacy | Current | +|---------|--------|---------| +| Init | `LDAIClient(ld_client)` / `initAi(ldClient)` | Lazy init from `LD_SDK_KEY`, or `init_client()` / `initClient()` | +| Completion | `completion_config()` / `completionConfig()` | `config().invoke()` or a convenience function | +| Agent | `agent_config()` / `agentConfig()` | `config().invoke()`; mode comes from the variation | +| Multi-agent | `agent_configs()` / `agentConfigs()` | `graph()` or a native framework runner | +| Managed objects | `create_model()`, `create_agent()` / `createModel().run()`, `createAgent()` | Convenience function or `config().invoke()` | +| Metrics | `create_tracker()`, `track_metrics_of`, `track_*` / `createTracker()`, `trackMetricsOf`, `track*` | Generation metrics are automatic on `invoke()` / `stream()`; a background `run_judge()` / `runJudge()` returns judge metrics that the worker must track | +| Feedback | `tracker.track_feedback()` / `tracker.trackFeedback()` | `get_client().track()` / `getClient().track()` with `result.track_data` / `result.trackData` (in Python, convert the context dict with `ldclient.Context.from_dict(context)`) | +| Model parameters | Passed manually to provider client (`model=config.model.name`, `**config.model.parameters`) | Handler packages forward a per-handler allowlist of `model.parameters` (mapping `max_tokens` → `max_output_tokens` on OpenAI) and drop unknown keys; move app-scoped knobs to `model.custom`, and use an explicit allowlist in custom handlers | +| Content capture | N/A (app owned any tracing) | Prompts, outputs, tool payloads, and judge reasoning are captured on OpenTelemetry spans by default in 1.0+ (`capture_content=True`); opt out with `configure(options=SDKOptions(capture_content=False))` / `configure({ options: { captureContent: false } })` | +| Fallback | Default config object with `enabled=False` | `try`/`except` around `invoke()` (checking `inspect_config()` / `inspectConfig()` before falling back); `inspect_config()` / `inspectConfig()` to read state | +| Template variables | Fourth argument to `completion_config` / `completionConfig` | `variables=` (convenience, Python), `{ variables }` (convenience, Node), or third argument to `invoke()` (`{{name}}` without spaces; `{{ldContext.*}}` replaces `{{ldctx.*}}`) | +| Judges | `create_judge()`, `judge_config()` / `createJudge()`, `judgeConfig()` | Judges attached to the variation; `skip_judges` / `skipJudges` plus a durable worker that runs and tracks `run_judge()` / `runJudge()` results | +| Graphs | `agent_graph()`, `create_agent_graph()`, traversal helpers | `graph()`, `resolve_graph()` / `resolveGraph()` | +| Shutdown | `ld_client.close()` / `ldClient.close()` | `shutdown()` | + +## Packages + +| Language | Remove | Add | +|----------|--------|-----| +| Python | `launchdarkly-server-sdk-ai`, `launchdarkly-server-sdk-ai-openai`, `launchdarkly-server-sdk-ai-langchain` (imports `ldai`, `ldai_openai`, `ldai_langchain`) | `"launchdarkly-ai-server[ld]"` (or `launchdarkly-server-sdk` + `launchdarkly-ai-server`, or `"launchdarkly-ai-python[ld]"`); plus one `launchdarkly-ai--` per handler | +| Node.js | `@launchdarkly/server-sdk-ai`, `@launchdarkly/server-sdk-ai-openai`, `@launchdarkly/server-sdk-ai-langchain`, `@launchdarkly/server-sdk-ai-vercel` | `@launchdarkly/ai-node` (bundles `@launchdarkly/node-server-sdk`), or `@launchdarkly/ai-server` for edge runtimes; plus one `@launchdarkly/ai--` per handler | + +```bash +# Python (3.12+) +pip uninstall launchdarkly-server-sdk-ai launchdarkly-server-sdk-ai-openai +pip install "launchdarkly-ai-server[ld]" launchdarkly-ai-openai-messages + +# Node.js +npm uninstall @launchdarkly/server-sdk-ai @launchdarkly/server-sdk-ai-openai +npm install @launchdarkly/ai-node @launchdarkly/ai-openai-messages +``` + +Python modules are the package name with underscores (`launchdarkly_ai_server`, `launchdarkly_ai_openai_messages`). If you install `launchdarkly-ai-python`, import the core API from `launchdarkly_ai_python` instead. + +## Initialization + +Legacy: + +```python +from ldclient import set_config, get +from ldclient.config import Config +from ldai import LDAIClient + +set_config(Config("YOUR_SDK_KEY")) +aiclient = LDAIClient(get()) +``` + +```typescript +import { init } from '@launchdarkly/node-server-sdk'; +import { initAi } from '@launchdarkly/server-sdk-ai'; + +const ldClient = init('YOUR_SDK_KEY'); +await ldClient.waitForInitialization({ timeout: 10 }); +const aiClient = initAi(ldClient); +``` + +Current: delete both. The first AI call initializes the client from `LD_SDK_KEY`. Pre-warm or pass options only if needed, and always flush on exit: + +```python +from launchdarkly_ai_server import init_client, shutdown + +await init_client() # optional +... +await shutdown() +``` + +```typescript +import { initClient, shutdown } from '@launchdarkly/ai-node'; + +await initClient(); // optional +// Edge or custom runtime: await initClient(preInitializedClient); +... +await shutdown(); +``` + +If the app also evaluates ordinary feature flags, keep its existing server SDK client for those. To share one client, pass it to `init_client(client=ld_client)` / `initClient(ldClient)`. + +In 1.0+, OpenTelemetry spans capture prompts, model output, tool arguments/results, and judge reasoning by default (`capture_content=True`). To opt out process-wide, call `configure()` before the first AI call: + +```python +from launchdarkly_ai_server import SDKOptions, configure + +configure(options=SDKOptions(capture_content=False)) +``` + +```typescript +import { configure } from '@launchdarkly/ai-node'; + +configure({ options: { captureContent: false } }); +``` + +## Completion call: convenience function + +Use when the config serves one provider in one mode. + +Legacy: + +```python +from ldai import AICompletionConfigDefault +from ldai_openai import get_ai_metrics_from_response + +config = aiclient.completion_config( + "example-config-key", + context, + AICompletionConfigDefault(enabled=False), + {"example_custom_variable": "example_custom_value"}, +) +tracker = config.create_tracker() + +if config.enabled: + messages = [] if config.messages is None else config.messages + completion = tracker.track_metrics_of( + get_ai_metrics_from_response, + lambda: openai_client.chat.completions.create( + model=config.model.name, + messages=[m.to_dict() for m in messages] + [{"role": "user", "content": user_input}], + ), + ) + text = completion.choices[0].message.content +``` + +```typescript +import { getAIMetricsFromResponse } from '@launchdarkly/server-sdk-ai-openai'; + +const aiConfig = await aiClient.completionConfig( + 'example-config-key', + context, + { enabled: false }, + { exampleCustomVariable: 'exampleCustomValue' }, +); +const tracker = aiConfig.createTracker(); +const response = await tracker.trackMetricsOf(getAIMetricsFromResponse, () => + openai.chat.completions.create({ + model: aiConfig.model?.name || 'gpt-4', + messages: [...(aiConfig.messages || []), { role: 'user', content: userInput }], + }), +); +const text = response.choices[0].message.content; +``` + +Current: + +```python +from launchdarkly_ai_openai_messages import openai_messages + +result = await openai_messages( + "example-config-key", + user_input, + {"kind": "user", "key": "user-123"}, + variables={"example_custom_variable": "example_custom_value"}, +) +text = result.response +``` + +```typescript +import { openaiMessages } from '@launchdarkly/ai-openai-messages'; + +const result = await openaiMessages( + 'example-config-key', + userInput, + { kind: 'user', key: 'user-123' }, + { variables: { exampleCustomVariable: 'exampleCustomValue' } }, +); +const text = result.response; +``` + +Other convenience functions follow the same shape: `openai_agents` / `openaiAgents`, `claude_messages` / `claudeMessages`, `claude_agents` / `claudeAgents`, and so on. Check the handler package's exports. + +## Completion call: multi-provider handlers + +Use when variations of one config can target different providers or modes. The SDK routes to the handler that matches the variation's provider and mode. + +```python +from launchdarkly_ai_server import config +from launchdarkly_ai_openai_messages import create_openai_messages_handler +from launchdarkly_ai_claude_messages import create_claude_messages_handler + +result = await config( + key="example-config-key", + handler=[create_openai_messages_handler(), create_claude_messages_handler()], +).invoke( + user_input, + {"kind": "user", "key": "user-123"}, + {"example_custom_variable": "example_custom_value"}, +) +print(result.response, result.usage) +``` + +```typescript +import { config } from '@launchdarkly/ai-node'; +import { createOpenAIMessagesHandler } from '@launchdarkly/ai-openai-messages'; +import { createClaudeMessagesHandler } from '@launchdarkly/ai-claude-messages'; + +const result = await config({ + key: 'example-config-key', + handler: [createOpenAIMessagesHandler(), createClaudeMessagesHandler()], +}).invoke( + userInput, + { kind: 'user', key: 'user-123' }, + { exampleCustomVariable: 'exampleCustomValue' }, +); +console.log(result.response, result.usage); +``` + +## Fallbacks and inspection + +There is no fallback argument. `invoke()` raises when the variation is disabled (`RuntimeError: Variation '' is disabled.` in Python, `Error: Variation is not enabled` in Node.js) or when LaunchDarkly cannot be reached. Because handler packages also raise `RuntimeError` / `Error` when a run fails after starting (for example, hitting the tool-loop step limit), check `inspect_config()` / `inspectConfig()` in the error handler and re-raise when `enabled` is true so real run failures are not swallowed by the disabled-config fallback: + +```python +from launchdarkly_ai_server import inspect_config +from launchdarkly_ai_openai_messages import openai_messages + +FALLBACK = "Sorry, the assistant is unavailable right now." + +try: + result = await openai_messages("example-config-key", user_input, context) + text = result.response +except Exception: + if (await inspect_config("example-config-key", context))["enabled"]: + raise + logger.warning("AI config disabled or unreachable; using fallback") + text = FALLBACK +``` + +```typescript +import { inspectConfig } from '@launchdarkly/ai-node'; +import { openaiMessages } from '@launchdarkly/ai-openai-messages'; + +let text: string; +try { + text = (await openaiMessages('example-config-key', userInput, context)).response; +} catch (err) { + if ((await inspectConfig('example-config-key', context)).enabled) throw err; + logger.warn('AI config disabled or unreachable; using fallback', err); + text = FALLBACK; +} +``` + +To read state up front without raising and without calling a provider or emitting metrics, use `inspect_config()` / `inspectConfig()`. It returns `enabled`, `config` (the evaluated variation, or `None` / `null`), and `meta`: + +```python +state = await inspect_config("example-config-key", context) +if not state["enabled"]: + return FALLBACK +``` + +```typescript +const state = await inspectConfig('example-config-key', context); +if (!state.enabled) return FALLBACK; +``` + +Inspecting first and then invoking evaluates the flag twice; keep the `try`/`except` either way, because LaunchDarkly can become unreachable between the two calls. + +## Agent mode and tools + +Legacy agent mode used a separate evaluation call, and the app ran its own tool loop and tracked metrics: + +```python +from ldai import AIAgentConfigDefault + +agent = aiclient.agent_config( + "example-config-key", context, AIAgentConfigDefault(enabled=False), + {"example_custom_variable": "example_custom_value"}, +) +if agent.enabled: + tracker = agent.create_tracker() + result = run_my_tool_loop(agent.instructions, tools={"get-prefs": get_preferences_fn}) + tracker.track_success() +``` + +```typescript +const agent = await aiClient.agentConfig( + 'example-config-key', context, { enabled: false }, + { exampleCustomVariable: 'exampleCustomValue' }, +); +if (agent.enabled) { + // agent.instructions + hand-written tool loop + agent.tracker.track*() +} +``` + +Current: completion and agent mode share `config()`. Register tool implementations by name; the handler runs the tool loop. Keys must match the tool names in the variation exactly, including case. Each tool function receives a single `args` dict (Python) or object (Node.js), not unpacked keyword arguments—wrap multi-argument functions (`lambda args: get_preferences_fn(**args)`) or LangChain `@tool` objects (`lambda args: get_preferences_tool.invoke(args)`). + +```python +from launchdarkly_ai_server import config +from launchdarkly_ai_openai_agents import create_openai_agents_handler + +result = await config( + key="example-config-key", + handler=[create_openai_agents_handler()], + tool_handlers={"get-prefs": get_preferences_fn}, +).invoke( + "What are my preferences?", + {"kind": "user", "key": "user-123"}, + {"example_custom_variable": "example_custom_value"}, +) +``` + +```typescript +import { config } from '@launchdarkly/ai-node'; +import { createOpenAIAgentsHandler } from '@launchdarkly/ai-openai-agents'; + +const result = await config({ + key: 'example-config-key', + handler: [createOpenAIAgentsHandler()], + toolHandlers: { 'get-prefs': getPreferencesFn }, +}).invoke( + 'What are my preferences?', + { kind: 'user', key: 'user-123' }, + { exampleCustomVariable: 'exampleCustomValue' }, +); +``` + +Before deleting the loop, confirm that every tool it defined in code exists on each variation in LaunchDarkly. A tool missing from the variation is silently unavailable to the migrated agent; nothing fails. Do the same for a step cap or default model the old loop set: move it to the variation (or a custom handler) or tell the user. Then delete the hand-written loop, the provider tool-schema construction, and the manual `track_tool_call` / `trackToolCall` calls. + +## Registries + +Register handlers and tools once, then pass the registry to every call site. Use a scoped `Registry` when parts of the app need different handlers or tools, and `global_registry` / `globalRegistry` when one set fits the whole app. + +```python +from launchdarkly_ai_server import Registry, config +from launchdarkly_ai_openai_messages import create_openai_messages_handler +from launchdarkly_ai_openai_agents import create_openai_agents_handler + +openai_registry = Registry( + handlers=[create_openai_messages_handler(), create_openai_agents_handler()], + tools={"get-prefs": get_preferences_fn}, +) + +result = await config(key="example-config-key", registry=openai_registry).invoke(user_input, context) +``` + +```typescript +import { Registry, config } from '@launchdarkly/ai-node'; +import { createOpenAIMessagesHandler } from '@launchdarkly/ai-openai-messages'; +import { createOpenAIAgentsHandler } from '@launchdarkly/ai-openai-agents'; + +const openaiRegistry = new Registry({ + handlers: [createOpenAIMessagesHandler(), createOpenAIAgentsHandler()], + tools: { 'get-prefs': getPreferencesFn }, +}); + +const result = await config({ key: 'example-config-key', registry: openaiRegistry }).invoke(userInput, context); +``` + +Some providers expose built-in tools (for example Claude web search) that you assign as tool implementations instead of writing a function. + +## Streaming and history + +Replace `trackStreamMetricsOf` / manual time-to-first-token tracking with `stream()`. It takes the same arguments as `invoke()`. + +```python +async for event in config(key="example-config-key", registry=openai_registry).stream(user_input, context): + if event["type"] == "chunk": + print(event["text"], end="") + elif event["type"] == "done": + final_text, usage = event["response"], event["usage"] +``` + +```typescript +for await (const event of config({ key: 'example-config-key', registry: openaiRegistry }).stream(userInput, context)) { + if (event.type === 'chunk') process.stdout.write(event.text); + if (event.type === 'done') console.log(event.usage); +} +``` + +For multi-turn chat, stop appending to a message array you merge with `config.messages`. Pass earlier turns as the fourth argument, `history`: + +```python +history = [ + {"role": "user", "content": "Hi"}, + {"role": "assistant", "content": "Hello! How can I help?"}, +] +result = await config(key="example-config-key", registry=openai_registry).invoke( + user_input, context, None, history +) +``` + +## User feedback + +The current SDKs do not have a `track_feedback` / `trackFeedback` helper on a tracker, because there is no tracker. Instead, `invoke()` returns `result.track_data` (Python) / `result.trackData` (Node.js) identifying the config, variation, version, and model. Pass it to `get_client().track()` / `getClient().track()` when the user gives thumbs up or down: + +Legacy: + +```python +from ldai.tracker import FeedbackKind + +tracker.track_feedback({"kind": FeedbackKind.Positive}) +``` + +```typescript +import { LDFeedbackKind } from '@launchdarkly/server-sdk-ai'; + +tracker.trackFeedback({ kind: LDFeedbackKind.Positive }); +``` + +Current: + +```python +from ldclient import Context +from launchdarkly_ai_server import config, get_client + +result = await config(key="example-config-key", registry=openai_registry).invoke(user_input, context) + +# When the user responds with thumbs up / down: +event = "$ld:ai:feedback:user:positive" if thumbs_up else "$ld:ai:feedback:user:negative" +get_client().track(event, Context.from_dict(context), result.track_data, 1) +``` + +```typescript +import { config, getClient } from '@launchdarkly/ai-node'; + +const result = await config({ key: 'example-config-key', registry: openaiRegistry }).invoke(userInput, context); + +// When the user responds with thumbs up / down: +const event = thumbsUp ? '$ld:ai:feedback:user:positive' : '$ld:ai:feedback:user:negative'; +getClient().track(event, context, result.trackData, 1); +``` + +`stream()` does not currently return `track_data`, so feedback on streamed responses cannot yet be attributed to a variation. + +## Judges + +Legacy apps created a judge and called it after each response. Now judges are attached to the config variation in LaunchDarkly and run inline by default on every `invoke()`. If the legacy app judged only some calls on a config, it will now judge all of them; pass `skip_judges=True` / `skipJudges: true` on the calls that shouldn't be judged. Delete `create_judge` / `createJudge` call sites once the judges are attached to the variation (ask the user to attach them if they aren't). + +To keep latency low, skip inline judges and put the returned tasks on a durable +worker or queue. `run_judge()` / `runJudge()` executes the judge call but does +**not** emit the judge's evaluation metric. The worker must track the returned +score and supplied track data. Do not use a request-scoped `create_task()` or an +unobserved promise: the request can end before either the judge or its metric +finishes. + +Python example for a long-lived process (use a durable queue instead when the +process can be suspended or terminated after the request): + +```python +import asyncio +import threading +from ldclient import Context +from launchdarkly_ai_server import ( + config, + get_client, + global_registry, + resolve_handlers, + run_judge, +) + +result = await config( + key="example-config-key", registry=global_registry, skip_judges=True +).invoke(user_input, context) + +handlers = resolve_handlers(global_registry, None) or [] + +def run_and_track_judge(task): + judge_result = asyncio.run(run_judge(task, handlers)) + if judge_result and task.evaluation_metric_key: + get_client().track( + task.evaluation_metric_key, + Context.from_dict(task.user_context), + judge_result.track_data, + judge_result.score, + ) + +for task in result.judge_tasks or []: + threading.Thread( + target=run_and_track_judge, + args=(task,), + daemon=False, + ).start() +``` + +Node.js worker example. Enqueue each `judgeTask` from the request handler, then +run this function in the queue consumer: + +```typescript +import { + config, + getClient, + globalRegistry, + runJudge, +} from '@launchdarkly/ai-node'; +import { createOpenAIMessagesHandler } from '@launchdarkly/ai-openai-messages'; + +// The handlers your judge configs' providers need. +const handlers = [createOpenAIMessagesHandler()]; + +const { response, judgeTasks } = await config({ + key: 'example-config-key', + registry: globalRegistry, + skipJudges: true, +}).invoke(userInput, context); + +// Request path: enqueue the serializable task; do not run it as an unobserved promise. +for (const task of judgeTasks ?? []) { + await judgeQueue.enqueue(task); +} + +// Durable worker / queue consumer: +async function processJudgeTask(task) { + const judgeResult = await runJudge(task, handlers); + if (judgeResult && task.evaluationMetricKey) { + getClient().track( + task.evaluationMetricKey, + task.userContext, + judgeResult.trackData, + judgeResult.score, + ); + } +} +``` + +`handlers` is the list of provider handlers the judge configs can use. Remove +the legacy `track_judge_result` / `trackJudgeResult` calls, but preserve their +behavior with the current client's `track()` call shown above. + +## Agent graphs + +Replace `agent_configs` / `agentConfigs`, `agent_graph`, `create_agent_graph`, and hand-written traversal with `graph().invoke()`: + +```python +from launchdarkly_ai_server import graph +from launchdarkly_ai_claude_agents import create_claude_agents_handler +from launchdarkly_ai_openai_agents import create_openai_agents_handler + +result = await graph( + "support-graph", + handlers=[create_claude_agents_handler(), create_openai_agents_handler()], + tool_handlers={"lookup-invoice": lookup_invoice}, +).invoke("I was double charged", {"kind": "user", "key": "user-123"}, {"account_tier": "pro"}) +print(result.response) +``` + +```typescript +import { graph } from '@launchdarkly/ai-node'; +import { createClaudeAgentsHandler } from '@launchdarkly/ai-claude-agents'; + +const result = await graph('support-graph', { + handlers: [createClaudeAgentsHandler()], + toolHandlers: { 'lookup-invoice': lookupInvoice }, +}).invoke('I was double charged', { kind: 'user', key: 'user-123' }, { account_tier: 'pro' }); +console.log(result.response); +``` + +To keep running on a framework the app already uses, call `resolve_graph()` / `resolveGraph()` and pass the result to that framework's runner: `to_lang_graph` / `toLangGraph`, `to_openai_agents` / `toOpenAIAgents`, or `to_claude_agents` / `toClaudeAgents`. Check the handler package's exports and README for the exact call. + +If the app walks the resolved topology by hand instead of using `graph().invoke()` or a native runner: +- Node and edge accessors are attributes now: `graph.enabled`, `graph.root`, `node.key`, `node.config` (a plain dict), `node.edges`, and `edge.target_key` / `edge.targetKey` (replacing `is_enabled()`, `root()`, `get_key()`, `get_config()`, `get_edges()`, and `edge.target_config`). +- `resolve_graph()` / `resolveGraph()` returns topology only and emits **no** telemetry on its own. Pass `handlers=` to `resolve_graph()` and execute nodes with `await graph.run_node(node, input, opts)` / `await graph.runNode(node, input, opts)` or `graph.route(node, input, opts)`, or emit the `$ld:ai:graph:*` events yourself (`$ld:ai:graph:node`, `$ld:ai:graph:handoff_success`, `$ld:ai:graph:handoff_failure`, `$ld:ai:graph:invocation_success`, `$ld:ai:graph:invocation_failure`, `$ld:ai:graph:duration:total`, `$ld:ai:graph:total_tokens`, plus `$ld:ai:tool_call`). These match the legacy SDKs except that `$ld:ai:graph:path` became `$ld:ai:graph:node` and `$ld:ai:graph:redirect` is gone, so only LaunchDarkly metrics built on those two keys need updating. diff --git a/tests/test_legacy_sdk_migrate.py b/tests/test_legacy_sdk_migrate.py new file mode 100644 index 00000000..fdb7e240 --- /dev/null +++ b/tests/test_legacy_sdk_migrate.py @@ -0,0 +1,33 @@ +import unittest +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[1] +SKILL = ROOT / "skills/agentcontrol/legacy-sdk-migrate/SKILL.md" +API_MAPPING = ( + ROOT + / "skills/agentcontrol/legacy-sdk-migrate/references/api-mapping.md" +) + + +class LegacySdkMigrateTests(unittest.TestCase): + def test_background_judge_examples_track_returned_metrics(self): + mapping = API_MAPPING.read_text(encoding="utf-8") + + self.assertIn("get_client().track", mapping) + self.assertIn("getClient().track", mapping) + self.assertNotIn("asyncio.create_task(judge_worker(task))", mapping) + self.assertNotIn("void runJudge(task, handlers)", mapping) + + def test_skill_preserves_tracking_for_background_judges(self): + skill = SKILL.read_text(encoding="utf-8") + + self.assertIn("Background judges are the exception", skill) + self.assertIn("Do not fire and forget", skill) + self.assertIn("returned score", skill) + self.assertIn("launchdarkly-server-sdk", skill) + self.assertNotIn("must be a tuple", skill) + + +if __name__ == "__main__": + unittest.main()