Skip to content

fix(solve): keep the remaining plan in view after each step fold - #1969

Open
KKandLL-Forever wants to merge 2 commits into
HKUDS:devfrom
KKandLL-Forever:fix/solve-fold-keeps-plan
Open

KKandLL-Forever wants to merge 2 commits into
HKUDS:devfrom
KKandLL-Forever:fix/solve-fold-keeps-plan

Conversation

@KKandLL-Forever

Copy link
Copy Markdown

Description

solve_finish_step folds the finished step's tool chatter (intended), but the same cut also dropped the solve_plan result and the step's own next pointer. From step 2 onward the model only saw [S1] <goal> — done. and had to reconstruct the remaining steps from memory; the persisted model turn lost the plan too.

This PR:

  • lets a context checkpoint carry an optional state next to summary. Summaries still accumulate before the fold boundary; state is one system message after it, so each fold replaces it instead of stacking stale copies;
  • has solve_finish_step pass SolveSession.progress_text() (every step marked done / next / todo) as that state, so the current plan, including after a replan, stays in view and is persisted with the model turn.

Checkpoints without state behave exactly as before; _context_checkpoint is only emitted by solve.

Related Issues

Module(s) Affected

  • agents
  • tests
  • Other: capabilities (solve)

Validation

  • New tests: test_context_checkpoint_state_replaces_instead_of_stacking and test_solve_checkpoint_fold_keeps_remaining_plan (tests/agents/chat/test_agent_loop.py); test_session_progress_text_marks_each_step, test_session_progress_text_after_last_step_and_replan, test_finish_step_checkpoint_carries_plan_state (tests/capabilities/test_solve_capability.py). All five fail on dev and pass here; the existing test_context_checkpoint_folds_completed_tool_rounds passes unchanged.
  • Full suite as in CI (pytest -q tests deeptutor/learning/tests, server + partners requirements, LightRAG/GraphRAG on 3.12): 10,325 passed, 11 skipped, 2 failed. Both failures fail identically on untouched dev (e60fb158) on my macOS machine and are unrelated: tests/runtime/test_launcher.py::test_noninteractive_restart_recovers_an_actual_orphaned_backend_listener, tests/services/sandbox/test_sandbox.py::test_runner_server_executes_and_truncates_output.
  • ruff check ., ruff format --check . (0.16.0), lint-imports (3 kept, 0 broken) and scripts/check_architecture.py pass.
  • pre-commit run --all-files: all hooks pass on the changed files; end-of-file-fixer flags web/public/provider-icons/onomeo.svg (missing trailing newline, from 47e10db), which this PR leaves untouched.
  • No live-model calls; the reproduction uses the suite's scripted client.

Checklist

  • This PR targets dev (not main)
  • I have read and followed the contribution guidelines.
  • My code follows the project's coding standards.
  • I have run pre-commit run --all-files and fixed any issues. (One pre-existing finding in an unrelated file, noted above.)
  • I have added relevant tests for my changes.
  • I have updated the documentation where necessary (module docstrings in solve/session.py and solve/tools.py).
  • My changes do not introduce any known new security vulnerabilities.

A checkpoint may now carry an optional `state` next to `summary`. The fold
keeps appending summaries before the boundary as before and places `state`
in one system message after it, so the next fold replaces the state instead
of stacking stale copies. Checkpoints without `state` behave exactly as before.
solve_finish_step folds the finished step's tool chatter, and the same cut
dropped the solve_plan result and the step's own `next` pointer, leaving the
model only "[S1] ... — done.". Pass SolveSession.progress_text() as the
checkpoint state so every fold keeps the current plan (done / next / todo) in
view, including after a replan, and the persisted model turn carries it into
follow-up turns.

Closes HKUDS#1968
@KKandLL-Forever KKandLL-Forever changed the title Fix/solve fold keeps plan fix(solve): keep the remaining plan in view after each step fold Oct 15, 2026
@KKandLL-Forever

Copy link
Copy Markdown
Author

Real-model check, in addition to the scripted tests: I ran the same open-ended deep_solve prompt 5 times on dev (e60fb158) and 5 times on this branch.

Setup

  • Model: deepseek-flash; identical settings; a separate data home per arm.
  • Command: deeptutor run deep_solve "<prompt>" -l zh, with the prompt "Fully study f(x) = (x³ − 3x² + 4)/(x² − 1) and sketch it; verify every key value". The model chooses its own decomposition, so the remaining steps are not recoverable from the question.
  • exec was unavailable in this CLI setup for both arms.
dev (n=5) this PR (n=5)
Failed turns 1 (two rounds of reasoning-only output) 0
Turns that re-planned (solve_plan > 1) 5/5 (6, 4, 2, 7, 2 calls) 0/4 (one run answered without planning)
Steps finished again after a re-plan 17 (one run finished S1 six times) 0
Final plan finished (completed turns) 2/4 4/4
Median time 177 s 95 s
Median tokens 147,635 88,392
Median tool calls 13 7
Answer checklist (hole at (−1, −4.5), asymptotes x = 1 / y = x − 3, extrema, no inflection) 24/24 on completed turns 30/30

What this shows

  • On dev, once the first fold removes the plan, the model calls solve_plan again, which overwrites the session, and it redoes S1.
  • With this PR, it plans once and walks the steps in order.
  • Answers stay correct on both sides for this model, so the gain here is fewer wasted rounds (about 40% fewer tokens). Correctness did not change.

Limitations

  • n = 5 per arm.
  • The arms ran concurrently, so timings are noisy.
  • The PR column's medians include the one run that skipped planning. Without it, the medians are 107 s and 89,590 tokens; the conclusion is unchanged.
  • Whether the failed dev turn was caused by the lost plan is plausible but not proven.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant