Skip to content

feat: scale Copilot release research with paginated evidence - #5

Merged
SSmallMonster merged 2 commits into
mainfrom
improve/scalable-release-research
Oct 26, 2026
Merged

SSmallMonster merged 2 commits into
mainfrom
improve/scalable-release-research

Conversation

@SSmallMonster

@SSmallMonster SSmallMonster commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Large release ranges can exceed maxPromptBytes before Copilot starts, while per-PR findings can retain behavior that later PRs already corrected. Copilot now reads frozen evidence through paginated tools, checkpoints findings by problem, and reconciles them against the complete final diff before composing notes.

  • Keep full source metadata and diffs in artifacts instead of the initial prompt. Track contiguous read coverage, retaining deletion-only changes and refusing incomplete investigations.
  • Enable SDK context compaction. Saved findings and the run's policy remain available through tools after compaction; findings can be replaced or combined without losing source attribution.
  • Separate initial-instruction size from cumulative research budgets, record model/context metrics, and read frozen source files from Git objects instead of depending on GitHub Contents inline limits. Default research budgets are 300 calls and 4,000,000 returned bytes; explicit existing limits remain effective.
  • Preserve release identity, run-owned snapshots, latest-policy reruns and publication guards. Compatible endpoints retain their existing supplied-evidence mode.

Validation: 76 TypeScript tests, 7 Python tests, type checking, build and CLI checks pass. Coverage includes large diffs, Unicode/long lines, deletions, unread gaps, partial PR reads, checkpoint failures, frozen large-file reads, and updating the same release PR with a new policy. A live gpt-6.1-sol preview of the previously failing Operator range reduced the initial prompt from 277,602 to 21,454 bytes, completed all required evidence reads, and successfully resumed composition after actual compaction from 166,191 to 8,876 tokens. The generated notes combine Operator #399 and #413 and omit the superseded environment-removal breaking change.

An additional live synthetic single-file test read all 419,109 characters and completed after 18 automatic compactions. It used deliberately low test thresholds (2%/4%); production remains at 80%/95%. Runtime-reported context capacity is checked because it can be smaller than the model catalog maximum. GitHub CI also passes on the latest commit.

…nt findings

Signed-off-by: SSmallMonster <mingming.zhou@daocloud.io>
Signed-off-by: SSmallMonster <mingming.zhou@daocloud.io>
@SSmallMonster
SSmallMonster marked this pull request as ready for review October 26, 2026 10:27
@SSmallMonster
SSmallMonster merged commit 08fb609 into main Oct 26, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant