[rishabhsinha17]

Stacked on #611 (base branch feat/priority-preemption). Draft: it should merge after the stack, and the case has not run on GPUs.

The #585 discussion settled on a two-instance acceptance case as its own PR once priority support exists, two instances on one GPU with the lower priority shrinking once the higher posts its shortfall. #608, #609 and #611 add the watermark, the reclaim plan and preemption, with CPU unit tests and hand-run A100 and L4 numbers, but no CI case serves real models against them. Elastic checks release and reuse between equals, limit needs an operator to lower it, and nothing exercises the demand-driven path end to end.

Weekly gains priority-vllm-qwen05b+vllm-qwen35_4b, built on the elastic and limit harness. L, a vLLM Qwen2.5-0.5B at priority 0, starts with its cache retention cap lifted (KVCACHED_MAX_CACHED_TOKENS=-1) and plays the idle-cache giver of the #609 validation. H, a vLLM Qwen3.5-4B at priority 1, is the burst. A GPU run passes phase by phase.

Setup: both servers share the run GPU with a batch limit of 128, and kvctl priority publishes both priorities before any load (priority_set). Fill: L gets distinct 3000-word prompts with 1 output token, 8 at a time, sized to 70% of the free memory. Every fill request must succeed (l_requests_ok), L must grow by more than half of the free memory (l_filled_memory), and it must retain half of that growth until the burst begins (l_held_at_burst), which fails the case if the retention cap was not lifted. Burst: H gets distinct 3000-word prompts with 1024 output tokens and ignore_eos, all at once, sized to 70% of the free memory plus what L retains, more than fits without L. Every burst request must succeed (h_requests_ok). During it, the JSON kvctl list prints, sampled every 2 s, must show a shortfall posted by H (h_shortfall_posted) and a watermark on L (l_watermark_raised). L must end the burst with less than half of its fill growth still charged (l_released_for_h), and H must grow past what was free at the start of the burst (h_reused_l_memory), possible only with pages L physically released, the acceptance bar shipiyouniao set in #585. Probes and shutdown: greedy probes before and after must give the same tokens on both servers, both server processes must survive, and the shutdown and GPU-release checks of the other cases apply.

Following the #585 acceptance-process proposal, the case declares co-location and real serving as its layers, records its workload and checks in setup.json, and the CI results issue renders a Priority tests table like the elastic one. Fair fixed-load throughput comparisons stay with the perf runs. This case is the pass or fail acceptance of the semantics. Case logs keep both servers p50 and max request latencies, the memory numbers, the sampled ledger timeline and the burst-phase offsets of both server logs. Debugging a failed run starts from those offsets. Ledger reads go only through kvctl, the reader kvcached provides. I projected the sizing from the measured #577 weekly numbers and did not run this pair: on a 23 GiB L4 the two servers idle at about 12 GiB, L fills about 7 of the 10 GiB free, and the burst needs about 7 GiB against about 3 GiB still free, roughly 20 requests of 352 MiB. Larger GPUs take priority-gpu-mib, the ballast flag the elastic case already has.

CPU-checkable parts carry the local evidence. New tests in tests/test_e2e_harness.py cover the pair naming and weekly wiring, ledger_stats on kvctl JSON (fields summed per GPU, a missing field counted as 0, foreign or failed output ignored), one failing arm per priority_checks check, including the lifted-cap precondition, and set_priority taking only an accepted publish, since kvctl exits 0 on a rejected one. A tests/test_ci_report.py test covers the Priority tests section, the failed-case brief and part merging. All five fail at 8bb3b8a, an import error for the harness ones and a missing section for the report one, and pass here, 38 tests across the three CI test files. On the full CPU manifest at 0760701, 1643 pass with a failure set identical to the bare stack at 8bb3b8a on this machine (its 13 known env cases), 1643 against 1638. ruff, isort, codespell, the test classification and CI-mode mypy (3.10 and 3.13) are clean. After the stack merges, the first weekly run is the arbiter of the GPU criteria above, with the #562 hold on /ci run weekly for changes under tests/e2e/.

Refs #585, #608, #609, #611