Skip to content

feat(sdk): add checkpointable Compose integration on current SDK - #127

Draft
hky1999 wants to merge 4 commits into
inclusionAI:developfrom
hky1999:hky1999/sdk-managed-compose
Draft

hky1999 wants to merge 4 commits into
inclusionAI:developfrom
hky1999:hky1999/sdk-managed-compose

Conversation

@hky1999

@hky1999 hky1999 commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Harbor needs to run unchanged Dockerfile/Compose tasks while retaining whole-sandbox checkpoint/restore. Move reusable image preparation, Docker runtime startup, Compose lifecycle and captured service-process handles into the AKernel SDK. Harbor keeps task/resource/verifier adaptation, model access and aligned harness checkpoint decisions.

Review baseline and implementation

Based on inclusionAI/AKernel:develop at 5ded29d5809f7f279c7ae122b3312d6912207fc6; current source-preview head is d13e774. The change retains the merged snapshot/lifecycle SDK, command identity, conservative creation ownership, cancellation/rollback and fork cleanup contracts. Backend changes forward validated extra_config through restore and fork; they do not replace the snapshot implementation.

  • ComposeProject freezes original project inputs, prepares startup-declared images, exposes explicit start/build/pull/up boundaries and serializable descriptors, and owns service exec/transfers. Original Dockerfiles and Compose bytes stay unchanged.
  • DockerRuntime and builder/image-bridge validate OCI metadata and register Sandboxd's existing shared ROFS lower roots with private managed upper storage. They add no second registry layer cache.
  • ServiceHandle / ServiceProcessHandle preserve container ID/start time and process PID/start time. Restore attaches captured processes without replaying builds, up or the agent launch; uncertain launches retain an observation token. Trees thaw descendants before waiting ancestors.
  • Native default/custom IPv4 bridges, DNS/aliases and userland TCP/UDP mappings remain intact. The SDK manages its own forwarding/SNAT and reconnects those rules after a new outer TAP. A peer-restricted service gateway reuses the existing reverse tunnel and forwards HTTP/auth/SSE bytes unchanged.
  • Image-only services such as image: name:tag@sha256:... reuse their prepared ROFS roots. Native Compose's missing pull policy still pulls digest references, which otherwise enters forbidden layer extraction. Automatic pull() now verifies pinned aliases and committed snapshots; up() verifies them and disables repeat pulls. Missing/changed registrations fail without an extraction fallback. Ordinary nonautomatic Compose keeps native pull behavior. The current SDK requires the source-matched schema-2 Go helper; older helpers are rejected before project inputs are sent.

Start with component boundaries and integration, then _compose.py, _docker_runtime.py, _image_bridge_guest.py, _service_process.py and the bridge helper.

Dependencies and scope

The managed file upper requires the Sandboxd contract from Sandboxd #83. Its reviewed server revision must be pinned and qualified by the AKernel consumer. AKernel #126 separately exposes explicit Helm reflink configuration. Compatible same-mount storage, durable synchronization and Docker 28 runsc packet-socket flags are operator prerequisites.

This PR changes no Sandboxd gitlink, runtime release pin, deployment or Coordinator. It introduces no Compose-aware Agent DX server API. Harbor delivery branch hky1999/harbor-compose-native-network now pins this SDK; its audited runtime code is 88dc5ace, delivered with tests/documentation at 76791d384b0f7b5a777379948c49bc6f4bfdfbbd.

Delivery hardening in this head

  • Require helper discovery schema 2 before dispatching project input; confine Dockerfile/context/.env reads to the frozen project and reject unsupported include/extends files. Discovery uses only explicit environment inputs. Registry credentials remain client-side.
  • Validate selected OCI config/platform/diff IDs/chain relationships, allow different immutable targets sharing one rootfs chain, filter unused manual aliases, and retain the automatic marker for scratch-only plans.
  • Preserve unknown command outcomes and command identities through cancellation/secondary kill failure; image verification follows the zero-timeout background wait contract. Do not replay uncertain commands.
  • Harbor independently settles dispatched SDK work across repeated cancellation, bounds retention debt, backs off failed/unknown captures, persists owned cleanup IDs/names and checks native process start times. These harness changes are in the Harbor branch, not the SDK.

Validation of this head

  • make sdk-check: 477 SDK unit tests + 35 benchmark-driver tests, Ruff and mypy passed. Bridge gofmt, Go tests and vet passed.
  • Wheel/sdist build succeeded; wheel installed into a new venv with public Compose/service/process APIs and packaged guest helpers. Harbor used the immutable Git-installed SDK, without a source-path override.
  • 366 Harbor backend/Claude regressions + 38 Codex tests + 4 generic native-harness tests passed. Whole-repository Ruff passed; ty retains 146 existing baseline diagnostics and is not green.
  • Static strict parsing accepted all 100 locally available task Dockerfiles. This did not execute their dependency downloads, RUN instructions or agents.
  • Owned-cluster boundary tests passed for scratch-only builds and two immutable image targets sharing one rootfs chain: build/up, snapshot, mutation, source deletion and restore preserved memory/PID/container identity.
  • Four concurrent complete Harbor trials: two original 0003 tasks and two kv-live-surgery copies with the colleague's exact 3148-byte image-only Compose; remaining task archive bytes unchanged. Automatic main/verifier mode selection passed. Harbor installed real Claude 2.1.278 using the existing model resource, LiteLLM 1.93.0 and reverse tunnel.
  • Each trial committed 1 baseline and 5 periodic snapshots (2 before loss, 3 after recovery), automatically restored once after source deletion, and retained launch count 1. Original process identity, RAM, file offset, internal TCP and private-upper rollback/writes passed; both Compose container IDs survived. Active old-ancestor references occurred; other stale points were reclaimed, peak tracked snapshots were 4 per trial, and final cleanup journals were empty.
  • All four independent verifiers ran; four valid 3-step trajectories matched token/cost fields. Original 0003 reward was 1 for both trials. kv remained 4/6, reward 0: 5/6 accumulated drops and 15788/15723 ms peak RTT fail the unchanged zero-downtime/latency SLOs. Final 30-second measurement windows had zero drops. This is a real-Claude bounded process-memory fixture followed by the original reference workload, not autonomous solve or uninterrupted-service qualification.
  • Periodic checkpoint APIs: direct 1.731–2.242 s, Compose 2.491–2.785 s. Environment restore: direct 5.490/5.768 s, Compose 9.322/9.314 s. Full harness recovery: direct 8.778/8.999 s, Compose 14.643/14.444 s. These are separate boundaries and functional samples, not a performance distribution or strict A/B.
  • Cleaned 16 owned environments / 26 snapshots, including boundary tests; global inventory 0/0 and deployment fingerprints unchanged. No deployment, Coordinator, gitlink or runtime pin was updated.

Full delivery review records configuration, test scope and operational limits.

Release and scale limits

This remains a source preview pending component review and normal release qualification. The matching Linux/amd64 helper attachment was source-built and anonymously downloaded/hash-verified. Its non-SDK release tag triggers the generic Python release workflow's tag-format rejection; subsequent build/publish steps do not run and no PyPI package was published. This is not an official CI SDK release.

Four concurrent trials do not qualify 100/1000 cold starts, registry quotas or long-run tail latency. Metadata remains per-project and the client freezes/uploads the full project context. Unsupported direct-Dockerfile syntax and external Compose sources fail closed; external TCP address migration, local Harbor restart recovery and exactly-once external model calls are not promised. A prior prototype also had one 300-second checkpoint timeout without an established root cause. DSH/Codex real-model E2E was not repeated with this head.

@hky1999 hky1999 changed the title feat(sdk): own managed Docker image preparation and bridge runtime feat(sdk): manage Compose workloads and checkpointable service processes Oct 28, 2026
@hky1999 hky1999 changed the title feat(sdk): manage Compose workloads and checkpointable service processes feat(sdk): manage checkpointable Compose workloads with native networks Oct 28, 2026
Move Compose preparation, shared ROFS bridge bootstrap and service/process handles into the SDK while preserving the merged snapshot, ownership and command identity contracts. Forward managed runtime configuration through restore and fork without replaying fresh setup.

Keep native IPv4 Compose networking and service-scoped model tunnel entries, and reconnect only owned network rules after restoration. Preserve original process identities and child-first thaw. Add migration regressions, a maintained example, component boundary documentation and helper/distribution CI checks.

Signed-off-by: 月可 <hukeyang.hky@antgroup.com>
@hky1999
hky1999 force-pushed the hky1999/sdk-managed-compose branch from d932540 to 61606ce Compare October 28, 2026 14:59
@hky1999 hky1999 changed the title feat(sdk): manage checkpointable Compose workloads with native networks feat(sdk): add checkpointable Compose integration on current SDK Oct 28, 2026
Compose pull with the missing policy still pulls digest references. In the
managed runtime this enters ordinary layer extraction even though the
complete immutable image root is already mounted by sandboxd.

Verify pinned Docker references and committed image snapshots instead of
pulling prepared inputs again, and prevent repeat pulls during automatic
Compose up. Preserve native pulls without automatic image preparation and
keep missing or changed registrations visible without an extraction fallback.

Cover mixed build/image services and failed registration checks, and document
the preparation-time pinning contract. Validate with the full SDK quality
gate, bridge tests and an isolated runtime using the reported loadgen digest.

Signed-off-by: 月可 <hukeyang.hky@antgroup.com>
Validate immutable manifest/config/chain relationships while allowing images
with distinct target digests to share an identical filesystem chain. Preserve
empty automatic plans for scratch builds and filter unused manual aliases.

Require discovery schema 2 before dispatching project input. Constrain host
Dockerfile, build-context and dotenv reads, isolate discovery environment
variables, and reject unsupported external Compose includes and extends.

Retain unknown command outcomes and cancellation cleanup errors, and preserve
unbounded verification waits without a foreground zero-timeout call. Cover
these cases with regression tests and document the matching-helper boundary.

Signed-off-by: 月可 <hukeyang.hky@antgroup.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant