Skip to content

ci: make Lychee validate repository links against the checkout - #3143

Open
rwgk wants to merge 2 commits into
rwgk/lychee-pre-commit-cifrom
rwgk/lychee-checkout-aware-links
Open

rwgk wants to merge 2 commits into
rwgk/lychee-pre-commit-cifrom
rwgk/lychee-checkout-aware-links

Conversation

@rwgk

@rwgk rwgk commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Highlights

Let Lychee validate default-branch (main) GitHub links against the proposed checkout, so new or moved paths can pass link checks before merging.

  • Paths and links can land together: Lychee checks links to files and directories introduced by the same PR against that PR's checkout.
  • Stable URLs in documentation: source files and published documentation keep their intended GitHub URLs. Only Lychee's validation targets are remapped.
  • Earlier feedback during development: authored checks resolve link targets in the working tree, including uncommitted and unpushed additions. Deleted or missing targets fail even if an earlier check succeeded.
  • Default branches discovered automatically: support the public repository and the repository being checked, including synchronized repositories whose default is ctk-next. Each URL is matched against its own repository's default branch.

This builds on #3142's shared Lychee pre-commit hooks, adding repository-link handling to the existing development, PR, documentation-build, and nightly checks.

How it works

For example, a PR can add new-directory/ and immediately link to https://github.com/NVIDIA/cuda-python/tree/main/new-directory. Lychee validates the proposed target before it exists on remote main.

Documentation being checked Where Lychee validates the repository link Fragment policy
Authored Markdown and reST Files and directories in the working tree Existing policy: fragments are not checked
Rendered HTML GitHub's file or directory view at the actual checked-out commit Fragment checks are retained

The checked-out commit can be a CI mirror or merge revision. The current working branch and PR base do not determine which source URLs are remapped.

Scope and tradeoffs

Remapping applies to https://github.com/.../blob/<default-branch>/... and .../tree/<default-branch>/... links naming the canonical public repository or the selected checkout repository. Version tags, explicit commits, other branches, unrelated repositories, and URLs embedded inside other URLs retain their targets.

Local hooks discover defaults from cached Git remote HEAD references, without making discovery requests. A checkout may therefore need its remote metadata initialized or refreshed. Missing, dangling, or conflicting defaults fail with specific repair instructions. Rendered checks still require the checked-out commit to be available on GitHub; authored checks cover local unpublished edits.

These rules model the proposed default-branch tree. Maintenance branches need an explicit policy before adopting them when their links intentionally refer to a different tree.

Implementation details and suggested review order

Repository discovery and matching

CI prepares repository metadata once per job, using the workflow event's repository identity and default branch. When checking another repository, preparation also queries the public repository's advertised default branch once. Each hook invocation reuses that metadata.

Local discovery selects upstream when present, otherwise origin. An explicit git config lychee.repository OWNER/REPOSITORY selects the repository when a different arrangement is needed. Matching remotes supply its cached default; the helper reports commands such as git remote set-head upstream --auto when a refresh is required. Synchronized checkouts also need cached default metadata for the public repository.

The wrapper passes anchored --remap rules to native Lychee. Rules match the supported repository and its complete default-branch prefix, preserving the remaining path, query, and fragment. Authored rules point to file URIs; rendered rules use the current repository and git rev-parse HEAD.

The new helper participates in the cache policy hash, so caches produced under the previous mapping policy are not reused. The three-day lifetime, separate authored/rendered caches, and publication policy come from #3142.

Suggested review order

  1. Repository-link guide: intended behavior, local setup, and maintenance-branch boundary.
  2. Default-branch discovery and remapping: repository selection, metadata validation, and the two target modes.
  3. Native Lychee wrapper: add the resolved remaps when invoking the existing hook executable.
  4. The pre-commit, nightly link, and documentation-build workflows: prepare the same metadata through the named LYCHEE REPOSITORY METADATA synchronized block.
  5. Cache descriptor: include the new helper in the policy fingerprint.

Validation

All-files pre-commit passed locally on 983ac757fea1, after restacking on the refreshed #3142. The feature diff is unchanged. The LYCHEE REPOSITORY METADATA blocks in pre-commit.yml, lychee.yml, and build-docs.yml were compared byte-for-byte again.

Earlier validation of this implementation

Before the latest stack refresh:

  • 292 CI-tool tests and 36 subtests passed locally, covering default discovery, public/synchronized repository pairs, encoded paths and branches, preserved targets, checked-out commit selection, and dangling remote-HEAD detection.
  • Full all-files pre-commit and the permanent offline hook smoke check passed. Native Lychee probes checked Markdown/reST remapping, rejected deleted targets after a cached success, and exercised actual public/synchronized repository metadata.
  • A native rendered probe accepted a newly added README heading at the published feature commit and rejected a missing fragment.
  • Linux and Windows PR checks passed, including the configured hook and all three Windows shells.
  • Full CI passed all 95 jobs, including the documentation build and rendered-link check.

Checklist

  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@rwgk rwgk added this to the cuda.bindings next milestone Oct 17, 2026 — with ChatGPT Codex Connector
@copy-pr-bot

copy-pr-bot Bot commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@rwgk rwgk added enhancement Any code-related improvements CI/CD CI/CD infrastructure labels Oct 17, 2026 — with ChatGPT Codex Connector
@rwgk rwgk self-assigned this Oct 17, 2026
@rwgk
rwgk added this pull request to stack #3144 October 17, 2026 00:03
@rwgk
rwgk force-pushed the rwgk/lychee-checkout-aware-links branch from 280c425 to 672e513 Compare October 17, 2026 00:14
@rwgk

rwgk commented Oct 17, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 672e513

@github-actions

Copy link
Copy Markdown
Contributor

@rwgk
rwgk force-pushed the rwgk/lychee-checkout-aware-links branch from 672e513 to d24e19f Compare October 18, 2026 04:41
rwgk added 2 commits October 17, 2026 22:16
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
@rwgk
rwgk force-pushed the rwgk/lychee-checkout-aware-links branch from d24e19f to 983ac75 Compare October 18, 2026 05:18
@rwgk rwgk changed the title ci: validate default-branch repository links against the checkout ci: make Lychee validate repository links against the checkout Oct 18, 2026
@rwgk
rwgk marked this pull request as ready for review October 18, 2026 23:33
@rwgk rwgk added the PR review get-together Mark PRs you'd like the team to review at the weekly PR review get-together. label Oct 18, 2026
@coderabbitai

coderabbitai Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

📝 Summary

Summary by CodeRabbit

  • Improvements
    • Documentation link checks now resolve supported repository links against the proposed checkout, helping catch broken links in authored and rendered documentation.
  • Documentation
    • Added guidance on supported repository links and how link checks resolve repository and branch information.

The change adds repository default-branch discovery and URL remapping for supported repository links in Lychee checks. CI workflows publish repository metadata, while local hooks resolve cached remote HEADs. Authored links map to checkout paths; rendered links map to the checked-out commit. Documentation and tests cover these paths.

Changes: Repository Link Checks

Layer / File(s) Summary
Resolve and publish repository defaults
.github/workflows/*.yml, ci/tools/checkout_link_defaults.py, ci/tools/tests/test_checkout_link_defaults.py
The script validates repository and branch metadata, resolves cached local remote defaults or CI defaults, and writes CI metadata to the environment file. The documentation tests cover cached-default validation and CI branch discovery.
Apply repository link remaps
ci/tools/checkout_link_defaults.py, ci/tools/run_lychee.py, ci/tools/lychee_cache.py, ci/tools/tests/*, ci/README.md
Lychee receives remap arguments for supported default-branch URLs. Authored links map to working-tree paths, while rendered links map to the checked-out commit. The cache digest includes the remapping script, and the documentation describes supported URL forms and default discovery.

Priority: ⬇️ Low

Change: Feature

Merge Risk: 🟡 Moderate · up to 983ac

Nightly link checks can fail before they run. Add a default-branch fallback before merging.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments: 1
ci/tools/run_lychee.py: 1

66-68: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

suggestion: Exercise nonempty remaps at the subprocess boundary.

The run_lychee.main() tests replace repo_link_remaps with [], so they do not detect a regression that drops remap options before subprocess.run. The helper tests cover remap construction separately, but not this caller wiring. Add nonempty remaps to both authored and rendered tests and assert their exact position in the Lychee command.

Suggested fix
-            patch("run_lychee.repo_link_remaps", return_value=[]),
+            patch("run_lychee.repo_link_remaps", return_value=["--remap", "source=target"]),
...
-        assert command[1:4] == ["--format", "json", "--output"]
-        report = Path(command[4])
+        assert command[1:3] == ["--remap", "source=target"]
+        assert command[3:6] == ["--format", "json", "--output"]
+        report = Path(command[5])
...
-        patch("run_lychee.repo_link_remaps", return_value=[]),
+        patch("run_lychee.repo_link_remaps", return_value=["--remap", "source=target"]),
...
         [
             str(tmp_path / "lychee"),
+            "--remap",
+            "source=target",
             "--files-from",

ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA/cuda-python/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: 13a83dd1-9c60-4b9d-84a2-a179372b52cb
📥 Commits

Reviewing files that changed from the base of the PR and between d7d7bd8 and 983ac75.

📒 Files selected for processing: 9
  • .github/workflows/build-docs.yml
  • .github/workflows/lychee.yml
  • .github/workflows/pre-commit.yml
  • ci/README.md
  • ci/tools/checkout_link_defaults.py
  • ci/tools/lychee_cache.py
  • ci/tools/run_lychee.py
  • ci/tools/tests/test_checkout_link_defaults.py
  • ci/tools/tests/test_run_lychee.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

# BEGIN SYNCED LYCHEE REPOSITORY METADATA
env:
LYCHEE_REPOSITORY: ${{ github.repository }}
LYCHEE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🔎 Supported by static analysis

🏁 Script executed:

#!/bin/bash
fd -e yml -e yaml . .github/workflows --exec sh -c 'echo "== $1"; sed -n "1,45p" "$1" | rg -n "on:|schedule|cron|workflow_call|workflow_dispatch|push|pull_request|uses: ./.github/workflows/(lychee|pre-commit)" ' _ {}

Repository: NVIDIA/cuda-python

Length of output: 6774


🏁 Script executed:

set -e
printf '%s\n' '== workflow triggers and caller references =='
for f in .github/workflows/lychee.yml .github/workflows/pre-commit.yml .github/workflows/ci-nightly.yml .github/workflows/ci.yml .github/workflows/build-docs.yml; do
  echo "== $f"
  nl -ba "$f" | sed -n '1,180p'
done
printf '%s\n' '== relevant helper and prepare_ci references =='
rg -n -F --glob '*.py' -e 'def _branch' -e 'LYCHEE_DEFAULT_BRANCH' -e 'prepare_ci' ci

Repository: NVIDIA/cuda-python

Length of output: 41037


🏁 Script executed:

set -e
nl -ba ci/tools/checkout_link_defaults.py | sed -n '35,205p'

Repository: NVIDIA/cuda-python

Length of output: 8299


🌐 Web query:

GitHub Actions schedule event payload empty github.event.repository default_branch reusable workflow github context caller

💡 Result:

Yes—**a `schedule` event has no webhook payload**, so `github.event.repository.default_branch` is not available. GitHub’s event reference lists the schedule payload as “Not applicable”; the event object isn’t a repository object. ([github.com](https://github.com/github/docs/blob/main/content/actions/reference/workflows-and-actions/events-that-trigger-workflows.md?utm_source=openai))

In a reusable workflow called by that scheduled workflow, **`github` still refers to the caller’s context**. So `github.event.repository.default_branch` remains unavailable there; calling a reusable workflow doesn’t add the missing payload fields. ([docs.github.com](https://docs.github.com/en/actions/reference/workflows-and-actions/reusing-workflow-configurations?utm_source=openai))

Use the caller’s `github.repository` to identify the repo and, if you need its default branch, query repository metadata (for example, with the GitHub API). The event’s `github.ref` is the default branch ref for a scheduled run, but it is not itself the `default_branch` property. ([github.com](https://github.com/github/docs/blob/main/content/actions/reference/workflows-and-actions/events-that-trigger-workflows.md?utm_source=openai))

Citations:

- 1: https://github.com/github/docs/blob/main/content/actions/reference/workflows-and-actions/events-that-trigger-workflows.md?utm_source=openai
- 2: https://docs.github.com/en/actions/reference/workflows-and-actions/reusing-workflow-configurations?utm_source=openai
- 3: https://github.com/github/docs/blob/main/content/actions/reference/workflows-and-actions/events-that-trigger-workflows.md?utm_source=openai

🏁 Script executed:

set -e
printf '%s\n' '== workflow references =='
rg -n -F --glob '*.yml' --glob '*.yaml' -e './.github/workflows/lychee.yml' -e './.github/workflows/pre-commit.yml' -e './.github/workflows/build-docs.yml' .github/workflows
printf '%s\n' '== metadata blocks =='
for f in .github/workflows/lychee.yml .github/workflows/pre-commit.yml .github/workflows/build-docs.yml; do
  echo "== $f"
  rg -n -C 8 -F 'LYCHEE_DEFAULT_BRANCH:' "$f"
done

Repository: NVIDIA/cuda-python

Length of output: 3010


important: Add a schedule fallback for the default branch.

ci-nightly.yml runs on schedule and calls lychee.yml. Scheduled events do not provide github.event.repository.default_branch, so the helper receives an empty branch and fails before the nightly link checks run.

pre-commit.yml is not scheduled, and build-docs.yml guards this step to pull-request refs. Apply the same fallback to all synced blocks:

Suggested fix
--- a/.github/workflows/lychee.yml
+++ b/.github/workflows/lychee.yml
@@
-          LYCHEE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+          LYCHEE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch || (github.event_name == 'schedule' && github.ref_name) || '' }}
--- a/.github/workflows/pre-commit.yml
+++ b/.github/workflows/pre-commit.yml
@@
-          LYCHEE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+          LYCHEE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch || (github.event_name == 'schedule' && github.ref_name) || '' }}
--- a/.github/workflows/build-docs.yml
+++ b/.github/workflows/build-docs.yml
@@
-          LYCHEE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+          LYCHEE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch || (github.event_name == 'schedule' && github.ref_name) || '' }}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
LYCHEE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
LYCHEE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch || (github.event_name == 'schedule' && github.ref_name) || '' }}

Source: Path instructions

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure enhancement Any code-related improvements PR review get-together Mark PRs you'd like the team to review at the weekly PR review get-together.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant