Skip to content

Preserve messages and stack traces of external multiple failures - #2438

Open
liyifan9642-creator wants to merge 1 commit into
spockframework:masterfrom
liyifan9642-creator:fix/multiple-failures-diagnostics-2175
Open

liyifan9642-creator wants to merge 1 commit into
spockframework:masterfrom
liyifan9642-creator:fix/multiple-failures-diagnostics-2175

Conversation

@liyifan9642-creator

Copy link
Copy Markdown

Fixes #2175.

MasterRunSupervisor currently unwraps every MultipleFailuresError. The collector then creates a fresh aggregate with an empty heading, losing the original aggregate's message and originating stack trace. This affects exhausted @Retry failures, among other externally created aggregates.

This change keeps external aggregate diagnostics while retaining the existing processing of their children:

  • Preserve the original message, stack trace, cause, and suppressed exceptions in a SpockMultipleFailuresError containing the transformed children. The original message is not reused as a heading, since it already includes the child failure text.
  • Continue flattening internal condition groups, converting child equality assertions for IDE comparisons, and notifying Spock listeners of leaf failures.
  • Keep a preserved aggregate intact if it passes through the collector again.
  • Keep the existing public constructor's null-heading behavior.

Externally thrown single-failure aggregates also remain aggregates, including those containing a TestAbortedException. Plain aborted/skipped exceptions and their listener handling are unchanged. Tests cover this boundary as well as existing retry/abortion interoperability.

Verification

Before the production change, the two retry-mode cases and three supervisor regression cases failed on current master; the existing selected tests passed. After the fix, all local runs below completed with zero failures and errors:

Groovy Test Java Core specs JUnit4 suite
2.5 11 130 selected, 0 skipped 77 total, 1 skipped
3.0 17 130 selected, 0 skipped 77 total, 1 skipped
4.0 17 3,825 total, 72 skipped 74 total, 1 skipped
5.0 17 130 selected, 0 skipped 74 total, 1 skipped
6.0 17 130 selected, 0 skipped 74 total, 1 skipped

The selected core specs cover MasterRunSupervisorSpec, RetryFeatureExtensionSpec, RetryTestAbortedExceptionInteropSpec, VerifyAll*, VerifyEach*, RunListenerSpec, and AutoCleanupExtension. Added tests also check nested aggregates, cause/suppressed preservation, listener context, repeated collection, and comparison values.

Tests ran on Windows using the Gradle wrapper; core Java compilation retains the configured release 8 target. git diff --check passes. The complete multi-module allVariants build and Linux/macOS runs were not performed locally.

AI assistance

This change was developed with significant AI assistance, including implementation and regression tests.

Retain aggregate messages, stack traces, causes, and suppressed errors while transforming child assertions and notifying listeners. Keep internal condition groups flattened and add regression coverage for both retry modes.

Fixes spockframework#2175

Developed with AI assistance.
@coderabbitai

coderabbitai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 4d112629-bfaa-456a-8e5c-2cce57b1177b

📥 Commits

Reviewing files that changed from the base of the PR and between ce7e8af and 4130442.


📒 Files selected for processing (5)
  • spock-core/src/main/java/org/spockframework/runtime/ErrorInfoCollector.java
  • spock-core/src/main/java/org/spockframework/runtime/MasterRunSupervisor.java
  • spock-core/src/main/java/org/spockframework/runtime/SpockMultipleFailuresError.java
  • spock-specs/src/test/groovy/org/spockframework/runtime/MasterRunSupervisorSpec.groovy
  • spock-specs/src/test/groovy/org/spockframework/smoke/extension/RetryFeatureExtensionSpec.groovy

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.



📝 Walkthrough

Walkthrough

The runtime now preserves diagnostic information from external multiple-failure aggregates while expanding untransformed Spock aggregates. New tests cover aggregate handling and retry modes.

Changes

Aggregate preservation

Layer / File(s) Summary
Retain aggregate metadata
spock-core/src/main/java/org/spockframework/runtime/SpockMultipleFailuresError.java, spock-core/src/main/java/org/spockframework/runtime/ErrorInfoCollector.java, spock-specs/src/test/groovy/org/spockframework/runtime/MasterRunSupervisorSpec.groovy
SpockMultipleFailuresError can retain an original aggregate’s message, stack trace, cause, and suppressed exceptions. ErrorInfoCollector uses getExceptions() when creating an aggregate. A test checks the public constructor with a null heading.
Transform and verify collected aggregates
spock-core/src/main/java/org/spockframework/runtime/MasterRunSupervisor.java, spock-specs/src/test/groovy/org/spockframework/runtime/MasterRunSupervisorSpec.groovy, spock-specs/src/test/groovy/org/spockframework/smoke/extension/RetryFeatureExtensionSpec.groovy
MasterRunSupervisor expands untransformed Spock aggregates and wraps other processed aggregates. Tests check preserved diagnostics, nested aggregates, aborted failures, repeated collection, listener notifications, and retry-mode comparison failures.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium


Merge Risk: ⚪ Minimal · up to 41304

The change preserves messages and stack traces of external multiple-failure errors, with tests covering the main paths. No actionable merge-blocking risk was found; the full multi-module build was not run by the author, which is normal CI-level verification.

Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 3 files. (2 skipped: 2… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: preserving messages and stack traces for external multiple failures.
Description check Passed The description directly explains the reported issue, the implementation, preserved behavior, test coverage, and verification results. It is clearly related to the changeset.
Linked Issues check Passed Issue #2175 requires preservation of an external MultipleFailuresError message and stack trace. MasterRunSupervisor now wraps external aggregates in SpockMultipleFailuresError. The wrapper copie…
Out of Scope Changes check Passed The production changes are limited to aggregate collection and diagnostic preservation in MasterRunSupervisor, ErrorInfoCollector, and SpockMultipleFailuresError. The added tests exercise issue …

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 3 files. (2 skipped: 2 unsupported.)


  • Fix all pre-merge checks with AI
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit watched the failures flow
Their messages stayed, their traces glow
The nested leaves came into view
The retry tests checked each mode too
Then hopped away beneath the moon

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable defects were found.

Summary

Preserves messages and originating stack traces for external MultipleFailuresError groups while keeping transformed children.

  • External failure groups keep their original diagnostics.

liyifan9642-creator explicitly states that externally thrown single-failure aggregates remain aggregates, including those containing TestAbortedException. This behavior is intentional.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A["MasterRunSupervisor.error"] --> B{"MultipleFailuresError?"}
  B -- No --> C["Transform and filter leaf"]
  C --> D["Notify listener unless aborted or skipped"]
  D --> E["Add leaf to collector"]
  B -- Yes --> F{"Untransformed SpockMultipleFailuresError?"}
  F -- Yes --> G["Process children in outer collector"]
  F -- No --> H["Process children in nested collector"]
  H --> I["Copy original diagnostics into preserved group"]
  I --> J["Filter and add group to outer collector"]
Loading

Reviews (1) · Last reviewed commit: "Preserve diagnostics of external multipl..." · Reviewed by Greptile

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Message and stacktrace of MultipleFailuresError like from RetryExtension get lost

1 participant