Skip to content

Fix #11123: preserve representable subnormal erfc results - #11124

Merged
thewilsonator merged 1 commit into
dlang:masterfrom
jmh530:codex/fix-erfc-subnormal
Oct 18, 2026
Merged

thewilsonator merged 1 commit into
dlang:masterfrom
jmh530:codex/fix-erfc-subnormal

Conversation

@jmh530

@jmh530 jmh530 commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

Rationale

With 80-bit real, erfc(106.625L) returns zero instead of approximately 1.892705840797887e-4940, a representable subnormal value. The existing cutoff uses the normal-range limit and discards part of the subnormal output range.

Replace that cutoff with a conservative zero bound derived from the floating-point format. In the extreme tail, evaluate z = exp(-x*x/2) and form (z * erfce(x)) * z, keeping intermediate factors normal and rounding into the subnormal range only at the final multiplication. Ordinary inputs retain the original calculation.

Feature request or issue tracking

Closes #11123.

Validation

  • Error-function module tests passed in debug and optimized builds with Windows DMD 2.113.0 (80-bit real), Windows LDC 1.43.0 (64-bit real), and Linux LDC 1.36.0 (80-bit real).
  • Focused public-API checks passed on all three configurations.
  • Added regression tests for subnormal results, the transition to zero, negative arguments, and large finite inputs.
  • Checked 2,001 boundary inputs per configuration against independent high-precision references, plus 2,001 additional fractional inputs and three adjacent inputs around the new path's threshold on each Windows compiler. All reference checks passed, with no sampled lost tails or incorrect nonzero results below half the smallest subnormal.

These are targeted checks, not a full Phobos test-suite run or a correctly-rounded accuracy guarantee. Reference tolerances allow the existing exponential's argument-dependent error. IEEE quadruple precision was not tested locally.

Performance

Optimized local microbenchmarks compared this commit with 6f9665c6c. Each sample made 100,000 calls through function pointers. Times below are medians of six alternating baseline/PR pairs after discarding the first pair, in nanoseconds per call.

Configuration Input Original PR
Windows LDC, 64-bit real 2 17.24 18.27
Windows LDC, 64-bit real 26.625 130.18 65.28
Windows LDC, 64-bit real 27 130.62 66.10
Windows DMD, 80-bit real 106.5 361.64 352.84
Windows DMD, 80-bit real 106.625 107.50 513.03
Linux LDC, 80-bit real 106.5 175.57 180.00
Linux LDC, 80-bit real 106.625 8.00 302.56

Sampled ordinary-case timings varied approximately -7% to +6%. Windows LDC subnormal cases improved by about 2x, consistent with avoiding subnormal intermediate arithmetic. The substantial slowdown at 106.625 with 80-bit real replaces an incorrect immediate zero with a computed subnormal result. These short local measurements are not performance guarantees.

Pre-review checklist

LLM/AI disclosure

This PR was written entirely by an LLM/AI.

AI-Assisted-By: Codex (GPT-6 Astra Medium)

Replace the normal-range cutoff with a conservative zero cutoff
derived from the real format. The previous cutoff returned zero for
representable 80-bit subnormal results, including erfc(106.625).

In the extreme tail, evaluate exp(-x*x/2) and multiply its two factors
with the scaled erfc approximation so only the final product enters
the subnormal range. This avoids premature intermediate rounding
while retaining the original calculation for ordinary inputs.

Add regression tests for 64-bit and 80-bit subnormal results, the
transition to zero, negative arguments, and large finite inputs.
Validate module tests and public-API checks with Windows DMD/LDC
and Linux LDC, plus high-precision boundary sweeps.

Ordinary-case performance remains broadly similar. Producing the
previously discarded 80-bit subnormal results costs more than the
incorrect early return; sampled Windows LDC subnormal cases improve.

Fixes dlang#11123.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
@thewilsonator
thewilsonator merged commit 3a75e5f into dlang:master Oct 18, 2026
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

erfc returns zero for representable subnormal results with 80-bit real

3 participants