Skip to content

Serve the remaining framework library code from .NET 11 IL - #424

Draft
jonathanperis wants to merge 91 commits into
mainfrom
feat/flip-framework-il
Draft

jonathanperis wants to merge 91 commits into
mainfrom
feat/flip-framework-il

Conversation

@jonathanperis

Copy link
Copy Markdown
Owner

Fixes #423

One PR for every remaining library flip, as the owner asked. Each area lands as its own commit series, checked locally before it is pushed; the checklist and per-area status are in docs/il-first/remaining-flips.md, and each area adds its docs/il-first/<library>.md row with the NativeAOT column.

Draft until every area is done and the five test jobs and the Bootstrap checks are green on the final head. This description will list, per area: what was deleted, the compiler gaps fixed (with regression samples), size and run-time numbers against NativeAOT, and anything left as an explicit "not supported by csharpc yet" diagnostic.

The owner asked for every remaining library flip to land in one PR (#423).
This file is that PR's checklist; each area's commits update its row.
@jonathanperis jonathanperis added this to the Drop-in for large apps milestone Nov 14, 2026
@jonathanperis jonathanperis added enhancement New feature or request area: bcl Library surface (corelib) area: il-frontend IL front-end (program or libraries as assemblies) labels Nov 14, 2026
@jonathanperis jonathanperis self-assigned this Nov 14, 2026
…hadows are deleted

- SpinWait is sliced from CoreLib (upstream-types/System.Threading.txt); the corelib's
  SpinWaitImpl is deleted. Thread.OptimalMaxSpinWaitsPerSpinIteration (internal upstream)
  is a ShadowMembers property (8), and Thread.SpinWait(n) now spins n processor pauses
  (csrt_thread_spin_wait) instead of compiling to nothing.
- System.Threading.dll is taken as IL with a renamed twin (TwinReference), as
  System.Collections is: its own types (Barrier, BarrierPostPhaseException,
  CountdownEvent, ReaderWriterLock ...) come from the framework, the rest stays forwarded
  to CoreLib. The Barrier shadow is deleted; CountdownEvent is no longer the runtime's
  event object (its lowering by name, the CountdownEventWaits members and
  CancelableWaits are removed).

Sample 1338 (recorded from .NET): SpinWait's count and yield threshold, SpinUntil,
CountdownEvent's counts and exceptions, Barrier phases across threads, the post-phase
action and its exception, participants, disposal.
…NET); 1337/1338 need threads, skipped on wasi
… Barrier row; remaining-flips: sync types done
System.Numerics.BFloat16, Decimal32, Decimal64 and Decimal128 now come from the pinned framework's CoreLib IL
(upstream-types/System.Numerics.IeeeTypes.txt), and the corelib copies are deleted (BFloat16.cs, Decimal32/64/128.cs,
IeeeDecimal.cs, Number.IeeeDecimal.cs: 1 830 lines). Their parsing, formatting, rounding, conversions and generic math are
upstream's by construction.

General compiler pieces the IL needed:
- The slicer hosts a generic interface's static default members as generic methods over the interface's type parameters
  (!n becomes !!n in the member's own context, not in the signatures it references). A call Roslyn resolves to the reference
  pack's body-less default member goes to the hosted body (HostedDefaultInterfaceMember). This covers every generic-math
  interface's defaults (upstream-types/System.Numerics.InterfaceDefaults.txt): INumberBase's UTF-8 Parse/TryParse, Create*,
  the checked operators, INumber's Clamp/Min/Max/Sign, IFloatingPoint's rounding and the rest, for user number types too.
- An interface member over a CoreLib type that the program sees as its slice replacement maps to the replacement's interface.
- Decimal's private _flags/_hi32/_lo64 are lowered like Int128's halves (FunctionCodeGen.BuiltinPrivateFields.cs).
- Unary operators through T resolve the static abstract operator like binary ones, and LibraryOperator calls a concrete
  operator that is compiled code.
- checked(-x) with T a signed primitive checked the subtraction as unsigned (the unsubstituted T), so any value threw
  OverflowException. It now uses the substituted type.
- IBinaryInteger<T>.GetByteCount() on a built-in integer is a constant.

The corelib supplies the CoreLib-internal contract the IL calls: decimal's IsNegative, High/Low/Mid, DecDivMod1E9 and
VarBFloat16FromDec (DecimalContract.cs), NumberFormatInfo's *TChar symbols with upstream's NUL-terminated UTF-8 cache, and
CompareInfo.IsPrefixUtf8.

Samples, recorded from .NET:
- 1333: generic math, UTF-16/UTF-8 parsing and formatting, every conversion, edge values, canonical and non-canonical
  encodings. Main does not compile it (30 diagnostics).
- 1338: INumberBase's default members on user types. Before this change: 13 invalid-IR externals.
- 1339: de-DE formatting and parsing. Skipped on wasi, which has no named cultures.
- 1340: unary operators through T. Before this change: 19 "operator not supported" diagnostics.
- 1341: checked negation of signed primitives through T. Main throws OverflowException for 41.

Local checks:
- Those samples with 34 neighbours: 74/74 at -O0/-O2, 37/37 via IL, 66/66 on wasm32-wasip1.
- tests/slicer.sh: 11/11.
- tests/size.sh: green. json's budget is raised to measured + 3 % (1 571 032 to 1 643 608 B). The program names the
  types only in typeof, but a struct's method table lowers ToString and the dispatched interface slots without a box (#420).
- Against NativeAOT (ieee_types_throughput): 843 520 / 1 480 584 B; run 554 / 784 ms (0.71x); 4.75 G / 6.55 G
  instructions.

Remaining unary gaps found along the way are tracked in #425.
…s IL (#405)

# Conflicts:
#	docs/il-first/remaining-flips.md
#	src/Csharpc/Frontend/Il/IlAssemblySet.cs
…work's IL

Stream, MemoryStream, UnmanagedMemoryStream, BufferedStream, TextReader,
TextWriter, StreamReader, StreamWriter, StringReader, StringWriter,
BinaryReader, BinaryWriter and IndentedTextWriter come from CoreLib's IL
(upstream-types/System.IO.txt); their shadows are deleted. Stream's
asynchronous defaults, which upstream runs on Task internals, are
csharpc's contract (corelib/IO/StreamContract.cs), as are the few
internal Task, ValueTask, Decimal and Encoding members the IL calls.

General fixes: the slicer forwards an instance member's this to its
replacement; [AsyncMethodBuilder] on a compiled async method is not a
dropped semantic; CoreLib.MapForReceiver no longer binds base(...) of a
corelib class deriving from a framework-served class to the class's own
constructor (infinite recursion: 199_debug_assert_failfast,
548_trace_listeners).
… serve the Win32 calls ICU mode makes, keep WASI's root collation (#424, #426)

The first #424 CI round (on 871c8d7) failed on two platforms after the globalization flip.

Windows: the Windows CoreLib carries the NLS back end next to ICU's and picks one by GlobalizationMode.UseNls. csharpc, like .NET by
default, uses ICU (its GlobalizationMode answers false), but CultureData and CompareInfo are root types, so every NLS member was cut
into the slice and lowered: Interop.Kernel32 calls csharpc does not serve, char.IsBetween, and the CultureData/JapaneseCalendar nested
types that made the 64-bit and 32-bit slices serve different types (the pointer-size check failed the compiler build).

- The slicer folds configured static bool getters (Cli.Fold; the csharpc profile folds GlobalizationMode.UseNls to false, as
  .NET's default): the emitted IL loads the constant, so csharpc never lowers the ruled-out branch; the slicer does not follow it;
  and the members and nested types of root types that only folded branches use are not rooted. The Unix CoreLibs contain no UseNls
  call, so their slices are unchanged (identical maps).
- What .NET still asks Windows in ICU mode goes to the corelib's own imports of the same functions (corelib/UpstreamWindows.cs):
  GetLocaleInfoEx (the locale check after ICU's, user overrides, code pages), GetCalendarInfoEx, GetUserGeoID/GetGeoInfo,
  GetUserPreferredUILanguages, LCIDToLocaleName, MultiByteToWideChar, and HijriCalendar's AddHijriDate registry value (advapi32,
  the same parsing as CoreLib's HijriCalendar.Win32.cs). Interop.BOOL is redirected to a corelib enum.
- char.IsBetween is hosted from its IL.

WASI: there is no ICU, so CoreLib's CompareInfo turned linguistic comparisons ordinal ("Banana" < "apple"), where the corelib copy
used the runtime's approximation of ICU's root collation. Replace keys can now name one overload, and the slicer's --wasi option
(passed for the 32-bit slice, which only wasm32 uses) points CompareInfo.Compare over spans at corelib/UpstreamWasiCollation.cs:
CoreLib's IL step for step, with csrt_root_collation_compare for no option or IgnoreCase in invariant mode, as before the flip.
Every linguistic comparison of two strings arrives there; main approximated nothing else.

Security: each Win32 import gets the caller's own buffer and the length CoreLib computed for it, strings go in NUL-terminated, and
the registry read sizes its buffer from what RegQueryValueExW reports (retrying on ERROR_MORE_DATA) without trusting a length
beyond the buffer it allocated.

Local: build 0 errors; slices cut locally from the win-x64, linux-arm and linux-x64 CoreLibs: pointer-size check passes for both
hosts, no Interop.Kernel32/Advapi32 member left unserved in the Windows contract; tests/size.sh 12/12 (unchanged); tests/slicer.sh
10/10; wasm32: the samples CI failed (102_collections, 85_sorting, 516_arraylist, 83_bitoperations_array_methods,
493_enum_icomparable_object_comparer, 1236_roslyn_survey_members, 100_list, 547_bcl_members_for_xml) 8/8, conformance2
collections/004 and strings/003 2/2; Linux: the 130 globalization and number samples of the culture gate plus the 8 above, 272/272 at -O0 and -O2; wasm32 on the 12 of them it runs, 12/12. Windows itself is CI's to confirm.
…ded out, Win32 calls of ICU mode served) and WASI (root collation kept) (#424, #426)
…flips row, json/jsonwriter/regexoptions budgets lowered to measured (#423, #435)
…rs of structs (#423)

Rune, StringRuneEnumerator and SpanRuneEnumerator are served from the framework's IL
(upstream-types/System.Text.Rune.txt); corelib/Rune.cs (565 lines) is deleted. string.EnumerateRunes
and MemoryExtensions.EnumerateRunes stay corelib members that build the framework's enumerators
through their internal constructors, and string.CreateFromChar joins the contract members.

[UnsafeAccessor(UnsafeAccessorKind.Constructor)] now creates a struct too: the constructor runs on a
zeroed value whose address is `this` (sample 1406, recorded from .NET; not supported on main).
Sample 1407 covers Rune from IL; on main it stops at the explicit "not supported" for non-ASCII
numeric values and non-BMP categories, which the framework's Unicode data now answers.
…ds those callers (sample 1393); the inlined/tail-called framework frames by a walk from the outermost frame
…System.Native contract (#315)

File, Directory, FileInfo, DirectoryInfo, FileSystemInfo, FileStream, FileStreamOptions, Path, RandomAccess,
SafeFileHandle and the enumeration types come from CoreLib's IL, Interop and Interop.Sys included; their 57
libSystem.Native entry points are corelib/IO/SystemNative.cs, pal_io.c's behaviour function by function
(option (b); linking the pack's shim stays the owner's decision, docs/21-ownership.md). The File, FileStatus,
FileStream, Path, PathWindows and PathSpans shadows are deleted and Posix.cs keeps only what Process, Console,
Environment, FileSystemWatcher and MemoryMappedFiles use.

Security review in docs/il-first/System.IO.md: O_CLOEXEC and O_NOCTTY on every descriptor, modes through the
umask, rwx-only permission copy, symlink following as libSystem.Native, NUL and length handling, EINTR, buffer
bounds. Samples 1391 (the contract's surface) and 1393 (#436). Run time at parity with NativeAOT; against the
shadows 1.31x and 2.67x size (#438).
…er (#437)

GlobalizationMode.EnsureIcuLoaded checked s_mode without a guard, so threads
that reached their first culture-aware call at the same time (PLINQ workers in
552_plinq_attached_children) each ran GlobalizationNative_LoadICU. Each call
probes and reassigns the libicuuc/libicui18n handles while another thread binds
symbols from them, which aborted with "Cannot get symbol uenum_close_74 from
libicuuc". .NET loads ICU once, in GlobalizationMode.Settings' static
constructor; csharpc now does the same under a lock with a double check.

Sample 1420 releases 16 threads into their first culture-aware call together.
Before the fix (6121169): 1420 failed 14/30 at -O2, 552 failed 5/30 at -O0 and
9/30 at -O2. After: 1420 and 552 30/30 at -O0 and -O2, 44 culture/collation/PLINQ
neighbours at -O0/-O2, both via IL, size.sh green. 1420 is skipped on wasm32
(no threads there).
…y .NET formats them (#424, #426)

Sample 1342 regressed on every platform once the text flip made the slicer keep SR.Format's IL (a non-resource format text in
UTF32Encoding), so CoreLib's messages went through the corelib's string.Format again: a key implementing ISpanFormattable came out
through IFormattable.ToString, where .NET calls TryFormat.

- string.Format (ValueStringBuilder.AppendFormatHelper) gives an ISpanFormattable item, when nothing is padded before it, the space
  left in its buffer, once; when that is too small, or the item is right-aligned, it uses IFormattable.ToString. The capacity is
  observable, so the corelib's CharBuilder now grows exactly as ValueStringBuilder does (256 stack chars, EnsureCapacity(format
  length + 8 per argument), ArrayPool<char> power-of-two arrays), and string.Format's items go through Number.AppendCompositeItem.
- The SR.Format replacement (UpstreamSupport.FormatItems) and the collections' key messages (SR.Format with no provider, as
  CoreLib's ThrowHelper calls it) use the same buffer and rule; a key of known text (string, int, long, uint, ulong, char) is still
  spliced in, decided per instantiation so the other path is not compiled for those.
- Interpolation holes of a reference type or an IFormattable struct follow DefaultInterpolatedStringHandler.AppendFormatted<T>:
  TryFormat first (into a buffer grown until it fits), then IFormattable.ToString(null, null), then ToString(); they used ToString().
- A boxed built-in value's ISpanFormattable thunk (BoxedSpanFormattable.TryFormatOf) is its IFormattable text, so testing
  ISpanFormattable does not link a second formatter per boxed type. No type test on the boxed numbers in these paths: testing a
  boxed type refers to its method table, which compiles that type's interface thunks and formatters into the program.

Sample 1397 (string.Format with and without alignment, a provider, StringBuilder.AppendFormat, FormattableString, interpolation,
items that do not fit the space left at several format lengths, an IFormattable-only type, Dictionary key messages), recorded
from .NET, fails without the fix at -O0 and -O2.

Size: the net program grows because ISpanFormattable dispatch now reaches the TryFormat of the program's own ISpanFormattable types
(IPAddress, IPEndPoint, IPNetwork, UTF-8 too), as it does under .NET; linux-x86_64 net 272 760 -> 326 280 B and regexoptions
2 202 944 -> 2 207 808 B, budgets raised with that reason; every other program within 200 B of the branch (async 265 920, collections
251 360, numbers 231 736).

Local: build 0 errors; 1342, 1397 and 44 composite-format, interpolation, boxed-formattable and dictionary samples (184, 1252, 1404,
19, 370, 376, 420, 490, 547, 390, 1309, 864, 101, 843, 1307, 250, 700, 848 and every *interpol*/*tostring*/*formattable* sample)
46/46 at -O0 and -O2; conformance formatting/* 5/5 and conformance2 formatting/* 7/7; wasm32 18/18; via IL 8/8; tests/size.sh
12/12 with the two budgets above.
…polation call ISpanFormattable.TryFormat as .NET does (#424, #426)

# Conflicts:
#	tests/size/budgets/regexoptions.txt
The value of a call that returns by reference was loaded at the call's type as
written. Inside a generic type that type can still name the type parameter
(AsyncLocalImpl<T>.Value.set passes Unsafe.As<AsyncLocalValueChangedArgsImpl<T>,
AsyncLocalValueChangedArgs<T>>(ref args) to a delegate), so the corelib check,
which binds the corelib from source, failed on "open generic struct
AsyncLocalValueChangedArgsImpl<T>" once a flip made hello-world reach that
setter. Every other value load reads the type through the substitution
(TypeOf); this one now does too.

Evidence: the corelib check (CSHARPC_CHECK_CORELIB=1, tests/programs.sh) fails
before and passes after. Sample 1421 covers ref-returned generic struct values
as delegate arguments, locals and return values; it passes on both builds (user
code's generic structs reach the load substituted already; the corelib check is
the regression test for the shadow-mapped case). 110 generic/Unsafe.As/ref
samples plus 1230, 1391, 540, 1342, 1397, 1420 at -O0/-O2, 3 via IL,
tests/slicer.sh 10/10, tests/size.sh green.
… ref-return loads at the instantiation's type (#424)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: bcl Library surface (corelib) area: il-frontend IL front-end (program or libraries as assemblies) enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Serve the remaining framework library code from .NET 11 IL and delete csharpc's copies

1 participant