You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 2d76874
Browse filesBrowse the repository at this point in the historyBrowse files
System.Net.Primitives from the framework's IL; delete the IPAddress/IPNetwork shadows
IPAddress, IPEndPoint, EndPoint and IPNetwork now come from System.Net.Primitives'
own IL (ADR 0023, Phase 1 of ADR 0025); corelib/Net/IPAddress.cs and IPNetwork.cs
(633 lines) are deleted. SslClientHelloInfo, a System.Net.Security type that lived
in IPNetwork.cs, moves to Net/SslStream.cs.
The shadow's hand-written parser was silently wrong on IPv6 scope ids: "fe80::1%",
"fe80::1%nosuchif0", "fe80::1%2%3", "fe80::1%4294967296" and "fe80::1%-1" are valid
addresses with scope 0 in .NET and were rejected. It also lacked the span and UTF-8
Parse/TryParse, TryFormat and EndPoint.Serialize/Create.
What stays csharpc's is the socket engine's contract (ADR 0024 boundary A):
SocketException (native error and text from the OS layer) and the two internal
classes of the assembly that P/Invoke libSystem.Native, SocketAddressPal (struct
sockaddr, the same layout NetNative uses, per target OS) and InterfaceInfoPal
(if_nametoindex), in the new corelib/Net/SocketAddressPal.cs. Every sockaddr access
is bounded by the buffer length, as libSystem.Native's EFAULT checks are.
General pieces the IL needed:
- UTF-8 Parse/TryParse for short, sbyte, byte, ushort, uint and ulong;
- a boxed Int128/UInt128's ISpanFormattable slot formats only that type instead of
going through the switch over every primitive (decimal, DateTime and the
calendars): tests/size/net.cs 393 536 to 255 976 B, json.cs 1 572 320 to
1 559 040 B;
- a program that names any System.Net namespace takes System.Net.Primitives, since
the corelib's Socket, Dns and TLS shadows take and return its types.
(The branch's earlier commits brought MemoryExtensions.TryWriteInterpolatedStringHandler
from the CoreLib slice, which IPNetwork.TryFormat uses, and Vector128 array/span forms.)
Numbers (linux-x86_64, -O2 stripped): tests/size/net.cs 171 480 (shadow) to 255 976 B,
NativeAOT 1 365 832; IPAddress.Parse/ToString loop (2 M, median of 7) shadow 336 ms,
IL 237 ms, NativeAOT 295 ms; compile of net.cs 4.8 to 6.3 s user. New budget
linux-x86_64 net 264192 (measured + max(4 KiB, 3 %)).
Samples 1316 (parsing corner cases) and 1317 (IPEndPoint.Serialize/Create,
SocketAddress) are recorded from real .NET; the 46 networking samples pass at -O0
and -O2.
Copy file name to clipboardExpand all lines: docs/16-interop-io.md
+5-4Lines changed: 5 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -232,12 +232,13 @@ to a method expecting one) is the corelib object and dispatches virtually.
232
232
## System.Net (sockets)
233
233
234
234
`src/Csharpc/corelib/Net/` implements the basic networking types in C# over the OS's BSD socket API
235
-
(libc on Linux and macOS, Winsock/`ws2_32` on Windows): `IPAddress`/`IPEndPoint` (parsing and
236
-
formatting follow .NET: inet_aton-style IPv4 input such as `127.1` or `0x7f.1`, RFC 5952 IPv6
237
-
output with embedded IPv4 for mapped addresses), `Socket` (TCP and UDP; `new
235
+
(libc on Linux and macOS, Winsock/`ws2_32` on Windows): `Socket` (TCP and UDP; `new
238
236
Socket(SocketType, ProtocolType)` and `new TcpClient()` are dual-mode IPv6 sockets, so IPv4 peers
239
237
appear as `::ffff:a.b.c.d`, as in .NET), `NetworkStream`, `TcpListener`, `TcpClient`, `UdpClient`
240
-
and `Dns.GetHostAddresses`/`GetHostName` (getaddrinfo). Constants, `sockaddr` layouts (macOS has a
238
+
and `Dns.GetHostAddresses`/`GetHostName` (getaddrinfo). `IPAddress`, `IPEndPoint`, `IPNetwork` and `SocketAddress`
239
+
are System.Net.Primitives' own code, compiled from the framework's IL (ADR 0023, #326); its two internal classes that
240
+
call libSystem.Native are served by the corelib over the same `sockaddr` layout (`corelib/Net/SocketAddressPal.cs`:
241
+
`SocketAddressPal`, and `InterfaceInfoPal`, whose `if_nametoindex` resolves a scope such as `fe80::1%eth0`). Constants, `sockaddr` layouts (macOS has a
241
242
length byte first) and errno values are per target OS. Errors are `SocketException`s with .NET's
242
243
`SocketErrorCode` (Winsock numbering, mapped from errno), the native error as `ErrorCode`/
243
244
`NativeErrorCode`, the OS's message text, and — for failed connects — the endpoint appended, as .NET
Copy file name to clipboardExpand all lines: docs/21-ownership.md
+2-1Lines changed: 2 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,8 @@ System.Private.CoreLib. Only the code that is necessary stays.
9
9
10
10
> The tables below are the snapshot taken before the Regex flip (#206) and the System.Text.Json flip (#286): the Regex and
11
11
> Json shadows (the `Regex` and `Json` rows; `Json/ArrayBufferWriter.cs` went with System.Memory, #306) are deleted and `ownership.tsv` no longer lists them, nor the Linq ones (`Enumerable.cs`, `LinqIterators.cs`: #267;
12
-
> `LinqSpans.cs` is now `RuntimeSpans.cs`); the totals are regenerated by `tools/il-first/ownership.sh` on the next refresh.
12
+
> `LinqSpans.cs` is now `RuntimeSpans.cs`), nor `Net/IPAddress.cs` and `Net/IPNetwork.cs` (System.Net.Primitives from IL, #326; the
13
+
> socket engine's sockaddr shims are the new `Net/SocketAddressPal.cs`, GLUE); the totals are regenerated by `tools/il-first/ownership.sh` on the next refresh.
13
14
14
15
This page is the map from today's `src/Csharpc/corelib` (354 files, 169,705 lines, 8.1 MB) to that principle. The
15
16
per-file data is [`tools/il-first/ownership.tsv`](../tools/il-first/ownership.tsv); regenerate it, or print the tables
Copy file name to clipboardExpand all lines: docs/adr/0023-framework-il-first.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -100,6 +100,7 @@ samples are tiny and start-up dominated.
100
100
| `System.Collections.Generic.Dictionary<TKey,TValue>`, `HashSet<T>` and `System.Runtime.InteropServices.CollectionsMarshal` (CoreLib types, #274) | the 116 samples that mention them (`Dictionary`, `HashSet`, `ToLookup`, `GroupBy`, `Distinct`, JSON, Linq, Roslyn library members...) pass at -O0 and -O2, 36 Int128/Vector128/BigInteger samples too, `tests/slicer.sh` (11 slice runs) passes; three new samples recorded from real .NET for what the flip exposed (1305: boxed `Index`/`Range` as keys and `object`, an `Int128` in a boxed struct and a `Dictionary<Int128,V>` enumerator boxed through `IEnumerable`, `Comparer<Int128>.Default` which threw `InvalidOperationException`; 1306: `Comparer` of string collections and `ReadOnlySpan<char>` alternate lookups; 1307: the missing-key and duplicate-key messages for 18 key types), found along the way: #277 (SortedDictionary duplicate-key message) | **+5 % to +14 % on programs that format a number**, because the corelib's culture caches are `Dictionary<string, ...>` and now carry the real `Resize`/`Remove`/enumerator code and tables: linux-x86_64 async 156 544 to 172 144 B (+10.0 %), collections 143 448 to 163 032 (+13.7 %), linq 174 600 to 189 864 (+8.7 %), numbers 129 096 to 143 912 (+11.5 %), regex 825 600 to 870 304 (+5.4 %), regexoptions 1 949 520 to 2 093 104 (+7.4 %); hello, exceptions, typeinfo unchanged. A 4-collection program (`Dictionary<string,int>`, `HashSet<int>`, `HashSet<string>`, `Dictionary<int,int>`) 145 056 to 155 744 B (+7.4 %). Without the two profile replacements below it was +46 % (212 368 B) | collections 3.92 to 4.08 s (+4 %), linq 4.59 to 4.95 s (+8 %) CPU | instructions, same program, work beyond start-up: `Dictionary<string,int>` set/get/remove 33.5 M to 33.9 M (**+1.4 %**), `HashSet<int>` add/contains 69.2 M to 67.4 M (-2.5 %), `HashSet<string>` 28.1 M to 31.0 M (**+10 %**), `Dictionary<int,int>` 93.7 M to 84.3 M (**-10 %**); the spike before the type-guarded default comparer (#269) was 21 to 31 % behind | 1 425 deleted and 71 added in `corelib` (`HashSet.cs` and `CollectionsMarshal.cs` deleted, `DictionaryImpl` out of `Dictionary.cs`; added: contract members in `UpstreamSupport.cs`), against 1 431 deleted and 218 added over `src` and `tools` (the compiler side: boxed `Index`/`Range` method tables, 8-aligned `Int128` aggregates, `Comparer<Int128>`, IL folding of `typeof(A) == typeof(B)`, generic forwarding in the slicer) | **Flipped**. Two things the flip decides: (1) the string comparer family stays redirected to the corelib, with a wrapper (`UpstreamWrappedDefaultStringComparer`) for the default comparer so alternate lookups and `Comparer` behave as .NET's; the randomized comparer upstream switches to after many collisions (hash-flooding mitigation) is not there, as it was not there for the shadows: not weaker than before, first thing to add when the string comparers are flipped. (2) The throw helpers that format the key with `string.Format` are replaced (`csharpc` profile) by direct text for `string`, `int`, `long`, `uint`, `ulong` and `char` keys and `ToString`/`IFormattable` for the rest, so a program reading a Dictionary does not carry composite formatting and every number formatter. The size cost is the owner's call under the standing authorisation; a follow-up could replace the culture caches' Dictionaries with something smaller. |
101
101
| `System.Text.StringBuilder` and its chunk enumerator (CoreLib type, #304) | all 69 samples that mention StringBuilder, plus the record, interpolation, format, tuple and regex samples (244 at -O0 and -O2) pass; sample 1309 (recorded from real .NET) for the generic shapes the framework's handler uses on boxed values; found along the way: #305 (interface tests on boxed Nullable, enums and Half in generic code) | **neutral overall**, smaller where StringBuilder is used by the corelib: linux-x86_64 regexoptions -11.5 KB (-0.5 %), regex -3.5 KB, collections -3.8 KB, async/linq/numbers +0.3 KB; hello, exceptions, typeinfo unchanged. **A program that appends numbers and interpolations to a StringBuilder grows**: the benchmark program 140 000 to 209 280 B (+49 %, the real handler's generic formatting) | linq 6.84 to 7.11 s CPU (+4 %) | instructions per operation, 100 000 to 300 000 operations: `Append(char)` equal, `Insert` equal, `ToString` +8 %, `Append(string)` **+99 %** (27 to 53 per call: `Buffer.Memmove` is a call to libc for short strings, the shadow copied inline), `Append(int)` +46 %, `AppendLine` +88 %, `Append($"...")` **+75 %** (10.6 M to 18.8 M; was 10x slower until three IL folds: `box T; isinst I` and `castclass I` on a known T are constants, `box T; callvirt I.M` is T's own method, `typeof(T).IsEnum/IsValueType` prune branches) | 1 498 deleted and 44 added in `corelib` (`StringBuilder.cs` out; `StringBuilderRuntime.cs` and a `Rune.AsSpan` contract member in) | **Flipped**. Security: bounds, capacity and `ArgumentOutOfRangeException` checks of `Append`/`Insert`/`ToString` are now upstream's own code, not a port; `Buffer.Memmove<T>`'s element count is `checked` (no silent truncation). The speed and size costs of appending strings and numbers are the price; the compiler's own interpolation lowering (the shadow's fast path) cannot stand beside the real handler, closing the gap is a follow-up. |
102
102
| System.Memory (`System.Memory`: `ArrayBufferWriter<T>`, `ReadOnlySequence<T>`, `SequenceReader<T>`, `SequencePosition`, `MemoryPool<T>`, `EncodingExtensions`) | 14 of 14 samples that use these types byte-identical to real .NET at -O0, -O2 and `--via-il` (1051, 1250 both, 1251, 1266, 1281, 310, 483, 505, 506, 834, 835, 846, 849); the ownership map's `Memory<T>` blocker (csharpc's `Memory<T>` does not record its array) did not show in any of them. System.Memory defines types and forwards others to CoreLib: it takes the renamed reference twin of #287 (`TwinReference.SupportedAssemblies`), nothing System.Memory-specific in the compiler | an `ArrayBufferWriter` program (1281) 136 376 to 267 264 B (1.96x), a `ReadOnlySequence`/`SequenceReader` program (505) 247 040 to 329 736 B (1.33x), the JSON writer sample (310) 649 736 to 723 184 B (1.11x), a `MemoryPool` program (483) 193 608 to 165 016 B (**0.85x**); `tests/size`: regex 875 664 to 894 096 B and regexoptions 2 097 888 to 2 130 616 B (both within their budgets), the other size programs unchanged | compile cpu 4.1 to 6.7 s (1281), 7.5 to 8.6 s (505), 4.4 to 4.0 s (483), 15.8 to 17.5 s (310) | not measured: no sample has a hot path in these types | 3 146 (`Buffers/ReadOnlySequence.cs` 2 931, `MemoryPool.cs` 87, `Json/ArrayBufferWriter.cs` 88, the `EncodingExtensions` members of `Text/Encoding.cs` 40); `git diff --shortstat`: 1 insertion, 3 146 deletions over 5 files in `src` | **Flipped (#306)**. Security: the framework's own bounds and overflow checks replace the ports' (`ArrayBufferWriter`'s growth, which the port once got wrong (#194), `SequenceReader`'s advance and rewind, `ReadOnlySequence`'s slicing across segments, `EncodingExtensions.GetString` decoding a character split across segments); no native code, no unsafe code of csharpc's. |
103
+
| System.Net.Primitives (`IPAddress`, `IPEndPoint`, `EndPoint`, `IPNetwork`; the rest of the assembly was IL already, ADR 0013) | the 46 networking samples (IPAddress/IPEndPoint/IPNetwork, Dns, Socket, HttpClient, SslStream, cookies, credentials) pass at -O0 and -O2 (92 runs), nine of them also with `--via-il`; two new samples recorded from real .NET: 1316 (parsing of untrusted text: inet_aton limits, IPv6 compression and embedded IPv4, scope ids, brackets and ports, the span and UTF-8 overloads, `TryFormat`) and 1317 (`IPEndPoint.Serialize/Create`, `SocketAddress`). **The shadow was silently wrong** on scope ids: `fe80::1%`, `fe80::1%nosuchif0`, `fe80::1%2%3`, `fe80::1%4294967296` and `fe80::1%-1` are valid addresses with scope 0 in .NET and were rejected; it also lacked the span/UTF-8 `Parse`/`TryParse`, `TryFormat` and `Serialize`/`Create` (compile errors). `SocketException` stays a shadow (the socket engine's native error and text), and the two internal classes that P/Invoke libSystem.Native, `SocketAddressPal` and `InterfaceInfoPal`, are a 196-line contract over the socket engine's `sockaddr` layout and `if_nametoindex` (`corelib/Net/SocketAddressPal.cs`) | `tests/size/net.cs` 171 480 to 255 976 B (1.49x; **NativeAOT 1 365 832**), the speed program 148 872 to 246 104 (NativeAOT 1 357 608), sample 195 182 440 to 267 208, 615 175 608 to 259 640: the IL's generic parser and formatter over `char` and `byte`, `UInt128` in `IPNetwork` and the number formatting they reach. Before a general fix in this PR it was 393 536: a boxed `UInt128`'s `ISpanFormattable` slot went through the formatter switch of every primitive (decimal, DateTime with the calendars); now it formats only `UInt128` (`tests/size/json.cs` 1 572 320 to 1 559 040 from the same fix). The other `tests/size` programs are unchanged | `net.cs` 4.8 to 6.3 s user (wall 4.6 to 5.4; NativeAOT publish 9.9 s user, 13.4 wall) | `IPAddress.Parse` + `ToString` + `GetAddressBytes`, 2 M iterations over 8 addresses, median of 7: shadow 336 ms, IL **237 ms (0.71x)**, NativeAOT 295 ms; the checksum equals real .NET | 633 (`Net/IPAddress.cs` 509, `Net/IPNetwork.cs` 124; `SslClientHelloInfo`, a System.Net.Security type, moved to `Net/SslStream.cs`); 196 added for the sockaddr contract | **Flipped (#326)**. Security: the hand-written parsers of untrusted text (`IPAddress`, `IPEndPoint`, `IPNetwork`) are replaced by the framework's own `IPAddressParser`/`IPv4AddressHelper`/`IPv6AddressHelper`, which are fuzzed upstream and bound every component (sample 1316 pins the limits); cookie parsing was already the framework's IL. The new contract code bounds every `sockaddr` access by the buffer length (`ArgumentOutOfRangeException("socketAddressSize")`, libSystem.Native's EFAULT) and passes `if_nametoindex` a NUL-terminated copy. General pieces added on the way: `MemoryExtensions.TryWriteInterpolatedStringHandler` from the CoreLib slice (nested types of a type outside the slice are hoisted), UTF-8 `Parse`/`TryParse` for the remaining integer types, `Vector128` array and span `Create`/`CopyTo`, a program that names any System.Net namespace takes System.Net.Primitives (the corelib's Socket and Dns shadows return its types). |
103
104
104
105
**Assemblies that define types and also forward CoreLib types (System.Collections, System.Collections.Concurrent)** needed one more piece of the IL
105
106
front-end (#275). The implementation forwards `List<T>`, `Queue<T>`, `ConcurrentQueue<T>`... to System.Private.CoreLib, and binding against it in place of
0 commit comments