Skip to content

feat: add reader-configured filtering and parsed-value reuse - #399

Merged
Seddryck merged 2 commits into
mainfrom
feat/reader-filtering
Nov 2, 2026
Merged

Seddryck merged 2 commits into
mainfrom
feat/reader-filtering

Conversation

@Seddryck

@Seddryck Seddryck commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Rows can now be filtered on the reader before object construction or output-row materialization. Add named/ordinal equality, sanitized-span, and typed cell conditions, plus read-only multi-field row predicates, through the shared sync/async pipeline for every supported format and CSV batches.

Typed conversions are cached lazily per current row, keyed by ordinal, result type, and effective parser. Filters, getters, and default CSV constructor projection reuse compatible conversions; explicit span-parser overrides remain independent. Document null/error/lifetime rules, source numbering, and allocation tradeoffs.

close #397

Validation:

  • dotnet build PocketCsvReader.full.sln --no-restore: passed on net8.0/net9.0/net10.0 (existing test nullable and Profiler dependency/entry-point warnings).
  • dotnet format PocketCsvReader.full.sln analyzers --verify-no-changes --no-restore: passed via the pre-push hook.
  • Core tests on all three frameworks: filter FullyQualifiedName~ReaderFilteringTest|FullyQualifiedName~SpanParser|FullyQualifiedName~SpanCustomParser|FullyQualifiedName~AsyncTypedRecordStreaming|FullyQualifiedName~CsvObjectReaderTest|FullyQualifiedName~RowNumberTest|FullyQualifiedName~ObjectArrayRowsTest|FullyQualifiedName~DynamicRowsTest|FullyQualifiedName~CsvBatchDataReaderTest|FullyQualifiedName~AsyncDataReaderTest|FullyQualifiedName~DefaultParserTests|FullyQualifiedName~CsvArrayStringTest (216/framework), followed by rerunning ReaderFilteringTest after adding the custom-mapper/legacy-array check (21/framework): passed.
  • NDJSON, JSON, fixed-width, key-value, web-log, and security-log test projects on all three frameworks: filter FullyQualifiedName~ReaderFilteringTest|FullyQualifiedName~ParsedRowsTest|FullyQualifiedName~RowShapeParserCachingTest|FullyQualifiedName~LazyJsonDecodingTests|FullyQualifiedName~SecurityLogReaderTests (1002 executions): passed. NDJSON filtering rerun after adding nullable stable-shape coverage (3/framework): passed.
  • Release/net10.0 --reader-filtering-matrix: passed; 50,000 rows, span filtering 0.24 allocated bytes/source row versus 0.22 without filters. Typed filter/projection used 50,000 parser calls versus 100,000 manually, at 24.26 bytes/row from boxed values. For this cheap integer parser, cache overhead made the configured case slower (56.52 ms versus 37.84 ms); this is a smoke measurement, not a controlled throughput benchmark.
  • Full test suites and the full BenchmarkDotNet matrix were not run. An initial filtered solution-wide test invocation also launched the Profiler executable and failed on its relative resource path; validation was rerun successfully against the test projects explicitly.

Documentation alignment and rebase:

  • Rebased onto main at a6c4542 (documentation reorganization).
  • Moved filtering into docs/_docs/reader-filtering.md, added it under Reading data, and linked it from profile, reader, output, validation, source-position, and batch guides.
  • Moved filtering measurements into the performance guide and clarified generated row-number gaps after filtering.
  • Kept main's concise README and added the published filtering-guide link.
  • Validation after rebase: 8 documentation-validator unit tests; source/navigation/link checks; all 23 compiled documentation examples on .NET 10 Release; full solution build; 102 ReaderFilteringTest executions across seven projects on .NET 8/9/10. All passed.
  • Full test suites and benchmark matrix were not rerun for this documentation update. Jekyll rendering and rendered-route/anchor checks were not run locally because Ruby/Bundler is unavailable; the documentation CI workflow provides those checks.

@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a9f24ea6-4512-423f-8fb2-bcd684e003fa
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@Seddryck
Seddryck force-pushed the feat/reader-filtering branch from 4823eef to cf929f1 Compare November 2, 2026 20:18
@Seddryck
Seddryck force-pushed the feat/reader-filtering branch from cf929f1 to e8c88e7 Compare November 2, 2026 20:28
@Seddryck
Seddryck merged commit 183299f into main Nov 2, 2026
5 of 7 checks passed
@Seddryck
Seddryck deleted the feat/reader-filtering branch November 2, 2026 20:28
@sonarqubecloud

sonarqubecloud Bot commented Nov 2, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add reader-configured filtering with span, parsed-value, and row predicates

1 participant