[jmcphers]

Addresses #3302

Summary

This PR adds a search box to the Data Explorer that filters the table to rows where any column matches the search term. It works with R and Python data frames, and with CSV, TSV, Parquet, and Excel files opened in the file viewer. Matching cells are highlighted, so you can see why a row is shown.

This is the Positron half of the feature. It includes the protocol, the frontend, the Python backend, and the DuckDB file viewer, and it bumps ark to pick up the R backend in posit-dev/ark#1463. See that PR for the R matching details.

How it works

The search is sent to the backend as a new row filter type, global_search, that has no column. Because it goes through the existing row filter path, it stacks with column filters, is reflected in summary stats, and is re-applied when the data changes. The search always runs after the column filters, so it only looks at rows that passed them. The search is never included in Convert to Code.

Matching rules

• Text and factor columns match a case-insensitive substring.
• Numeric columns match by the precision of the term, using the same rounding the grid displays. So 3 matches 3.7, and 3.1 matches 3.14159 but not 3. Grouped commas (1,234), a leading dot (.5), and signs are accepted. Note that this behavior differs from RStudio, which treats numbers as text.
• Dates, times, and other types match on their displayed text.
• Missing values match when the term is a null marker such as NA, null, or None.
• Row names and string row labels are searched too.
• List and nested columns are skipped.

Handling numerics

RStudio's search basically converts every column to text and then searches the text, which I've always thought in retrospect was pretty dumb for numeric values; if you search for "5" you are probably not looking for "153". As noted above, we now match numbers with as much precision as you specify, so "5" finds all numbers that are 5. If you type a more specific number like 5.1, you get a more precise result.

Numbers still match inside text columns, so you can do numeric & text searches at the same time. This feature is designed to be more about low-friction, fast searching than designing a careful search.

Large tables

To keep a search from blocking the kernel for too long, R and Python only scan up to 10 million cells by default (the dataExplorer.search.maxCells setting). When a search hits the limit, the status bar says so and offers a Search All button. The DuckDB file viewer ignores the limit, since it searches off the extension host thread.

UI

• The search box is at the right end of the row filter bar. Press Cmd/Ctrl+F in the grid to focus it. Escape clears it.
• After typing, the search waits briefly before it runs.
• When nothing matches, an empty state offers to clear the search, or to search all rows if the search was limited.
• Clear Filters does not clear the search.

Compatibility

All new protocol fields are optional. The search box only appears when the backend lists global_search as a supported filter type, so older backends and other data sources (for example, the database connections and third-party kernels) are unaffected and do not show it. Backends that do not return per-cell matches fall back to highlighting done in the frontend.

Release Notes

New Features

• The Data Explorer can now search for text and numbers across all columns (Data Explorer: Search across all columns for keywords #3302)

Bug Fixes

• N/A

Validation Steps

@:data-explorer @:duck-db @:web @:win

New e2e tests are in test/e2e/tests/data-explorer/data-explorer-search.test.ts.

Python:

import pandas as pd
import numpy as np
df = pd.DataFrame({
    "name": ["Alice", "Bob", "Carol", None],
    "score": [3.14159, 3.7, 42.0, np.nan],
    "when": pd.to_datetime(["2024-01-01", "2024-06-15", None, "2025-03-03"]),
})

R:

df <- data.frame(name = c("Alice", "Bob", NA), score = c(3.14159, 3.7, 42), when = as.Date(c("2024-01-01", NA, "2025-03-03")))
View(df)

1. Open the data frame in the Data Explorer and type al in the search box. Rows with "Alice" and "Carol" remain and the matching cells are highlighted.
2. Search 3, then 3.1, and check that numeric precision works as described above.
3. Search NA (or None in Python) and check that rows with missing values are shown.
4. Add a column filter and check that the search stacks with it and the row count in the status bar is correct.
5. Check that Convert to Code ignores the search.
6. Open a CSV or Parquet file and repeat the text and number searches.
7. Open a large data frame (for example, 2 million rows by 10 columns) and search for something rare. Check that the status bar shows the limit notice and that Search All searches every row.

[github-actions]

E2E Tests 🚀
This PR will run tests tagged with: @:critical @:data-explorer @:duck-db @:web @:win @:ark @:console @:interpreter

Why these tags?

Tag	Source
@:critical	Always runs (required)
@:data-explorer	PR description
@:duck-db	PR description
@:web	PR description
@:win	PR description
@:ark	Changed files
@:console	Changed files
@:interpreter	Changed files

More on automatic tags from changed files.

Test count: Linux: 411 tests across 3 shards, Windows: 292 tests across 2 shards, Web: 331 tests across 2 shards.

^{readme  valid tags}

[jmcphers]

/explore drive this around a bit and see if you can find anything broken, awkward, or missing

[github-actions]

🔎 Exploratory testing dedea1c

Focus: drive this around a bit and see if you can find anything broken, awkward, or missing

1 moderate · 1 minor
View report →

[jmcphers]

From the moderate finding:

Convert to Code leaves out the active search, so the code returns rows the grid hides

This is intentional; the code necessary to include the search would be prohibitively complicated since we match different values in different column types and would need to explicitly list each column and all of the matching we do, string conversion, etc. to get it to a state where we can apply the search.