[jmcphers]

Fixes #16472

Summary

Opens SAS (.sas7bdat, .xpt), SPSS, Stata, ORC, OpenDocument spreadsheet, and common geospatial files from the Explorer pane straight into the Data Explorer, the same way .csv and .xlsx open today. No R or Python session is needed.

These formats are read by DuckDB extensions (mainly the community read_stat extension) that are downloaded the first time a matching file is opened. A new setting, dataExplorer.fileReaders, maps a file extension to the DuckDB extension that reads it, as [core|community/]extension[:function]. Positron ships defaults for the formats above.

Users can add their own readers in the Settings editor, or set a value to an empty string to turn one off.

How it works

• The editor resolver registers the Data Explorer for the built-in formats plus the setting's keys, and re-registers when the setting changes, so no reload is needed. Text formats that people also edit (.geojson, .kml, etc.) are only offered under "Open With..." so they don't shadow the text editor.
• The extension is installed with INSTALL ... FROM community (or core for spatial) into Positron's global storage, not ~/.duckdb. If the download takes more than a second, a cancellable progress notification appears. If it fails, the error is logged to a new "DuckDB" output channel and an error notification offers "Show Logs". Failures aren't cached, so reopening the file retries.
• GEOMETRY columns from spatial are converted to WKT text so the grid can show them.
• "Import Data" now has haven (R) and pyreadstat (Python) importers for .sas7bdat and .xpt.

Security, and why there are two DuckDB workers

The existing DuckDB worker runs with allow_unsigned_extensions on macOS so it can load our vendored, Apple-signed Excel extension. DuckDB only lets you set that at startup, so loading downloaded extensions into that worker would skip DuckDB's signature check. Instead, files matched by dataExplorer.fileReaders are opened in a second worker that keeps the signature check on and sets lock_configuration after startup, so later SQL can't turn the check off or move the extension directory. The existing worker is unchanged.

Having two workers is cheap:

• Workers start lazily on the first query. Users who never open one of these files never start the second worker.
• Each worker shuts itself down after two minutes with no open Data Explorers, independently of the other. So in practice there is at most one extra in-memory DuckDB process while a SAS/Stata/etc. file is open, and none after.
• Both workers reuse the same DuckDBInstance code for spawning, crash recovery, and idle shutdown. A crash in one only drops the views backed by that worker; the other keeps running.
• Each worker only loads the tables for its own files, so data isn't duplicated between them.

Admin control and offline use

• The setting is restricted and machine-scoped, so an untrusted workspace can't make Positron download extensions.
• Workbench admins can turn the feature off by enforcing "dataExplorer.fileReaders": {} through POSITRON_ENFORCED_SETTINGS.
• Downloads need network access, so in airgapped environments these formats fail with the download error (built-in formats like CSV and XSLX are unaffected).

Interaction with the Positron SAS extension

posit-dev/positron-sas-extension also opens .sas7bdat and .xpt files from the Explorer into the Data Explorer, but through a live SAS session. Its custom editors are registered with "priority": "default", which outranks this PR's builtin registration. For users who have it installed:

• Double-clicking a .sas7bdat or .xpt opens the SAS extension's viewer, which fails with "No active SAS session" if no session is running.
• Because both editors match, VS Code shows its "multiple default editors" prompt until the user picks one with workbench.editorAssociations.

There's no conflict for users without the SAS extension, and no overlap in Import Data (the SAS extension doesn't register a data importer).

The SAS extension still handles some things read_stat probably doesn't, such as PROC CPORT transport files, multi-member .xpt files, files from the SAS Content and Clinical Repository views, and SAS-side formats. So the two should coexist rather than one replacing the other. Suggested changes on the SAS extension side:

• Lower its custom editors to "priority": "option" (or make them the default only while a SAS session is running). A plain double-click would then use Positron's viewer, and "Open With..." would still offer the SAS viewer.
• Optionally, when no SAS session is running, fall back to Positron's viewer instead of failing.

Release Notes

New Features

• Open SAS (.sas7bdat, .xpt), SPSS, Stata, ORC, OpenDocument spreadsheet, and geospatial files directly in the Data Explorer. The DuckDB extension that reads each format is downloaded on first use and can be configured with the dataExplorer.fileReaders setting (Feature Request: Data Explorer support for .sas7bdat and .xpt files. #16472)
• Import Data can generate haven (R) and pyreadstat (Python) code for .sas7bdat and .xpt files (Feature Request: Data Explorer support for .sas7bdat and .xpt files. #16472)

Bug Fixes

• N/A

Validation Steps

@:data-explorer @:duck-db

New tests: fileReaders.vitest.ts (setting parsing), resolver tests in positronDataExplorerEditorResolution.vitest.ts, importer registry tests, haven/pyreadstat importer unit tests, and positron-duckdb extension host tests that open small .sas7bdat and .xpt files through the signed worker (these need network access).

1. Double-click a .sas7bdat or .xpt file in the Explorer pane. The first time, a "Downloading..." notification may appear briefly, then the data opens in the Data Explorer.
2. Open a second file of the same type. It should open without a download.
3. With an R or Python session running, use Import Data on the open file and check the generated haven or pyreadstat code.
4. Set "dataExplorer.fileReaders": {} in settings. .sas7bdat and .xpt should no longer open in the Data Explorer (no reload needed). .csv should still work.
5. Add a reader that doesn't exist, e.g. "foo": "not_a_real_ext", and open a .foo file. You should see an error notification with "Show Logs" rather than a hang.
6. Turn off your network and open a .dta file for the first time. You should get the same error notification.

[github-actions]

E2E Tests 🚀
This PR will run tests tagged with: @:critical @:data-explorer @:duck-db @:interpreter @:console @:ark

Why these tags?

Tag	Source
@:critical	Always runs (required)
@:data-explorer	PR description
@:duck-db	PR description
@:interpreter	Changed files
@:console	Changed files
@:ark	Changed files

More on automatic tags from changed files.

Test count: Linux: 408 tests across 3 shards.

^{readme  valid tags}