Skip to content

Bump scouter - #393

Merged
thorrester merged 28 commits into
mainfrom
bump-scouter
Apr 29, 2026
Merged

thorrester merged 28 commits into
mainfrom
bump-scouter

Conversation

@thorrester

@thorrester thorrester commented Apr 28, 2026 •

Copy link
Copy Markdown
Member

Pull Request

Short Summary

Promotes scouter-client from the genai-semantics feature branch to main, and ships everything that depended on that promotion: new GenAI and observability proxy routes in the Rust server, a rebuilt OTel-compliant tracing layer in Python, a new ServiceMapMiddleware for recording service-to-service connections, and a full faceted filter sidebar for the trace UI.

Context

The genai-semantics branch of scouter has been merged to main. The dependency pin in Cargo.toml is now dead weight — this PR removes it. Everything else follows from that unlock.

Rust — GenAI proxy routes
A new crates/opsml_server/src/core/scouter/genai/route.rs adds ten-plus endpoints that proxy Scouter's GenAI analytics surface: agent dashboards, model usage, token metrics, error breakdowns, operation breakdowns, tool activity, conversation lookup, and span filtering. The shared post_proxy / get_proxy helpers in that file handle the AssertUnwindSafe + catch_unwind pattern for the Scouter client calls. A parallel observability/route.rs does the same for the infra-level observability metrics, spans, and traces that were previously only reachable by going to Scouter directly.

Rust — trace route cleanup
The existing trace handlers were missing AuditContext extensions and had copy-pasted error strings ("Failed to acknowledge drift alerts" in the trace handler — that was a bug). Both are fixed. A new get_trace_spans_from_filters endpoint and a facet endpoint round out the trace surface.

Rust — profile split
profiles BFF logic is split from drift-profile hydration into separate handlers so each endpoint has a single responsibility.

Python — tracing overhaul
tracing/__init__.py gains ScouterSpan — an OTel-compliant wrapper around the Rust ActiveSpan — and ScouterTracingMiddleware for ASGI apps. _capture_arguments introspects function signatures to record call arguments as span attributes. OTel import fallbacks are restructured so the module degrades cleanly when opentelemetry is absent.

Python — ServiceMapMiddleware
New ASGI middleware at opsml/scouter/service_map/middleware.py that intercepts HTTP requests and responses, captures source_service → destination_service + endpoint + method + status + latency_ms + trace_id, and writes ServiceConnectionRecord rows to Bifrost. Schema inference (infer_schema) and trace ID extraction (extract_trace_id) are delegated to Rust via PyO3.

Python — deps
Adds opentelemetry as an optional dependency group (opentelemetry-sdk, opentelemetry-exporter-otlp-proto-common, opentelemetry-instrumentation). The genai extras group drops duplicate OTel pins and adds crewai, openai-agents, openinference-instrumentation-crewai, and opentelemetry-instrumentation-openai-agents-v2.

Agent examples
Removes music_recommender, recipe_generator, and single_agent examples (all ADK-specific, now stale). Replaces them with three parallel examples (crewai/, openai/, google/) sharing a common setup.py, scenarios.jsonl, and a top-level run.py that dispatches by framework.

UI — faceted trace filtering
The trace dashboard gets a full sidebar filter system: FacetSidebar, FacetSection, FacetRow, AttributeFacet, DurationFacet, ChipBar, AddFilterMenu, and filterState.svelte.ts (Svelte 5 runes-based). Filters accumulate as chips and drive the TraceTable query. TraceDashboard.svelte and TraceTable.svelte are substantially rewritten to wire this up.

UI — agent task view
TaskDetailView.svelte is rebuilt (343 lines changed). AgentAssertionPill.svelte is new — renders assertion pass/fail status with a consistent pill design. ComparisonView.svelte is updated to match.

UI — mock data mode
A dev-only mock mode (mockMode.svelte.ts, devMock.ts, src/lib/server/mock/mode.ts) lets the UI render with static fixtures without a live backend, useful during rapid UI iteration.

UI — new BFF endpoints
Ten new +server.ts files under src/routes/api/scouter/genai/ proxy the new Rust GenAI routes to the frontend.

Tests
Rust: tests/api/scouter/genai.rs (258 lines), observability.rs (91 lines), updated trace.rs and profile.rs. UI: Vitest tests for filterState, task.ts assertions, charts.ts, facets.ts, trace/utils.ts, agent inference helpers, and user schema validation.

File Change
Cargo.toml scouter-client branch: genai-semantics → main
crates/opsml_server/src/core/scouter/genai/route.rs New — 10+ GenAI proxy endpoints
crates/opsml_server/src/core/scouter/observability/route.rs New — observability metrics/spans/trace proxy
crates/opsml_server/src/core/scouter/trace/route.rs Added AuditContext, new spans + facet endpoints, fixed error strings
crates/opsml_server/src/core/scouter/profile/route.rs Split profiles BFF from hydration
crates/opsml_server/tests/api/scouter/genai.rs New integration tests
crates/opsml_server/tests/api/scouter/observability.rs New integration tests
py-opsml/python/opsml/scouter/tracing/__init__.py ScouterSpan, ScouterTracingMiddleware, _capture_arguments
py-opsml/python/opsml/scouter/service_map/middleware.py New — ServiceMapMiddleware + ServiceConnectionRecord
py-opsml/python/opsml/scouter/trace.py New — trace context helpers
py-opsml/pyproject.toml opentelemetry optional group; genai extras updated
py-opsml/examples/genai/agent/ Replaced per-framework examples with unified crewai/openai/google/ structure
crates/opsml_server/opsml_ui/src/lib/components/trace/filters/ New — full faceted filter sidebar (8 components + state)
crates/opsml_server/opsml_ui/src/lib/components/trace/TraceDashboard.svelte Major rewrite to wire filter sidebar
crates/opsml_server/opsml_ui/src/lib/components/scouter/agent/task/TaskDetailView.svelte Rebuilt agent task detail layout
crates/opsml_server/opsml_ui/src/lib/components/scouter/agent/task/AgentAssertionPill.svelte New — assertion pill component
crates/opsml_server/opsml_ui/src/routes/api/scouter/genai/ 10 new BFF +server.ts endpoints
crates/opsml_server/opsml_ui/src/lib/components/settings/mockMode.svelte.ts New — dev mock mode

Is this a Breaking Change?

No. Existing Python imports (opsml.scouter.tracing, opsml.scouter.drift, opsml.scouter.evaluate) are unchanged — new exports are additive. The Rust API surface adds new routes; no existing routes are removed or have their request/response contracts changed. The scouter-client branch switch is internal to the server binary and is not visible to callers.

Separate concerns: GET /api/scouter/profiles lists profiles, POST /api/scouter/profiles/hydrate runs drift hydration. Routes.ts: removed PROFILE_GET duplicate, moved MONITORING_PROFILES to /hydrate path.
Rename setup_agent_layout to setupAgentLayout; replace bg-white/text-gray/border-gray/backdrop-blur with theme tokens in TraceTable; add matchesTags to mock filteredItems in CardSearch.
Covers: correct URL, cursor field stripping, non-ok response error, body error propagation.
Covers getAssertion (all variants + invalid throws), isTraceAssertionValue, isAgentAssertionValue, AgentEvalProfileHelper (all 4 getters, missing tasks.agent/trace graceful handling).
…R-9)

Replace Matcher::Any with typed UrlEncoded matchers; assert body fields on success/error. Fix duplicate rand dep in opsml-server dev-dependencies (rand = 0.* conflicted with workspace rand = 0.9). Use serde_json::Value for facets handler return until TraceFacetsResponse lands on stable scouter branch.
@github-actions

github-actions Bot commented Apr 28, 2026 •

Copy link
Copy Markdown

Dependency Review

The following issues were found:
  • ❌ 1 vulnerable package(s)
  • ✅ 0 package(s) with incompatible licenses
  • ✅ 0 package(s) with invalid SPDX license definitions
  • ⚠️ 20 package(s) with unknown licenses.
See the Details below.

Vulnerabilities

py-opsml/uv.lock

NameVersionVulnerabilitySeverity
litellm1.67.1LiteLLM: Authentication bypass via OIDC userinfo cache key collisioncritical
LiteLLM: Privilege escalation via unrestricted proxy configuration endpointhigh
LiteLLM: Password hash exposure and pass-the-hash authentication bypasshigh
Only included vulnerabilities with severity high or higher.

License Issues

py-opsml/uv.lock

PackageVersionLicenseIssue Type
auth0-python5.3.0NullUnknown License
build1.4.4NullUnknown License
chromadb1.5.8NullUnknown License
google-adk1.31.1NullUnknown License
google-genai1.73.1NullUnknown License
griffelib2.0.2NullUnknown License
importlib-resources7.1.0NullUnknown License
instructor1.15.1NullUnknown License
ipython8.39.0NullUnknown License
json-repair0.59.4NullUnknown License
json50.14.0NullUnknown License
openai2.32.0NullUnknown License
openai-agents0.14.5NullUnknown License
openinference-instrumentation0.1.47NullUnknown License
openinference-instrumentation-crewai1.1.3NullUnknown License
openinference-semantic-conventions0.1.29NullUnknown License
opentelemetry-util-genai0.3b0NullUnknown License
orjson3.11.8NullUnknown License
uv0.11.7NullUnknown License
pypdfium25.7.1NullUnknown License

Scanned Manifest Files

Cargo.lock
  • openssl@0.10.77
  • rustls-webpki@0.103.12
  • alloc-no-stdlib@2.0.4
  • alloc-stdlib@0.2.2
  • ar_archive_writer@0.5.1
  • arrayref@0.3.9
  • arrow@58.1.0
  • arrow-arith@58.1.0
  • arrow-array@58.1.0
  • arrow-buffer@58.1.0
  • arrow-cast@58.1.0
  • arrow-csv@58.1.0
  • arrow-data@58.1.0
  • arrow-ipc@58.1.0
  • arrow-json@58.1.0
  • arrow-ord@58.1.0
  • arrow-row@58.1.0
  • arrow-schema@58.1.0
  • arrow-select@58.1.0
  • arrow-string@58.1.0
  • async-compression@0.4.42
  • aws-config@1.8.16
  • aws-runtime@1.7.3
  • aws-sdk-sso@1.98.0
  • aws-sdk-ssooidc@1.100.0
  • aws-sdk-sts@1.103.0
  • aws-sigv4@1.4.3
  • aws-types@1.3.15
  • baked-potato@0.23.0
  • bigdecimal@0.4.10
  • blake3@1.8.5
  • block-buffer@0.12.0
  • brotli@8.0.2
  • brotli-decompressor@5.0.0
  • buoyant_kernel@0.21.103
  • buoyant_kernel_derive@1.0.0
  • bzip2@0.6.1
  • camino@1.2.2
  • cargo-platform@0.1.9
  • cargo_metadata@0.14.2
  • cc@1.2.61
  • chacha20@0.10.0
  • chrono-tz@0.10.4
  • cmov@0.5.3
  • comfy-table@7.2.2
  • compression-codecs@0.4.38
  • compression-core@0.4.32
  • const-oid@0.10.2
  • constant_time_eq@0.4.2
  • convert_case@0.9.0
  • cpufeatures@0.3.0
  • crc-catalog@2.5.0
  • crypto-common@0.2.1
  • ctor@0.10.1
  • ctor-proc-macro@0.0.13
  • ctutils@0.4.2
  • darling@0.20.11
  • darling_core@0.20.11
  • darling_macro@0.20.11
  • dashmap@5.5.3
  • data-encoding@2.11.0
  • datafusion@53.1.0
  • datafusion-catalog@53.1.0
  • datafusion-catalog-listing@53.1.0
  • datafusion-common@53.1.0
  • datafusion-common-runtime@53.1.0
  • datafusion-datasource@53.1.0
  • datafusion-datasource-arrow@53.1.0
  • datafusion-datasource-csv@53.1.0
  • datafusion-datasource-json@53.1.0
  • datafusion-datasource-parquet@53.1.0
  • datafusion-doc@53.1.0
  • datafusion-execution@53.1.0
  • datafusion-expr@53.1.0
  • datafusion-expr-common@53.1.0
  • datafusion-functions@53.1.0
  • datafusion-functions-aggregate@53.1.0
  • datafusion-functions-aggregate-common@53.1.0
  • datafusion-functions-nested@53.1.0
  • datafusion-functions-table@53.1.0
  • datafusion-functions-window@53.1.0
  • datafusion-functions-window-common@53.1.0
  • datafusion-macros@53.1.0
  • datafusion-optimizer@53.1.0
  • datafusion-physical-expr@53.1.0
  • datafusion-physical-expr-adapter@53.1.0
  • datafusion-physical-expr-common@53.1.0
  • datafusion-physical-optimizer@53.1.0
  • datafusion-physical-plan@53.1.0
  • datafusion-proto@53.1.0
  • datafusion-proto-common@53.1.0
  • datafusion-pruning@53.1.0
  • datafusion-session@53.1.0
  • datafusion-sql@53.1.0
  • deltalake@0.32.0
  • deltalake-core@0.32.0
  • deltalake-derive@1.0.0
  • digest@0.11.2
  • dtor@0.8.1
  • dtor-proc-macro@0.0.13
  • error-chain@0.12.4
  • fraction@0.15.4
  • glob@0.3.3
  • hmac@0.13.0
  • humantime@2.3.0
  • hybrid-array@0.4.11
  • ident_case@1.0.1
  • integer-encoding@3.0.4
  • jsonschema@0.46.2
  • libbz2-rs-sys@0.2.3
  • libc@0.2.186
  • liblzma@0.4.6
  • liblzma-sys@0.4.6
  • libmimalloc-sys@0.1.47
  • link-section@0.2.1
  • lz4_flex@0.13.0
  • mimalloc@0.1.50
  • mini-moka@0.10.3
  • num_cpus@1.17.0
  • object@0.37.3
  • object_store@0.13.2
  • openssl@0.10.78
  • openssl-sys@0.9.114
  • ordered-float@2.10.1
  • parquet@58.1.0
  • percent-encoding-rfc3986@0.1.3
  • phf@0.12.1
  • phf_shared@0.12.1
  • potato-agent@0.23.0
  • potato-head@0.23.0
  • potato-provider@0.23.0
  • potato-spec@0.23.0
  • potato-state@0.23.0
  • potato-type@0.23.0
  • potato-util@0.23.0
  • potato-workflow@0.23.0
  • potatohead-macro@0.23.0
  • psm@0.1.31
  • pulldown-cmark@0.9.6
  • quick-xml@0.39.2
  • rand@0.10.1
  • rand_core@0.10.1
  • recursive@0.1.1
  • recursive-proc-macro-impl@0.1.1
  • referencing@0.46.2
  • roaring@0.11.4
  • rustls@0.23.39
  • rustls-pki-types@1.14.1
  • rustls-webpki@0.103.13
  • scouter-dataframe@0.25.0
  • seq-macro@0.3.6
  • sha2@0.11.0
  • skeptic@0.13.7
  • snap@1.1.1
  • sqlparser@0.61.0
  • sqlparser_derive@0.5.0
  • stacker@0.1.24
  • strum@0.27.2
  • strum_macros@0.27.2
  • system-configuration@0.7.0
  • system-configuration-sys@0.6.0
  • tagptr@0.2.0
  • thrift@0.17.0
  • triomphe@0.1.15
  • twox-hash@2.1.2
  • unicode-segmentation@1.13.2
  • validator@0.19.0
  • validator_derive@0.19.0
  • windows-registry@0.6.1
  • winnow@1.0.2
  • z85@3.0.7
  • zstd@0.13.3
  • zstd-safe@7.2.4
  • zstd-sys@2.0.16+zstd.1.5.7
  • arrow@57.3.0
  • arrow-arith@57.3.0
  • arrow-array@57.3.0
  • arrow-buffer@57.3.0
  • arrow-cast@57.3.0
  • arrow-csv@57.3.0
  • arrow-data@57.3.0
  • arrow-ipc@57.3.0
  • arrow-json@57.3.0
  • arrow-ord@57.3.0
  • arrow-row@57.3.0
  • arrow-schema@57.3.0
  • arrow-select@57.3.0
  • arrow-string@57.3.0
  • aws-config@1.8.15
  • aws-runtime@1.7.2
  • aws-sdk-sso@1.97.0
  • aws-sdk-ssooidc@1.99.0
  • aws-sdk-sts@1.102.0
  • aws-sigv4@1.4.2
  • aws-types@1.3.14
  • baked-potato@0.22.0
  • cc@1.2.60
  • crc-catalog@2.4.0
  • data-encoding@2.10.0
  • fraction@0.15.3
  • jsonschema@0.46.1
  • libc@0.2.185
  • libmimalloc-sys@0.1.44
  • mimalloc@0.1.48
  • openssl-sys@0.9.113
  • potato-agent@0.22.0
  • potato-head@0.22.0
  • potato-provider@0.22.0
  • potato-spec@0.22.0
  • potato-state@0.22.0
  • potato-type@0.22.0
  • potato-util@0.22.0
  • potato-workflow@0.22.0
  • potatohead-macro@0.22.0
  • referencing@0.46.1
  • rustls@0.23.38
  • rustls-pki-types@1.14.0
  • winnow@1.0.1
crates/opsml_server/Cargo.toml
  • rand@>= 0.0.0, < 1.0.0
py-opsml/uv.lock
  • litellm@1.67.1
  • google-adk@1.25.0
  • appdirs@1.4.4
  • asttokens@3.0.1
  • auth0-python@5.3.0
  • bcrypt@5.0.0
  • blinker@1.9.0
  • build@1.4.4
  • chromadb@1.5.8
  • crewai@0.118.0
  • decorator@5.2.1
  • durationpy@0.10
  • et-xmlfile@2.0.0
  • google-adk@1.31.1
  • google-auth@2.49.2
  • google-cloud-aiplatform@1.148.1
  • google-cloud-dataplex@2.18.0
  • google-cloud-iam@2.22.0
  • google-genai@1.73.1
  • griffelib@2.0.2
  • httptools@0.7.1
  • importlib-resources@7.1.0
  • instructor@1.15.1
  • ipython@8.39.0
  • ipython@9.10.1
  • ipython@9.12.0
  • ipython-pygments-lexers@1.1.1
  • jedi@0.19.2
  • json-repair@0.59.4
  • json5@0.14.0
  • jsonpickle@4.1.1
  • jsonref@1.1.0
  • kubernetes@35.0.0
  • matplotlib-inline@0.2.1
  • oauthlib@3.3.1
  • openai@2.32.0
  • openai-agents@0.14.5
  • openinference-instrumentation@0.1.47
  • openinference-instrumentation-crewai@1.1.3
  • openinference-semantic-conventions@0.1.29
  • openpyxl@3.1.5
  • opentelemetry-exporter-otlp-proto-grpc@1.38.0
  • opentelemetry-instrumentation-openai-agents-v2@0.1.0
  • opentelemetry-util-genai@0.3b0
  • orjson@3.11.8
  • overrides@7.7.0
  • parso@0.8.6
  • pdfminer-six@20251230
  • pdfplumber@0.11.9
  • pexpect@4.9.0
  • ptyprocess@0.7.0
  • pure-eval@0.2.3
  • pybase64@1.4.3
  • pypdfium2@5.7.1
  • pypika@0.51.1
  • pyproject-hooks@1.2.0
  • pyvis@0.3.2
  • requests-oauthlib@2.0.0
  • stack-data@0.6.3
  • tiktoken@0.12.0
  • traitlets@5.14.3
  • uv@0.11.7
  • uvloop@0.22.1
  • watchfiles@1.1.1
  • websocket-client@1.9.0
  • google-auth@2.47.0
  • google-cloud-aiplatform@1.133.0
  • google-genai@1.57.0
  • openai@2.15.0
  • rsa@4.9.1

@thorrester
thorrester marked this pull request as ready for review April 28, 2026 02:04
devin-ai-integration[bot]

This comment was marked as resolved.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 17 additional findings in Devin Review.

Open in Devin Review

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@thorrester
thorrester merged commit 329c9c8 into main Apr 29, 2026
28 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant