Repository navigation
Time range components - #394
Conversation
Dependency Review✅ No vulnerabilities or license issues found.Scanned Manifest Files |
…/card/[space]/[name]/[version]/evaluation/+page.ts Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
| const path = promptEvalPath(e); | ||
| const profile = e.monitoringData.profile as AgentEvalProfile; | ||
| e.monitoringData.selectedData.workflows?.items?.forEach(w => | ||
| items.push({ ...w, _agentName: e.promptCard.name, _evalPath: path, _profile: profile }) | ||
| items.push({ | ||
| ...w, | ||
| _agentName: e.promptCard.name, | ||
| _evalPath: path, | ||
| _profile: profile, | ||
| _traceId: recordTraceMap.get(w.record_uid), |
There was a problem hiding this comment.
🟡 _traceId lookup fails when records and workflows are on different pages
The workflowPage derived block builds a recordTraceMap from the current page of records (e.monitoringData.selectedData.records?.items), then sets _traceId: recordTraceMap.get(w.record_uid) for each workflow. Since records and workflows are paginated independently via handleRecordPageChange and handleWorkflowPageChange (crates/opsml_server/opsml_ui/src/lib/components/card/agent/evaluation/AgentEvalDashboard.svelte:98-122), after the user pages forward on either table, the map only contains UIDs from the current record page. Any workflow referencing a record on a different page will have _traceId set to undefined, silently hiding the trace link even when the record actually has a trace_id.
(Refers to lines 293-314)
Prompt for agents
The _traceId for each workflow is looked up from a map built from only the currently-visible page of records. Since records and workflows paginate independently (handleRecordPageChange vs handleWorkflowPageChange), a workflow on page N may reference a record_uid that is on a different records page, causing the map lookup to return undefined and the trace link to disappear.
Possible approaches:
1. Maintain a cumulative record trace map (Map<string, string>) across all pages ever loaded, not just the current page. Append to it in the effect whenever new records arrive.
2. Include trace_id directly on the AgentEvalWorkflowResult type from the backend, so workflows carry their own trace_id without needing to cross-reference records.
3. If the relationship is always 1:1, denormalize by having the backend embed trace_id in the workflow response.
Files involved: AgentEvalDashboard.svelte (workflowPage derived block), types.ts (WorkflowWithAgent), and potentially the backend workflow endpoint.
Was this helpful? React with 👍 or 👎 to provide feedback.
Pull Request
Short Summary
Adds a faceted filter system to the trace dashboard — service, status code, duration, has-errors, and span attributes — plus a reusable
DashboardTimeBarcomponent that consolidates the time-range picker and refresh button. Also extends the Rust Scouter proxy with three new trace endpoints, a full GenAI metrics route module, and an observability metrics route.Context
Trace filter system
The trace dashboard previously had a free-text search box and no way to filter by service, status code, or duration without knowing the exact query syntax. This PR replaces that with a proper faceted filter layer:
filterState.svelte.ts— pure derive function that mapsTracePageFilterto anActiveFilter[]chip list, plus aremoveActiveFilterhelper. No reactive class, no global store — just functions the dashboard calls.FacetSidebar.svelte— persistent sidebar listing service names and status codes with counts from the backend, plus duration range and attribute inputs.ChipBar.svelte+AddFilterMenu.svelte— chip strip showing active filters with individual X buttons; the+button opensAddFilterMenuwhich lets you pick service, status, has-errors, attribute key=value, or a duration range.AttributeFacet,DurationFacet,FacetRow,FacetSection— composable primitives used by both the sidebar and the add-filter menu.The dashboard fetches
TraceFacetsResponse(service counts, status code counts) alongside metrics and page data on every time-range change or manual refresh. Facet data feeds both the sidebar and the chip bar.DashboardTimeBarcomponentThe inline
<TimeRangeFilter>+<button>Refresh</button>pair was duplicated across dashboards.DashboardTimeBar.sveltewraps both with a consistent layout and exposesonRangeChange/onRefreshprops.TraceDashboardand the agent observability page now use it.timeState.svelte.tsgainsisRefreshingstate withbeginRefresh()/endRefresh()guards to prevent overlapping fetches.ModeTabs— search vs analyticsThe dashboard now has two modes:
search(trace list + filters) andanalytics(metric charts). AModeTabscomponent switches between them. Previously, charts and table were always both rendered.Rust: new Scouter proxy endpoints
scouter/trace/route.rsadds three routes:POST /scouter/trace/spans/filtersTraceFiltersbodyPOST /scouter/trace/facetsGET /scouter/trace/{id}/spansscouter/genai/route.rsis a new ~526-line module proxying all GenAI Scouter routes (token metrics, operation breakdown, model usage, agent activity, tool activity, error breakdown, spans, conversations).scouter/observability/route.rsproxies the observability metrics endpoint.ScouterApiClientinclient.rswas refactored —execute_requestandbuild_route_urlextracted from the inline match arms, andrequest_with_pathadded for routes that include dynamic path segments (e.g.trace/{id}/spans).Existing trace endpoints now attach
AuditContextextensions — they were missing them before.Python / Scouter SDK
service_map/middleware.py— new service-map middleware for automatic trace enrichment.tracing/middleware.py— tracing middleware refactored into its own file.bifrost/— split frombifrost.pyinto a package (_client.py+__init__.py).evaluate/runner.py— eval runner gets concurrency improvements.py-opsml/examples/genai/agent/are replaced:music_recommender,recipe_generator, andsingle_agent(bespoke, divergent setups) are removed;crewai/,google/, andopenai/examples replace them with a sharedsetup.py/scenarios.jsonl/run.pyharness.Test additions
Vitest unit tests cover
filterState,DashboardTimeBar, agent assertion helpers,dashboard/utils,a2aOperations,agentInference, traceutils, and tracefacets. Rust integration tests cover the new GenAI, observability, trace, and profile endpoints.trace/filters/filterState.svelte.tsTracePageFiltertrace/filters/FacetSidebar.sveltetrace/filters/ChipBar.sveltetrace/filters/AddFilterMenu.svelteutils/DashboardTimeBar.svelteutils/timeState.svelte.tsisRefreshingguard withbeginRefresh/endRefreshtrace/TraceDashboard.sveltetrace/TraceTable.sveltetrace_idURL param on sidebar closescouter/trace/route.rsscouter/genai/route.rsscouter/observability/route.rsscouter/client.rsexecute_request/build_route_url; addsrequest_with_pathpy-opsml/python/opsml/scouter/service_map/py-opsml/python/opsml/scouter/tracing/middleware.pypy-opsml/examples/genai/agent/Is this a Breaking Change?
No. All new Svelte components are additive;
TraceDashboardandTraceTableprops retain backward-compatible defaults. New Rust endpoints are additive. TheScouterApiClientrefactor preserves the existingrequestmethod signature. Python example deletions remove content not imported by library consumers.