Repository navigation
Description
Observed behavior
Every Flowstate evaluation runs with cost tracking on and throws the measurement away: evalProgram (pkg/flowstate/v1/celenv.go:243) reads out, _, err := prg.ContextEval(...), and the five policy sites do the same (taskpolicy.go:437, netpolicy/rules.go:103, auth/assume.go:229, auth/secretpolicy.go, runfilter.go:217). EvalDetails.ActualCost() is read only in tests (celcost_test.go:277-366, celenv_optional_test.go:98). No production code uses cel.OptTrackState, interpreter.EvalState or EvalStateObserver (grep at 40cc365).
What the discarded data would answer, on surfaces that exist and ask the question by other means:
- How much of the budget did this step's expressions spend? The task span carries the task name, the step id and a secret-reference count (
taskspan.go:238-284) and no expression cost; a run one comprehension away fromDefaultCostLimitlooks identical to one spending 1%. cel: the runtime cost observer makes every large comprehension quadratic in wall time — a 50k-element map costs ~2.5s CPU while the budget reads it as affordable #1357 is a cost-shaped incident nothing in telemetry could have shown. - Why did this rule decide? cli: no CEL policy surface can be tested against cases — a
flow policy testandflow policy explainover the five activations that already exist #1468 asks forflow policy explainreporting "which rule decided and why"; Policy: one vocabulary, composable modules, a test verb, and enforcement below the layer that can bypass it #548 finding 3 asks which rule denied a request. cel-go'sOptTrackStaterecords the value of every expression node by id (interpreter/evalstate.go,EvalStateObserveras the planner option), andParsedExpr.source_infomaps ids to source positions — the trace is one option away. - What did this sub-expression evaluate to?
flowtest'scheckWitnesses(flowtest/check.go:524) re-evaluates each maximal reference path of a failing claim to printpath = value; the DAPevaluaterequest returns cel-go's message on failure and nothing else (flowdap/server.go:577-600); dx: nothing outside an editor answers "what names are in scope here" or "what does${steps.x.y}resolve to" — no generated scope reference, no CLI verb, no MCP tool #1438 wants "what does this resolve to" for a position. All three approximate the state record cel-go already keeps. - Which limit fired?
interpreter.EvalCancelledErrorcarries aCause(ContextCancelled,CostLimitExceeded, andMemoryLimitExceededafter the bump);errors.goclassifies expression failures by outer judgement (errors.go:305-336) and never reads it.
Desired outcome
Evaluator keeps what cel-go measured and offers it in two forms: a per-evaluation outcome (actual cost, peak memory once available, cancellation cause) that the step executor aggregates into the task span and the task metrics on both drivers; and an opt-in explain evaluation that plans with OptTrackState, joins the state map to source_info positions, and renders position → value through the same redaction seams flowdebug uses (flowdebug/inspect.go:172-233; celcomplete/doc.go:23-30 states why values never leave without them). flow policy explain, DAP evaluate on failure and flowtest witnesses read the one trace.
Acceptance criteria
- A task span carries the summed expression cost for the step and the run metrics carry a histogram of per-evaluation cost, on both drivers, pinned by the observe conformance case.
-o jsonforflow run localdoes not carry cost: it is interpreter-version-dependent and would make a run document differ across builds; the run document stays a projection of durable state.flow policy explain(cli: no CEL policy surface can be tested against cases — aflow policy testandflow policy explainover the five activations that already exist #1468) reports the deciding rule and the value of each sub-expression of that rule, redacted, from one evaluation rather than a re-evaluation.- A failing DAP
evaluateand a failingexpect.check:show the sameposition → valuerendering, produced by one function. ErrorKindExpressionfailures name the cause (cost,memory,cancelled) read fromEvalCancelledError.Cause, with a conformance case per cause.
Constraints and dependencies
- Cost and state tracking are per-program options; the program cache (
celenv.go:372) keys on env and expression, so an explain program must be a distinct cache entry or uncached (it is a debugging path; uncached is fine). OptTrackStateretains every intermediate value for the life of the evaluation, so explain is bounded byMaxCommandBytes-class limits and never enabled on the hot path.- Nothing measured here may enter
RunStateor the run document (invariant 10 and the version-pinning argument indocs/ARCHITECTURE.md); spans and metrics are the only sinks. - cel-go v0.32.0's "observers report every evaluation step" (Report every evaluation step to every observer cel-expr/cel-go#1419) changes what a state observer sees; the trace format should be pinned after the bump, not before.
Activity
picatz commentedon Sep 7, 2026
One attribute to carry on the cost metric when this lands: the run's tenant (the identity's namespace, which the memo already records). At
7530b29every evaluation site goes throughDefaultEvaluator()with one process-wideLimits(call.go:59,wait.go:393,concurrency.go:131,value.go:53,loop.go:260,switch.go:204,eval_task_http.go:658and the rest), so on a shared worker (#236's Tier 1a) there is no per-tenant budget and no per-tenant number. #236's gap 6 deliberately declines an app-layer limiter and points at Temporal namespace rate limits for volume; expression cost is the one resource those limits cannot see, because it is spent inside a workflow task. A histogram keyed by tenant makes a noisy neighbour visible without building a limiter, and is the evidence a later decision about per-tenantLimits(Tier 1b) would need. Nothing here changes what entersRunState.Generated by Claude Code