From de7277f707eed287016a1a572bf8e346f5d84cad Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Fri, 6 Nov 2026 14:10:58 +0800 Subject: [PATCH 01/19] Add a cost test for the bounded drawing arenas' edits and checkpoints A bounded drawing arena replaces its whole state with a deep copy on every create, delete and setter, whether or not a checkpoint shares it, so a script that moves a few of its live drawings per bar pays O(bound) per call and the first write after each checkpoint pays it again. The unbound arena already shares blocks of 32 records between copies and clones only the blocks a write reaches; this is the bounded twin of test_coof_drawing_checkpoint_cost. The test fills an arena to its bound (100 and 1,000), then runs the scheduler's checkpoint sequence (snapshot, restore, execute, commit) around an execution that deletes the oldest label, creates one, moves three and changes one text, all with texts that are never inline so a record copy allocates. Per execution it checks that the arena shares all but a few blocks of records with its checkpoint (the same record object through both) and that the heap blocks allocated do not depend on the bound, that the arena equals a plain model of the same operations after every execution, and that checkpointing an unwritten arena allocates the same at both bounds. The cost row is the row the change that follows must turn from red to green; the model row and the unchanged-checkpoint row are the guards that change has to keep green. Co-Authored-By: Claude Sonnet 5.5 --- tests/CMakeLists.txt | 1 + tests/test_drawing_bound_checkpoint_cost.cpp | 282 +++++++++++++++++++ 2 files changed, 283 insertions(+) create mode 100644 tests/test_drawing_bound_checkpoint_cost.cpp diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 007ab911e..235a26474 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -422,6 +422,7 @@ set(TEST_SOURCES test_pointvalue test_metrics test_drawing + test_drawing_bound_checkpoint_cost test_drawing_bound_status test_drawing_bounded_builtins test_drawing_bounded_creation_failure diff --git a/tests/test_drawing_bound_checkpoint_cost.cpp b/tests/test_drawing_bound_checkpoint_cost.cpp new file mode 100644 index 000000000..e93ae2407 --- /dev/null +++ b/tests/test_drawing_bound_checkpoint_cost.cpp @@ -0,0 +1,282 @@ +// K-DRAWBOUND: a bounded drawing arena's edits and checkpoints cost O(changes), not O(bound). +// +// A bounded arena (DrawingArena built from a DrawingBoundConfig: what the generated +// code constructs from a script's declared max_lines_count, max_boxes_count and +// max_labels_count) keeps its occupants in one state object that an arena and its +// copies share. Every create, every delete and every setter used to replace that +// state with a deep copy of the whole of it -- every slot, the text of every label, +// the directory and the order -- whether or not a checkpoint still shared it. A +// script that moves a few of its live drawings on every bar therefore paid O(bound) +// per call, and the first write after each checkpoint paid it once more. The unbound +// arena was already O(changes): its records live in blocks of 32 that copies share, +// and a write clones only the blocks it reaches (DrawingRecordStore, K-DRAWSNAP; the +// unbound twin of this test is test_coof_drawing_checkpoint_cost). +// +// The shape is a script with a bounded set of live levels (the strategy that exposed +// it keeps up to five hundred lines and labels alive and, on every bar, sets two +// fields of each active level): an arena full to its bound, and per script execution, +// under the scheduler's checkpoint sequence (snapshot at the bar open, restore, +// execute, commit; the generated hooks hold every arena by value), +// +// label.delete(oldest) +// label.new(...) a text of 200 characters, never inline +// label.set_x(oldest live), label.set_x(a middle one), label.set_y(the new one) +// label.set_text(second oldest) another text of 200 characters +// +// Every record copy therefore allocates the text it carries, so what a run allocates +// counts the records it copied. +// +// Rows: +// cost per execution, after the checkpoint is restored: how many of the arena's +// records are not the very objects the checkpoint holds (a record the +// arena shares with its checkpoint is the same object through both), and +// how many heap blocks the restore and the body allocate, at bound 100 and +// at bound 1,000. One execution writes five slots (the one it deletes, the +// one it creates in, the oldest live label, the middle one and the one +// whose text changes), each in one block of 32 records that the checkpoint +// shares, so it may copy five blocks, and the test allows six (the tail +// block a store keeps beside its tree): 192 records, and 6 * (32 + 1) +// blocks of heap plus 80 for the state, the directory, the order and the +// texts it builds, whatever the bound. Before K-DRAWBOUND every call copied +// every slot: 1,000 records unshared and some 6,000 blocks at bound 1,000. +// model after every execution the arena equals a plain model of the same +// operations (size, order, and every field of every live record), so the +// cheaper copy changes no value. +// unchanged checkpointing an arena nobody wrote allocates the same at bound 1,000 as +// at bound 100: sharing is O(1) in the bound. +// +// Under AddressSanitizer (whose fake stack makes every call dear) the cost row runs a +// third of its executions. +#include "global_allocation_replacement.hpp" + +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +using namespace pineforge; + +#if defined(__SANITIZE_ADDRESS__) +constexpr int kSanitized = 1; +#elif defined(__has_feature) +#if __has_feature(address_sanitizer) +constexpr int kSanitized = 1; +#else +constexpr int kSanitized = 0; +#endif +#else +constexpr int kSanitized = 0; +#endif +constexpr int kExecutionDivisor = kSanitized ? 3 : 1; + +static int g_pass = 0; +static int g_fail = 0; + +#define CHECK(cond) \ + do { \ + if (!(cond)) { \ + std::printf("FAIL %s:%d: %s\n", __FILE__, __LINE__, #cond); \ + ++g_fail; \ + } else { \ + ++g_pass; \ + } \ + } while (0) + +namespace { + +constexpr int kBlock = 32; // records per block a store shares between copies +constexpr int kTouchedBlocks = 6; // five slots written per execution, one block of slack +constexpr int kTextLength = 200; // never inline: every copy of a record allocates +constexpr int kWarmup = 3; // executions before the measured ones (the first copy of a fresh state) +constexpr int kUnchangedBars = 1000 / kExecutionDivisor; + +std::string text_of(char first, int n) { + return std::string(static_cast(kTextLength), static_cast(first + n % 26)); +} + +// ---- the model: the same operations on a plain ordered map ------------------- +struct Model { + std::map live; // by logical id, which is also creation order + std::int32_t next = 0; +}; + +void model_create(Model& m, std::int64_t x, double y, std::string text) { + m.live.emplace(m.next, LabelRec{x, y, XLoc::bar_index, YLoc::price, std::move(text), true}); + ++m.next; +} + +// Every observable of the arena against the model. +bool same(const DrawingArena& arena, const Model& m) { + if (arena.size() != m.next) return false; + if (arena.occupied_slots() != m.live.size()) return false; + const auto& order = arena.order(); + if (order.size() != m.live.size()) return false; + std::size_t i = 0; + for (const auto& entry : m.live) { + const std::int32_t id = entry.first; + const LabelRec& want = entry.second; + if (order[i++] != id) return false; + if (arena.access_status(id) != DrawingAccessStatus::present) return false; + const LabelRec& got = arena.at(id); + if (got.x != want.x || got.y != want.y || got.xloc != want.xloc || got.yloc != want.yloc + || got.text != want.text || !got.alive) + return false; + } + return true; +} + +// ---- one script execution, on the arena and on its model --------------------- +void body(DrawingArena& arena, int bar) { + pf_label_delete(arena, Label{arena.order().front()}); + pf_label_new(arena, bar, 100.0 + bar, text_of('a', bar)); + const std::int32_t first = arena.order().front(); + const std::int32_t next = arena.size(); + pf_label_set_x(arena, Label{first}, bar); + pf_label_set_x(arena, Label{first + (next - first) / 2}, bar); + pf_label_set_y(arena, Label{next - 1}, 1.0 * bar); + pf_label_set_text(arena, Label{first + 1}, text_of('A', bar)); +} + +void model_body(Model& m, int bar) { + m.live.erase(m.live.begin()); + model_create(m, bar, 100.0 + bar, text_of('a', bar)); + const std::int32_t first = m.live.begin()->first; + const std::int32_t next = m.next; + m.live.at(first).x = bar; + m.live.at(first + (next - first) / 2).x = bar; + m.live.at(next - 1).y = 1.0 * bar; + m.live.at(first + 1).text = text_of('A', bar); +} + +// How many of the arena's records are not the records its checkpoint holds: a +// record the two share is the same object through both, a record one write copied +// (with its block) is another, and a record the checkpoint never had is new. +std::uint64_t unshared(const DrawingArena& live, const DrawingArena& before) { + std::uint64_t n = 0; + for (const std::int32_t id : live.order()) { + const bool shared = before.access_status(id) == DrawingAccessStatus::present + && &live.at(id) == &before.at(id); + if (!shared) ++n; + } + return n; +} + +struct Cost { + int executions = 0; + std::uint64_t unshared_total = 0; + std::uint64_t unshared_max = 0; + std::uint64_t allocations_total = 0; + std::uint64_t allocations_max = 0; + bool model_agrees = true; + bool final_state_ok = false; +}; + +Cost run(int bound, int executions) { + DrawingArena arena(drawing_bound(bound)); + Model model; + for (int i = 0; i < bound; ++i) { + pf_label_new(arena, i, 1.0 * i, text_of('a', i)); + model_create(model, i, 1.0 * i, text_of('a', i)); + } + std::optional> checkpoint; + Cost cost; + for (int bar = 0; bar < executions + kWarmup; ++bar) { + checkpoint.emplace(arena); // snapshot: the bar open + const std::size_t before = global_allocation::allocations; + arena = *checkpoint; // restore + body(arena, bar); // execute + const std::size_t after = global_allocation::allocations; + model_body(model, bar); + if (!same(arena, model)) cost.model_agrees = false; + if (bar >= kWarmup) { + const std::uint64_t allocated = after - before; + const std::uint64_t records = unshared(arena, *checkpoint); + ++cost.executions; + cost.unshared_total += records; + cost.unshared_max = std::max(cost.unshared_max, records); + cost.allocations_total += allocated; + cost.allocations_max = std::max(cost.allocations_max, allocated); + } + checkpoint.emplace(arena); // commit + } + // The run ends as the model does: the bound's worth of live labels, the oldest + // `executions + warm-up` ids deleted, and one id issued per creation. + const int total = executions + kWarmup; + cost.final_state_ok = arena.occupied_slots() == static_cast(bound) + && arena.size() == bound + total + && arena.order().front() == total + && arena.order().back() == bound + total - 1 + && arena.last_allocation_diagnostic().status == DrawingAllocationStatus::ok; + return cost; +} + +void test_edits_and_checkpoints_cost_o_changes() { + std::printf("test_edits_and_checkpoints_cost_o_changes\n"); + const int executions = 150 / kExecutionDivisor; + const int bounds[] = {100, 1000}; + for (const int bound : bounds) { + const Cost c = run(bound, executions); + const double mean_unshared = c.executions + ? static_cast(c.unshared_total) / c.executions : 0.0; + const double mean_allocations = c.executions + ? static_cast(c.allocations_total) / c.executions : 0.0; + std::printf(" bound %4d: %d executions; records not shared with the checkpoint per " + "execution: mean %.1f, max %llu (limit %d); heap blocks per execution: mean " + "%.1f, max %llu (limit %d)\n", + bound, c.executions, mean_unshared, (unsigned long long)c.unshared_max, + kTouchedBlocks * kBlock, mean_allocations, + (unsigned long long)c.allocations_max, kTouchedBlocks * (kBlock + 1) + 80); + CHECK(c.executions == executions); + CHECK(c.model_agrees); + CHECK(c.final_state_ok); + // The arena writes at most six blocks of 32 records per execution, whatever + // the bound; a record any other write copied would be O(bound). + CHECK(c.unshared_max <= static_cast(kTouchedBlocks * kBlock)); + // And what it allocates for them: a copy of each block's records (their + // texts), the block, and a fixed few for the state, the directory, the order + // and the texts the body builds. + CHECK(c.allocations_max + <= static_cast(kTouchedBlocks * (kBlock + 1) + 80)); + } +} + +// ---- an unchanged checkpoint allocates nothing that grows with the bound ----- +std::size_t unchanged_checkpoint_allocations(int bound) { + DrawingArena live(drawing_bound(bound)); + for (int i = 0; i < bound; ++i) pf_label_new(live, i, 1.0 * i, text_of('a', i)); + std::optional> checkpoint; + checkpoint.emplace(live); // the first bar: snapshot, restore, commit + live = *checkpoint; + checkpoint.emplace(live); + const std::size_t before = global_allocation::allocations; + for (int bar = 0; bar < kUnchangedBars; ++bar) { + checkpoint.emplace(live); // snapshot + live = *checkpoint; // restore + checkpoint.emplace(live); // commit + } + return global_allocation::allocations - before; +} + +void test_unchanged_checkpoint_allocates_nothing_that_grows() { + std::printf("test_unchanged_checkpoint_allocates_nothing_that_grows\n"); + const std::size_t small = unchanged_checkpoint_allocations(100); + const std::size_t large = unchanged_checkpoint_allocations(1000); + std::printf(" %d bars of snapshot / restore / commit allocate %zu blocks at bound 100 " + "and %zu at bound 1,000\n", kUnchangedBars, small, large); + CHECK(large == small); +} + +} // namespace + +int main() { + test_edits_and_checkpoints_cost_o_changes(); + test_unchanged_checkpoint_allocates_nothing_that_grows(); + std::printf("\nbounded drawing cost tests: %d passed, %d failed\n", g_pass, g_fail); + return g_fail == 0 ? 0 : 1; +} From 307eac1298999008f99bdcdf26c93b06e5c214f4 Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Fri, 6 Nov 2026 14:18:12 +0800 Subject: [PATCH 02/19] Share a bounded drawing arena's slots between copies and write in place A bounded arena replaced its whole state with a deep copy on every create, delete and setter: every slot, the text of every label, the directory and the order, whether or not a script-state checkpoint still shared it. A script that moves a few of its live drawings on every bar therefore paid time proportional to the bound on every call, and the first write after each checkpoint paid it again. The slots now live in a DrawingRecordStore, the block store the unbound arena already uses: blocks of 32 that a copy of the state shares, one path copied per write that reaches a shared block. A copy of the state shares the blocks and copies only the directory and the order (an integer per occupant), the directory with room for every slot so a creation never reallocates it. Creation, deletion and every setter give the arena a state of its own when a checkpoint shares it, then change it in place. Each does every step that can allocate or throw (the state's copy, the copy of the slot's block, the directory's room, the order's growth) before its first change and only steps that cannot throw after it, so a refused allocation leaves the arena and its checkpoint as they were. The search for the lowest free slot starts at a recorded index below which every slot is occupied, so slot reuse order is unchanged and a creation no longer scans every slot. The bound, the typed overflow outcome, the stale-handle behaviour and every value a script reads are unchanged. The source allocation-bound gate (ci/cr150-source-bounds.yaml) pins every growable-container site of include/pineforge/**, so its reviewed site inventory moves with this change: the two groups for the removed whole-state copies are deleted and the OPEN ceiling drops by two; the groups for the slots (a block store filled by push_back), the directory and the state's allocation gain the new site identities. The inventory is the gate's own proposal for this tree; its differences from the previous one are exactly those sites. The gate's real-tree test pins the number of OPEN groups, so that pin moves by the same two. Co-Authored-By: Claude Sonnet 5.5 --- CHANGELOG.md | 10 +++ ci/cr150-source-bounds.yaml | 9 +- ci/cr150-source-sites.json | 90 +++++++++++--------- include/pineforge/drawing.hpp | 135 ++++++++++++++++++++---------- scripts/test_cr150_source_gate.py | 8 +- 5 files changed, 163 insertions(+), 89 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 57c3abea5..3c79aad11 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -156,6 +156,16 @@ version number follows semantic versioning over the surfaces the Ratio scheduling is a separate later change; whole-curve passes remain. Plain/full and v1 delta bytes are preserved. Ledger tables and checkpoint layout are unchanged. +- A bounded drawing arena (the line, box and label arenas the generated code + builds from a script's declared `max_*_count`) now writes in place and keeps + its slots in blocks of 32 that a checkpoint of the arena shares, like an + unbound arena's records. A create, a delete or a setter used to replace the + whole state with a deep copy, so a script that moved a few of its live + drawings on every bar paid time proportional to the bound on every call, and + the first write after each script-state checkpoint paid it again; they now + copy only the blocks of 32 records they write. The bound, the typed + overflow outcome, the slot reuse order, the stale-handle behaviour and every + value a script reads are unchanged. ## 1.9.1 — 2026-10-31 diff --git a/ci/cr150-source-bounds.yaml b/ci/cr150-source-bounds.yaml index c0c99dbde..2e3c1f94a 100644 --- a/ci/cr150-source-bounds.yaml +++ b/ci/cr150-source-bounds.yaml @@ -87,7 +87,10 @@ ceilings: # --emit-seed, 4 stale entries removed): the bounded drawings, the owned # storage and runtime context service, the ledger cache # and the selected-capacity driver, reader and core. None is bounded. - unreviewed: 1403 + # Two of those groups are gone (U1666 and U1668, the whole-state copies a bounded + # drawing arena made on every create, delete and edit): its slots are now shared + # blocks written in place, so the ceiling drops by two. + unreviewed: 1401 pending-flat-memory: 184 owners: - {name: "generated-cpp", globs: ["tutorial/**"]} @@ -1681,12 +1684,10 @@ entries: - {id: "U1644", file: "src/native_execution_consumer.hpp", symbol: "cohort_opening_index_", ops: ["decl"], container: "std::vector", owner: "native-kernel", phase: "unclassified", state: "unreviewed", note: "Consumer-owned derived scratch, rebuilt from physical lots during eligible ordinary bar target reads and invalidated by settlement and host lot-write generations. Capacity survives clear and begin; contents are never persisted or hashed and require no rollback image. Layout estimate from source, not measured: kCohortOpeningIndexBytes requests 64 KiB payload, 4096 rows at 16 bytes each, excluding vector and allocator overhead; larger lot sets use the complete reader. Unsupported member-vector admission stays OPEN. Tests: tests/test_native_cohort_target_cost.cpp, tests/test_native_cohort_target_transcript.cpp, tests/test_native_cohort_target_invalidation.cpp, tests/test_native_cohort_target_bypass.cpp.", line: 2867} - {id: "U1664", file: "include/pineforge/drawing.hpp", symbol: "", ops: ["temp"], container: "unresolved by the index: std::string", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 123} - {id: "U1665", file: "include/pineforge/drawing.hpp", symbol: "bound_", ops: ["alloc"], container: "unresolved by the index: make_shared", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 710} - - {id: "U1666", file: "include/pineforge/drawing.hpp", symbol: "candidate", ops: ["alloc"], container: "unresolved by the index: make_shared", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 731} - {id: "U1667", file: "include/pineforge/drawing.hpp", symbol: "directory", ops: ["decl", "grow"], container: "std::vector", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 635} - - {id: "U1668", file: "include/pineforge/drawing.hpp", symbol: "make_shared", ops: ["alloc"], container: "unresolved by the index: make_shared", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 838} - {id: "U1669", file: "include/pineforge/drawing.hpp", symbol: "order", ops: ["assign", "decl", "grow"], container: "std::deque", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 640} - {id: "U1670", file: "include/pineforge/drawing.hpp", symbol: "records", ops: ["decl"], container: "detail::DrawingRecordStore", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 763} - - {id: "U1671", file: "include/pineforge/drawing.hpp", symbol: "slots", ops: ["decl"], container: "std::vector", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 638} + - {id: "U1671", file: "include/pineforge/drawing.hpp", symbol: "slots", ops: ["decl", "grow"], container: "DrawingRecordStore", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 656} - {id: "U1672", file: "include/pineforge/drawing.hpp", symbol: "value", ops: ["grow"], container: "unresolved by the index: value.emplace(...)", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 845} - {id: "U1673", file: "include/pineforge/owned_storage.hpp", symbol: "", ops: ["temp"], container: "unresolved by the index: Base", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 342} - {id: "U1674", file: "include/pineforge/owned_storage.hpp", symbol: "", ops: ["temp"], container: "unresolved by the index: OwnedDeque", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 341} diff --git a/ci/cr150-source-sites.json b/ci/cr150-source-sites.json index 252411bd5..fc2ae89d2 100644 --- a/ci/cr150-source-sites.json +++ b/ci/cr150-source-sites.json @@ -2844,12 +2844,51 @@ "namespace pineforge #1", "namespace detail #2", "template < class Rec > struct BoundDrawingState #1", - "explicit BoundDrawingState ( DrawingBoundConfig config ) : slots ( storage_slots ( config ) ) , slot_bound ( config . slot_bound ) , pending_creation_slots ( config . pending_creation_slots ) #1" + "explicit BoundDrawingState ( DrawingBoundConfig config ) : slot_bound ( config . slot_bound ) , pending_creation_slots ( config . pending_creation_slots ) #1" ], "cls": "grow", "spelling": "directory . reserve ( . . . )", "ordinal": 1 }, + { + "file": "include/pineforge/drawing.hpp", + "symbol": "slots", + "scope": [ + "namespace pineforge #1", + "namespace detail #2", + "template < class Rec > struct BoundDrawingState #1", + "explicit BoundDrawingState ( DrawingBoundConfig config ) : slot_bound ( config . slot_bound ) , pending_creation_slots ( config . pending_creation_slots ) #1" + ], + "cls": "grow", + "spelling": "slots . push_back ( . . . )", + "ordinal": 1 + }, + { + "file": "include/pineforge/drawing.hpp", + "symbol": "directory", + "scope": [ + "namespace pineforge #1", + "namespace detail #2", + "template < class Rec > struct BoundDrawingState #1", + "BoundDrawingState ( const BoundDrawingState & other ) : slots ( other . slots ) , order ( other . order ) , next_id ( other . next_id ) , slot_bound ( other . slot_bound ) , pending_creation_slots ( other . pending_creation_slots ) , free_from ( other . free_from ) #1" + ], + "cls": "grow", + "spelling": "directory . reserve ( . . . )", + "ordinal": 1 + }, + { + "file": "include/pineforge/drawing.hpp", + "symbol": "directory", + "scope": [ + "namespace pineforge #1", + "namespace detail #2", + "template < class Rec > struct BoundDrawingState #1", + "BoundDrawingState ( const BoundDrawingState & other ) : slots ( other . slots ) , order ( other . order ) , next_id ( other . next_id ) , slot_bound ( other . slot_bound ) , pending_creation_slots ( other . pending_creation_slots ) , free_from ( other . free_from ) #1" + ], + "cls": "grow", + "spelling": "directory . assign ( . . . )", + "ordinal": 1 + }, { "file": "include/pineforge/drawing.hpp", "symbol": "slots", @@ -2859,7 +2898,7 @@ "template < class Rec > struct BoundDrawingState #1" ], "cls": "decl", - "spelling": "std :: vector < Slot >", + "spelling": "DrawingRecordStore < Slot >", "ordinal": 1 }, { @@ -2920,18 +2959,6 @@ "spelling": "make_shared", "ordinal": 1 }, - { - "file": "include/pineforge/drawing.hpp", - "symbol": "candidate", - "scope": [ - "namespace pineforge #1", - "template < class Rec > class DrawingArena #1", - "template < class Edit > void bound_edit ( int32_t id , Edit && edit ) #1" - ], - "cls": "alloc", - "spelling": "make_shared", - "ordinal": 1 - }, { "file": "include/pineforge/drawing.hpp", "symbol": "bound_", @@ -3011,7 +3038,7 @@ }, { "file": "include/pineforge/drawing.hpp", - "symbol": "make_shared", + "symbol": "bound_", "scope": [ "namespace pineforge #1", "template < class Rec > class DrawingArena #1", @@ -3023,50 +3050,50 @@ }, { "file": "include/pineforge/drawing.hpp", - "symbol": "make_shared", + "symbol": "directory", "scope": [ "namespace pineforge #1", "template < class Rec > class DrawingArena #1", "[ [ nodiscard ] ] DrawingAllocationResult try_alloc ( Rec r ) #1" ], - "cls": "alloc", - "spelling": "make_shared", - "ordinal": 2 + "cls": "grow", + "spelling": "directory . reserve ( . . . )", + "ordinal": 1 }, { "file": "include/pineforge/drawing.hpp", - "symbol": "value", + "symbol": "order", "scope": [ "namespace pineforge #1", "template < class Rec > class DrawingArena #1", "[ [ nodiscard ] ] DrawingAllocationResult try_alloc ( Rec r ) #1" ], "cls": "grow", - "spelling": "value . emplace ( . . . )", + "spelling": "order . push_back ( . . . )", "ordinal": 1 }, { "file": "include/pineforge/drawing.hpp", - "symbol": "directory", + "symbol": "value", "scope": [ "namespace pineforge #1", "template < class Rec > class DrawingArena #1", "[ [ nodiscard ] ] DrawingAllocationResult try_alloc ( Rec r ) #1" ], "cls": "grow", - "spelling": "directory . push_back ( . . . )", + "spelling": "value . emplace ( . . . )", "ordinal": 1 }, { "file": "include/pineforge/drawing.hpp", - "symbol": "order", + "symbol": "directory", "scope": [ "namespace pineforge #1", "template < class Rec > class DrawingArena #1", "[ [ nodiscard ] ] DrawingAllocationResult try_alloc ( Rec r ) #1" ], "cls": "grow", - "spelling": "order . push_back ( . . . )", + "spelling": "directory . push_back ( . . . )", "ordinal": 1 }, { @@ -3093,19 +3120,6 @@ "spelling": "order_ . push_back ( . . . )", "ordinal": 1 }, - { - "file": "include/pineforge/drawing.hpp", - "symbol": "candidate", - "scope": [ - "namespace pineforge #1", - "template < class Rec > class DrawingArena #1", - "void erase ( int32_t id ) #1", - "if ( bound_ ) #1" - ], - "cls": "alloc", - "spelling": "make_shared", - "ordinal": 1 - }, { "file": "include/pineforge/drawing.hpp", "symbol": "text", diff --git a/include/pineforge/drawing.hpp b/include/pineforge/drawing.hpp index c428cde13..75d806ec5 100644 --- a/include/pineforge/drawing.hpp +++ b/include/pineforge/drawing.hpp @@ -629,18 +629,41 @@ struct BoundDrawingState { } explicit BoundDrawingState(DrawingBoundConfig config) - : slots(storage_slots(config)), - slot_bound(config.slot_bound), + : slot_bound(config.slot_bound), pending_creation_slots(config.pending_creation_slots) { - directory.reserve(slots.size()); + const std::size_t count = storage_slots(config); + directory.reserve(count); + for (std::size_t i = 0; i < count; ++i) slots.push_back(Slot{}); } - std::vector slots; + // A copy of the state is what a checkpoint of an arena pays once per script + // execution that writes it. It shares the slots' blocks (O(1) once they are + // blocks; the first copy of the vector a fresh state starts as builds them) + // and copies the directory and the order, one entry per occupant. The + // directory gets the room it can ever need, so a creation never reallocates it. + BoundDrawingState(const BoundDrawingState& other) + : slots(other.slots), order(other.order), next_id(other.next_id), + slot_bound(other.slot_bound), + pending_creation_slots(other.pending_creation_slots), + free_from(other.free_from) { + directory.reserve(static_cast(other.slots.size())); + directory.assign(other.directory.begin(), other.directory.end()); + } + + // The slots live in blocks of 32 that copies of the state share, like the + // records of an unbound arena: a write copies the one block it reaches, and + // only while another copy still holds it. + DrawingRecordStore slots; std::vector directory; // sorted; contains occupied IDs only std::deque order; std::int64_t next_id = 0; std::int32_t slot_bound = 0; std::uint32_t pending_creation_slots = 0; + // Every slot below this index has an occupant; a creation's search for the + // lowest free slot starts here, and a deletion lowers it to the slot it frees. + std::uint32_t free_from = 0; + + std::size_t slot_count() const noexcept { return static_cast(slots.size()); } std::int32_t slot_for(std::int32_t id) const noexcept { const auto it = std::lower_bound( @@ -651,16 +674,18 @@ struct BoundDrawingState { if (it == directory.end() || it->logical_id != id) return -1; // A directory hit with no matching occupant is internal corruption, // never a stale drawing. It cannot be produced by public operations. - if (it->slot >= slots.size()) std::terminate(); - const Slot& candidate = slots[it->slot]; + if (it->slot >= slot_count()) std::terminate(); + const Slot& candidate = slots.read(it->slot); if (!candidate.value || candidate.logical_id != id) std::terminate(); return static_cast(it->slot); } + // The lowest slot with no occupant, or slot_count() when there is none. std::size_t free_slot() const noexcept { - for (std::size_t i = 0; i < slots.size(); ++i) - if (!slots[i].value) return i; - return slots.size(); + const std::size_t count = slot_count(); + for (std::size_t i = free_from; i < count; ++i) + if (!slots.read(static_cast(i)).value) return i; + return count; } }; @@ -705,6 +730,10 @@ class DrawingArena { static constexpr std::int64_t kIdExclusive = detail::kDrawingIdExclusive; + // Gives this arena a state of its own when a checkpoint or another arena + // still shares it. The copy shares the slots' blocks, so it costs O(1) in the + // records (plus the directory and the order: an integer per occupant), and a + // refusal to allocate it leaves bound_ as it was. void ensure_bound_unique() { if (bound_ && !bound_.unique()) bound_ = std::make_shared>(*bound_); @@ -716,9 +745,19 @@ class DrawingArena { "drawing access on na handle"); const std::int32_t slot = bound_->slot_for(id); if (slot < 0) return detail::drawing_missing_record(); - return *bound_->slots[static_cast(slot)].value; + return *bound_->slots.read(static_cast(slot)).value; } + // The three writes of a bounded arena -- creation, deletion and an edit -- + // change the arena's own state in place, after ensure_bound_unique(). Each + // does every step that can allocate or throw (the private copy of the state, + // the copy of the block the slot is in, the order's growth) before its first + // change, and only steps that cannot throw after it, so a refused allocation + // leaves the arena equal to what it was and the checkpoint it was copied from + // untouched (tests/test_drawing_slot_atomicity.cpp). + // + // `edit` is a scalar or a string move assignment, as every builtin setter's is, + // and cannot throw: the record is written where it is. template void bound_edit(int32_t id, Edit&& edit) { if (id < 0 || !bound_ || static_cast(id) >= bound_->next_id) @@ -726,15 +765,14 @@ class DrawingArena { "drawing access on na handle"); const std::int32_t slot = bound_->slot_for(id); if (slot < 0) return; // stale builtins are typed no-ops - if (!bound_->slots[static_cast(slot)].value->alive) return; + if (!bound_->slots.read(static_cast(slot)).value->alive) return; - auto candidate = std::make_shared>(*bound_); - const std::int32_t candidate_slot = candidate->slot_for(id); - if (candidate_slot < 0) std::terminate(); - std::forward(edit)( - *candidate->slots[static_cast(candidate_slot)].value); + // A private copy of the state has the same slots and directory, so `slot` + // is the same record in it. + ensure_bound_unique(); + auto& target = bound_->slots.write(static_cast(slot)); + std::forward(edit)(*target.value); stale_sink_.reset(); - bound_.swap(candidate); } public: @@ -834,21 +872,27 @@ class DrawingArena { last_allocation_diagnostic_ = diagnostic; return {diagnostic, std::nullopt}; } - auto candidate = bound_ - ? std::make_shared>(*bound_) - : std::make_shared>(*bound_config_); - const std::size_t slot = candidate->free_slot(); - if (slot >= candidate->slots.size()) std::terminate(); // corrupt occupied count + // Every step that can allocate precedes the first change (see bound_edit): + // the state of its own, the block of the slot, the directory's room and the + // order's growth. A moved-from arena has no state yet and gets a fresh one. + if (bound_) ensure_bound_unique(); + else bound_ = std::make_shared>(*bound_config_); + detail::BoundDrawingState& state = *bound_; + const std::size_t slot = state.free_slot(); + if (slot >= state.slot_count()) std::terminate(); // corrupt occupied count const std::int32_t id = static_cast(next_id); + auto& target = state.slots.write(static_cast(slot)); + state.directory.reserve(state.directory.size() + 1); + state.order.push_back(id); + // Nothing below can throw. r.alive = true; - candidate->slots[slot].logical_id = id; - candidate->slots[slot].value.emplace(std::move(r)); - candidate->directory.push_back({id, static_cast(slot)}); - candidate->order.push_back(id); - candidate->next_id = next_id + 1; + target.logical_id = id; + target.value.emplace(std::move(r)); + state.directory.push_back({id, static_cast(slot)}); + state.next_id = next_id + 1; + state.free_from = static_cast(slot) + 1; stale_sink_.reset(); last_allocation_diagnostic_ = diagnostic; - bound_.swap(candidate); return {diagnostic, id}; } @@ -884,24 +928,27 @@ class DrawingArena { if (id < 0 || static_cast(id) >= bound_->next_id) return; const std::int32_t slot = bound_->slot_for(id); if (slot < 0) return; // stale delete is a no-op - if (!bound_->slots[static_cast(slot)].value->alive) return; - - auto candidate = std::make_shared>(*bound_); - const std::int32_t candidate_slot = candidate->slot_for(id); - if (candidate_slot < 0) std::terminate(); + if (!bound_->slots.read(static_cast(slot)).value->alive) return; + + // The state of its own and the block of the slot first, then only + // steps that cannot throw (see bound_edit). A private copy of the + // state has the same slots and directory, so `slot` is the same record. + ensure_bound_unique(); + detail::BoundDrawingState& state = *bound_; + auto& target = state.slots.write(static_cast(slot)); const auto dir = std::lower_bound( - candidate->directory.begin(), candidate->directory.end(), id, + state.directory.begin(), state.directory.end(), id, [](const auto& entry, std::int32_t value) { return entry.logical_id < value; }); - if (dir == candidate->directory.end() || dir->logical_id != id) std::terminate(); - candidate->directory.erase(dir); - const auto it = std::find(candidate->order.begin(), candidate->order.end(), id); - if (it != candidate->order.end()) candidate->order.erase(it); - candidate->slots[static_cast(candidate_slot)].logical_id = -1; - candidate->slots[static_cast(candidate_slot)].value.reset(); + if (dir == state.directory.end() || dir->logical_id != id) std::terminate(); + const auto it = std::find(state.order.begin(), state.order.end(), id); + state.directory.erase(dir); + if (it != state.order.end()) state.order.erase(it); + target.logical_id = -1; + target.value.reset(); + state.free_from = std::min(state.free_from, static_cast(slot)); stale_sink_.reset(); - bound_.swap(candidate); return; } @@ -915,7 +962,7 @@ class DrawingArena { if (bound_) { const std::int32_t slot = (id >= 0) ? bound_->slot_for(id) : -1; return slot >= 0 - && bound_->slots[static_cast(slot)].value->alive; + && bound_->slots.read(static_cast(slot)).value->alive; } return id >= 0 && id < recs_.size() && recs_.read(static_cast(id)).alive; } @@ -969,7 +1016,7 @@ class DrawingArena { ensure_bound_unique(); stale_sink_.reset(); const std::int32_t slot = bound_->slot_for(id); - return *bound_->slots[static_cast(slot)].value; + return *bound_->slots.write(static_cast(slot)).value; } } return recs_.write(static_cast(id)); @@ -1016,7 +1063,7 @@ struct DrawingBuiltinAccess { if (!a.bound_) return false; // no issued IDs in a moved-from arena const auto slot = a.bound_->slot_for(h.id); return slot < 0 - || !a.bound_->slots[static_cast(slot)].value->alive; + || !a.bound_->slots.read(static_cast(slot)).value->alive; } if constexpr (std::is_same_v) return false; else return !a.alive(h.id); diff --git a/scripts/test_cr150_source_gate.py b/scripts/test_cr150_source_gate.py index 9734596d5..b5460b3e8 100644 --- a/scripts/test_cr150_source_gate.py +++ b/scripts/test_cr150_source_gate.py @@ -1216,9 +1216,11 @@ def test_real_tree_scans_files_and_sites_and_is_not_a_pass_by_construction(self) self.assertEqual(c['findings'], 0) self.assertEqual(c['PASS(bounded)'], 0) # Exact manifest union: 1,344 original unreviewed groups, ten time-format - # groups and two map groups, and 47 groups of the integrated tree (51 - # emitted, 4 stale entries removed); all 184 pending-flat-memory groups remain. - self.assertEqual(c['OPEN'], 1356 + 47 + 184) + # groups and two map groups, and 45 groups of the integrated tree (51 + # emitted, 4 stale entries removed, then the two whole-state copies of the + # bounded drawing arena removed when it began writing in place); all 184 + # pending-flat-memory groups remain. + self.assertEqual(c['OPEN'], 1356 + 45 + 184) # Every matched entry is exactly one of PASS(bounded) / OPEN. self.assertEqual(c['PASS(bounded)'] + c['OPEN'], c['entries_matched']) strict = run_gate(ROOT, '--strict') From a9b73e02f7120d1361a0af8afc8e18e050426590 Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Fri, 6 Nov 2026 19:57:13 +0800 Subject: [PATCH 03/19] Say what the bounded drawing change does not do, and refresh manifest lines A bounded drawing arena's edits and checkpoints do no per-record work in the bound: they copy only the blocks of 32 records they write. They do not cost O(changes): the first write after a checkpoint still copies the slot directory and the creation order (one 8-byte integer per occupant), and deleting the oldest drawing moves the directory's entries. The test header and the CHANGELOG now say so, and the test's function is renamed to match. The source allocation-bound manifest keeps its line fields as diagnostics. The entries of drawing.hpp that the change moved (U1665, U1669, U1670, U1671, U1672) carry the first line the gate finds for them again. No identity changes. Co-Authored-By: Claude Sonnet 5.5 --- CHANGELOG.md | 9 ++++--- ci/cr150-source-bounds.yaml | 10 +++---- tests/test_drawing_bound_checkpoint_cost.cpp | 28 +++++++++++++------- 3 files changed, 29 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3c79aad11..2438ba548 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -163,9 +163,12 @@ version number follows semantic versioning over the surfaces the whole state with a deep copy, so a script that moved a few of its live drawings on every bar paid time proportional to the bound on every call, and the first write after each script-state checkpoint paid it again; they now - copy only the blocks of 32 records they write. The bound, the typed - overflow outcome, the slot reuse order, the stale-handle behaviour and every - value a script reads are unchanged. + copy only the blocks of 32 records they write, so they do no per-record work + in the bound (the first write after a checkpoint still copies the slot + directory and the creation order, one 8-byte integer per occupant, and + deleting the oldest drawing moves the directory's entries). The bound, the + typed overflow outcome, the slot reuse order, the stale-handle behaviour and + every value a script reads are unchanged. ## 1.9.1 — 2026-10-31 diff --git a/ci/cr150-source-bounds.yaml b/ci/cr150-source-bounds.yaml index 2e3c1f94a..be57925b8 100644 --- a/ci/cr150-source-bounds.yaml +++ b/ci/cr150-source-bounds.yaml @@ -1683,12 +1683,12 @@ entries: - {id: "U1643", file: "src/native_execution_consumer.cpp", symbol: "cohort_opening_index_", ops: ["grow"], container: "unresolved by the index: cohort_opening_index_.reserve(...)", owner: "native-kernel", phase: "unclassified", state: "unreviewed", note: "Derived scratch populated from physical lots during eligible ordinary bar target reads. The lot count must fit row_budget before reserve(max_rows) and emplace_back; over-budget input or bad_alloc uses the complete reader. Reserve requests the fixed payload on rebuild and retains the first successful allocation across commands and begin. Layout estimate from source, not measured: 4096 rows of 16 bytes, 64 KiB payload, excluding vector and allocator overhead. Settlement and host lot-write generations invalidate the view; no rollback image, hash fold or persistence. Member-vector reserve and loop growth are outside the gate's bounded admission form, so this stays OPEN. Tests: tests/test_native_cohort_target_cost.cpp, tests/test_native_cohort_target_transcript.cpp, tests/test_native_cohort_target_invalidation.cpp, tests/test_native_cohort_target_bypass.cpp.", line: 14724} - {id: "U1644", file: "src/native_execution_consumer.hpp", symbol: "cohort_opening_index_", ops: ["decl"], container: "std::vector", owner: "native-kernel", phase: "unclassified", state: "unreviewed", note: "Consumer-owned derived scratch, rebuilt from physical lots during eligible ordinary bar target reads and invalidated by settlement and host lot-write generations. Capacity survives clear and begin; contents are never persisted or hashed and require no rollback image. Layout estimate from source, not measured: kCohortOpeningIndexBytes requests 64 KiB payload, 4096 rows at 16 bytes each, excluding vector and allocator overhead; larger lot sets use the complete reader. Unsupported member-vector admission stays OPEN. Tests: tests/test_native_cohort_target_cost.cpp, tests/test_native_cohort_target_transcript.cpp, tests/test_native_cohort_target_invalidation.cpp, tests/test_native_cohort_target_bypass.cpp.", line: 2867} - {id: "U1664", file: "include/pineforge/drawing.hpp", symbol: "", ops: ["temp"], container: "unresolved by the index: std::string", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 123} - - {id: "U1665", file: "include/pineforge/drawing.hpp", symbol: "bound_", ops: ["alloc"], container: "unresolved by the index: make_shared", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 710} + - {id: "U1665", file: "include/pineforge/drawing.hpp", symbol: "bound_", ops: ["alloc"], container: "unresolved by the index: make_shared", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 739} - {id: "U1667", file: "include/pineforge/drawing.hpp", symbol: "directory", ops: ["decl", "grow"], container: "std::vector", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 635} - - {id: "U1669", file: "include/pineforge/drawing.hpp", symbol: "order", ops: ["assign", "decl", "grow"], container: "std::deque", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 640} - - {id: "U1670", file: "include/pineforge/drawing.hpp", symbol: "records", ops: ["decl"], container: "detail::DrawingRecordStore", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 763} - - {id: "U1671", file: "include/pineforge/drawing.hpp", symbol: "slots", ops: ["decl", "grow"], container: "DrawingRecordStore", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 656} - - {id: "U1672", file: "include/pineforge/drawing.hpp", symbol: "value", ops: ["grow"], container: "unresolved by the index: value.emplace(...)", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 845} + - {id: "U1669", file: "include/pineforge/drawing.hpp", symbol: "order", ops: ["assign", "decl", "grow"], container: "std::deque", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 658} + - {id: "U1670", file: "include/pineforge/drawing.hpp", symbol: "records", ops: ["decl"], container: "detail::DrawingRecordStore", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 801} + - {id: "U1671", file: "include/pineforge/drawing.hpp", symbol: "slots", ops: ["decl", "grow"], container: "DrawingRecordStore", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 636} + - {id: "U1672", file: "include/pineforge/drawing.hpp", symbol: "value", ops: ["grow"], container: "unresolved by the index: value.emplace(...)", owner: "script-state", phase: "unclassified", state: "unreviewed", line: 890} - {id: "U1673", file: "include/pineforge/owned_storage.hpp", symbol: "", ops: ["temp"], container: "unresolved by the index: Base", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 342} - {id: "U1674", file: "include/pineforge/owned_storage.hpp", symbol: "", ops: ["temp"], container: "unresolved by the index: OwnedDeque", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 341} - {id: "U1675", file: "include/pineforge/owned_storage.hpp", symbol: "", ops: ["temp"], container: "unresolved by the index: std::deque>", owner: "engine-core", phase: "unclassified", state: "unreviewed", line: 338} diff --git a/tests/test_drawing_bound_checkpoint_cost.cpp b/tests/test_drawing_bound_checkpoint_cost.cpp index e93ae2407..306133187 100644 --- a/tests/test_drawing_bound_checkpoint_cost.cpp +++ b/tests/test_drawing_bound_checkpoint_cost.cpp @@ -1,4 +1,5 @@ -// K-DRAWBOUND: a bounded drawing arena's edits and checkpoints cost O(changes), not O(bound). +// K-DRAWBOUND: a bounded drawing arena's edits and checkpoints do no per-record work in +// the bound. // // A bounded arena (DrawingArena built from a DrawingBoundConfig: what the generated // code constructs from a script's declared max_lines_count, max_boxes_count and @@ -6,11 +7,18 @@ // copies share. Every create, every delete and every setter used to replace that // state with a deep copy of the whole of it -- every slot, the text of every label, // the directory and the order -- whether or not a checkpoint still shared it. A -// script that moves a few of its live drawings on every bar therefore paid O(bound) -// per call, and the first write after each checkpoint paid it once more. The unbound -// arena was already O(changes): its records live in blocks of 32 that copies share, -// and a write clones only the blocks it reaches (DrawingRecordStore, K-DRAWSNAP; the -// unbound twin of this test is test_coof_drawing_checkpoint_cost). +// script that moves a few of its live drawings on every bar therefore paid time +// proportional to the bound per call, and the first write after each checkpoint paid it +// once more. The unbound arena already did no per-record work in the bound: its records +// live in blocks of 32 that copies share, and a write clones only the blocks it reaches +// (DrawingRecordStore, K-DRAWSNAP; the unbound twin of this test is +// test_coof_drawing_checkpoint_cost). +// +// What this test claims is no per-record work in the bound, not O(changes): the first +// write after a checkpoint still copies the state's directory and order (one 8-byte +// integer per occupant), and deleting the oldest occupant moves the directory's entries. +// That is O(bound) in integers, with no record and no text copied, and the rows below +// count records and heap blocks, so they do not see it. // // The shape is a script with a bounded set of live levels (the strategy that exposed // it keeps up to five hundred lines and labels alive and, on every bar, sets two @@ -43,7 +51,7 @@ // operations (size, order, and every field of every live record), so the // cheaper copy changes no value. // unchanged checkpointing an arena nobody wrote allocates the same at bound 1,000 as -// at bound 100: sharing is O(1) in the bound. +// at bound 100: sharing the state costs the same at every bound. // // Under AddressSanitizer (whose fake stack makes every call dear) the cost row runs a // third of its executions. @@ -216,8 +224,8 @@ Cost run(int bound, int executions) { return cost; } -void test_edits_and_checkpoints_cost_o_changes() { - std::printf("test_edits_and_checkpoints_cost_o_changes\n"); +void test_edits_and_checkpoints_do_no_per_record_work() { + std::printf("test_edits_and_checkpoints_do_no_per_record_work\n"); const int executions = 150 / kExecutionDivisor; const int bounds[] = {100, 1000}; for (const int bound : bounds) { @@ -275,7 +283,7 @@ void test_unchanged_checkpoint_allocates_nothing_that_grows() { } // namespace int main() { - test_edits_and_checkpoints_cost_o_changes(); + test_edits_and_checkpoints_do_no_per_record_work(); test_unchanged_checkpoint_allocates_nothing_that_grows(); std::printf("\nbounded drawing cost tests: %d passed, %d failed\n", g_pass, g_fail); return g_fail == 0 ? 0 : 1; From 2beb60dfed2de6d7166c0ee0bb5fc39607a86460 Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Fri, 6 Nov 2026 20:21:15 +0800 Subject: [PATCH 04/19] Name the copied directory and order entries exactly The CHANGELOG entry and the cost test header of the bounded drawing change said that the first write after a checkpoint copies "one 8-byte integer per occupant". The slot directory holds two 32-bit integers per occupant (the id and the slot) and the creation order one 32-bit integer, so both now say "a directory entry and an order index per occupant". Words only: no code, no assertion and no manifest line changes. Co-Authored-By: Claude Sonnet 5.5 --- CHANGELOG.md | 8 ++++---- tests/test_drawing_bound_checkpoint_cost.cpp | 8 ++++---- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2438ba548..2bac5e558 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -165,10 +165,10 @@ version number follows semantic versioning over the surfaces the the first write after each script-state checkpoint paid it again; they now copy only the blocks of 32 records they write, so they do no per-record work in the bound (the first write after a checkpoint still copies the slot - directory and the creation order, one 8-byte integer per occupant, and - deleting the oldest drawing moves the directory's entries). The bound, the - typed overflow outcome, the slot reuse order, the stale-handle behaviour and - every value a script reads are unchanged. + directory and the creation order, a directory entry and an order index per + occupant, and deleting the oldest drawing moves the directory's entries). + The bound, the typed overflow outcome, the slot reuse order, the + stale-handle behaviour and every value a script reads are unchanged. ## 1.9.1 — 2026-10-31 diff --git a/tests/test_drawing_bound_checkpoint_cost.cpp b/tests/test_drawing_bound_checkpoint_cost.cpp index 306133187..4a0e93d05 100644 --- a/tests/test_drawing_bound_checkpoint_cost.cpp +++ b/tests/test_drawing_bound_checkpoint_cost.cpp @@ -15,10 +15,10 @@ // test_coof_drawing_checkpoint_cost). // // What this test claims is no per-record work in the bound, not O(changes): the first -// write after a checkpoint still copies the state's directory and order (one 8-byte -// integer per occupant), and deleting the oldest occupant moves the directory's entries. -// That is O(bound) in integers, with no record and no text copied, and the rows below -// count records and heap blocks, so they do not see it. +// write after a checkpoint still copies the state's directory and order (a directory +// entry and an order index per occupant), and deleting the oldest occupant moves the +// directory's entries. That is O(bound) in integers, with no record and no text copied, +// and the rows below count records and heap blocks, so they do not see it. // // The shape is a script with a bounded set of live levels (the strategy that exposed // it keeps up to five hundred lines and labels alive and, on every bar, sets two From 6ea32cd22baa782decf18a13129d1c7d3e794e03 Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Fri, 6 Nov 2026 12:40:36 +0800 Subject: [PATCH 05/19] Test signed short trade sizes Assert open and closed short accessors preserve direction from a real fill. --- tests/test_engine_trade_accessors.cpp | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/tests/test_engine_trade_accessors.cpp b/tests/test_engine_trade_accessors.cpp index ec974d72a..066c9f31e 100644 --- a/tests/test_engine_trade_accessors.cpp +++ b/tests/test_engine_trade_accessors.cpp @@ -238,6 +238,8 @@ class ShortProbe : public pineforge::source::PineStrategyHost { public: double profit_at_close = 0; double pct_at_close = 0; + double open_size_at_close = 0; + double closed_size_after = 0; ShortProbe() { source::PineStrategyConfig config; config.initial_capital = 100000; @@ -255,6 +257,11 @@ class ShortProbe : public pineforge::source::PineStrategyHost { // Read the snapshots BEFORE we close profit_at_close = open_trade_profit(0); pct_at_close = open_trade_profit_percent(0); + open_size_at_close = open_trade_size(0); + strategy_close("S1"); + } + if (bar_index_ == 4) { + closed_size_after = closed_trade_size(0); } } }; @@ -267,8 +274,8 @@ static void test_open_trade_short_path() { {100, 102, 98, 100, 1000, 60'000}, {100, 102, 98, 100, 1000, 120'000}, // entry "S1" placed → fills next bar at open { 99, 100, 94, 95, 500, 180'000}, // S1 short fills at open=99 - { 95, 96, 90, 92, 500, 240'000}, // snapshot bar (bar_index_ == 3); current_bar_.close=92 - { 92, 95, 90, 93, 500, 300'000}, + { 95, 96, 90, 92, 500, 240'000}, // snapshot at close=92, then request close + { 92, 95, 90, 93, 500, 300'000}, // close fills at open=92; read closed size }; strat.run(bars, 5); @@ -280,6 +287,8 @@ static void test_open_trade_short_path() { // profit / entry_cost * 100 = 14 / (99*2) * 100 ≈ 7.0707% // (was the gross short price-ratio (99/92-1)*100 ≈ 7.6087%). CHECK(near(strat.pct_at_close, 14.0 / (99.0 * 2.0) * 100.0, 1e-6)); + CHECK(near(strat.open_size_at_close, -2.0)); + CHECK(near(strat.closed_size_after, -2.0)); } // Zero-price guard branches in open_trade_profit_percent / From adf504432dfa7144dbf7e44107e90bc648feede0 Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Fri, 6 Nov 2026 12:57:56 +0800 Subject: [PATCH 06/19] Preserve direction in trade size accessors Report signed quantities for open and closed short trades while preserving long, flat and bounds behavior. --- include/pineforge/engine.hpp | 3 ++- src/engine_trade_accessors.cpp | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/include/pineforge/engine.hpp b/include/pineforge/engine.hpp index bc83246cf..d5d432bf8 100644 --- a/include/pineforge/engine.hpp +++ b/include/pineforge/engine.hpp @@ -1789,7 +1789,8 @@ class BacktestEngine { } double closed_trade_size(int idx) const { if (idx < 0 || idx >= (int)trades_.size()) return std::numeric_limits::quiet_NaN(); - return trades_.qty_at(idx); + const double qty = trades_.qty_at(idx); + return trades_.is_long_at(idx) ? qty : -qty; } double closed_trade_max_runup(int idx) const { if (idx < 0 || idx >= (int)trades_.size()) return 0.0; diff --git a/src/engine_trade_accessors.cpp b/src/engine_trade_accessors.cpp index e23ec40fb..b06d21214 100644 --- a/src/engine_trade_accessors.cpp +++ b/src/engine_trade_accessors.cpp @@ -90,7 +90,8 @@ int64_t BacktestEngine::open_trade_entry_time(int idx) const { double BacktestEngine::open_trade_size(int idx) const { if (position_side_ == PositionSide::FLAT || idx < 0 || idx >= (int)pyramid_entries_.size()) return na(); - return pyramid_entries_[(size_t)idx].qty; + const double qty = pyramid_entries_[(size_t)idx].qty; + return position_side_ == PositionSide::LONG ? qty : -qty; } // pe.max_drawdown / pe.max_runup are tracked in price-points × qty (the From eddfb2b40c697eda5a7b332532b92ea72f32e240 Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Fri, 6 Nov 2026 21:07:02 +0800 Subject: [PATCH 07/19] Docs: name signed-size corrections for the doc-reverts guard The signed trade-size accessor fix intentionally removes two obsolete public statements from docs/pages/pine-to-native.md. The sentence introduced by 7328f4d is wrong after the fix because Pine's size accessor carries direction in its sign; the sentence introduced by cced2aa is the same obsolete unsigned/raw-quantity description. The replacement rows state positive long and negative short and distinguish raw magnitude fields with is_long. Name 7328f4d and cced2aa so the doc-reverts guard can verify why those historical sentences were removed. --- docs/pages/pine-to-native.md | 4 ++-- tests/p2cx_tledger_fixture.hpp | 3 ++- .../test_aggregated_entry_bar_index_tape.cpp | 3 ++- ...st_bracket_lifecycle_declined_reversal.cpp | 10 +++++---- ...racket_lifecycle_declined_reversal_l4c.cpp | 10 +++++---- tests/test_closed_trade_text_reads.cpp | 4 +++- tests/test_cr277_close_reissue.cpp | 15 +++++++------ ...st_direct_short_reversal_affordability.cpp | 16 ++++++++------ ...t_exit_bracket_position_cycle_lifetime.cpp | 7 +++--- ...it_bracket_position_cycle_lifetime_l4c.cpp | 7 +++--- tests/test_margin_call.cpp | 10 ++++++++- tests/test_margin_call_1x_long_entry_fill.cpp | 9 +++++--- ...est_margin_call_1x_long_entry_fill_l4a.cpp | 9 +++++--- tests/test_margin_call_gap_open.cpp | 14 +++++++----- tests/test_margin_call_gap_open_l4a.cpp | 14 +++++++----- .../test_margin_call_intrabar_chronology.cpp | 22 ++++++++++--------- ...st_margin_call_intrabar_chronology_l4a.cpp | 22 ++++++++++--------- ...test_margin_call_trail_exit_chronology.cpp | 10 +++++---- ..._margin_call_trail_exit_chronology_l4a.cpp | 10 +++++---- tests/test_margin_rules_forward_replay.cpp | 8 +++++-- tests/test_native_open_lots.cpp | 2 +- tests/test_pyramiding_count_partial_drain.cpp | 4 +++- ...est_pyramiding_count_partial_drain_l4d.cpp | 4 +++- tests/test_short_reversal_emission.cpp | 16 +++++++++++--- tests/test_sizing_basis_mintick.cpp | 10 +++++---- tests/test_sizing_basis_mintick_l4b.cpp | 10 +++++---- tests/test_trade_ledger.cpp | 3 ++- 27 files changed, 159 insertions(+), 97 deletions(-) diff --git a/docs/pages/pine-to-native.md b/docs/pages/pine-to-native.md index 1605b6892..a91307308 100644 --- a/docs/pages/pine-to-native.md +++ b/docs/pages/pine-to-native.md @@ -593,7 +593,7 @@ native_order.hpp:467 names. | `strategy.opentrades.entry_bar_index()` | `entry_bar_index` native_host.hpp:336 | `pf_native_open_lot_v1` native_c_api.h:1779 | `native_open_lots_strategy.cpp` | Script-bar index of the opening fill. | | `strategy.opentrades.entry_time()` | `entry_time_ms` native_host.hpp:335 | `pf_native_open_lot_v1` native_c_api.h:1779 | `native_open_lots_strategy.cpp` | Effective time of the opening fill, Unix ms. | | `strategy.opentrades.entry_price()` | `entry_price` native_host.hpp:337 | `pf_native_open_lot_v1` native_c_api.h:1779 | `native_open_lots_strategy.cpp` | The booked price: slippage and the price grid are already applied. | -| `strategy.opentrades.size()` | `signed_units` native_host.hpp:338 | `pf_native_open_lot_v1` native_c_api.h:1779 | `native_open_lots_strategy.cpp` | Pine's is unsigned with a separate direction; here `> 0` is long, `< 0` short, and `side` native_host.hpp:332 says the same. | +| `strategy.opentrades.size()` | `signed_units` native_host.hpp:338 | `pf_native_open_lot_v1` native_c_api.h:1779 | `native_open_lots_strategy.cpp` | Pine and native open-lot sizes are signed: positive long, negative short; `side` native_host.hpp:332 also records direction. Raw `Trade.qty` / `pf_trade_t::qty` use a magnitude plus `is_long`. | | `strategy.opentrades.commission()` | `entry_commission` native_host.hpp:339 | `pf_native_open_lot_v1` native_c_api.h:1779 | `native_open_lots_strategy.cpp` | The entry fee still on the lot, account currency. A partial close takes its share onto the closed row: the example's FIFO reduce leaves 1.5 units carrying 1.5 of a 2.00 ticket. | | `strategy.opentrades.profit()` | `unrealized_pnl` native_host.hpp:341 | `pf_native_open_lot_v1` native_c_api.h:1779 | `native_open_lots_strategy.cpp` | The move from `entry_price` to `mark` in account currency, **net of `entry_commission`** — the lot's own term of the marked equity. Gross is `unrealized_pnl + entry_commission`. | | `strategy.opentrades.profit_percent()` | derive: `unrealized_pnl` native_host.hpp:341 over entry cost | — | `native_open_lots_strategy.cpp` | Entry cost is `entry_price * abs(signed_units) * point_value * account_fx`. | @@ -637,7 +637,7 @@ rows of that array. | `strategy.closedtrades.exit_bar_index()` | `exit_bar_index` engine.hpp:198 | `pf_trade_t` pineforge.h:203 | `native_open_lots_strategy.cpp` | | | `strategy.closedtrades.exit_time()` | `exit_time` engine.hpp:190 | `pf_trade_t` pineforge.h:203 | `native_open_lots_strategy.cpp` | | | `strategy.closedtrades.exit_price()` | `exit_price` engine.hpp:192 | `pf_trade_t` pineforge.h:203 | `native_open_lots_strategy.cpp` | | -| `strategy.closedtrades.size()` | `qty` engine.hpp:193 with `is_long` engine.hpp:196 | `pf_trade_t` pineforge.h:203 | `native_open_lots_strategy.cpp` | Unsigned, with the direction beside it. A partial close books a row for the closed slice only. | +| `strategy.closedtrades.size()` | `qty` engine.hpp:193 with `is_long` engine.hpp:196 | `pf_trade_t` pineforge.h:203 | `native_open_lots_strategy.cpp` | Pine's accessor is signed: positive long, negative short. Raw `Trade.qty` / `pf_trade_t::qty` are magnitudes with separate `is_long`; apply that direction to match Pine. A partial close books only the closed slice. | | `strategy.closedtrades.profit()` | `pnl` engine.hpp:194 | `pf_trade_t` pineforge.h:203 | `native_open_lots_strategy.cpp` | Net of `commission` engine.hpp:217. `pnl + commission` is the gross move, which equals the lot's `unrealized_pnl + entry_commission` at a mark equal to the exit price, scaled to the closed slice. | | `strategy.closedtrades.profit_percent()` | `pnl_pct` engine.hpp:195 | `pf_trade_t` pineforge.h:203 | `native_open_lots_strategy.cpp` | Net return on entry cost. | | `strategy.closedtrades.commission()` | `commission` engine.hpp:217 | `pf_trade_t` pineforge.h:203 | `native_open_lots_strategy.cpp` | Entry share plus exit share. One cash-per-execution ticket is split by units over every slice of that execution. | diff --git a/tests/p2cx_tledger_fixture.hpp b/tests/p2cx_tledger_fixture.hpp index d64d5fe1d..11a348dd9 100644 --- a/tests/p2cx_tledger_fixture.hpp +++ b/tests/p2cx_tledger_fixture.hpp @@ -121,7 +121,8 @@ struct Reader : Host { CHECK(same_bits(closed_trade_exit_price(i), expected.exit_price)); CHECK(closed_trade_entry_time(i) == expected.entry_time); CHECK(closed_trade_exit_time(i) == expected.exit_time); - CHECK(same_bits(closed_trade_size(i), expected.qty)); + // Keep the quantity bits, including zero, and apply the fixture side. + CHECK(same_bits(closed_trade_size(i), expected.is_long ? expected.qty : -expected.qty)); CHECK(same_bits(closed_trade_max_runup(i), expected.max_runup)); CHECK(same_bits(closed_trade_max_drawdown(i), expected.max_drawdown)); const double cost = expected.entry_price * expected.qty; diff --git a/tests/test_aggregated_entry_bar_index_tape.cpp b/tests/test_aggregated_entry_bar_index_tape.cpp index e9375a850..9923c0625 100644 --- a/tests/test_aggregated_entry_bar_index_tape.cpp +++ b/tests/test_aggregated_entry_bar_index_tape.cpp @@ -366,7 +366,8 @@ void test_tape_rows() { && r.entry_ms == tv.entry_ms && r.exit_ms == tv.exit_ms && same_price(r.entry_price, tv.entry_price) && same_price(r.exit_price, tv.exit_price) - && r.qty == tv.qty + // Tape quantities are magnitudes with an independent side. + && r.qty == (tv.is_long ? tv.qty : -tv.qty) && r.entry_id == tv.entry_signal && r.exit_comment == tv.exit_signal && cents(r.profit, tv.net_pnl) && (mag || (cents(r.runup, tv.favorable) && cents(-r.drawdown, tv.adverse))) diff --git a/tests/test_bracket_lifecycle_declined_reversal.cpp b/tests/test_bracket_lifecycle_declined_reversal.cpp index c4fc278c0..836fceba3 100644 --- a/tests/test_bracket_lifecycle_declined_reversal.cpp +++ b/tests/test_bracket_lifecycle_declined_reversal.cpp @@ -132,6 +132,8 @@ class Probe : public pineforge::source::PineStrategyHost { std::string x_comment(int i) const { return closed_trade_exit_comment(i); } std::string x_id(int i) const { return closed_trade_exit_id(i); } double x_price(int i) const { return closed_trade_exit_price(i); } + // expectation corrected: 47.0588235294/52.9411764706 -> their negatives, + // because both margin and bracket slices close ShortMcProbe's short. double t_size(int i) const { return closed_trade_size(i); } int x_bar(int i) const { return closed_trade_exit_bar_index(i); } using BacktestEngine::position_qty_; @@ -300,11 +302,11 @@ static void test_revive_B_margin_call_partial_revives() { CHECK(p.trade_count() == 2); if (p.trade_count() == 2) { CHECK(p.x_comment(0) == std::string("Margin call")); - CHECK_NEAR(p.t_size(0), 47.0588235294, 1e-6); + CHECK_NEAR(p.t_size(0), -47.0588235294, 1e-6); CHECK_NEAR(p.x_price(0), 170.0, 1e-9); CHECK(p.x_bar(0) == 3); CHECK(p.x_id(1) == std::string("X")); // revived bracket fill - CHECK_NEAR(p.t_size(1), 52.9411764706, 1e-6); + CHECK_NEAR(p.t_size(1), -52.9411764706, 1e-6); CHECK_NEAR(p.x_price(1), 180.0, 1e-9); // ORIGINAL armed price CHECK(p.x_bar(1) == 4); } @@ -323,12 +325,12 @@ static void test_cascade_marketable_revived_stop() { CHECK(p.trade_count() == 2); if (p.trade_count() == 2) { CHECK(p.x_comment(0) == std::string("Margin call")); - CHECK_NEAR(p.t_size(0), 47.0588235294, 1e-6); + CHECK_NEAR(p.t_size(0), -47.0588235294, 1e-6); CHECK_NEAR(p.x_price(0), 170.0, 1e-9); // NOT the 165 open CHECK(p.x_bar(0) == 3); CHECK(p.x_id(1) == std::string("X")); CHECK(p.x_comment(1) != std::string("Margin call")); - CHECK_NEAR(p.t_size(1), 52.9411764706, 1e-6); + CHECK_NEAR(p.t_size(1), -52.9411764706, 1e-6); CHECK_NEAR(p.x_price(1), 170.0, 1e-9); // MC event price CHECK(p.x_bar(1) == 3); // same bar as the slice } diff --git a/tests/test_bracket_lifecycle_declined_reversal_l4c.cpp b/tests/test_bracket_lifecycle_declined_reversal_l4c.cpp index 431797283..341746028 100644 --- a/tests/test_bracket_lifecycle_declined_reversal_l4c.cpp +++ b/tests/test_bracket_lifecycle_declined_reversal_l4c.cpp @@ -136,6 +136,8 @@ class Probe : public pineforge::source::PineStrategyHost { std::string x_comment(int i) const { return closed_trade_exit_comment(i); } std::string x_id(int i) const { return closed_trade_exit_id(i); } double x_price(int i) const { return closed_trade_exit_price(i); } + // expectation corrected: 47.0588235294/52.9411764706 -> their negatives, + // because both margin and bracket slices close ShortMcProbe's short. double t_size(int i) const { return closed_trade_size(i); } int x_bar(int i) const { return closed_trade_exit_bar_index(i); } using BacktestEngine::position_qty_; @@ -302,11 +304,11 @@ static void test_revive_B_margin_call_partial_revives() { CHECK(p.trade_count() == 2); if (p.trade_count() == 2) { CHECK(p.x_comment(0) == std::string("Margin call")); - CHECK_NEAR(p.t_size(0), 47.0588235294, 1e-6); + CHECK_NEAR(p.t_size(0), -47.0588235294, 1e-6); CHECK_NEAR(p.x_price(0), 170.0, 1e-9); CHECK(p.x_bar(0) == 3); CHECK(p.x_id(1) == std::string("X")); // revived bracket fill - CHECK_NEAR(p.t_size(1), 52.9411764706, 1e-6); + CHECK_NEAR(p.t_size(1), -52.9411764706, 1e-6); CHECK_NEAR(p.x_price(1), 180.0, 1e-9); // ORIGINAL armed price CHECK(p.x_bar(1) == 4); } @@ -325,12 +327,12 @@ static void test_cascade_marketable_revived_stop() { CHECK(p.trade_count() == 2); if (p.trade_count() == 2) { CHECK(p.x_comment(0) == std::string("Margin call")); - CHECK_NEAR(p.t_size(0), 47.0588235294, 1e-6); + CHECK_NEAR(p.t_size(0), -47.0588235294, 1e-6); CHECK_NEAR(p.x_price(0), 170.0, 1e-9); // NOT the 165 open CHECK(p.x_bar(0) == 3); CHECK(p.x_id(1) == std::string("X")); CHECK(p.x_comment(1) != std::string("Margin call")); - CHECK_NEAR(p.t_size(1), 52.9411764706, 1e-6); + CHECK_NEAR(p.t_size(1), -52.9411764706, 1e-6); CHECK_NEAR(p.x_price(1), 170.0, 1e-9); // MC event price CHECK(p.x_bar(1) == 3); // same bar as the slice } diff --git a/tests/test_closed_trade_text_reads.cpp b/tests/test_closed_trade_text_reads.cpp index 6f4680737..5e6811b26 100644 --- a/tests/test_closed_trade_text_reads.cpp +++ b/tests/test_closed_trade_text_reads.cpp @@ -1155,7 +1155,9 @@ struct SealedReader : Reader { SEALED_EXPECT(double_bytes(closed_trade_exit_price(i), expected.exit_price)); SEALED_EXPECT(closed_trade_entry_time(i) == expected.entry_time); SEALED_EXPECT(closed_trade_exit_time(i) == expected.exit_time); - SEALED_EXPECT(double_bytes(closed_trade_size(i), expected.qty)); + // The fixture's raw quantity stays unchanged; Pine adds its direction. + SEALED_EXPECT(double_bytes(closed_trade_size(i), + expected.is_long ? expected.qty : -expected.qty)); SEALED_EXPECT(double_bytes(closed_trade_max_runup(i), expected.max_runup)); SEALED_EXPECT(double_bytes(closed_trade_max_drawdown(i), expected.max_drawdown)); // Independent cost = input entry price * input quantity * pointvalue(2). diff --git a/tests/test_cr277_close_reissue.cpp b/tests/test_cr277_close_reissue.cpp index 72ccfc91e..d86adf2f1 100644 --- a/tests/test_cr277_close_reissue.cpp +++ b/tests/test_cr277_close_reissue.cpp @@ -330,7 +330,7 @@ void check_row(const Row& row) { CHECK(host.physical_position().signed_units == -expected_short); CHECK(host.physical_position().lot_count == 1); CHECK(host.open_trade_entry_id(0) == "X"); - CHECK(host.open_trade_size(0) == expected_short); + CHECK(host.open_trade_size(0) == -expected_short); CHECK(host.open_trade_entry_price(0) == 100.0); CHECK(host.trade_count() == 1); if (host.trade_count() == 1) { @@ -515,7 +515,8 @@ int open_id_count(const ScopeProbeHost& host, const char* id) { return count; } -double open_id_units(const ScopeProbeHost& host, const char* id) { +// Signed units for the matching live lots, as in physical_position(). +double open_id_signed_units(const ScopeProbeHost& host, const char* id) { double units = 0.0; for (std::size_t index = 0; index < host.physical_position().lot_count; ++index) { if (host.open_trade_entry_id(static_cast(index)) == id) @@ -597,7 +598,7 @@ void ScopeProbeHost::stage_reversal_close_all_capture() { CHECK(physical_position().signed_units == seed_units); CHECK(physical_position().lot_count == 1); CHECK(open_id_count(*this, "L") == 1); - CHECK(open_id_units(*this, "L") == seed_units); + CHECK(open_id_signed_units(*this, "L") == seed_units); CHECK(fill_id_count(*this, "S") == 0); // Exactly the three order commands of script bar 190. Observations @@ -686,7 +687,7 @@ void ScopeProbeHost::check_reversal_close_all_release() { CHECK(s_lots == 1); CHECK(physical_position().lot_count == 1); CHECK(physical_position().signed_units == -seed_units); - CHECK(open_id_units(*this, "S") == seed_units); + CHECK(open_id_signed_units(*this, "S") == -seed_units); CHECK(open_id_count(*this, "L") == 0); CHECK(fill_id_count(*this, "__close__") == 1); CHECK(trade_count() == 1); @@ -874,8 +875,8 @@ void check_scope_row(const ScopeRow& row) { CHECK(host.physical_position().lot_count == 2); CHECK(open_id_count(host, "X") == 1); CHECK(open_id_count(host, "Y") == 1); - CHECK(open_id_units(host, "X") == seed_units); - CHECK(open_id_units(host, "Y") == seed_units); + CHECK(open_id_signed_units(host, "X") == -seed_units); + CHECK(open_id_signed_units(host, "Y") == -seed_units); CHECK(fill_id_count(host, "X") == 1); CHECK(fill_id_count(host, "Y") == 1); } else if (row.scenario == ScopeCase::QueuedReplacement) { @@ -893,7 +894,7 @@ void check_scope_row(const ScopeRow& row) { CHECK(host.physical_position().signed_units == -reissue_units); CHECK(host.physical_position().lot_count == 1); CHECK(open_id_count(host, "X") == 1); - CHECK(open_id_units(host, "X") == reissue_units); + CHECK(open_id_signed_units(host, "X") == -reissue_units); CHECK(fill_id_count(host, "X") == 1); } CHECK(strategy_pending_orders_len(&host) == 0); diff --git a/tests/test_direct_short_reversal_affordability.cpp b/tests/test_direct_short_reversal_affordability.cpp index b227e8da2..dcdf71867 100644 --- a/tests/test_direct_short_reversal_affordability.cpp +++ b/tests/test_direct_short_reversal_affordability.cpp @@ -83,6 +83,8 @@ class DirectShortReversalProbe : public pineforge::source::PineStrategyHost { && std::isnan(opening_obligations_.raw_fill_base()); } + // expectation corrected: +slice -> -slice, because every checked margin + // slice here closes a short; the original quantity and price oracles stay. std::vector margin_quantities() const { std::vector out; for (int index = 0; index < trade_count(); ++index) { @@ -172,9 +174,9 @@ void test_direct_reversal_runs_opening_check_and_one_adverse_retry() { CHECK(qty.size() == 2U); CHECK(price.size() == 2U); if (qty.size() == 2U && price.size() == 2U) { - CHECK_NEAR(qty[0], 0.0376, 1e-9); + CHECK_NEAR(qty[0], -0.0376, 1e-9); CHECK_NEAR(price[0], 3145.01, 1e-9); - CHECK_NEAR(qty[1], 0.6204, 1e-9); + CHECK_NEAR(qty[1], -0.6204, 1e-9); CHECK_NEAR(price[1], 3154.20, 1e-9); } CHECK_NEAR(probe.position_size(), -30.8219, 1e-9); @@ -228,9 +230,9 @@ void test_direct_reversal_floor_zero_full_residual_yields_to_slice() { CHECK(one_contract_price.size() == 2U); if (one_contract_qty.size() == 2U && one_contract_price.size() == 2U) { - CHECK_NEAR(one_contract_qty[0], 0.0392, 1e-9); + CHECK_NEAR(one_contract_qty[0], -0.0392, 1e-9); CHECK_NEAR(one_contract_price[0], 4514.70, 1e-9); - CHECK_NEAR(one_contract_qty[1], 1.0, 1e-9); + CHECK_NEAR(one_contract_qty[1], -1.0, 1e-9); CHECK_NEAR(one_contract_price[1], 4539.00, 1e-9); } CHECK_NEAR(one_contract.position_size(), -1.7346, 1e-9); @@ -251,9 +253,9 @@ void test_direct_reversal_floor_zero_full_residual_yields_to_slice() { CHECK(full_residual_price.size() == 2U); if (full_residual_qty.size() == 2U && full_residual_price.size() == 2U) { - CHECK_NEAR(full_residual_qty[0], 0.0392, 1e-9); + CHECK_NEAR(full_residual_qty[0], -0.0392, 1e-9); CHECK_NEAR(full_residual_price[0], 4514.70, 1e-9); - CHECK_NEAR(full_residual_qty[1], 1.0, 1e-9); + CHECK_NEAR(full_residual_qty[1], -1.0, 1e-9); CHECK_NEAR(full_residual_price[1], 4539.00, 1e-9); } CHECK_NEAR(full_residual.position_size(), -1.7346, 1e-9); @@ -308,7 +310,7 @@ void test_true_flat_floor_zero_control_slices_one_contract() { const std::vector qty = probe.margin_quantities(); CHECK(qty.size() == 1U); if (qty.size() == 1U) { - CHECK_NEAR(qty[0], 1.0, 1e-9); + CHECK_NEAR(qty[0], -1.0, 1e-9); } CHECK_NEAR(probe.position_size(), -2.6930, 1e-9); CHECK(probe.has_live_short_position()); diff --git a/tests/test_exit_bracket_position_cycle_lifetime.cpp b/tests/test_exit_bracket_position_cycle_lifetime.cpp index eb04d4489..925cf45bb 100644 --- a/tests/test_exit_bracket_position_cycle_lifetime.cpp +++ b/tests/test_exit_bracket_position_cycle_lifetime.cpp @@ -104,6 +104,7 @@ class CycleProbe : public pineforge::source::PineStrategyHost { std::string entry_id(int i) const { return closed_trade_entry_id(i); } std::string exit_id(int i) const { return closed_trade_exit_id(i); } double exit_price(int i) const { return closed_trade_exit_price(i); } + // expectation corrected: 1/2 -> -1/-2, because these cycles are short. double size(int i) const { return closed_trade_size(i); } double position_size() const { return signed_position_size(); } @@ -275,7 +276,7 @@ static void test_orphaned_bucket_leg_still_fires() { CHECK(eng.rows_with_exit_id("T2") == 1); // the orphaned leg fired CHECK(eng.rows_at_exit_price(96.0) == 2); CHECK(eng.rows_at_exit_price(90.0) == 2); - for (int i = 0; i < 4; ++i) CHECK(near(eng.size(i), 1.0)); + for (int i = 0; i < 4; ++i) CHECK(near(eng.size(i), -1.0)); } // ---- B: reaching flat re-admits the next cycle's same-id entry ------------- @@ -293,7 +294,7 @@ static void test_flat_readmits_next_cycle_entry() { CHECK(eng.trade_count() == 5); if (eng.trade_count() < 5) return; CHECK(eng.entry_id(4) == std::string("S")); - CHECK(near(eng.size(4), 2.0)); + CHECK(near(eng.size(4), -2.0)); CHECK(near(eng.position_size(), 0.0)); } @@ -312,7 +313,7 @@ static void test_leg_for_unfilled_entry_id_is_removed() { CHECK(eng.trade_count() == 1); if (eng.trade_count() < 1) return; CHECK(eng.exit_id(0) == std::string("REAL")); - CHECK(near(eng.size(0), 2.0)); + CHECK(near(eng.size(0), -2.0)); CHECK(near(eng.exit_price(0), 90.0)); CHECK(near(eng.position_size(), 0.0)); } diff --git a/tests/test_exit_bracket_position_cycle_lifetime_l4c.cpp b/tests/test_exit_bracket_position_cycle_lifetime_l4c.cpp index e7ce7003e..1d3e5aa42 100644 --- a/tests/test_exit_bracket_position_cycle_lifetime_l4c.cpp +++ b/tests/test_exit_bracket_position_cycle_lifetime_l4c.cpp @@ -108,6 +108,7 @@ class CycleProbe : public pineforge::source::PineStrategyHost { std::string entry_id(int i) const { return closed_trade_entry_id(i); } std::string exit_id(int i) const { return closed_trade_exit_id(i); } double exit_price(int i) const { return closed_trade_exit_price(i); } + // expectation corrected: 1/2 -> -1/-2, because these cycles are short. double size(int i) const { return closed_trade_size(i); } double position_size() const { return signed_position_size(); } @@ -279,7 +280,7 @@ static void test_orphaned_bucket_leg_still_fires() { CHECK(eng.rows_with_exit_id("T2") == 1); // the orphaned leg fired CHECK(eng.rows_at_exit_price(96.0) == 2); CHECK(eng.rows_at_exit_price(90.0) == 2); - for (int i = 0; i < 4; ++i) CHECK(near(eng.size(i), 1.0)); + for (int i = 0; i < 4; ++i) CHECK(near(eng.size(i), -1.0)); } // ---- B: reaching flat re-admits the next cycle's same-id entry ------------- @@ -297,7 +298,7 @@ static void test_flat_readmits_next_cycle_entry() { CHECK(eng.trade_count() == 5); if (eng.trade_count() < 5) return; CHECK(eng.entry_id(4) == std::string("S")); - CHECK(near(eng.size(4), 2.0)); + CHECK(near(eng.size(4), -2.0)); CHECK(near(eng.position_size(), 0.0)); } @@ -316,7 +317,7 @@ static void test_leg_for_unfilled_entry_id_is_removed() { CHECK(eng.trade_count() == 1); if (eng.trade_count() < 1) return; CHECK(eng.exit_id(0) == std::string("REAL")); - CHECK(near(eng.size(0), 2.0)); + CHECK(near(eng.size(0), -2.0)); CHECK(near(eng.exit_price(0), 90.0)); CHECK(near(eng.position_size(), 0.0)); } diff --git a/tests/test_margin_call.cpp b/tests/test_margin_call.cpp index 305ae0adf..46f6ac7af 100644 --- a/tests/test_margin_call.cpp +++ b/tests/test_margin_call.cpp @@ -77,7 +77,15 @@ class MCEngine : public pineforge::source::PineStrategyHost { std::string exit_comment(int i) const { return closed_trade_exit_comment(i); } double exit_price(int i) const { return closed_trade_exit_price(i); } double entry_price(int i) const { return closed_trade_entry_price(i); } - double trade_size(int i) const { return closed_trade_size(i); } + // The liquidation budgets, lot-grid checks and sums below use magnitudes. + // Validate Pine's signed size against the stored row before projecting it. + double trade_size(int i) const { + const Trade trade = get_trade(i); + const double signed_size = closed_trade_size(i); + CHECK(trade.qty >= 0.0); + CHECK(signed_size == (trade.is_long ? trade.qty : -trade.qty)); + return trade.is_long ? signed_size : -signed_size; + } int entry_bar(int i) const { return closed_trade_entry_bar_index(i); } int exit_bar(int i) const { return closed_trade_exit_bar_index(i); } double position_size() const { return signed_position_size(); } diff --git a/tests/test_margin_call_1x_long_entry_fill.cpp b/tests/test_margin_call_1x_long_entry_fill.cpp index 66ba69622..a45871fb1 100644 --- a/tests/test_margin_call_1x_long_entry_fill.cpp +++ b/tests/test_margin_call_1x_long_entry_fill.cpp @@ -201,7 +201,8 @@ static void test_one_contract_slice_before_same_bar_stop() { // stop closing the reduced remainder. CHECK(eng.trade_count() == 3); CHECK(margin_call_rows(eng) == 1); - CHECK(near(eng.trade_size(0), 3.3333)); + // expectation corrected: 3.3333 -> -3.3333, because this closes the short. + CHECK(near(eng.trade_size(0), -3.3333)); CHECK(near(eng.exit_price(0), 2997.50)); CHECK(eng.exit_comment(1) == std::string("Margin call")); CHECK(near(eng.trade_size(1), 1.0)); @@ -297,11 +298,13 @@ static void test_short_one_x_mirror_trims_before_stop() { CHECK(eng.trade_count() == 2); CHECK(margin_call_rows(eng) == 1); CHECK(eng.exit_comment(0) == std::string("Margin call")); - CHECK(near(eng.trade_size(0), 0.1996)); + // expectation corrected: 0.1996/99.7504 -> -0.1996/-99.7504, + // because both rows close the commissioned short mirror. + CHECK(near(eng.trade_size(0), -0.1996)); CHECK(near(eng.exit_price(0), 100.0)); CHECK(eng.exit_bar(0) == 1); CHECK(eng.exit_id(1) == std::string("X")); - CHECK(near(eng.trade_size(1), 99.7504)); + CHECK(near(eng.trade_size(1), -99.7504)); CHECK(near(eng.exit_price(1), 105.0)); CHECK(near(eng.position_size(), 0.0)); } diff --git a/tests/test_margin_call_1x_long_entry_fill_l4a.cpp b/tests/test_margin_call_1x_long_entry_fill_l4a.cpp index 5c436d3fa..e9cebc4ae 100644 --- a/tests/test_margin_call_1x_long_entry_fill_l4a.cpp +++ b/tests/test_margin_call_1x_long_entry_fill_l4a.cpp @@ -204,7 +204,8 @@ static void test_one_contract_slice_before_same_bar_stop() { // stop closing the reduced remainder. CHECK(eng.trade_count() == 3); CHECK(margin_call_rows(eng) == 1); - CHECK(near(eng.trade_size(0), 3.3333)); + // expectation corrected: 3.3333 -> -3.3333, because this closes the short. + CHECK(near(eng.trade_size(0), -3.3333)); CHECK(near(eng.exit_price(0), 2997.50)); CHECK(eng.exit_comment(1) == std::string("Margin call")); CHECK(near(eng.trade_size(1), 1.0)); @@ -300,11 +301,13 @@ static void test_short_one_x_mirror_trims_before_stop() { CHECK(eng.trade_count() == 2); CHECK(margin_call_rows(eng) == 1); CHECK(eng.exit_comment(0) == std::string("Margin call")); - CHECK(near(eng.trade_size(0), 0.1996)); + // expectation corrected: 0.1996/99.7504 -> -0.1996/-99.7504, + // because both rows close the commissioned short mirror. + CHECK(near(eng.trade_size(0), -0.1996)); CHECK(near(eng.exit_price(0), 100.0)); CHECK(eng.exit_bar(0) == 1); CHECK(eng.exit_id(1) == std::string("X")); - CHECK(near(eng.trade_size(1), 99.7504)); + CHECK(near(eng.trade_size(1), -99.7504)); CHECK(near(eng.exit_price(1), 105.0)); CHECK(near(eng.position_size(), 0.0)); } diff --git a/tests/test_margin_call_gap_open.cpp b/tests/test_margin_call_gap_open.cpp index 856c7ab07..dbdab5b69 100644 --- a/tests/test_margin_call_gap_open.cpp +++ b/tests/test_margin_call_gap_open.cpp @@ -69,6 +69,8 @@ class MCEngine : public pineforge::source::PineStrategyHost { std::string exit_comment(int i) const { return closed_trade_exit_comment(i); } double exit_price(int i) const { return closed_trade_exit_price(i); } double entry_price(int i) const { return closed_trade_entry_price(i); } + // expectation corrected: short +quantity -> -quantity, because Pine size + // encodes the side; the fill-derived magnitudes below are unchanged. double trade_size(int i) const { return closed_trade_size(i); } int exit_bar(int i) const { return closed_trade_exit_bar_index(i); } double position_size() const { return signed_position_size(); } @@ -146,7 +148,7 @@ static void test_gap_open_single_slice_at_open() { CHECK(eng.exit_comment(0) == std::string("Margin call")); CHECK(eng.exit_bar(0) == 1); CHECK(near(eng.exit_price(0), 104.0)); // AT THE OPEN, not the high - CHECK(near(eng.trade_size(0), 3.0769230769, 1e-6)); // open-priced 4x shortfall + CHECK(near(eng.trade_size(0), -3.0769230769, 1e-6)); // open-priced 4x shortfall } CHECK(near(eng.position_size(), -(10.0 - 3.0769230769), 1e-6)); } @@ -172,9 +174,9 @@ static void test_gap_open_then_extreme_second_slice() { CHECK(eng.exit_bar(0) == 1); CHECK(eng.exit_bar(1) == 1); CHECK(near(eng.exit_price(0), 104.0)); - CHECK(near(eng.trade_size(0), 3.0769230769, 1e-6)); + CHECK(near(eng.trade_size(0), -3.0769230769, 1e-6)); CHECK(near(eng.exit_price(1), 130.0)); - CHECK(near(eng.trade_size(1), 3.6923076923, 1e-6)); + CHECK(near(eng.trade_size(1), -3.6923076923, 1e-6)); } } @@ -194,7 +196,7 @@ static void test_no_open_breach_keeps_extreme_only() { CHECK(count_margin_calls(eng) == 1); if (eng.trade_count() >= 1) { CHECK(near(eng.exit_price(0), 105.0)); - CHECK(near(eng.trade_size(0), 3.80952381, 1e-4)); + CHECK(near(eng.trade_size(0), -3.80952381, 1e-4)); CHECK(eng.exit_bar(0) == 1); } } @@ -216,7 +218,7 @@ static void test_gap_open_whole_share_grid() { if (eng.trade_count() >= 1) { CHECK(eng.exit_comment(0) == std::string("Margin call")); CHECK(near(eng.exit_price(0), 104.0)); - CHECK(near(eng.trade_size(0), 1.0)); + CHECK(near(eng.trade_size(0), -1.0)); CHECK(eng.exit_bar(0) == 1); } } @@ -235,7 +237,7 @@ static void test_gap_open_whole_share_grid() { if (eng.trade_count() >= 1) { CHECK(eng.exit_comment(0) == std::string("Margin call")); CHECK(near(eng.exit_price(0), 112.0)); - CHECK(near(eng.trade_size(0), 8.0)); + CHECK(near(eng.trade_size(0), -8.0)); } // Survivor 2 @112.5: equity = 1000 - 12*8 - 12.5*2 = 879, required // 225 -> no second slice on this bar. diff --git a/tests/test_margin_call_gap_open_l4a.cpp b/tests/test_margin_call_gap_open_l4a.cpp index a5f7fae77..db6161f41 100644 --- a/tests/test_margin_call_gap_open_l4a.cpp +++ b/tests/test_margin_call_gap_open_l4a.cpp @@ -72,6 +72,8 @@ class MCEngine : public pineforge::source::PineStrategyHost { std::string exit_comment(int i) const { return closed_trade_exit_comment(i); } double exit_price(int i) const { return closed_trade_exit_price(i); } double entry_price(int i) const { return closed_trade_entry_price(i); } + // expectation corrected: short +quantity -> -quantity, because Pine size + // encodes the side; the fill-derived magnitudes below are unchanged. double trade_size(int i) const { return closed_trade_size(i); } int exit_bar(int i) const { return closed_trade_exit_bar_index(i); } double position_size() const { return signed_position_size(); } @@ -149,7 +151,7 @@ static void test_gap_open_single_slice_at_open() { CHECK(eng.exit_comment(0) == std::string("Margin call")); CHECK(eng.exit_bar(0) == 1); CHECK(near(eng.exit_price(0), 104.0)); // AT THE OPEN, not the high - CHECK(near(eng.trade_size(0), 3.0769230769, 1e-6)); // open-priced 4x shortfall + CHECK(near(eng.trade_size(0), -3.0769230769, 1e-6)); // open-priced 4x shortfall } CHECK(near(eng.position_size(), -(10.0 - 3.0769230769), 1e-6)); } @@ -175,9 +177,9 @@ static void test_gap_open_then_extreme_second_slice() { CHECK(eng.exit_bar(0) == 1); CHECK(eng.exit_bar(1) == 1); CHECK(near(eng.exit_price(0), 104.0)); - CHECK(near(eng.trade_size(0), 3.0769230769, 1e-6)); + CHECK(near(eng.trade_size(0), -3.0769230769, 1e-6)); CHECK(near(eng.exit_price(1), 130.0)); - CHECK(near(eng.trade_size(1), 3.6923076923, 1e-6)); + CHECK(near(eng.trade_size(1), -3.6923076923, 1e-6)); } } @@ -197,7 +199,7 @@ static void test_no_open_breach_keeps_extreme_only() { CHECK(count_margin_calls(eng) == 1); if (eng.trade_count() >= 1) { CHECK(near(eng.exit_price(0), 105.0)); - CHECK(near(eng.trade_size(0), 3.80952381, 1e-4)); + CHECK(near(eng.trade_size(0), -3.80952381, 1e-4)); CHECK(eng.exit_bar(0) == 1); } } @@ -219,7 +221,7 @@ static void test_gap_open_whole_share_grid() { if (eng.trade_count() >= 1) { CHECK(eng.exit_comment(0) == std::string("Margin call")); CHECK(near(eng.exit_price(0), 104.0)); - CHECK(near(eng.trade_size(0), 1.0)); + CHECK(near(eng.trade_size(0), -1.0)); CHECK(eng.exit_bar(0) == 1); } } @@ -238,7 +240,7 @@ static void test_gap_open_whole_share_grid() { if (eng.trade_count() >= 1) { CHECK(eng.exit_comment(0) == std::string("Margin call")); CHECK(near(eng.exit_price(0), 112.0)); - CHECK(near(eng.trade_size(0), 8.0)); + CHECK(near(eng.trade_size(0), -8.0)); } // Survivor 2 @112.5: equity = 1000 - 12*8 - 12.5*2 = 879, required // 225 -> no second slice on this bar. diff --git a/tests/test_margin_call_intrabar_chronology.cpp b/tests/test_margin_call_intrabar_chronology.cpp index bc8a97b06..7aa61bf53 100644 --- a/tests/test_margin_call_intrabar_chronology.cpp +++ b/tests/test_margin_call_intrabar_chronology.cpp @@ -78,6 +78,8 @@ class MCEngine : public pineforge::source::PineStrategyHost { std::string exit_comment(int i) const { return closed_trade_exit_comment(i); } double exit_price(int i) const { return closed_trade_exit_price(i); } double entry_price(int i) const { return closed_trade_entry_price(i); } + // expectation corrected: short +quantity -> -quantity, because Pine size + // encodes the side; the fill-derived magnitudes below are unchanged. double trade_size(int i) const { return closed_trade_size(i); } int exit_bar(int i) const { return closed_trade_exit_bar_index(i); } double position_size() const { return signed_position_size(); } @@ -168,12 +170,12 @@ static void test_gap_event_slices_before_tp_exit() { CHECK(eng.trade_count() == 2); CHECK(margin_call_rows(eng) == 1); CHECK(eng.exit_comment(0) == std::string("Margin call")); - CHECK(near(eng.trade_size(0), 0.0084, 1e-9)); + CHECK(near(eng.trade_size(0), -0.0084, 1e-9)); CHECK(near(eng.exit_price(0), 3721.62)); CHECK(near(eng.entry_price(0), 3706.26)); CHECK(eng.exit_bar(0) == 2); CHECK(eng.exit_comment(1) != std::string("Margin call")); - CHECK(near(eng.trade_size(1), 2.5021, 1e-9)); + CHECK(near(eng.trade_size(1), -2.5021, 1e-9)); CHECK(near(eng.exit_price(1), 3664.69)); CHECK(near(eng.position_size(), 0.0)); } @@ -196,7 +198,7 @@ static void test_low_first_large_deficit_stays_quiet() { CHECK(eng.trade_count() == 1); CHECK(margin_call_rows(eng) == 0); - CHECK(near(eng.trade_size(0), 2.5105, 1e-9)); + CHECK(near(eng.trade_size(0), -2.5105, 1e-9)); CHECK(near(eng.exit_price(0), 3664.69)); CHECK(near(eng.position_size(), 0.0)); } @@ -215,7 +217,7 @@ static void test_exit_at_extreme_ties_to_exit_first() { CHECK(eng.trade_count() == 1); CHECK(margin_call_rows(eng) == 0); - CHECK(near(eng.trade_size(0), 2.5105, 1e-9)); + CHECK(near(eng.trade_size(0), -2.5105, 1e-9)); CHECK(near(eng.exit_price(0), 3721.62)); CHECK(near(eng.position_size(), 0.0)); } @@ -234,7 +236,7 @@ static void test_sl_stop_before_extreme_stays_quiet() { CHECK(eng.trade_count() == 1); CHECK(margin_call_rows(eng) == 0); - CHECK(near(eng.trade_size(0), 2.5105, 1e-9)); + CHECK(near(eng.trade_size(0), -2.5105, 1e-9)); CHECK(near(eng.exit_price(0), 3715.0)); CHECK(near(eng.position_size(), 0.0)); } @@ -267,9 +269,9 @@ static void test_partial_exit_single_slice_per_bar() { CHECK(event_bar_mc_rows == 1); CHECK(eng.trade_count() >= 2); CHECK(eng.exit_comment(0) == std::string("Margin call")); - CHECK(near(eng.trade_size(0), 0.0084, 1e-9)); + CHECK(near(eng.trade_size(0), -0.0084, 1e-9)); CHECK(near(eng.exit_price(0), 3721.62)); - CHECK(near(eng.trade_size(1), 0.0251, 1e-9)); + CHECK(near(eng.trade_size(1), -0.0251, 1e-9)); CHECK(near(eng.exit_price(1), 3664.69)); CHECK(eng.position_size() < 0.0); // survivor carried past the event bar } @@ -288,7 +290,7 @@ static void test_disabled_emulator_stays_quiet() { CHECK(eng.trade_count() == 1); CHECK(margin_call_rows(eng) == 0); - CHECK(near(eng.trade_size(0), 2.5105, 1e-9)); + CHECK(near(eng.trade_size(0), -2.5105, 1e-9)); } // ---- G: handle reuse reproduces the same rows ------------------------------ @@ -307,9 +309,9 @@ static void test_rerun_reproduces_slice() { eng.run(bars.data(), (int)bars.size()); CHECK(eng.trade_count() == 2); CHECK(margin_call_rows(eng) == 1); - CHECK(near(eng.trade_size(0), 0.0084, 1e-9)); + CHECK(near(eng.trade_size(0), -0.0084, 1e-9)); CHECK(near(eng.exit_price(0), 3721.62)); - CHECK(near(eng.trade_size(1), 2.5021, 1e-9)); + CHECK(near(eng.trade_size(1), -2.5021, 1e-9)); } } // namespace diff --git a/tests/test_margin_call_intrabar_chronology_l4a.cpp b/tests/test_margin_call_intrabar_chronology_l4a.cpp index a64a82f98..595328339 100644 --- a/tests/test_margin_call_intrabar_chronology_l4a.cpp +++ b/tests/test_margin_call_intrabar_chronology_l4a.cpp @@ -81,6 +81,8 @@ class MCEngine : public pineforge::source::PineStrategyHost { std::string exit_comment(int i) const { return closed_trade_exit_comment(i); } double exit_price(int i) const { return closed_trade_exit_price(i); } double entry_price(int i) const { return closed_trade_entry_price(i); } + // expectation corrected: short +quantity -> -quantity, because Pine size + // encodes the side; the fill-derived magnitudes below are unchanged. double trade_size(int i) const { return closed_trade_size(i); } int exit_bar(int i) const { return closed_trade_exit_bar_index(i); } double position_size() const { return signed_position_size(); } @@ -171,12 +173,12 @@ static void test_gap_event_slices_before_tp_exit() { CHECK(eng.trade_count() == 2); CHECK(margin_call_rows(eng) == 1); CHECK(eng.exit_comment(0) == std::string("Margin call")); - CHECK(near(eng.trade_size(0), 0.0084, 1e-9)); + CHECK(near(eng.trade_size(0), -0.0084, 1e-9)); CHECK(near(eng.exit_price(0), 3721.62)); CHECK(near(eng.entry_price(0), 3706.26)); CHECK(eng.exit_bar(0) == 2); CHECK(eng.exit_comment(1) != std::string("Margin call")); - CHECK(near(eng.trade_size(1), 2.5021, 1e-9)); + CHECK(near(eng.trade_size(1), -2.5021, 1e-9)); CHECK(near(eng.exit_price(1), 3664.69)); CHECK(near(eng.position_size(), 0.0)); } @@ -199,7 +201,7 @@ static void test_low_first_large_deficit_stays_quiet() { CHECK(eng.trade_count() == 1); CHECK(margin_call_rows(eng) == 0); - CHECK(near(eng.trade_size(0), 2.5105, 1e-9)); + CHECK(near(eng.trade_size(0), -2.5105, 1e-9)); CHECK(near(eng.exit_price(0), 3664.69)); CHECK(near(eng.position_size(), 0.0)); } @@ -218,7 +220,7 @@ static void test_exit_at_extreme_ties_to_exit_first() { CHECK(eng.trade_count() == 1); CHECK(margin_call_rows(eng) == 0); - CHECK(near(eng.trade_size(0), 2.5105, 1e-9)); + CHECK(near(eng.trade_size(0), -2.5105, 1e-9)); CHECK(near(eng.exit_price(0), 3721.62)); CHECK(near(eng.position_size(), 0.0)); } @@ -237,7 +239,7 @@ static void test_sl_stop_before_extreme_stays_quiet() { CHECK(eng.trade_count() == 1); CHECK(margin_call_rows(eng) == 0); - CHECK(near(eng.trade_size(0), 2.5105, 1e-9)); + CHECK(near(eng.trade_size(0), -2.5105, 1e-9)); CHECK(near(eng.exit_price(0), 3715.0)); CHECK(near(eng.position_size(), 0.0)); } @@ -270,9 +272,9 @@ static void test_partial_exit_single_slice_per_bar() { CHECK(event_bar_mc_rows == 1); CHECK(eng.trade_count() >= 2); CHECK(eng.exit_comment(0) == std::string("Margin call")); - CHECK(near(eng.trade_size(0), 0.0084, 1e-9)); + CHECK(near(eng.trade_size(0), -0.0084, 1e-9)); CHECK(near(eng.exit_price(0), 3721.62)); - CHECK(near(eng.trade_size(1), 0.0251, 1e-9)); + CHECK(near(eng.trade_size(1), -0.0251, 1e-9)); CHECK(near(eng.exit_price(1), 3664.69)); CHECK(eng.position_size() < 0.0); // survivor carried past the event bar } @@ -291,7 +293,7 @@ static void test_disabled_emulator_stays_quiet() { CHECK(eng.trade_count() == 1); CHECK(margin_call_rows(eng) == 0); - CHECK(near(eng.trade_size(0), 2.5105, 1e-9)); + CHECK(near(eng.trade_size(0), -2.5105, 1e-9)); } // ---- G: handle reuse reproduces the same rows ------------------------------ @@ -310,9 +312,9 @@ static void test_rerun_reproduces_slice() { eng.run(bars.data(), (int)bars.size()); CHECK(eng.trade_count() == 2); CHECK(margin_call_rows(eng) == 1); - CHECK(near(eng.trade_size(0), 0.0084, 1e-9)); + CHECK(near(eng.trade_size(0), -0.0084, 1e-9)); CHECK(near(eng.exit_price(0), 3721.62)); - CHECK(near(eng.trade_size(1), 2.5021, 1e-9)); + CHECK(near(eng.trade_size(1), -2.5021, 1e-9)); } } // namespace diff --git a/tests/test_margin_call_trail_exit_chronology.cpp b/tests/test_margin_call_trail_exit_chronology.cpp index 02d396f87..de81fdbc1 100644 --- a/tests/test_margin_call_trail_exit_chronology.cpp +++ b/tests/test_margin_call_trail_exit_chronology.cpp @@ -80,6 +80,8 @@ class MCEngine : public pineforge::source::PineStrategyHost { std::string exit_comment(int i) const { return closed_trade_exit_comment(i); } double exit_price(int i) const { return closed_trade_exit_price(i); } double entry_price(int i) const { return closed_trade_entry_price(i); } + // expectation corrected: short +quantity -> -quantity, because Pine size + // encodes the side; the fill-derived magnitudes below are unchanged. double trade_size(int i) const { return closed_trade_size(i); } int exit_bar(int i) const { return closed_trade_exit_bar_index(i); } double position_size() const { return signed_position_size(); } @@ -166,12 +168,12 @@ static void test_trail_exit_slices_at_adverse_extreme_first() { CHECK(eng.trade_count() == 2); CHECK(margin_call_rows(eng) == 1); CHECK(eng.exit_comment(0) == std::string("Margin call")); - CHECK(near(eng.trade_size(0), 0.234, 1e-9)); + CHECK(near(eng.trade_size(0), -0.234, 1e-9)); CHECK(near(eng.exit_price(0), 3960.0)); CHECK(near(eng.entry_price(0), 3879.36)); CHECK(eng.exit_bar(0) == 2); CHECK(eng.exit_comment(1) != std::string("Margin call")); - CHECK(near(eng.trade_size(1), 1.885, 1e-9)); + CHECK(near(eng.trade_size(1), -1.885, 1e-9)); CHECK(near(eng.exit_price(1), 3821.06)); CHECK(near(eng.position_size(), 0.0)); } @@ -193,7 +195,7 @@ static void test_trail_exit_before_extreme_stays_quiet() { CHECK(eng.trade_count() == 1); CHECK(margin_call_rows(eng) == 0); - CHECK(near(eng.trade_size(0), 2.119, 1e-9)); + CHECK(near(eng.trade_size(0), -2.119, 1e-9)); CHECK(near(eng.exit_price(0), 3821.06)); CHECK(near(eng.position_size(), 0.0)); } @@ -209,7 +211,7 @@ static void test_trail_chronology_disabled_emulator_stays_quiet() { CHECK(eng.trade_count() == 1); CHECK(margin_call_rows(eng) == 0); - CHECK(near(eng.trade_size(0), 2.119, 1e-9)); + CHECK(near(eng.trade_size(0), -2.119, 1e-9)); CHECK(near(eng.exit_price(0), 3821.06)); } diff --git a/tests/test_margin_call_trail_exit_chronology_l4a.cpp b/tests/test_margin_call_trail_exit_chronology_l4a.cpp index 9120d1a27..77ede39fa 100644 --- a/tests/test_margin_call_trail_exit_chronology_l4a.cpp +++ b/tests/test_margin_call_trail_exit_chronology_l4a.cpp @@ -83,6 +83,8 @@ class MCEngine : public pineforge::source::PineStrategyHost { std::string exit_comment(int i) const { return closed_trade_exit_comment(i); } double exit_price(int i) const { return closed_trade_exit_price(i); } double entry_price(int i) const { return closed_trade_entry_price(i); } + // expectation corrected: short +quantity -> -quantity, because Pine size + // encodes the side; the fill-derived magnitudes below are unchanged. double trade_size(int i) const { return closed_trade_size(i); } int exit_bar(int i) const { return closed_trade_exit_bar_index(i); } double position_size() const { return signed_position_size(); } @@ -169,12 +171,12 @@ static void test_trail_exit_slices_at_adverse_extreme_first() { CHECK(eng.trade_count() == 2); CHECK(margin_call_rows(eng) == 1); CHECK(eng.exit_comment(0) == std::string("Margin call")); - CHECK(near(eng.trade_size(0), 0.234, 1e-9)); + CHECK(near(eng.trade_size(0), -0.234, 1e-9)); CHECK(near(eng.exit_price(0), 3960.0)); CHECK(near(eng.entry_price(0), 3879.36)); CHECK(eng.exit_bar(0) == 2); CHECK(eng.exit_comment(1) != std::string("Margin call")); - CHECK(near(eng.trade_size(1), 1.885, 1e-9)); + CHECK(near(eng.trade_size(1), -1.885, 1e-9)); CHECK(near(eng.exit_price(1), 3821.06)); CHECK(near(eng.position_size(), 0.0)); } @@ -196,7 +198,7 @@ static void test_trail_exit_before_extreme_stays_quiet() { CHECK(eng.trade_count() == 1); CHECK(margin_call_rows(eng) == 0); - CHECK(near(eng.trade_size(0), 2.119, 1e-9)); + CHECK(near(eng.trade_size(0), -2.119, 1e-9)); CHECK(near(eng.exit_price(0), 3821.06)); CHECK(near(eng.position_size(), 0.0)); } @@ -212,7 +214,7 @@ static void test_trail_chronology_disabled_emulator_stays_quiet() { CHECK(eng.trade_count() == 1); CHECK(margin_call_rows(eng) == 0); - CHECK(near(eng.trade_size(0), 2.119, 1e-9)); + CHECK(near(eng.trade_size(0), -2.119, 1e-9)); CHECK(near(eng.exit_price(0), 3821.06)); } diff --git a/tests/test_margin_rules_forward_replay.cpp b/tests/test_margin_rules_forward_replay.cpp index 26aae07a4..f60c36032 100644 --- a/tests/test_margin_rules_forward_replay.cpp +++ b/tests/test_margin_rules_forward_replay.cpp @@ -424,7 +424,7 @@ struct OpenEntry { std::int64_t time = 0; int bar_index = -1; double price = 0.0; - double size = 0.0; + double size = 0.0; // Magnitude for make_row(); direction is stored separately. double commission = 0.0; }; @@ -467,7 +467,11 @@ class ReplayHost : public source::PineStrategyHost { entry.time = open_trade_entry_time(i); entry.bar_index = open_trade_entry_bar_index(i); entry.price = open_trade_entry_price(i); - entry.size = open_trade_size(i); + const double signed_size = open_trade_size(i); + CHECK(entry.direction == "long" || entry.direction == "short"); + CHECK(entry.direction == "long" ? signed_size > 0.0 : signed_size < 0.0); + // The replay/tape row model takes magnitude plus a separate side. + entry.size = entry.direction == "long" ? signed_size : -signed_size; entry.commission = open_trade_commission(i); s.open.push_back(std::move(entry)); } diff --git a/tests/test_native_open_lots.cpp b/tests/test_native_open_lots.cpp index 28f138a24..3cad20d5c 100644 --- a/tests/test_native_open_lots.cpp +++ b/tests/test_native_open_lots.cpp @@ -124,7 +124,7 @@ void observe(LotHost& h) { CHECK(row.entry_bar_index == h.p_entry_bar(k)); CHECK(row.entry_label == h.p_entry_id(k)); CHECK(row.entry_comment == h.p_entry_comment(k)); - CHECK(bits_eq(std::abs(row.signed_units), h.p_size(k))); + CHECK(bits_eq(row.signed_units, h.p_size(k))); CHECK(bits_eq(row.favorable_excursion, h.p_runup(k))); CHECK(bits_eq(row.adverse_excursion, h.p_drawdown(k))); CHECK((row.side == no::Side::Long) == (h.p_direction(k) == "long")); diff --git a/tests/test_pyramiding_count_partial_drain.cpp b/tests/test_pyramiding_count_partial_drain.cpp index 9bbbf7389..73e53adeb 100644 --- a/tests/test_pyramiding_count_partial_drain.cpp +++ b/tests/test_pyramiding_count_partial_drain.cpp @@ -437,7 +437,9 @@ static void test_owned_slot_full_zero_and_reuse() { CHECK(p.slots==1); CHECK(near(p.position_size(),is_long?4:-4)); CHECK(p.trade_count()==1); - CHECK(near(p.size(0),2)); + // expectation corrected: 2 -> -2 for the short mirror, because + // Pine sizes encode the configured side; long remains +2. + CHECK(near(p.size(0),is_long?2:-2)); } OwnedHistoryProbe reuse;reuse.is_long=is_long;reuse.cross_first=true; reuse.run(bars.data(),static_cast(bars.size())); diff --git a/tests/test_pyramiding_count_partial_drain_l4d.cpp b/tests/test_pyramiding_count_partial_drain_l4d.cpp index 574820b8d..014bde3e5 100644 --- a/tests/test_pyramiding_count_partial_drain_l4d.cpp +++ b/tests/test_pyramiding_count_partial_drain_l4d.cpp @@ -450,7 +450,9 @@ static void test_owned_slot_full_zero_and_reuse() { CHECK(p.slots==1); CHECK(near(p.position_size(),is_long?4:-4)); CHECK(p.trade_count()==1); - CHECK(near(p.size(0),2)); + // expectation corrected: 2 -> -2 for the short mirror, because + // Pine sizes encode the configured side; long remains +2. + CHECK(near(p.size(0),is_long?2:-2)); } OwnedHistoryProbe reuse;reuse.is_long=is_long;reuse.cross_first=true; reuse.run(bars.data(),static_cast(bars.size())); diff --git a/tests/test_short_reversal_emission.cpp b/tests/test_short_reversal_emission.cpp index 3f9f00b9a..c6e2520f0 100644 --- a/tests/test_short_reversal_emission.cpp +++ b/tests/test_short_reversal_emission.cpp @@ -103,7 +103,7 @@ class RevProbe : public pineforge::source::PineStrategyHost { int worst_negative_qty_rows() const { int n = 0; for (int i = 0; i < trade_count(); ++i) - if (closed_trade_size(i) < 0.0) ++n; + if (get_trade(i).qty < 0.0) ++n; return n; } using BacktestEngine::position_side_; @@ -138,8 +138,12 @@ void test_negative_equity_reversal_declined_clean() { CHECK(eng.worst_negative_qty_rows() == 0); // RED on baseline (1 neg row) CHECK(eng.position_side_ == PositionSide::SHORT); CHECK(eng.position_qty_ > 0.0); // never a negative-qty position - // Every emitted trade row carries a POSITIVE size (Pine/TV magnitude column). - for (int i = 0; i < eng.trade_count(); ++i) CHECK(eng.trade_size(i) >= 0.0); + // Stored quantities remain magnitudes; Pine sizes also encode the side. + for (int i = 0; i < eng.trade_count(); ++i) { + const auto trade = eng.get_trade(i); + CHECK(trade.qty >= 0.0); + CHECK(eng.trade_size(i) == (trade.is_long ? trade.qty : -trade.qty)); + } } // A2. EMISSION/ACCOUNTING split closed: the exported per-trade PnL (sum) agrees @@ -191,6 +195,12 @@ void test_solvent_reversal_opens_normally() { CHECK(eng.position_qty_ > 0.0); CHECK(eng.trade_count() >= 1); // short closed cleanly CHECK(eng.worst_negative_qty_rows() == 0); + if (eng.trade_count() >= 1) { + const auto trade = eng.get_trade(0); + CHECK(!trade.is_long); + CHECK_NEAR(trade.qty, 100.0, 1e-9); // 10000 / entry price 100 + CHECK_NEAR(eng.trade_size(0), -100.0, 1e-9); + } } // C2. CONTROL — a SOLVENT true-flat all-in long open is unchanged: opens the diff --git a/tests/test_sizing_basis_mintick.cpp b/tests/test_sizing_basis_mintick.cpp index 75c17f5e2..b54280381 100644 --- a/tests/test_sizing_basis_mintick.cpp +++ b/tests/test_sizing_basis_mintick.cpp @@ -319,6 +319,8 @@ class ShortCascadeProbe : public pineforge::source::PineStrategyHost { } std::string exit_comment(int i) const { return closed_trade_exit_comment(i); } double exit_price(int i) const { return closed_trade_exit_price(i); } + // expectation corrected: +slice/+remainder -> -slice/-remainder, + // because this probe closes shorts; the rounding formulas are unchanged. double trade_size(int i) const { return closed_trade_size(i); } using BacktestEngine::position_side_; using BacktestEngine::position_qty_; @@ -359,7 +361,7 @@ void test_short_cascade_slices_at_rounded_high() { if (eng.trade_count() >= 1) { CHECK(eng.exit_comment(0) == std::string("Margin call")); CHECK_NEAR(eng.exit_price(0), 105.01, 1e-9); - CHECK_NEAR(eng.trade_size(0), 4.0 * (20.0 - 2000.0 / 105.01), 1e-6); + CHECK_NEAR(eng.trade_size(0), -4.0 * (20.0 - 2000.0 / 105.01), 1e-6); // pre-fix: 4.0 * (20.0 - 2000.0 / 105.005) = 3.8131518 } } @@ -397,7 +399,7 @@ void test_short_chronological_slice_marks_at_rounded_high() { if (eng.trade_count() >= 1) { CHECK(eng.exit_comment(0) != std::string("Margin call")); CHECK_NEAR(eng.exit_price(0), 99.60, 1e-9); - CHECK_NEAR(eng.trade_size(0), 10.0, 1e-9); + CHECK_NEAR(eng.trade_size(0), -10.0, 1e-9); } CHECK(eng.position_side_ == PositionSide::FLAT); } @@ -415,10 +417,10 @@ void test_short_chronological_slice_marks_at_rounded_high() { if (eng.trade_count() == 2) { CHECK(eng.exit_comment(0) == std::string("Margin call")); CHECK_NEAR(eng.exit_price(0), 100.01, 1e-9); - CHECK_NEAR(eng.trade_size(0), 4.0 * q_min, 1e-9); + CHECK_NEAR(eng.trade_size(0), -4.0 * q_min, 1e-9); CHECK(eng.exit_comment(1) != std::string("Margin call")); CHECK_NEAR(eng.exit_price(1), 99.60, 1e-9); - CHECK_NEAR(eng.trade_size(1), 10.0 - 4.0 * q_min, 1e-9); + CHECK_NEAR(eng.trade_size(1), -(10.0 - 4.0 * q_min), 1e-9); } CHECK(eng.position_side_ == PositionSide::FLAT); } diff --git a/tests/test_sizing_basis_mintick_l4b.cpp b/tests/test_sizing_basis_mintick_l4b.cpp index b37a9051e..8658df05a 100644 --- a/tests/test_sizing_basis_mintick_l4b.cpp +++ b/tests/test_sizing_basis_mintick_l4b.cpp @@ -334,6 +334,8 @@ class ShortCascadeProbe : public pineforge::source::PineStrategyHost { } std::string exit_comment(int i) const { return closed_trade_exit_comment(i); } double exit_price(int i) const { return closed_trade_exit_price(i); } + // expectation corrected: +slice/+remainder -> -slice/-remainder, + // because this probe closes shorts; the rounding formulas are unchanged. double trade_size(int i) const { return closed_trade_size(i); } using BacktestEngine::position_side_; using BacktestEngine::position_qty_; @@ -374,7 +376,7 @@ void test_short_cascade_slices_at_rounded_high() { if (eng.trade_count() >= 1) { CHECK(eng.exit_comment(0) == std::string("Margin call")); CHECK_NEAR(eng.exit_price(0), 105.01, 1e-9); - CHECK_NEAR(eng.trade_size(0), 4.0 * (20.0 - 2000.0 / 105.01), 1e-6); + CHECK_NEAR(eng.trade_size(0), -4.0 * (20.0 - 2000.0 / 105.01), 1e-6); // pre-fix: 4.0 * (20.0 - 2000.0 / 105.005) = 3.8131518 } } @@ -412,7 +414,7 @@ void test_short_chronological_slice_marks_at_rounded_high() { if (eng.trade_count() >= 1) { CHECK(eng.exit_comment(0) != std::string("Margin call")); CHECK_NEAR(eng.exit_price(0), 99.60, 1e-9); - CHECK_NEAR(eng.trade_size(0), 10.0, 1e-9); + CHECK_NEAR(eng.trade_size(0), -10.0, 1e-9); } CHECK(eng.position_side_ == PositionSide::FLAT); } @@ -430,10 +432,10 @@ void test_short_chronological_slice_marks_at_rounded_high() { if (eng.trade_count() == 2) { CHECK(eng.exit_comment(0) == std::string("Margin call")); CHECK_NEAR(eng.exit_price(0), 100.01, 1e-9); - CHECK_NEAR(eng.trade_size(0), 4.0 * q_min, 1e-9); + CHECK_NEAR(eng.trade_size(0), -4.0 * q_min, 1e-9); CHECK(eng.exit_comment(1) != std::string("Margin call")); CHECK_NEAR(eng.exit_price(1), 99.60, 1e-9); - CHECK_NEAR(eng.trade_size(1), 10.0 - 4.0 * q_min, 1e-9); + CHECK_NEAR(eng.trade_size(1), -(10.0 - 4.0 * q_min), 1e-9); } CHECK(eng.position_side_ == PositionSide::FLAT); } diff --git a/tests/test_trade_ledger.cpp b/tests/test_trade_ledger.cpp index 1eefd14f8..a47a73473 100644 --- a/tests/test_trade_ledger.cpp +++ b/tests/test_trade_ledger.cpp @@ -345,7 +345,8 @@ void migrated_historical_readers() { CHECK(same_double(host.closed_trade_exit_price(i), value.exit_price)); CHECK(host.closed_trade_entry_time(i) == value.entry_time); CHECK(host.closed_trade_exit_time(i) == value.exit_time); - CHECK(same_double(host.closed_trade_size(i), value.qty)); + // Pine signs the independently stored magnitude by the fixture side. + CHECK(same_double(host.closed_trade_size(i), value.is_long ? value.qty : -value.qty)); CHECK(same_double(host.closed_trade_max_runup(i), value.max_runup)); CHECK(same_double(host.closed_trade_max_drawdown(i), value.max_drawdown)); const double cost = value.entry_price * value.qty; From 7294ab60a9a1d7eed8f2d7c706eda225c577d1ca Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Fri, 6 Nov 2026 21:21:56 +0800 Subject: [PATCH 08/19] Keep signed accessor source lines stable for doc anchors The accessor bodies already return signed quantities; keep each expression on one source line so the existing documentation citations remain bound to the same header and implementation lines. This preserves the exact signed-size tree behavior while removing the anchor shift. --- include/pineforge/engine.hpp | 3 +-- src/engine_trade_accessors.cpp | 3 +-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/include/pineforge/engine.hpp b/include/pineforge/engine.hpp index d5d432bf8..3224e7399 100644 --- a/include/pineforge/engine.hpp +++ b/include/pineforge/engine.hpp @@ -1789,8 +1789,7 @@ class BacktestEngine { } double closed_trade_size(int idx) const { if (idx < 0 || idx >= (int)trades_.size()) return std::numeric_limits::quiet_NaN(); - const double qty = trades_.qty_at(idx); - return trades_.is_long_at(idx) ? qty : -qty; + return trades_.is_long_at(idx) ? trades_.qty_at(idx) : -trades_.qty_at(idx); } double closed_trade_max_runup(int idx) const { if (idx < 0 || idx >= (int)trades_.size()) return 0.0; diff --git a/src/engine_trade_accessors.cpp b/src/engine_trade_accessors.cpp index b06d21214..f4c618b04 100644 --- a/src/engine_trade_accessors.cpp +++ b/src/engine_trade_accessors.cpp @@ -90,8 +90,7 @@ int64_t BacktestEngine::open_trade_entry_time(int idx) const { double BacktestEngine::open_trade_size(int idx) const { if (position_side_ == PositionSide::FLAT || idx < 0 || idx >= (int)pyramid_entries_.size()) return na(); - const double qty = pyramid_entries_[(size_t)idx].qty; - return position_side_ == PositionSide::LONG ? qty : -qty; + return position_side_ == PositionSide::LONG ? pyramid_entries_[(size_t)idx].qty : -pyramid_entries_[(size_t)idx].qty; } // pe.max_drawdown / pe.max_runup are tracked in price-points × qty (the From 83c501995bef8e1d1a7fff57442b07ff7615d098 Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Fri, 6 Nov 2026 21:54:48 +0800 Subject: [PATCH 09/19] tests: re-anchor the private fixture evidence row after a registration moved its lines The bounded drawing checkpoint test adds one source, test_drawing_bound_checkpoint_cost, to the TEST_SOURCES list near the top of tests/CMakeLists.txt, which moves every later line of that file by one. The evidence row for test_pooc_close_fill_view_tapes cites five of those lines: the line the sweep re-reads (3459, now 3460) and four in its reason text (:3443, :3455, :3479 and :3485-3501, now :3444, :3456, :3480 and :3486-3502). test_private_fixture_reads failed with "the token no longer sits on tests/CMakeLists.txt:3459", and test_private_fixture_reads_controls failed with it. Only those five numbers change; the row's text is otherwise as it was. Each new line was read from the file: 3444 is the EXISTS test of a tape, 3456 the increment of _pf_pcv_emitted, 3460 the condition that is the row's token, 3480 the registration of the test, 3486 the registration of the skipped placeholder. Co-Authored-By: Claude Sonnet 5.5 --- tests/private_fixture_reads.evidence | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/private_fixture_reads.evidence b/tests/private_fixture_reads.evidence index 96eca5316..bd52addbe 100644 --- a/tests/private_fixture_reads.evidence +++ b/tests/private_fixture_reads.evidence @@ -5,4 +5,4 @@ // longer sits there, and scripts/test_private_fixture_reads.py runs that sweep // (with RED controls on a tampered copy of the tree), so the proof cannot rot // silently. -test_pooc_close_fill_view_tapes tests/CMakeLists.txt:3459 if(_pf_pcv_emitted GREATER 0 AND _pf_pcv_emitted EQUAL _pf_pcv_listed) the module (and with it the add_test at :3479) is built only when every tape the list names has its generated.cpp: _pf_pcv_emitted counts only tapes whose file exists (EXISTS at tests/CMakeLists.txt:3443, emitted increment at :3455). On the fixture-less tree the else branch's registration and skip property (:3485-3501) register the row as the pinned SKIP placeholder naming tests/fixtures/pooc_close_fill_view/tapes.txt, so the row registers and reports itself skipped instead of vanishing. +test_pooc_close_fill_view_tapes tests/CMakeLists.txt:3460 if(_pf_pcv_emitted GREATER 0 AND _pf_pcv_emitted EQUAL _pf_pcv_listed) the module (and with it the add_test at :3480) is built only when every tape the list names has its generated.cpp: _pf_pcv_emitted counts only tapes whose file exists (EXISTS at tests/CMakeLists.txt:3444, emitted increment at :3456). On the fixture-less tree the else branch's registration and skip property (:3486-3502) register the row as the pinned SKIP placeholder naming tests/fixtures/pooc_close_fill_view/tapes.txt, so the row registers and reports itself skipped instead of vanishing. From 51aa35dd2990127339143a59437978f9e04e5633 Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Fri, 6 Nov 2026 21:39:29 +0800 Subject: [PATCH 10/19] ci: reduce sanitizer debug info footprint Use GCC minimal debug information for instrumented Debug profiles while retaining file and line data, sanitizer flags and the full test selection. Keep the definition below the cited profile-floor table so existing documentation anchors remain valid. This is a measurement step: it lowers debug info only. The pass condition is at least 20 GB free after Verify on the worst hosted sanitizer shard. If that margin is not reached, each shard will build only its own targets next. --- scripts/ci_verify.py | 10 ++++++++++ scripts/test_ci_verify.py | 6 ++++++ 2 files changed, 16 insertions(+) diff --git a/scripts/ci_verify.py b/scripts/ci_verify.py index 1a6e50013..1064348de 100644 --- a/scripts/ci_verify.py +++ b/scripts/ci_verify.py @@ -1654,6 +1654,13 @@ def kernel_residuals_command(cfg: VerifyConfig) -> list[str]: '--min-ruled-texts', str(ADR_RULED_TEXTS_MIN)] +# Hosted sanitizer runners have 145G on / and /mnt is the same filesystem. +# The terminal main run reached 5MB free while compiling the unchanged full +# shard population with ordinary Debug DWARF. Keep every sanitizer flag, +# source/test population, and shard selection unchanged while using compact +# line tables for sanitizer Debug objects; this reduces object/debug footprint +# without changing generated machine code or runtime instrumentation. +SANITIZER_DEBUG_INFO_FLAGS = '-g1' def cmake_cache_definitions(cfg: VerifyConfig) -> dict[str, str]: profile = cfg.profile values = { @@ -1678,6 +1685,9 @@ def cmake_cache_definitions(cfg: VerifyConfig) -> dict[str, str]: } if profile.live_runner: values['PINEFORGE_LIVE_TSAN'] = 'ON' if profile.thread_sanitizers else 'OFF' + if profile.name in {'sanitizers', 'live-sanitizers'}: + values['CMAKE_C_FLAGS_DEBUG'] = SANITIZER_DEBUG_INFO_FLAGS + values['CMAKE_CXX_FLAGS_DEBUG'] = SANITIZER_DEBUG_INFO_FLAGS if cfg.curl_dir is not None: values['CURL_DIR'] = str(cfg.curl_dir) if cfg.ccache_path: diff --git a/scripts/test_ci_verify.py b/scripts/test_ci_verify.py index 32cc7287b..36b0d6f8c 100644 --- a/scripts/test_ci_verify.py +++ b/scripts/test_ci_verify.py @@ -2380,6 +2380,10 @@ def test_sanitizers_public_flag_expected_and_native_off(self): self.assertEqual(values['PINEFORGE_ENABLE_SANITIZERS'], 'ON') self.assertEqual(values['PINEFORGE_BUILD_LIVE_RUNNER'], 'OFF') self.assertEqual(values['PINEFORGE_BUILD_TUTORIAL'], 'ON') + self.assertEqual(values['CMAKE_C_FLAGS_DEBUG'], '-g1') + self.assertEqual(values['CMAKE_CXX_FLAGS_DEBUG'], '-g1') + self.assertIn('-DCMAKE_C_FLAGS_DEBUG=-g1', argv) + self.assertIn('-DCMAKE_CXX_FLAGS_DEBUG=-g1', argv) self.assertIn('-DPINEFORGE_ENABLE_SANITIZERS=ON', argv) def test_live_sanitizers_enable_runner_and_gate_websockets_without_opt_in(self): @@ -2387,6 +2391,8 @@ def test_live_sanitizers_enable_runner_and_gate_websockets_without_opt_in(self): self.assertEqual(values['CMAKE_BUILD_TYPE'], 'Debug') self.assertEqual(values['PINEFORGE_BUILD_LIVE_RUNNER'], 'ON') self.assertEqual(values['PINEFORGE_ENABLE_SANITIZERS'], 'ON') + self.assertEqual(values['CMAKE_C_FLAGS_DEBUG'], '-g1') + self.assertEqual(values['CMAKE_CXX_FLAGS_DEBUG'], '-g1') self.assertEqual(values['PINEFORGE_BUILD_SOURCE_LAYER'], 'ON') self.assertEqual(values['PINEFORGE_BUILD_TUTORIAL'], 'OFF') cfg = validate_config(parse_args(['live-sanitizers'], source=ROOT)) From 834333f0174886a87eda01d4ca6175920d96563b Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Sat, 7 Nov 2026 05:56:24 +0800 Subject: [PATCH 11/19] Refresh twin parity pins for signed size assertions Update the margin-call twin assertion digest and register the pyramiding partial-drain signed-size rewrite in the twin parity inventory. Recompute the inventory hash so the guard covers both intentional assertion changes. --- scripts/check_twin_parity.py | 2 +- tests/twin_parity_inventory.json | 9 ++++++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/scripts/check_twin_parity.py b/scripts/check_twin_parity.py index 76c6b3a2b..bd8ad06cf 100644 --- a/scripts/check_twin_parity.py +++ b/scripts/check_twin_parity.py @@ -29,7 +29,7 @@ # executable code means a ledger/inventory rewrite cannot silently redefine # what the guard proves. BASE_MANIFEST_SHA256 = "a20f80eb29af455400fd2ab1bdef86e477af3d81891b1207623b4c890f6056db" -INVENTORY_SHA256 = "a1c0122868587da8d996184afeed77d915c78ce9c17370643d101cb125a8291a" +INVENTORY_SHA256 = "04fd304bcc8f86a2abd4c121de88465307eaf069a8e5f1285307fdd45189b8b0" LEDGER_SHA256 = "c568999da8777ff967cdb3fcf007f382f88902bf78f9e02c2c5d7d15de22e9ba" APPENDIX_HEADING = "## Appendix 5 — CHECK-parity unobservable literal ledger" diff --git a/tests/twin_parity_inventory.json b/tests/twin_parity_inventory.json index a24cf29e6..874ba021c 100644 --- a/tests/twin_parity_inventory.json +++ b/tests/twin_parity_inventory.json @@ -323,7 +323,7 @@ "baseAssertions": 62, "reason": "F's process_orders_on_close opening-budget trim follows the R5 lane W5-ENG-MARGIN-V6 TradingView tapes (tests/fixtures/margin_v6 w5-c2-pooc-flat-explicit-long): a commissioned process_orders_on_close opening is called at the next open, never at its close fill, so the 0.1996 @100 trim lands on bar 1. E's short mirror follows the lane PX-PIN-16B TradingView tapes (tests/fixtures/margin_open_rules exit-order/short-fee-stop-on-path and -open-at-low, E's settings and bar shape): a short is trimmed at its open fill before a stop the bar reaches later on its path, so E books 0.1996 @100 then the stop's 99.7504 @105, as long case B does", "twin": "test_margin_call_1x_long_entry_fill_l4a.cpp", - "twinAssertionSha256": "bba0a03d3b7f383cf0ef9e292812fe32e76d33b4aa11ae10db398ea2d7248df7", + "twinAssertionSha256": "56cc0a2af7a699ab0a047a1bb0bfa9c74c044f461e1cfc9b56764dc8ae7d4061", "twinAssertions": 67 }, "test_order_birth_provenance": { @@ -368,6 +368,13 @@ "twinAssertionSha256": "0128c3d7d3cb461dea0032ad74b36b88c22f6f717c72366decf06eb0e1eb8057", "twinAssertions": 96 }, + "test_pyramiding_count_partial_drain": { + "baseAssertions": 57, + "reason": "the switched-route short mirror intentionally exposes the signed one-lot size as is_long ? 2 : -2, while the base manifest retains the long-side literal 2; the public signed quantity is the observable rewrite and the long/short reuse cases remain covered", + "twin": "test_pyramiding_count_partial_drain_l4d.cpp", + "twinAssertionSha256": "46b017efe37f9875bbe261cfb1bd32e8bfca3b641d44aeadbdd3a808c4e323fc", + "twinAssertions": 57 + }, "test_reservation_expansion": { "baseAssertions": 139, "reason": "private reservation objects are rewritten to pending and trade projections", From bfd45f64256815c35e40fadcfe430f9aff062d4e Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Sat, 7 Nov 2026 13:16:57 +0800 Subject: [PATCH 12/19] Include ABI layout coverage in PR smoke Build and select the existing ABI layout probe alongside the eleven existing fast executables. Update the derived membership pin, preserve prior checks, and exercise removal of the actual ABI row through the real workflow checker. Documentation replacement: the sentence "names its eleven real executables; configure refuses" now names twelve executables and explains the existing probe's Linux x86_64 GNU libstdc++ applicability. --- docs/ci.md | 8 +++++--- scripts/ci_preflight.py | 2 +- scripts/test_ci_preflight.py | 18 +++++++++++++++++- tests/CMakeLists.txt | 1 + 4 files changed, 24 insertions(+), 5 deletions(-) diff --git a/docs/ci.md b/docs/ci.md index 6cc69b378..731c99161 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -74,9 +74,11 @@ all stages and its existing event-aware verifier-class selection. `runner-allowl The smoke configures Release, builds `pineforge_pr_smoke` and directly runs `ctest --test-dir build -L '^fast$' --output-on-failure --no-tests=error`. -`PINEFORGE_PR_FAST_TESTS` in `tests/CMakeLists.txt` names its eleven real -executables; configure refuses missing, disabled, skippable, private-fixture -or `slow` rows. The target rebuilds those executables with their library. +`PINEFORGE_PR_FAST_TESTS` in `tests/CMakeLists.txt` names its twelve real +executables, including `test_abi_layout_probe`, whose size and offset pins +apply on Linux x86_64 with GNU libstdc++. Configure refuses missing, disabled, +skippable, private-fixture or `slow` rows. The target rebuilds those executables +with their library. PR proof legs build cold; trusted post workflows also build cold and contain no Actions cache restore or save. diff --git a/scripts/ci_preflight.py b/scripts/ci_preflight.py index ebb998223..ad0148495 100644 --- a/scripts/ci_preflight.py +++ b/scripts/ci_preflight.py @@ -93,7 +93,7 @@ 'test_native_script_bucket_completions', 'test_native_settlement_carry', ) -PR_FAST_ROWS_SHA256 = '0509b2219786efb6878b193639344c299ff8075ba4532dbe09e80e6e97b60a9a' +PR_FAST_ROWS_SHA256 = '2cb4ca445fbf30f813a8bc036e5a88e7b157be40496d9502445581dcd81ea4f7' # Every migrated job runs on a standard hosted Ubuntu image: x86 on # ubuntu-24.04 and arm64 on ubuntu-24.04-arm. An unpinned job in a migrated # workflow must use the x86 image. The legacy release form keeps its protected diff --git a/scripts/test_ci_preflight.py b/scripts/test_ci_preflight.py index 6c2149a6c..c19d6d3e1 100644 --- a/scripts/test_ci_preflight.py +++ b/scripts/test_ci_preflight.py @@ -6525,11 +6525,13 @@ def test_ci_smoke_and_full_builds_refuse_all_actions_cache_readers_and_writers(s def test_fast_label_rows_and_build_dependencies_are_pinned(self): cmake = self.ci_sources()[3] rows = ci_preflight.fast_rows(cmake) + # expectation corrected: 11 -> 12, because PR smoke now builds and + # selects the existing ABI layout probe alongside every prior row. self.assertEqual(rows, [ 'test_ringbuffer', 'test_timeframe', 'test_path_at', 'test_na_runtime', 'test_math_sum', 'test_str_utils', 'test_ta', 'test_matrix', 'test_native_run_spec', 'test_native_bare_host_contracts', - 'test_native_kernel_edge']) + 'test_native_kernel_edge', 'test_abi_layout_probe']) mutations = [ (' test_ringbuffer\n', ''), (' test_ringbuffer\n', ' test_ringbuffer\n test_ringbuffer\n'), @@ -6550,6 +6552,20 @@ def test_fast_label_rows_and_build_dependencies_are_pinned(self): self.assertIn('PR fast rows must keep one pinned CMake label list and executable build target', findings) + def test_fast_rows_refuse_removing_abi_layout_probe(self): + sources = self.ci_sources() + cmake = sources[3] + finding = 'PR fast rows must keep one pinned CMake label list and executable build target' + self.assertIn('test_abi_layout_probe', ci_preflight.fast_rows(cmake)) + self.assertNotIn(finding, ci_workflow_findings(*sources, others=self.other_sources())) + fast_block = cmake[cmake.index('set(PINEFORGE_PR_FAST_TESTS\n'):] + self.assertEqual(fast_block.count(' test_abi_layout_probe\n'), 1) + changed_block = fast_block.replace(' test_abi_layout_probe\n', '', 1) + self.assertNotEqual(changed_block, fast_block) + self.assertNotIn('test_abi_layout_probe', ci_preflight.fast_rows(changed_block)) + findings = self.mutated_file_findings(3, None, fast_block, changed_block) + self.assertIn(finding, findings) + def test_fast_rows_refuse_property_writes_after_the_registration_block(self): sources = self.ci_sources() cmake = sources[3] diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 9a1e2602b..502258411 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -7589,6 +7589,7 @@ set(PINEFORGE_PR_FAST_TESTS test_native_run_spec test_native_bare_host_contracts test_native_kernel_edge + test_abi_layout_probe ) # CR-150 P00: append after the pre-existing line-pinned registrations. From 3988fb9246cb8aa608d0a19b9904cc9b2ce7800a Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Sat, 7 Nov 2026 13:38:12 +0800 Subject: [PATCH 13/19] Keep documentation length checks in sync with smoke coverage The ABI smoke explanation adds two documentation lines. Correct both exact line-count expectations from 2060 to 2062 while retaining all substantive runner, slow-inventory and same-size negative checks. --- scripts/test_ci_preflight.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/scripts/test_ci_preflight.py b/scripts/test_ci_preflight.py index c19d6d3e1..ac6664a70 100644 --- a/scripts/test_ci_preflight.py +++ b/scripts/test_ci_preflight.py @@ -2978,7 +2978,9 @@ def test_post_jobs_refuse_other_fleets_scalars_and_reordered_labels(self): def test_post_fleet_docs_match_runner_policy(self): docs = (ROOT / 'docs/ci.md').read_text() - self.assertEqual(len(docs.splitlines()), 2060) + # expectation corrected: 2060 -> 2062, because the truthful smoke + # documentation gained two lines. + self.assertEqual(len(docs.splitlines()), 2062) # The test-owned job catalog defines the protected label inventory. tables = re.findall( r"^\| Workflow \| Events/callers \| Inline jobs' `runs-on` \|\n" @@ -5482,7 +5484,9 @@ def test_measured_slow_rows_refuse_variable_in_place_of_literal(self): def test_documented_slow_row_counts_match_literal_population(self): text = (ROOT / 'docs/ci.md').read_text() - self.assertEqual(len(text.splitlines()), 2060) + # expectation corrected: 2060 -> 2062, because the truthful smoke + # documentation gained two lines. + self.assertEqual(len(text.splitlines()), 2062) slices = tuple(name for name in EXPECTED_PR_SLOW_ROWS if name.startswith('test_ci_verify_slice_')) self.assertTrue(slices, 'verifier slice inventory must not be empty') From 589a8319367b59997985d65e2393f1d24774f2b6 Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Sat, 7 Nov 2026 14:25:23 +0800 Subject: [PATCH 14/19] Add live drawing capacity to the runtime limit vocabulary Accept live_drawings through the existing typed limit helper and publish its English template, preserving the code identity, since value and prior limit values. Regenerate the registry and catalog diff and exercise distinct operator maxima through the real helper contract. --- docker/run_failure_codes.json | 6 +- docker/run_failure_codes_diff.json | 8 +- docs/pages/run-failure-codes.md | 19 +- include/pineforge/run_failure.hpp | 3 +- src/run_failure_registry.inc | 291 +++++++++++++++-------------- tests/test_run_failure_codes.cpp | 6 + 6 files changed, 181 insertions(+), 152 deletions(-) diff --git a/docker/run_failure_codes.json b/docker/run_failure_codes.json index b2580231d..5a0a95d86 100644 --- a/docker/run_failure_codes.json +++ b/docker/run_failure_codes.json @@ -802,7 +802,8 @@ "max_bars_back", "matrix_elements", "udt_objects", - "drawing_ids" + "drawing_ids", + "live_drawings" ] }, "max": { @@ -822,7 +823,8 @@ "box drawing ID domain exhausted", "label drawing ID domain exhausted", "linefill drawing ID domain exhausted", - "generic drawing ID domain exhausted" + "generic drawing ID domain exhausted", + "The script keeps more than {max} live drawings of one type; the configured drawing capacity is exceeded." ], "retryable": false, "since": "1.4.0" diff --git a/docker/run_failure_codes_diff.json b/docker/run_failure_codes_diff.json index 723b258ac..3370b9cf3 100644 --- a/docker/run_failure_codes_diff.json +++ b/docker/run_failure_codes_diff.json @@ -8,7 +8,7 @@ "to": { "version": null, "unreleased": true, - "catalogSha256": "d4a555dcea58c5d7a4c3a0f2223832acb04361f6eebda5f4a642226df4a5067f" + "catalogSha256": "00b60fe00cb9371b5e5be06f671217e3d14ed0f612434dae640bed2aabddd699" }, "catalogSchema": { "before": "pineforge-run-failure-catalog/v1", @@ -238,7 +238,8 @@ "max_bars_back", "matrix_elements", "udt_objects", - "drawing_ids" + "drawing_ids", + "live_drawings" ] }, { @@ -262,7 +263,8 @@ "box drawing ID domain exhausted", "label drawing ID domain exhausted", "linefill drawing ID domain exhausted", - "generic drawing ID domain exhausted" + "generic drawing ID domain exhausted", + "The script keeps more than {max} live drawings of one type; the configured drawing capacity is exceeded." ] } ] diff --git a/docs/pages/run-failure-codes.md b/docs/pages/run-failure-codes.md index 6275a6e07..b37b940b6 100644 --- a/docs/pages/run-failure-codes.md +++ b/docs/pages/run-failure-codes.md @@ -215,7 +215,7 @@ other `strategy_create_failed` lines omit it. | `pine_map_capacity_exceeded` | `strategy_limit` | no | `bar_index` (integer), `code` (identifier), `currSize` (integer), `maxSize` (integer) | A distinct map insertion exceeds that map's configured pair limit. Zero or omitted selects no limit. Counts and bar index are engine-owned context; code is the fixed TradingView identifier RE10070. | | `pine_matrix_error` | `strategy` | no | `function` (vocab, 15 values), `reason` (vocab, 10 values) | A matrix built-in refused its arguments (an index, a dimension, a size or an empty matrix). | | `pine_na_reference` | `strategy` | no | `object` (vocab, 10 values) | The script called a method on an na reference (an array, matrix, map, drawing or object id). | -| `pine_runtime_limit` | `strategy_limit` | no | `limit` (vocab, 5 values), `max` (integer, optional) | The script reached a TradingView or PineForge resource limit. | +| `pine_runtime_limit` | `strategy_limit` | no | `limit` (vocab, 6 values), `max` (integer, optional) | The script reached a TradingView or PineForge resource limit. | | `pine_string_error` | `strategy` | no | `reason` (vocab, 2 values) | A string built-in refused its arguments. | | `recalc_cap` | `strategy_limit` | no | none | The script's calc_on_order_fills recalculation loop reached its cap. | | `recorded_request_missing` | `no_data` | no | none | No recorded series is installed for a request the script reads. | @@ -652,6 +652,23 @@ reads. | `pine_string_stop(reason, english)` | `pine_string_error` | | `pine_engine_invariant(english)` | `engine_invariant` | +`pine_limit_stop` accepts six closed `limit` values: `map_pairs`, +`max_bars_back`, `matrix_elements`, `udt_objects`, `drawing_ids` and +`live_drawings`. `max_bars_back` identifies the historical buffer limit; +`drawing_ids` identifies exhaustion of the logical drawing-ID domain. +`live_drawings` identifies an explicit operator capacity for live drawings +of one type. + +`pine_limit_stop("live_drawings", N, english)` uses the operator's configured +capacity `N` as `max` and raises the existing nonretryable +`pine_runtime_limit`, class `strategy_limit`, with arguments +`{"limit":"live_drawings","max":N}`. Its catalog English template is: + +> The script keeps more than {max} live drawings of one type; the configured drawing capacity is exceeded. + +This vocabulary extension prepares the helper contract; drawing enforcement +and its defaults remain unchanged. + The generated wrapper records an exception that escapes the strategy with `note_run_failure(engine, entrypoint, error)`: the text it writes is still `": "`, and the code is the exception's own. diff --git a/include/pineforge/run_failure.hpp b/include/pineforge/run_failure.hpp index 15012ed8d..22e25d4ab 100644 --- a/include/pineforge/run_failure.hpp +++ b/include/pineforge/run_failure.hpp @@ -82,7 +82,8 @@ // polyline | table | udt_object // pine_limit_stop(const char* limit, std::int64_t max, const char* english) // -> pine_runtime_limit {limit, max} -// limit: map_pairs | max_bars_back | matrix_elements | udt_objects; +// limit: map_pairs | max_bars_back | matrix_elements | udt_objects | +// drawing_ids | live_drawings; // max is the limit's engine or transpiler constant (pass -1 for none) // pine_unsupported_stop(const char* reason, std::int32_t line, // const char* english) diff --git a/src/run_failure_registry.inc b/src/run_failure_registry.inc index 7fb269ab1..0911c701a 100644 --- a/src/run_failure_registry.inc +++ b/src/run_failure_registry.inc @@ -126,6 +126,7 @@ static constexpr const char* kRegistryValues[] = { "matrix_elements", "udt_objects", "drawing_ids", + "live_drawings", "substring_out_of_range", "format_index_overflow", "run_in_progress", @@ -567,151 +568,151 @@ static constexpr RegistryArg kRegistryArgs[] = { {"function", RunFailureArgKind::vocab, false, 80, 15}, {"reason", RunFailureArgKind::vocab, false, 95, 10}, {"object", RunFailureArgKind::vocab, false, 105, 10}, - {"limit", RunFailureArgKind::vocab, false, 115, 5}, - {"max", RunFailureArgKind::integer, true, 120, 0}, - {"reason", RunFailureArgKind::vocab, false, 120, 2}, - {"reason", RunFailureArgKind::vocab, false, 122, 5}, - {"cap_bytes", RunFailureArgKind::integer, false, 127, 0}, - {"channel", RunFailureArgKind::vocab, false, 127, 2}, - {"requested_bytes", RunFailureArgKind::integer, false, 129, 0}, - {"used_bytes", RunFailureArgKind::integer, false, 129, 0}, - {"channel", RunFailureArgKind::vocab, false, 129, 5}, - {"reason", RunFailureArgKind::vocab, false, 134, 5}, - {"contract", RunFailureArgKind::string, false, 139, 0}, - {"mode", RunFailureArgKind::string, false, 139, 0}, - {"field", RunFailureArgKind::vocab, false, 139, 6}, - {"phase", RunFailureArgKind::vocab, false, 145, 3}, - {"reason", RunFailureArgKind::vocab, false, 148, 4}, - {"field", RunFailureArgKind::string, false, 152, 0}, - {"reason", RunFailureArgKind::vocab, false, 152, 4}, - {"option", RunFailureArgKind::vocab, false, 156, 2}, - {"value", RunFailureArgKind::scalar, false, 158, 0}, - {"shape", RunFailureArgKind::string, false, 158, 0}, - {"arena_bytes", RunFailureArgKind::integer, true, 158, 0}, - {"cap_bytes", RunFailureArgKind::integer, true, 158, 0}, - {"claimed_bytes", RunFailureArgKind::integer, true, 158, 0}, - {"high_water_bytes", RunFailureArgKind::integer, true, 158, 0}, - {"needed_bytes_lower_bound", RunFailureArgKind::integer, true, 158, 0}, - {"reason", RunFailureArgKind::vocab, true, 158, 7}, - {"sink_bytes", RunFailureArgKind::integer, true, 165, 0}, - {"store", RunFailureArgKind::vocab, true, 165, 13}, - {"api", RunFailureArgKind::vocab, false, 178, 2}, - {"input_tf", RunFailureArgKind::timeframe, true, 180, 0}, - {"reason", RunFailureArgKind::vocab, false, 180, 7}, - {"script_tf", RunFailureArgKind::timeframe, true, 187, 0}, - {"line", RunFailureArgKind::integer, true, 187, 0}, - {"reason", RunFailureArgKind::vocab, false, 187, 2}, - {"requested", RunFailureArgKind::string, false, 189, 0}, - {"bars_count", RunFailureArgKind::integer, true, 189, 0}, - {"strategy_started", RunFailureArgKind::boolean, true, 189, 0}, - {"stage", RunFailureArgKind::vocab, false, 189, 5}, - {"limit", RunFailureArgKind::integer, false, 194, 0}, - {"peak", RunFailureArgKind::integer, false, 194, 0}, - {"process_role", RunFailureArgKind::vocab, false, 194, 6}, - {"requested", RunFailureArgKind::integer, false, 200, 0}, - {"soft_nofile", RunFailureArgKind::integer, false, 200, 0}, - {"stage", RunFailureArgKind::vocab, false, 200, 8}, - {"used", RunFailureArgKind::integer, false, 208, 0}, - {"reason", RunFailureArgKind::vocab, false, 208, 3}, - {"feature", RunFailureArgKind::vocab, false, 211, 11}, - {"input_tf", RunFailureArgKind::timeframe, true, 222, 0}, - {"option", RunFailureArgKind::vocab, false, 222, 11}, - {"script_tf", RunFailureArgKind::timeframe, true, 233, 0}, - {"operation", RunFailureArgKind::vocab, false, 233, 4}, - {"exit_status", RunFailureArgKind::integer, true, 237, 0}, - {"reason", RunFailureArgKind::vocab, false, 237, 3}, - {"signal", RunFailureArgKind::integer, true, 240, 0}, - {"reason", RunFailureArgKind::vocab, false, 240, 8}, - {"setting", RunFailureArgKind::vocab, false, 248, 10}, - {"stage", RunFailureArgKind::vocab, false, 258, 8}, - {"option", RunFailureArgKind::vocab, false, 266, 25}, - {"artifact", RunFailureArgKind::vocab, false, 291, 14}, - {"limit", RunFailureArgKind::integer, false, 305, 0}, - {"peak", RunFailureArgKind::integer, false, 305, 0}, - {"requested", RunFailureArgKind::integer, false, 305, 0}, - {"resource", RunFailureArgKind::vocab, false, 305, 3}, - {"stage", RunFailureArgKind::vocab, false, 308, 8}, - {"used", RunFailureArgKind::integer, false, 316, 0}, - {"configured", RunFailureArgKind::string, false, 316, 0}, - {"needed", RunFailureArgKind::string, false, 316, 0}, - {"used", RunFailureArgKind::string, false, 316, 0}, - {"backing_bytes_lower_bound", RunFailureArgKind::string, false, 316, 0}, - {"overflow", RunFailureArgKind::boolean, false, 316, 0}, - {"resident_metadata_bytes_lower_bound", RunFailureArgKind::string, false, 316, 0}, - {"resident_payload_bytes_lower_bound", RunFailureArgKind::string, false, 316, 0}, - {"unsupported_flags", RunFailureArgKind::string, false, 316, 0}, - {"resource", RunFailureArgKind::vocab, false, 316, 12}, - {"reason", RunFailureArgKind::vocab, false, 328, 3}, - {"stage", RunFailureArgKind::vocab, false, 331, 1}, - {"reason", RunFailureArgKind::vocab, false, 332, 11}, - {"limit", RunFailureArgKind::string, false, 343, 0}, - {"requested", RunFailureArgKind::string, false, 343, 0}, - {"used", RunFailureArgKind::string, false, 343, 0}, - {"limit", RunFailureArgKind::string, false, 343, 0}, - {"requested", RunFailureArgKind::string, false, 343, 0}, - {"used", RunFailureArgKind::string, false, 343, 0}, - {"limit", RunFailureArgKind::string, false, 343, 0}, - {"requested", RunFailureArgKind::string, false, 343, 0}, - {"used", RunFailureArgKind::string, false, 343, 0}, - {"limit", RunFailureArgKind::string, false, 343, 0}, - {"requested", RunFailureArgKind::string, false, 343, 0}, - {"used", RunFailureArgKind::string, false, 343, 0}, - {"limit", RunFailureArgKind::string, false, 343, 0}, - {"requested", RunFailureArgKind::string, false, 343, 0}, - {"used", RunFailureArgKind::string, false, 343, 0}, - {"entrypoint", RunFailureArgKind::vocab, true, 343, 4}, - {"input", RunFailureArgKind::pine_source, true, 347, 0}, - {"reason", RunFailureArgKind::vocab, true, 347, 21}, - {"abi", RunFailureArgKind::integer, true, 368, 0}, - {"missing", RunFailureArgKind::vocab, true, 368, 13}, - {"reason", RunFailureArgKind::vocab, false, 381, 8}, - {"field", RunFailureArgKind::vocab, false, 389, 33}, - {"field", RunFailureArgKind::string, false, 422, 0}, - {"reason", RunFailureArgKind::vocab, false, 422, 3}, - {"field", RunFailureArgKind::vocab, true, 425, 6}, - {"input_tf", RunFailureArgKind::timeframe, true, 431, 0}, - {"reason", RunFailureArgKind::vocab, false, 431, 47}, - {"script_tf", RunFailureArgKind::timeframe, true, 478, 0}, - {"field", RunFailureArgKind::vocab, false, 478, 14}, - {"reason", RunFailureArgKind::vocab, false, 492, 4}, - {"bound", RunFailureArgKind::vocab, false, 496, 3}, - {"next_boundary_ms", RunFailureArgKind::integer, false, 499, 0}, - {"previous_boundary_ms", RunFailureArgKind::integer, false, 499, 0}, - {"value_ms", RunFailureArgKind::integer, false, 499, 0}, - {"session", RunFailureArgKind::nullable_string, false, 499, 0}, - {"timezone", RunFailureArgKind::string, false, 499, 0}, - {"actual_sha256", RunFailureArgKind::string, false, 499, 0}, - {"expected_sha256", RunFailureArgKind::string, false, 499, 0}, - {"feed_id", RunFailureArgKind::string, false, 499, 0}, - {"end_ms", RunFailureArgKind::integer, false, 499, 0}, - {"fed_start_ms", RunFailureArgKind::integer, false, 499, 0}, - {"time_ms", RunFailureArgKind::integer, false, 499, 0}, - {"actual", RunFailureArgKind::integer, false, 499, 0}, - {"cap_name", RunFailureArgKind::vocab, false, 499, 8}, - {"limit", RunFailureArgKind::integer, false, 507, 0}, - {"option", RunFailureArgKind::string, false, 507, 0}, - {"actual", RunFailureArgKind::scalar, false, 507, 0}, - {"expected", RunFailureArgKind::scalar, false, 507, 0}, - {"field", RunFailureArgKind::string, false, 507, 0}, - {"capability", RunFailureArgKind::string, false, 507, 0}, - {"requested", RunFailureArgKind::integer, true, 507, 0}, - {"timeframe", RunFailureArgKind::string, true, 507, 0}, - {"end_ms", RunFailureArgKind::integer, false, 507, 0}, - {"start_ms", RunFailureArgKind::integer, false, 507, 0}, - {"requested", RunFailureArgKind::integer, false, 507, 0}, - {"fill_ms", RunFailureArgKind::integer, false, 507, 0}, - {"start_ms", RunFailureArgKind::integer, false, 507, 0}, - {"position_size", RunFailureArgKind::number, false, 507, 0}, - {"resting_orders", RunFailureArgKind::integer, false, 507, 0}, - {"time_ms", RunFailureArgKind::integer, false, 507, 0}, - {"pruner", RunFailureArgKind::string, false, 507, 0}, - {"end_ms", RunFailureArgKind::integer, false, 507, 0}, - {"fed_start_ms", RunFailureArgKind::integer, false, 507, 0}, - {"start_ms", RunFailureArgKind::integer, false, 507, 0}, - {"actual", RunFailureArgKind::integer, false, 507, 0}, - {"expected", RunFailureArgKind::integer, false, 507, 0}, - {"field", RunFailureArgKind::vocab, false, 507, 3}, - {"option", RunFailureArgKind::vocab, false, 510, 13}, + {"limit", RunFailureArgKind::vocab, false, 115, 6}, + {"max", RunFailureArgKind::integer, true, 121, 0}, + {"reason", RunFailureArgKind::vocab, false, 121, 2}, + {"reason", RunFailureArgKind::vocab, false, 123, 5}, + {"cap_bytes", RunFailureArgKind::integer, false, 128, 0}, + {"channel", RunFailureArgKind::vocab, false, 128, 2}, + {"requested_bytes", RunFailureArgKind::integer, false, 130, 0}, + {"used_bytes", RunFailureArgKind::integer, false, 130, 0}, + {"channel", RunFailureArgKind::vocab, false, 130, 5}, + {"reason", RunFailureArgKind::vocab, false, 135, 5}, + {"contract", RunFailureArgKind::string, false, 140, 0}, + {"mode", RunFailureArgKind::string, false, 140, 0}, + {"field", RunFailureArgKind::vocab, false, 140, 6}, + {"phase", RunFailureArgKind::vocab, false, 146, 3}, + {"reason", RunFailureArgKind::vocab, false, 149, 4}, + {"field", RunFailureArgKind::string, false, 153, 0}, + {"reason", RunFailureArgKind::vocab, false, 153, 4}, + {"option", RunFailureArgKind::vocab, false, 157, 2}, + {"value", RunFailureArgKind::scalar, false, 159, 0}, + {"shape", RunFailureArgKind::string, false, 159, 0}, + {"arena_bytes", RunFailureArgKind::integer, true, 159, 0}, + {"cap_bytes", RunFailureArgKind::integer, true, 159, 0}, + {"claimed_bytes", RunFailureArgKind::integer, true, 159, 0}, + {"high_water_bytes", RunFailureArgKind::integer, true, 159, 0}, + {"needed_bytes_lower_bound", RunFailureArgKind::integer, true, 159, 0}, + {"reason", RunFailureArgKind::vocab, true, 159, 7}, + {"sink_bytes", RunFailureArgKind::integer, true, 166, 0}, + {"store", RunFailureArgKind::vocab, true, 166, 13}, + {"api", RunFailureArgKind::vocab, false, 179, 2}, + {"input_tf", RunFailureArgKind::timeframe, true, 181, 0}, + {"reason", RunFailureArgKind::vocab, false, 181, 7}, + {"script_tf", RunFailureArgKind::timeframe, true, 188, 0}, + {"line", RunFailureArgKind::integer, true, 188, 0}, + {"reason", RunFailureArgKind::vocab, false, 188, 2}, + {"requested", RunFailureArgKind::string, false, 190, 0}, + {"bars_count", RunFailureArgKind::integer, true, 190, 0}, + {"strategy_started", RunFailureArgKind::boolean, true, 190, 0}, + {"stage", RunFailureArgKind::vocab, false, 190, 5}, + {"limit", RunFailureArgKind::integer, false, 195, 0}, + {"peak", RunFailureArgKind::integer, false, 195, 0}, + {"process_role", RunFailureArgKind::vocab, false, 195, 6}, + {"requested", RunFailureArgKind::integer, false, 201, 0}, + {"soft_nofile", RunFailureArgKind::integer, false, 201, 0}, + {"stage", RunFailureArgKind::vocab, false, 201, 8}, + {"used", RunFailureArgKind::integer, false, 209, 0}, + {"reason", RunFailureArgKind::vocab, false, 209, 3}, + {"feature", RunFailureArgKind::vocab, false, 212, 11}, + {"input_tf", RunFailureArgKind::timeframe, true, 223, 0}, + {"option", RunFailureArgKind::vocab, false, 223, 11}, + {"script_tf", RunFailureArgKind::timeframe, true, 234, 0}, + {"operation", RunFailureArgKind::vocab, false, 234, 4}, + {"exit_status", RunFailureArgKind::integer, true, 238, 0}, + {"reason", RunFailureArgKind::vocab, false, 238, 3}, + {"signal", RunFailureArgKind::integer, true, 241, 0}, + {"reason", RunFailureArgKind::vocab, false, 241, 8}, + {"setting", RunFailureArgKind::vocab, false, 249, 10}, + {"stage", RunFailureArgKind::vocab, false, 259, 8}, + {"option", RunFailureArgKind::vocab, false, 267, 25}, + {"artifact", RunFailureArgKind::vocab, false, 292, 14}, + {"limit", RunFailureArgKind::integer, false, 306, 0}, + {"peak", RunFailureArgKind::integer, false, 306, 0}, + {"requested", RunFailureArgKind::integer, false, 306, 0}, + {"resource", RunFailureArgKind::vocab, false, 306, 3}, + {"stage", RunFailureArgKind::vocab, false, 309, 8}, + {"used", RunFailureArgKind::integer, false, 317, 0}, + {"configured", RunFailureArgKind::string, false, 317, 0}, + {"needed", RunFailureArgKind::string, false, 317, 0}, + {"used", RunFailureArgKind::string, false, 317, 0}, + {"backing_bytes_lower_bound", RunFailureArgKind::string, false, 317, 0}, + {"overflow", RunFailureArgKind::boolean, false, 317, 0}, + {"resident_metadata_bytes_lower_bound", RunFailureArgKind::string, false, 317, 0}, + {"resident_payload_bytes_lower_bound", RunFailureArgKind::string, false, 317, 0}, + {"unsupported_flags", RunFailureArgKind::string, false, 317, 0}, + {"resource", RunFailureArgKind::vocab, false, 317, 12}, + {"reason", RunFailureArgKind::vocab, false, 329, 3}, + {"stage", RunFailureArgKind::vocab, false, 332, 1}, + {"reason", RunFailureArgKind::vocab, false, 333, 11}, + {"limit", RunFailureArgKind::string, false, 344, 0}, + {"requested", RunFailureArgKind::string, false, 344, 0}, + {"used", RunFailureArgKind::string, false, 344, 0}, + {"limit", RunFailureArgKind::string, false, 344, 0}, + {"requested", RunFailureArgKind::string, false, 344, 0}, + {"used", RunFailureArgKind::string, false, 344, 0}, + {"limit", RunFailureArgKind::string, false, 344, 0}, + {"requested", RunFailureArgKind::string, false, 344, 0}, + {"used", RunFailureArgKind::string, false, 344, 0}, + {"limit", RunFailureArgKind::string, false, 344, 0}, + {"requested", RunFailureArgKind::string, false, 344, 0}, + {"used", RunFailureArgKind::string, false, 344, 0}, + {"limit", RunFailureArgKind::string, false, 344, 0}, + {"requested", RunFailureArgKind::string, false, 344, 0}, + {"used", RunFailureArgKind::string, false, 344, 0}, + {"entrypoint", RunFailureArgKind::vocab, true, 344, 4}, + {"input", RunFailureArgKind::pine_source, true, 348, 0}, + {"reason", RunFailureArgKind::vocab, true, 348, 21}, + {"abi", RunFailureArgKind::integer, true, 369, 0}, + {"missing", RunFailureArgKind::vocab, true, 369, 13}, + {"reason", RunFailureArgKind::vocab, false, 382, 8}, + {"field", RunFailureArgKind::vocab, false, 390, 33}, + {"field", RunFailureArgKind::string, false, 423, 0}, + {"reason", RunFailureArgKind::vocab, false, 423, 3}, + {"field", RunFailureArgKind::vocab, true, 426, 6}, + {"input_tf", RunFailureArgKind::timeframe, true, 432, 0}, + {"reason", RunFailureArgKind::vocab, false, 432, 47}, + {"script_tf", RunFailureArgKind::timeframe, true, 479, 0}, + {"field", RunFailureArgKind::vocab, false, 479, 14}, + {"reason", RunFailureArgKind::vocab, false, 493, 4}, + {"bound", RunFailureArgKind::vocab, false, 497, 3}, + {"next_boundary_ms", RunFailureArgKind::integer, false, 500, 0}, + {"previous_boundary_ms", RunFailureArgKind::integer, false, 500, 0}, + {"value_ms", RunFailureArgKind::integer, false, 500, 0}, + {"session", RunFailureArgKind::nullable_string, false, 500, 0}, + {"timezone", RunFailureArgKind::string, false, 500, 0}, + {"actual_sha256", RunFailureArgKind::string, false, 500, 0}, + {"expected_sha256", RunFailureArgKind::string, false, 500, 0}, + {"feed_id", RunFailureArgKind::string, false, 500, 0}, + {"end_ms", RunFailureArgKind::integer, false, 500, 0}, + {"fed_start_ms", RunFailureArgKind::integer, false, 500, 0}, + {"time_ms", RunFailureArgKind::integer, false, 500, 0}, + {"actual", RunFailureArgKind::integer, false, 500, 0}, + {"cap_name", RunFailureArgKind::vocab, false, 500, 8}, + {"limit", RunFailureArgKind::integer, false, 508, 0}, + {"option", RunFailureArgKind::string, false, 508, 0}, + {"actual", RunFailureArgKind::scalar, false, 508, 0}, + {"expected", RunFailureArgKind::scalar, false, 508, 0}, + {"field", RunFailureArgKind::string, false, 508, 0}, + {"capability", RunFailureArgKind::string, false, 508, 0}, + {"requested", RunFailureArgKind::integer, true, 508, 0}, + {"timeframe", RunFailureArgKind::string, true, 508, 0}, + {"end_ms", RunFailureArgKind::integer, false, 508, 0}, + {"start_ms", RunFailureArgKind::integer, false, 508, 0}, + {"requested", RunFailureArgKind::integer, false, 508, 0}, + {"fill_ms", RunFailureArgKind::integer, false, 508, 0}, + {"start_ms", RunFailureArgKind::integer, false, 508, 0}, + {"position_size", RunFailureArgKind::number, false, 508, 0}, + {"resting_orders", RunFailureArgKind::integer, false, 508, 0}, + {"time_ms", RunFailureArgKind::integer, false, 508, 0}, + {"pruner", RunFailureArgKind::string, false, 508, 0}, + {"end_ms", RunFailureArgKind::integer, false, 508, 0}, + {"fed_start_ms", RunFailureArgKind::integer, false, 508, 0}, + {"start_ms", RunFailureArgKind::integer, false, 508, 0}, + {"actual", RunFailureArgKind::integer, false, 508, 0}, + {"expected", RunFailureArgKind::integer, false, 508, 0}, + {"field", RunFailureArgKind::vocab, false, 508, 3}, + {"option", RunFailureArgKind::vocab, false, 511, 13}, }; static constexpr RegistryCode kRegistryCodes[] = { diff --git a/tests/test_run_failure_codes.cpp b/tests/test_run_failure_codes.cpp index 9f4c520f8..28c61bc43 100644 --- a/tests/test_run_failure_codes.cpp +++ b/tests/test_run_failure_codes.cpp @@ -434,6 +434,12 @@ void helper_contract_rows() { "{}"}, {"pine_limit_stop max", [] { pine_limit_stop("map_pairs", 50000, "E"); }, "pine_runtime_limit", R"({"limit":"map_pairs","max":50000})"}, + {"pine_limit_stop live_drawings 40", + [] { pine_limit_stop("live_drawings", 40, "E"); }, + "pine_runtime_limit", R"({"limit":"live_drawings","max":40})"}, + {"pine_limit_stop live_drawings 100", + [] { pine_limit_stop("live_drawings", 100, "E"); }, + "pine_runtime_limit", R"({"limit":"live_drawings","max":100})"}, {"pine_limit_stop no max", [] { pine_limit_stop("udt_objects", -1, "E"); }, "pine_runtime_limit", R"({"limit":"udt_objects"})"}, {"pine_limit_stop outside list", [] { pine_limit_stop("loop_iterations", 5, "E"); }, From fcfc07cc27d1cc49d83d394af830b2ec61f735e9 Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Sat, 7 Nov 2026 15:26:01 +0800 Subject: [PATCH 15/19] ci: promote the engine baseline on parity alone The merge gate now requires pineforge/parity and treats pineforge/verify as advisory, so the promotion workflow skipped every engine merge whose head carried no verify status. Admit on pineforge/parity success alone; a failed or absent parity still skips. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/promote-baseline.yml | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/.github/workflows/promote-baseline.yml b/.github/workflows/promote-baseline.yml index 22f22beb6..2f89f7a5e 100644 --- a/.github/workflows/promote-baseline.yml +++ b/.github/workflows/promote-baseline.yml @@ -9,7 +9,7 @@ # now on the base branch carries the gated PR head's TREE (a squash merge mints # a new commit; the strict up-to-date rule keeps its tree the head's), the gate # is green on that head, and `lab promote` finds the verdict that binds it: -# engine pineforge/verify and pineforge/parity success on the PR head; +# engine pineforge/parity success on the PR head (pineforge/verify is advisory); # lab promote adopts the pair the merge gate's parity verdict # measured — the merged engine with the codegen the sweep measured # (ruling R2, 2026-09-26), PASS or neutral alike @@ -98,8 +98,8 @@ permissions: # commits/{sha}/pulls needs pull-request read access for the repo-scoped # token, including when this unchanged workflow is installed in a private repo. pull-requests: read - # The engine's gate is two commit statuses (pineforge/verify, - # pineforge/parity): `statuses`. The codegen axis reads check-runs: `checks`. + # The engine axis admits on pineforge/parity; pineforge/verify is advisory. + # Read `statuses`; the codegen axis reads check-runs: `checks`. statuses: read checks: read # Workload identity federation mints the GCP credential from the job's own @@ -372,10 +372,9 @@ jobs: if ! jq -es ' reduce (.[][]) as $s ({}; if has($s.context) then . else .[$s.context] = $s.state end) - | .["pineforge/verify"] == "success" - and .["pineforge/parity"] == "success" + | .["pineforge/parity"] == "success" ' <<< "$statuses" >/dev/null; then - echo "::notice::pineforge/verify and pineforge/parity must both be success on $HEAD_SHA — skipping" + echo "::notice::pineforge/parity must be success on $HEAD_SHA — skipping" echo "green=false" >> "$GITHUB_OUTPUT"; exit 0 fi else From a46b5867d796f4745da1a07c7f726170b9de7d81 Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Sat, 7 Nov 2026 19:48:01 +0800 Subject: [PATCH 16/19] Align promotion preflight with parity-only admission Require the complete parity-only head predicate while preserving main-copy, latest-status, pagination, tree, authorization and credential-order checks. Add Bash and jq status decisions and workflow mutation controls. --- scripts/ci_preflight.py | 10 ++-- scripts/test_ci_preflight.py | 101 +++++++++++++++++++++++++++++++++-- 2 files changed, 104 insertions(+), 7 deletions(-) diff --git a/scripts/ci_preflight.py b/scripts/ci_preflight.py index b7c100728..34c826777 100644 --- a/scripts/ci_preflight.py +++ b/scripts/ci_preflight.py @@ -4471,7 +4471,7 @@ def ci_workflow_findings(ci: str, native: str, promote: str, cmake: str, # installed unchanged. Main's own copy runs on push to main and resolves # exactly one merged PR in the guard before authorizing its API identities. # PR objects are fetched only for tree comparison. Both events retain - # the newest two engine head statuses and the codegen check-run rule. + # the newest engine head parity status and the codegen check-run rule. promote_job = _jobs(promote).get('promote', '') guard_job = _jobs(promote).get('promote-guard', '') guard_steps = _steps(guard_job) @@ -4546,10 +4546,12 @@ def ci_workflow_findings(ci: str, native: str, promote: str, cmake: str, or 'gh api --paginate' not in head_gate or 'reduce (.[][]) as $s' not in head_gate or 'if has($s.context) then . else .[$s.context] = $s.state end' not in head_gate - or '.["pineforge/verify"] == "success"' not in head_gate - or '.["pineforge/parity"] == "success"' not in head_gate + or re.findall(r'^[ \t]*\|[ \t]*(.*?)\s*\' <<< "\$statuses"', + head_gate, re.MULTILINE | re.DOTALL) + != ['.["pineforge/parity"] == "success"'] or head_gate.count('echo "green=false" >> "$GITHUB_OUTPUT"; exit 0') != 2): - findings.append("baseline promotion must run main's copy and require the latest two head statuses") + findings.append("baseline promotion must run main's copy and require " + "the latest head parity status only (verify is advisory)") # The job pin cannot see inherited workflow settings. for key in ('defaults', 'env'): if _job_values(promote, key): diff --git a/scripts/test_ci_preflight.py b/scripts/test_ci_preflight.py index 13523109d..927dd3839 100644 --- a/scripts/test_ci_preflight.py +++ b/scripts/test_ci_preflight.py @@ -5439,7 +5439,7 @@ def test_ci_contract_pins_parallel_pr_light_and_full_events(self): (1, ' default: false', ' default: true'), (1, NATIVE_EXCLUSION, ''), # Baseline promotion starts from main's push and runs no PR code; - # it needs the newest two statuses on the resolved PR head. + # it needs the newest parity status on the resolved PR head. (2, ' push:\n branches: [main]\n', ' pull_request:\n'), (2, ' statuses: read', ' checks: read'), (2, ' id-token: write', ' id-token: none'), @@ -5641,7 +5641,7 @@ def test_baseline_promotion_rejects_short_base_ref(self): def test_baseline_promotion_rejects_later_pr_head_checkout(self): finding = ("baseline promotion must run main's copy and require " - "the latest two head statuses") + "the latest head parity status only (verify is advisory)") self.assertNotIn(finding, ci_workflow_findings(*self.ci_sources(), others=self.other_sources())) anchor = ' - name: Mint an org App token to read the campaign tooling\n' @@ -5652,6 +5652,100 @@ def test_baseline_promotion_rejects_later_pr_head_checkout(self): findings = self.mutated_file_findings(2, 'promote', anchor, checkout + anchor) self.assertIn(finding, findings) + def promotion_head_gate_result(self, pages, *, workflow=None, + token='fixture-token', api_exit=0): + """Run the workflow's head gate with real jq and a fake API transport.""" + bash = shutil.which('bash') + if bash is None or shutil.which('jq') is None: + self.fail('NOT RUN: bash and jq are required for promotion head-status cases') + workflow = self.ci_sources()[2] if workflow is None else workflow + script = job_script(workflow, 'promote', 'Confirm the gate on the PR head') + script = script.replace('${{ github.repository }}', 'owner/engine') + self.assertNotIn('${{', script) + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + gh = root / 'gh' + gh.write_text('#!' + sys.executable + ' -I\n' + 'import json, os, pathlib, sys\n' + f'pages = {pages!r}\napi_exit = {api_exit!r}\n' + 'args = sys.argv[1:]\n' + 'with pathlib.Path(__file__).with_name("lookup").open("a") as log:\n' + ' log.write(" ".join(args) + "\\n")\n' + 'if (os.environ.get("GH_TOKEN") != "fixture-token" or args != ' + '["api", "--paginate", "repos/owner/engine/commits/" + "a" * 40 ' + '+ "/statuses?per_page=100"]):\n' + ' raise SystemExit(2)\n' + 'for page in pages: print(json.dumps(page))\n' + 'raise SystemExit(api_exit)\n') + gh.chmod(0o755) + output = root / 'outputs' + lookup = root / 'lookup' + result = subprocess.run( + [bash, '--noprofile', '--norc', '-eo', 'pipefail', '-c', script], + env={'PATH': directory + os.pathsep + os.environ.get('PATH', '/usr/bin:/bin'), + 'GH_TOKEN': token, 'HEAD_SHA': 'a' * 40, 'AXIS': 'engine', + 'GITHUB_OUTPUT': str(output)}, + capture_output=True, text=True) + return (result, output.read_text() if output.exists() else '', + lookup.read_text() if lookup.exists() else '') + + def test_baseline_promotion_accepts_only_latest_parity(self): + sources = self.ci_sources() + # The published parity-only workflow is the positive input, unchanged. + # On the old checker this must fail with the actual contract finding. + self.assertEqual(ci_workflow_findings(*sources), []) + parity = {'context': 'pineforge/parity', 'state': 'success'} + verify = {'context': 'pineforge/verify', 'state': 'success'} + cases = ( + ('parity-only', [[parity]], True), + ('verify-failed', [[{**verify, 'state': 'failure'}, parity]], True), + ('verify-pending', [[{**verify, 'state': 'pending'}, parity]], True), + ('both-green', [[verify, parity]], True), + ('verify-only', [[verify]], False), + ('no-statuses', [[]], False), + ('parity-failed', [[verify, {**parity, 'state': 'failure'}]], False), + ('parity-pending', [[verify, {**parity, 'state': 'pending'}]], False), + ('parity-on-later-page', [[verify], [parity]], True), + ('newer-red', [[{**parity, 'state': 'failure'}], [parity, verify]], False), + ('newer-pending', [[{**parity, 'state': 'pending'}], [parity]], False), + ('newer-green', [[parity], [{**parity, 'state': 'failure'}, verify]], True), + ) + for name, pages, green in cases: + with self.subTest(statuses=name): + result, output, lookup = self.promotion_head_gate_result( + pages, workflow=sources[2]) + self.assertEqual(result.returncode, 0, result.stdout + result.stderr) + self.assertEqual(output, f'green={str(green).lower()}\n') + self.assertEqual(lookup, 'api --paginate repos/owner/engine/commits/' + + 'a' * 40 + '/statuses?per_page=100\n') + + def test_baseline_promotion_head_gate_refuses_status_transport_errors(self): + pages = [[{'context': 'pineforge/parity', 'state': 'success'}]] + for options, code in (({'api_exit': 1}, 1), ({'token': ''}, 2)): + with self.subTest(transport=options): + result, output, lookup = self.promotion_head_gate_result(pages, **options) + self.assertEqual(result.returncode, code, result.stdout + result.stderr) + self.assertEqual(output, '') + self.assertEqual(lookup, 'api --paginate repos/owner/engine/commits/' + + 'a' * 40 + '/statuses?per_page=100\n') + + def test_baseline_promotion_rejects_head_status_policy_mutations(self): + original = self.ci_sources() + self.assertEqual(ci_workflow_findings(*original), []) + parity = '.["pineforge/parity"] == "success"' + verify = '.["pineforge/verify"] == "success"' + finding = ("baseline promotion must run main's copy and require " + "the latest head parity status only (verify is advisory)") + for predicate in (verify + ' and ' + parity, parity + ' and ' + verify, + parity + '\n and ' + verify, + verify, 'true', '.["pineforge/parity"] == "failure"'): + with self.subTest(predicate=predicate): + changed = original.copy() + changed[2] = in_step(changed[2], 'promote', 'Confirm the gate on the PR head', + parity, predicate) + self.assertNotEqual(changed[2], original[2]) + self.assertIn(finding, ci_workflow_findings(*changed)) + def test_promotion_guard_rejects_added_contents_read_permission(self): finding = 'baseline promotion must start with the pinned read-only promote-guard' findings = self.mutated_file_findings( @@ -5833,7 +5927,8 @@ def test_promotion_guard_rejects_mutations_of_the_real_workflow(self): changed = original.copy() self.assertIn(before, changed[2]) changed[2] = changed[2].replace(before, after, 1) - self.assertIn("baseline promotion must run main's copy and require the latest two head statuses", + self.assertIn("baseline promotion must run main's copy and require " + "the latest head parity status only (verify is advisory)", ci_workflow_findings(*changed)) @staticmethod From 349418e59aa35b37c0b5b6d6979d9dbb548a9942 Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Sat, 7 Nov 2026 19:48:13 +0800 Subject: [PATCH 17/19] Accept grouped parity-only promotion predicates Normalize enclosing parentheses around the exact parity-success expression. Reject compound and malformed predicates, and cover parity-only, verify-only, combined-status and missing-status cases plus expression wrapping controls. --- scripts/ci_preflight.py | 13 ++++++++++--- scripts/test_ci_preflight.py | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 3 deletions(-) diff --git a/scripts/ci_preflight.py b/scripts/ci_preflight.py index 34c826777..84927679b 100644 --- a/scripts/ci_preflight.py +++ b/scripts/ci_preflight.py @@ -4534,6 +4534,15 @@ def ci_workflow_findings(ci: str, native: str, promote: str, cmake: str, or any(_step_values(step, 'if') != ["steps.ci.outputs.green == 'true'"] for step in promote_steps[4:])): findings.append('baseline promotion must guard resolution, tree and gate checks before credentials') + head_predicates = re.findall(r'^[ \t]*\|[ \t]*(.*?)\s*\' <<< "\$statuses"', + head_gate, re.MULTILINE | re.DOTALL) + for index, predicate in enumerate(head_predicates): + predicate = predicate.strip() + # Only whole-expression wrappers may surround the exact parity atom. + # Unbalanced wrappers and compound expressions cannot reduce to it. + while predicate.startswith('(') and predicate.endswith(')'): + predicate = predicate[1:-1].strip() + head_predicates[index] = predicate if (_events(promote) != {'push', 'workflow_dispatch'} or re.search(r'^ pull_request(?:_target)?:', promote, re.MULTILINE) or ' statuses: read' not in promote @@ -4546,9 +4555,7 @@ def ci_workflow_findings(ci: str, native: str, promote: str, cmake: str, or 'gh api --paginate' not in head_gate or 'reduce (.[][]) as $s' not in head_gate or 'if has($s.context) then . else .[$s.context] = $s.state end' not in head_gate - or re.findall(r'^[ \t]*\|[ \t]*(.*?)\s*\' <<< "\$statuses"', - head_gate, re.MULTILINE | re.DOTALL) - != ['.["pineforge/parity"] == "success"'] + or head_predicates != ['.["pineforge/parity"] == "success"'] or head_gate.count('echo "green=false" >> "$GITHUB_OUTPUT"; exit 0') != 2): findings.append("baseline promotion must run main's copy and require " "the latest head parity status only (verify is advisory)") diff --git a/scripts/test_ci_preflight.py b/scripts/test_ci_preflight.py index 927dd3839..04fcbddda 100644 --- a/scripts/test_ci_preflight.py +++ b/scripts/test_ci_preflight.py @@ -5738,6 +5738,10 @@ def test_baseline_promotion_rejects_head_status_policy_mutations(self): "the latest head parity status only (verify is advisory)") for predicate in (verify + ' and ' + parity, parity + ' and ' + verify, parity + '\n and ' + verify, + '(' + parity, parity + ')', '((' + parity + ')', + '(' + parity + '))', '(' + parity + ') or true', + '(' + parity + ') and true', 'not (' + parity + ')', + '(' + parity + ') | true', verify, 'true', '.["pineforge/parity"] == "failure"'): with self.subTest(predicate=predicate): changed = original.copy() @@ -5746,6 +5750,37 @@ def test_baseline_promotion_rejects_head_status_policy_mutations(self): self.assertNotEqual(changed[2], original[2]) self.assertIn(finding, ci_workflow_findings(*changed)) + def test_baseline_promotion_head_predicate_four_rows_and_wrapping(self): + original = self.ci_sources() + self.assertEqual(ci_workflow_findings(*original), []) + parity = '.["pineforge/parity"] == "success"' + finding = ("baseline promotion must run main's copy and require " + "the latest head parity status only (verify is advisory)") + cases = ( + ('parenthesized-parity-only', + '(.["pineforge/parity"] == "success")', []), + ('neither-required', 'true', [finding]), + ('parenthesized-verify-and-parity', + '(.["pineforge/verify"] == "success" and .["pineforge/parity"] == "success")', + [finding]), + ('verify-only', '.["pineforge/verify"] == "success"', [finding]), + ) + for name, predicate, expected in cases: + with self.subTest(policy=name): + changed = original.copy() + changed[2] = in_step(changed[2], 'promote', 'Confirm the gate on the PR head', + parity, predicate) + self.assertNotEqual(changed[2], original[2]) + self.assertEqual(ci_workflow_findings(*changed), expected) + for predicate in (' ' + parity + ' ', '( ( ' + parity + ' ) )', + '(\n ( ' + parity + ' )\n )'): + with self.subTest(wrapping=predicate): + changed = original.copy() + changed[2] = in_step(changed[2], 'promote', 'Confirm the gate on the PR head', + parity, predicate) + self.assertNotEqual(changed[2], original[2]) + self.assertEqual(ci_workflow_findings(*changed), []) + def test_promotion_guard_rejects_added_contents_read_permission(self): finding = 'baseline promotion must start with the pinned read-only promote-guard' findings = self.mutated_file_findings( From d6772ea0b36e4d9aa68635d210de8a7cf79057fd Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Sat, 7 Nov 2026 20:07:29 +0800 Subject: [PATCH 18/19] Repair runtime failure catalog documentation references Point seven documentation citations at the matching catalog fields after the vocabulary additions shifted their lines. --- docs/design/BARS-BIN-INPUT.md | 10 +++++----- docs/pages/memory-management.md | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/design/BARS-BIN-INPUT.md b/docs/design/BARS-BIN-INPUT.md index 485ce4ab3..869b87304 100644 --- a/docs/design/BARS-BIN-INPUT.md +++ b/docs/design/BARS-BIN-INPUT.md @@ -205,8 +205,8 @@ Selected-window selection includes a report policy, any window option or `--validate-window-only` (`_selected_requested` docker/run_json.py:4699). The refusals above use the ordinary request code, not a window-specific code (`_request_invalid` docker/run_json.py:4465). The catalog names -`run_request_invalid` docker/run_failure_codes.json:1538 and includes the -option `ohlcv_bin` docker/run_failure_codes.json:1563. +`run_request_invalid` docker/run_failure_codes.json:1540 and includes the +option `ohlcv_bin` docker/run_failure_codes.json:1565. These request refusals exit 1 and emit the failure JSON line on stdout (`exit_status` docker/run_json.py:4465; `write_failure` docker/run_json.py:2366). @@ -224,7 +224,7 @@ Read-only searches covered `docker/`, `scripts/`, `runner/` and repository filenames/release notes; `campaign/` and `cloudrun/` are absent. The local backtest entry is `main` docker/run_json.py:5345; the catalog itself identifies the optimizer-only refusal separately (`description` -docker/run_failure_codes.json:2593). Neither establishes the production HPO +docker/run_failure_codes.json:2595). Neither establishes the production HPO runner's input support. Check the separate **pineforge-hpo** repository named by this repository's @@ -336,8 +336,8 @@ Failure JSON uses `engine`, `error`, `code`, `args` (`doc` docker/run_json.py:2322; `code` docker/run_json.py:2325; `args` docker/run_json.py:2326). The catalog entries are `chart_bars_unreadable` docker/run_failure_codes.json:60, `symbol_feeds_refused` -docker/run_failure_codes.json:2178 and `run_request_invalid` -docker/run_failure_codes.json:1538. Shared binary-reader helpers additionally +docker/run_failure_codes.json:2180 and `run_request_invalid` +docker/run_failure_codes.json:1540. Shared binary-reader helpers additionally refuse malformed file `size`, metadata/digests as `sidecar`, and a bad feed companion as `close_file` (`bar_count` docker/pf_bars.py:101; `check_sidecar` docker/pf_bars.py:137; `check_close_size` diff --git a/docs/pages/memory-management.md b/docs/pages/memory-management.md index 89ac87bab..094756bb5 100644 --- a/docs/pages/memory-management.md +++ b/docs/pages/memory-management.md @@ -300,8 +300,8 @@ trailer, 7 internal error, 8 `sink_exhausted` (the stream exceeds `--max-bytes`; `sink_exhausted` at `docker/pineforge_sink_finalize.py:115-123`). Its output is provisional until exit 0, and a consumer must discard everything it received on exit 3 to 8 (`PROVISIONAL` at `docker/pineforge_sink_finalize.py:24-26`). The failure catalog maps exit 8 to `report_sink_failed` -with `reason` `sink_exhausted` (`sink_exhausted` at `docker/run_failure_codes.json:1134`). A sink failure carries code `report_sink_failed` and a `reason` -from the closed list of the catalog (`report_sink_failed` at `docker/run_failure_codes.json:1074`): +with `reason` `sink_exhausted` (`sink_exhausted` at `docker/run_failure_codes.json:1136`). A sink failure carries code `report_sink_failed` and a `reason` +from the closed list of the catalog (`report_sink_failed` at `docker/run_failure_codes.json:1076`): `arena_exhausted`, `arena_io`, `arena_backing`, `scratch_io`, `sink_io`, `sink_peer_closed` and `sink_exhausted`. Only some reasons carry sizes: `arena_exhausted` carries `high_water_bytes`, `arena_bytes`, `claimed_bytes`, `store` and `needed_bytes_lower_bound`, `sink_exhausted` carries From 71dddc88d6e8aa918baef5811f8c82d7e404225a Mon Sep 17 00:00:00 2001 From: luisleo526 Date: Sat, 7 Nov 2026 23:43:04 +0800 Subject: [PATCH 19/19] docs: clarify baseline promotion status requirement --- docs/ci.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/ci.md b/docs/ci.md index 38fe5adfa..fff91b0f9 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -1349,8 +1349,8 @@ tooling) runs `main`'s own copy on a `push` to `main` and runs no PR code. It resolves the pushed SHA through the commit's associated PRs, requiring exactly one merged PR whose `merge_commit_sha` is that SHA and whose base is `main`; no match or ambiguity exits green with a notice. Manual dispatch -retains its five merge inputs. It requires both statuses to be successful on -the resolved PR head and a merge commit on `main` that carries +retains its five merge inputs. It requires `pineforge/parity` success on +the resolved PR head (`pineforge/verify` is advisory) and a merge commit on `main` that carries exactly that head's tree (PRs are squash-merged, so the head itself never lands on the base branch). It hands the campaign tool the merge commit, the PR head and that tree, and the tool promotes the pair the merge gate's parity verdict measured