Skip to content

fix(typewriter): pin settlements to the head's timestamp, not a block the watch is catching up on - #145

Closed
kyscott18 wants to merge 1 commit into
mainfrom
fix/pin-head-timestamp
Closed

kyscott18 wants to merge 1 commit into
mainfrom
fix/pin-head-timestamp

Conversation

@kyscott18

Copy link
Copy Markdown
Collaborator

Fixes one of the two causes of the resync loop first reported in #137, which now fails seed 0x02 on main with no extra faults.

The bug:

  1. A settlement must land within MAX_SETTLEMENT_DELAY (60s) of the timestamp it pins. The runtime pinned the timestamp of the block the watch was processing.
  2. After a pause (the sim's 63s clock jumps) or slow providers, that block falls more than 60s behind the head.
  3. Every settlement executed during catch-up, including a resync's replays, is then already too late. It reverts with InvalidSettlementTimestamp, triggers another resync, and the cycle repeats. No error is ever raised.

The fix:

  • The watch already requests latest on every poll, and now carries the newest head's timestamp on each Added message.
  • The runtime pins that timestamp in revm and in settlements.
  • Block numbers stay as processed, so rebroadcast and stuck counting don't change.

Runs:

  • Seed 0x02 stalls on main in this loop, and passes here.
  • Seeds 0x01–0x20 (32 seeds) all pass.
  • Unit tests pass.

Not fixed — a second cause, slow signing:

  • With this patch, 0x600 on test(sim): take each transport down now and then; retry signing a settlement through it #137 (outages) and 0x27 on test(sim): add and remove credentials, and sign with them #143 (credentials) still stall.
  • 0x27's plan with slowRate set to 0 passes, both with and without this patch. So its failure comes from slow requests, not stale pinning.
  • My reading, not yet proven: settlements are signed one at a time, each with its own eth_fillTransaction. When requests take 5s timeouts or outages stall them, a backlog takes more than 60s to sign, so the tail misses the window it pinned at execution time.
  • Fixing that is a design choice:
    • sign settlements locally, fetching nonce and fees once instead of per settlement;
    • or re-execute what can't be sent in time.
  • I'd sign locally. I haven't built either.

🤖 Generated with Claude Code

… the watch is catching up on

A settlement must land within MAX_SETTLEMENT_DELAY (60s) of the
timestamp it pins. The runtime pinned the timestamp of the block the
watch was processing, which falls minutes behind the head after a pause
or slow providers. Then every settlement executed during catch-up,
including a resync's replays, reverted on landing and resynced again,
with no error. The watch now carries the newest head's timestamp with
each added block, and the runtime pins that. Seed 0x02 stalled on main
in this loop and now passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyscott18
kyscott18 marked this pull request as ready for review November 5, 2026 22:37
@kyscott18 kyscott18 closed this Nov 5, 2026
@kyscott18
kyscott18 deleted the fix/pin-head-timestamp branch November 6, 2026 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant