Repository navigation
Crosswalk detector via the open-data bootstrap: Phase 0 census, offset read and Vistas probe, then a Stage 1/2 type axis #274
Description
Umbrella: ProjectSidewalk/Planning#36. A candidate first label type beyond curb ramps (the order is Decision 1 there). Nothing trains until Phase 0 reports.
Why crosswalks
The validator's easiest type (AUC 0.972 held out; 0.941 on unseen deployments, n=138, sidewalk-validator-ai/README.md), and the richest supervision we can get without a labeling campaign: OSM highway=crossing nodes with crossing:markings, municipal marking inventories (often lines, which give both curb ends), 65k crowd Crosswalk labels (7.3k validated correct), the Vistas classes Crosswalk - Plain (8) and Lane Marking - Crosswalk (23) in facebook/mask2former-swin-large-mapillary-vistas-semantic (zero training; the #126 checkpoint), and aerial detection where ≤10 cm orthoimagery exists. A crossing's ends are also per-direction NoCurbRamp candidates (umbrella), which the corner route cannot produce.
Target convention. The Crosswalk label goes at the centre of the marked extent (Jon's convention; the labeling guide states none). The record keeps the full crossing line. An OSM crossing node sits on the road centreline, which is approximately the midpoint of a symmetric marked crossing; dual carriageways (two nodes) and crossings tagged on the junction node are the known exceptions, measured in Phase 0.
The bootstrap, stated as RampNet does it. Stage 1 consumes an inventory coordinate for its bearing from the pano (angular tolerance ±18.37°) and a crop model places the feature; range is discarded (docs/rampnet1_report.md §5.1). The crosswalk analogue: bearing from the midpoint, a crosswalk crop model trained on crowd Crosswalk crops (pano-grouped split; crowd clicks are level-frame, so the β ≈ 0.95 rig correction from Planning#34 applies when the model runs on rig-frame panos), extent from the Vistas mask. A projection-first arm (flat raycast of the midpoint, no crop model) is piloted in the umbrella's Phase 0 item 5 as a different mechanism.
Phase 0 (CPU; one afternoon of GPU for item 3)
- Census. Per PS city: validated Crosswalk labels; OSM crossing nodes and the share with
crossing:markings/crossing=marked; marking inventories in NYC, Portland, Bend and Seattle (endpoint recorded per row, asdocs/curb_ramp_data_sourcing.mddoes). Location precision rated on an aerial basemap, Table 1 style, before any city enters training. Known: OSM crossing coverage is uneven (sidewalk-validator-ai/docs/road-prior-2026-10.md: a Seattle-heavy proxy); Bend paints few crosswalks (Sidewalk location from aerial imagery (Tile2Net) projected into panos: a world-space surface layer for #47 and anchors for clustering sidewalk-auto-labeler#104). - Crowd-vs-OSM offset. The server's stored lat/lng for crowd Crosswalk labels joined to the nearest OSM crossing (no re-raycast): along- and across-crossing offset distributions. An aerial reference in one city separates OSM error from the crowd's placement. Output: the crowd's convention (centre vs end) and the matcher radius for item 3.
- Vistas probe. Classes 8 and 23 over the benchmark panos of
richmondandlaurens_mapillary/laurens_gsv(the splits whose cities have crowd Crosswalk labels) at the 1024 parity rig. Component rule, pinned now: union the two classes, take connected components, merge components whose centroids are within 0.03 of the pano width at the same row band, one point per merged component. Denominator: benchmark panos carrying ≥1 crowd Crosswalk label within the pano (count reported). Reading rule, stated now and independent of item 2: merged-component recall ≥ 0.8 of those crowd labels at a radius of 0.05 (wider than the ramp radius because a crossing spans tens of degrees) makes the mask a usable Stage 1 extent filter; below that, Stage 1 uses the crop model alone.
Phase 1 (after Phase 0 and the umbrella's Decision 1, gated on a decision comment here)
stage_one/gains alabel_typeaxis: a crossing source reader (points, lines, OSM nodes), the bearing rule, the crop model, the Vistas extent filter, negatives as today. No code import from the labeler; shared geometry is duplicated with a parity test until the imagery library (Planning#6) exists.stage_two/: a second heatmap channel on the released backbone (the RampNet 2.0: measurement, condition inference, and tagging — we already collect the supervision and discard it at ingest #86 item 6 shape), masked loss where supervision is sparse (RampNet 2.0: open-rig detector campaign, phase 0 (miss buckets, Lyon review, training plumbing) #263 Track 3), 3 seeds, read paired against the RampNet 2.0: open-rig detector campaign, phase 0 (miss buckets, Lyon review, training plumbing) #263 control arm.- GT. Per city, the orthoimagery source and GSD are named first (umbrella Phase 0 item 3). With ≤10 cm imagery: every marked crossing at every sampled intersection marked top-down, projected into the benchmark panos. Otherwise: a per-pano review of ~125 panos per city, as the ramp splits do. Either way the anchoring is measured against an un-anchored sample.
benchmark/<city>/verdicts.jsongains a type. - Scoreboard and HF release carry the type; the curb-ramp numbers are untouched.
Caveats carried from the ramp work
Open-data location precision is the binding constraint (Table 1: 3 of 8 cities passed). Vistas class semantics must be audited for these two classes, not assumed from the curb-cut result. Crowd labels are positives only, never recall GT. The crosswalk validator AUC measures crowd mistakes, not detector false positives (stop lines, bike boxes); #46-style FP taxonomy comes with the first scored run.
Refs #86, #126, #85, #263, #150.
🤖 Generated with Claude Code — Fable 5.1, claude-fable-5-1
Activity
jonfroehlich commentedon Oct 29, 2026
Phase 0 item 1: census of crowd Crosswalk / Signal labels vs OSM crossing and signal nodes (all 46 public cities)
Method. For every public city on
/v3/api/cities: all crowdCrosswalkandSignallabels fromrawLabels(every user, the server's stored lat/lng, no re-raycast); OSMhighway=crossingnodes (Signal:highway=traffic_signalsorcrossing=traffic_signals) pulled from Overpass over the 0.1° tiles that hold ≥ 1 label; haversine distance from each label to the nearest node.marked=crossing:markings≠no, orcrossing∈ {marked, zebra, lines, ladder, dashes, dots, surface};unmarked=crossing:markings=noorcrossing∈ {unmarked, no, informal}. "human ok/wrong" is a simplified margin rule on human votes only (net ≥ 2 Agree / net ≥ 2 Disagree); the AI validator's votes are ignored. Stdlib script, every pull cached; it goes in with the Phase 0 PR. Pulled 2026-10-28.Per-city table
Sorted by label count. "marked" and "tagged either way" are shares of the OSM nodes in the tiles (which cover more than the city). Distances are label → nearest node.
Total Crosswalk labels: 68014; Signal: 15309
Reading 1: OSM crossing coverage across our cities is bimodal, and that, not placement, drives the distances
The median label-to-node distance is either 2–5 m or 100–500 m, with almost nothing between. Counting OSM crossing nodes inside each city's own label footprint (1 km cells that hold ≥ 1 label) separates the two groups almost perfectly: above ~40 nodes per cell the crowd's labels sit on OSM nodes; below ~30 they do not, because there are no nodes to sit on. The far group is not crowd error: Teaneck (1,595 human-confirmed labels, 24 nodes/cell), Paterson, CDMX, Mendota, Oradell and Maywood are all heavily validated. Two cities have nodes but not where the crowd labels are, which is worth a look on an aerial basemap before trusting either side: Cuenca (86 nodes/cell, median 75 m) and Newberg (58/cell, 98 m; Bend-like, few painted crossings).
Reading 2: where a node exists, it is where the crowd puts the label, which supports the midpoint convention
In the 17 cities with ≥ 100 labels and a median ≤ 5 m, the median is 1.5–4.5 m, which is inside the server's own placement error for a label raycast from a pano. Restricted to human-confirmed labels in those cities (8,086 labels, Chicago 3,907 of them):
So the OSM crossing node is a usable Stage 1 bearing source (the ±18° tolerance is far wider than 2–5 m at any range we use) in the dense group, and the along/across decomposition of item 2 can be read from these same joins. St. Louis is the outlier to check (confirmed labels agree with nodes less often than unconfirmed ones).
Reading 3: the
crossing:markingstag disagrees with the crowd often enough that it cannot be the positive set on its ownPooled over confirmed labels with a node within 10 m, 36% sit on a node OSM tags unmarked or leaves untagged. Chicago drives that (56% of its 248k crossing nodes carry no markings tag at all; only 37% of its labels are within 10 m of a marked node); without Chicago it is 1 in 9, but in Columbus, Pittsburgh, Waltham and Detroit it is 1 in 2 or worse, and Richmond's 69 labels are all within 10 m of a node but only 19% within 10 m of a marked one. Either the tags are stale (a crossing painted since the node was tagged) or the crowd labels unmarked crossings as
Crosswalk. That is exactly what the one-city aerial reference in item 2 has to settle before any OSM-marked node is used as a training positive; until then, positives come from crowd-confirmed labels, and OSM nodes supply bearing and candidate crossings only.Reading 4: Signal labels sit ~9–12 m from OSM signal nodes
Across the cities with ≥ 100 Signal labels the median is 7–15 m and only 27–64% are within 10 m, against 2–4 m for crossings. An OSM signal node is tagged on the intersection node (or the stop line), while the PS label is on the pole at the corner, so a signal bearing rule needs a corner offset rather than the node itself. Parked until crosswalks are done; recorded here so the number is not re-derived.
What this settles for Phase 0
richmondand thelaurenssplits as pinned, with the caveat that both are small (69 and 20 crowd labels).🤖 Generated with Claude Code — Fable 5.1, claude-fable-5-1
jonfroehlich commentedon Oct 29, 2026
Phase 0 item 2: crowd-vs-OSM offset read, 17 dense cities
Method. In each of the 17 cities from the census with ≥ 100 Crosswalk labels and a median ≤ 5 m to the nearest OSM crossing node: every crowd label (server lat/lng, no re-raycast) is joined to its nearest
highway=crossingnode within 10 m; a seeded sample of up to 1,500 matched labels per city (all of them in Seattle); the ways through the sampled nodes are pulled from Overpass with geometry. The crossing axis is thefootway=crossing(or cycleway/path=crossing) way through the node when one exists, else the normal to the road way at the node. The offset vector label − node is projected on that axis (along the crossing, i.e. toward one kerb or the other) and on the road axis (across, i.e. up or down the street). Under a centre convention the along-crossing offset is small and symmetric about zero; under an end convention it has a mode near half the carriageway width (3–6 m) with either sign. Same script family as item 1; cached pulls; CSV per city withlabel_uid, node id, axis source, both components, the node's marking tag and the human verdict.Per-city table
| city | labels | matched <=10 m | sampled | axis from footway | median |along| m | p90 |along| | median |across| m | p90 |across| | |along| > 3 m | |across| > 3 m |
|---|---|---|---|---|---|---|---|---|---|---|
| chicago-il | 28091 | 25938 | 1500 | 98% | 0.87 | 3.07 | 0.69 | 2.19 | 10% | 6% |
| seattle-wa | 6283 | 6034 | 6034 | 99% | 1.3 | 4.3 | 1.04 | 3.3 | 21% | 12% |
| amsterdam | 2976 | 2571 | 1126 | 17% | 0.94 | 3.39 | 1.12 | 3.05 | 13% | 10% |
| st-louis-mo | 2890 | 2018 | 1500 | 98% | 1.47 | 4.75 | 1.28 | 4.06 | 23% | 17% |
| gainesville-fl | 2420 | 2227 | 1497 | 92% | 0.95 | 4.14 | 0.83 | 2.21 | 18% | 4% |
| burnaby | 1728 | 1693 | 1496 | 100% | 1.34 | 4.92 | 0.96 | 3.12 | 27% | 11% |
| pittsburgh-pa | 1018 | 819 | 819 | 97% | 1.08 | 3.99 | 0.9 | 3.11 | 17% | 10% |
| columbus-oh | 889 | 739 | 739 | 71% | 1.05 | 3.56 | 1.27 | 3.41 | 12% | 14% |
| tucson-az | 691 | 510 | 510 | 99% | 1.18 | 4.23 | 1.21 | 3.88 | 17% | 16% |
| danville-il | 488 | 419 | 419 | 99% | 1.45 | 5.34 | 1.41 | 3.24 | 26% | 13% |
| detroit-mi | 283 | 212 | 212 | 33% | 0.9 | 2.84 | 0.71 | 2.24 | 8% | 7% |
| waltham-ma | 281 | 251 | 251 | 96% | 0.99 | 3.61 | 0.76 | 2.19 | 15% | 5% |
| auckland | 273 | 254 | 252 | 97% | 1.61 | 4.61 | 1.11 | 3.71 | 24% | 13% |
| rancagua-chile | 148 | 112 | 111 | 12% | 1.37 | 5.3 | 2.58 | 5.69 | 22% | 42% |
| madison-wi | 137 | 107 | 107 | 99% | 1.6 | 5.06 | 0.89 | 2.95 | 25% | 8% |
| santiago-chile | 128 | 109 | 109 | 18% | 1.53 | 5.38 | 2.49 | 5.84 | 32% | 40% |
| sao-paulo-brazil | 105 | 80 | 80 | 56% | 0.94 | 3.3 | 2.5 | 6.72 | 15% | 41% |
Pooled cuts
Reading 1: the crowd labels the centre, and the OSM node is that centre
Pooled over 16,762 sampled labels the median offset is 1.17 m along the crossing and 1.00 m across it; the signed along-crossing median is -0.06 m, so the labels scatter symmetrically about the node rather than piling up at a kerb, and only 19% are more than 3 m from it along the crossing. There is no half-carriageway mode anywhere in the 17 cities (per-city medians 0.87–1.61 m along). Human-confirmed labels (3,512) read the same (1.18 m along, 1.03 m across), so this is not an artefact of unvalidated labels. These numbers are at or under the server's own placement error for a label raycast from a pano, which means the OSM node is, to the precision we can measure here, the midpoint the labeling convention asks for (the umbrella's Decision on the target convention stands: centre of the marked extent).
Reading 2: the caveats from the review are small or confined
oneway=yesroad (5,076) read 1.25 m along / 1.05 m across, against 1.17 / 1.00 overall: no separate mode. Each carriageway of a dual road tends to carry its own crossing node, and the crowd labels each half.unmarked(1,182) sit exactly like labels onmarkednodes (1.12 vs 1.22 m along). The crowd places them the same way; whether those are stale tags or unmarked crossings the crowd callsCrosswalkis the question only the aerial sheet answers (below).What this sets for item 3 and for Stage 1
crossing:markingscan be a positive source at all. It needs a rater and an imagery source, so it is the one item in Phase 0 that waits on a person.🤖 Generated with Claude Code — Fable 5.1, claude-fable-5-1
jonfroehlich commentedon Oct 29, 2026
Phase 0 item 2 addendum: the Seattle aerial sheet, rated (AI first pass)
The 120-label sheet (40 human-confirmed Seattle Crosswalk labels each on OSM nodes tagged marked / unmarked / untagged) was rated on King County's 2023 3-inch orthoimagery (36 m crops centred on the node, from the county's public map service) by Claude as a first pass; a person should spot-check it before it is leaned on. Two questions per row: is there a painted crossing at the node, and where is the label relative to the crossing. The population is nodes that already have a confirmed crowd label within 10 m, so this is the tag's precision on crossings the crowd found, not OSM's recall.
crossing=unmarked+crossing:markings=no)unmarked/crossing:markings=nois not a negative set, and excluding those nodes would cost recall. Item 1's reading 3 now reads: marked ∪ untagged-with-crowd-label is the positive set in Seattle.The rated sheet is
aerial_sample_seattle-wa.rated_ai.csvin the Phase 0 PR (#275), beside the unrated original; the crops are licensed imagery and are not committed.Phase 0 item 3: Vistas probe, classes 8 + 23 against crowd Crosswalk labels on benchmark panos
Amendment, stated before the numbers. The issue pinned the
richmondandlaurens_mapillary/laurens_gsvsplits. Their denominator is empty: no benchmark pano in any of the three carries a crowd Crosswalk label (Richmond has 70 Crosswalk labels city-wide and none on its 124 benchmark panos; the Laurens PS city has none on its 180). The two benchmark splits whose panos do carry crowd Crosswalk labels are paterson (16 of 125 panos, 44 labels) and gainesville (18 of 125, 23 labels), both GSV, so the probe ran there with the component rule and the reading rule exactly as pinned. The crowd positions are the server'spano_x / pano_width,pano_y / pano_height; the label's recorded pano size matches the benchmark record's on every row.Method.
facebook/mask2former-swin-large-mapillary-vistas-semantic(the #126 checkpoint; class 8 =Crosswalk - Plain, 23 =Lane Marking - Crosswalk, checked against the loadedid2label) on the 1024 parity rig (equirect_tiling.default_views(): 6 yaws, 90°×90°, pitch −30°, 1024×1024 views, source capped at 4096, processor size forced to 1024×1024), fp16 on an RTX 3070. Per view: union of the two classes, 8-connected components, min area 16 px (the #126 cache floor), each component's centroid mapped to pano-normalized coordinates. Across views: components whose pano centroids are within 0.03 of the pano width (RampNet's matcher metric, x wrapped at the seam and scaled by 1024, y by 512) are merged into one point, area-weighted. A label is matched when a merged component lies within 0.05 of the pano width in that metric. One departure fromVistasDetector._raw_detect: the class/mask reduction runs at the logits' 256×256 and the argmax map is upsampled, the memory-light order the #126 note recommends; it differs only at boundary pixels.correct= truecorrect= falseThe census's human-only margin rule confirms none of the 67 (they carry AI-validator votes only), so the server's
correctflag is the only validation cut available, and it is AI-validator dominated.What the mask produces. 6.2 merged components per labeled pano (211 on the 34 panos), of which 65 lie within 0.05 of a crowd label; crowd labels are positives only, so that is not a precision. 24 of the 55 matches merged components from two adjacent views, so the merge step is doing real work on crossings that straddle a view seam. Sensitivity (exploratory, never the rule): dropping components with mean class probability < 0.4 gives 0.81 at 5.4 per pano; < 0.5 gives 0.75 at 4.1; dropping components larger than 5% of a view (the road-surface blobs) gives 0.75. The exploratory pass over every pano of the five splits (labeled or not) reports components per pano by source: Paterson 3.8 per pano (median 2; 27 of 109 panos with none), Gainesville 2.3 (median 2; 35 of 107 with none), downtown Richmond (Mapillary) 5.2 (median 4; 16 of 124 with none), and the two Laurens splits 0.3 (GSV) and 0.2 (Mapillary) with 75 of 86 and 85 of 94 panos carrying no component at all. The rate tracks where painted crossings are (a small SC town paints few; the issue already notes Bend paints few), so the union mask is not firing on every road surface; the labeled-pano rate of 6.2 is the dense-intersection end of that range. 554 panos in all, 2 s each on the 3070.
Reading, under the rule pinned on the issue. 0.82 ≥ 0.8: the mask is a usable Stage 1 extent filter. The bar is cleared by one label in 67 and the interval spans it; Gainesville alone is under it; and at the ramp radius (0.022) recall is 0.58, so the mask locates a crossing to within a few degrees of bearing, not to the label point. That is the role the issue gave it: an extent read on crop-model candidates, not a positive source and not a placer. A confidence or area cut buys fewer components per pano at a recall cost that would fail the rule, so the filter runs on the raw union.
Phase 0 is complete with this. Scripts, tables, the doc (with the per-city aerial-imagery lookup for the Phase 1 GT plan) and figures are in #275. The umbrella's decisions (type order, rating budget, pilot-before-GT, repo split) are what gates Phase 1 now.
🤖 Generated with Claude Code — Fable 5.1, claude-fable-5-1
Aerial spot-check page for the Seattle rating sheet (#274 item 2)
Spot-check page: legend pills, position-only button wording (#274 ite…
Spot-check page: Next-unrated button; Save advances to the next unrat…
Spot-check page: say what each question decides; US spelling (#274 it…
Spot-check page: colored dot in each question header (#274 item 2)
Spot-check page: keep unsaved edits per row across a reload (#274 ite…
Spot-check page: 2023 + 2025 orthos side by side, the crowd's crop, p…
Spot-check page: cap panel height so the verdict card stays near the …
Spot-check page: "label is on a bike crossing" tag (B), counted by me…
Spot-check page: option 0 reads 'no paint to judge against', not 'no …
Seattle aerial spot-check: Jon's 120 verdicts (first pass), archive +…
Spot-check page: re-examine list (rule changes) shown per row and as …
Spot-check page: opens on the re-examine list; a re-saved row leaves …
Point code and docs at the spot-check replay archive on the lab share (…
Spot-check page: a clicked button no longer re-fires on Enter/Space (#…
Seattle aerial spot-check: Jon's final 120 verdicts after re-examinat…
Phase 0 review fixes, code and data (#274, PR #275 review)
Phase 0 doc: human rating headlines the aerial section; numbers corre…
jonfroehlich commentedon Nov 1, 2026
Phase 0 done; Phase 1 opens
Phase 0 items 1–3 and the Seattle aerial spot-check are on #275 (
docs/crosswalk_phase0_274.md). The human rating of the 120-row Seattle sheet (40 per OSM class, nodes that already carry a human-confirmed crowd label) confirms the first pass:markedunmarkedThe crowd's label sits on the crossing in 105 of 120 rows. So, for Stage 1: OSM
markednodes and untagged nodes with a confirmed crowd label are a usable positive set;unmarkedis not a negative set; the crowd labels the centre, as item 2 found. This is the tag's precision on crossings the crowd found in one city, not OSM's recall.Decision 1 on the umbrella plan is taken: the crosswalk detector comes first. Phase 1 opens in this order:
stage_onelabel_typeaxis: crossing source reader (OSM nodes, inventories), the bearing rule, candidate crossings, with a parity test on the shared geometry.GT is scored before any label goes to a server, test included.
🤖 Generated with Claude Code — Fable 5.1, claude-fable-5-1
Orthoimagery survey for the 17 dense-group cities; GT-city shortlist (#…
docs index: rows for the crosswalk Phase 0 doc and the orthoimagery s…
jonfroehlich commentedon Nov 1, 2026
Phase 1 item 1: orthoimagery survey, 17 dense-group cities
Desk survey of 2026-11-01 (every ArcGIS endpoint read via
?f=pjson, no imagery fetched; URLs and UNVERIFIED markers indocs/crosswalk_orthoimagery_survey.mdon #275; condensed table and reading indocs/crosswalk_phase0_274.md).10 of the 16 non-Seattle cities have a verified public source at ≤ 10 cm. Open licence and an open streaming service: Amsterdam (PDOK, 8 cm, CC BY 4.0, WMTS), Auckland (LINZ, 7.5 cm, CC BY 4.0, public S3 COGs), São Paulo (GeoSampa, 10 cm, CC BY-SA 4.0, WMS). Open ArcGIS ImageServers with disclaimer-only terms: Tucson (5.1 cm, 2022), Columbus (7.6 cm, 2025), Madison (7.6 cm, 2024 / 2026). Bulk only: Burnaby (7.5 cm, Open Government Licence, 4 GB MrSID), Rancagua (drone ~7 cm, no service). Pittsburgh: a 2017 partial 3-inch only. At 15 cm: Chicago, St. Louis, Gainesville, Danville, Detroit, Waltham. Santiago: 2012, 16 cm.
GT-round shortlist (the aerial + crop page as built for Seattle, several hundred rows): Amsterdam (the non-US check; the city where OSM and crowd agree most), Columbus (half its confirmed labels sit on untagged or
unmarkednodes, so the Seattle reading gets a second city at 7.6 cm), a second US city from Tucson / Madison once their terms are confirmed, and a 40-row São Paulo probe for the junction-tagged case. Chicago (28k labels) gets a 40-row test on the 15 cm Cook County service first: if presence of paint reads as well at 15 cm as at 7.5 cm, the 10 cm bar is for marking extents, not presence, and four more cities open up.Next: the multi-city GT page and the seeded draw (item 2 of the gate comment above).
🤖 Generated with Claude Code — Fable 5.1, claude-fable-5-1