Repository navigation
Lyon audit top-up: settle the two 'not decisive' groups (Grand Lyon Ladybug, other cameras) #278
Description
Follow-up to the Lyon label audit (#159, results in PR #277, comment).
All four groups are eligible on the posted rule: 0.55 precision ≥ 0.80 on the point estimate. Two of them are not decisive, meaning an interval still contains 0.80:
| group | P@0.55 | bootstrap 95% | Wilson 95% | random panos judged | top-up size from score |
run pool left |
|---|---|---|---|---|---|---|
| Grand Lyon Ladybug | 0.83 (84/101) | 0.75-0.90 | 0.75-0.89 | 59 | ~364 random panos (~623 detections) | 17,026 |
| other cameras | 0.87 (40/46) | 0.75-0.96 | 0.74-0.94 | 30 | ~69 random panos (~106 detections) | 1,483 |
Proposal (not decided): draw a top-up of random panos for each group from the same run, using the same sampler and seed scheme, with the 30 m never-train buffer extended to the new panos. Review them with the same rubric and score them as one pool with the original 200.
Considerations:
- Size. The top-up sizes assume the point estimate holds. Grand Lyon's ~364 panos is about twice the original review. Other cameras (~69) is cheap.
- Option: do other cameras only. That settles the cheap one. Grand Lyon could then enter training on the point estimate alone, or not at all, until it is settled.
- Recall needs no top-up. Recall at 0.55 is 0.26 in every group. Lyon's main problem is missing labels, not wrong ones (Lyon audit (#159): Jon's 200-pano review, scores, recall by elevation #277 section 10), and that affects how Lyon enters training whatever this issue decides.
- Process. Post the top-up plan and pins on Add Panoramax as a third imagery domain: Bayonne (first Panoramax PS city) as a benchmark split, Lyon as a training-only city (#158) #159 before reviewing, as was done for the first draw.
🤖 Generated with Claude Code — Opus 5.5, claude-opus-5-5
Activity