Benchmark Results — August 2026
Source-quality-weighted sweep results across 250 published benchmarks (276 matched to a catalog bike). Metrics, per-cluster bias, and methodology. Last updated 8 August 2026.
Open the interactive accuracy explorerAggregate Metrics (276 matched benchmarks)
Reference Baselines (hand-calibrated)
Three reference bikes from defaults.py are hand-calibrated and excluded from the ALL_KNOWN_BIKES auto-reseed. They serve as regression anchors — if any code change moves these by more than ±0.02s, something is wrong.
Top 10 Most Accurate Bikes
The 10 smallest residuals across the matched corpus — top-10 mean absolute error ±0.007s. Sources: Cycle World, Sport Rider, MCN, Motorcycle Consumer News. One row (rank 3) is a DERIVED estimate and is flagged as such. Conditions: race fuel, professional rider, ideal strip.
Per-Cluster Bias
Positive bias means the simulator predicts slower than reality. Clusters follow the cylinder-aware 12-cluster taxonomy introduced in the July 2026 retune — twins, singles, and small-cc I4 screamers are split out from the old displacement-only buckets and each cluster is re-centred against the full weighted corpus. A cluster is marked Closed only when its weighted bias is inside ±0.05s and its weighted |Δ| is inside 0.30s — a centred cluster that still scatters widely stays Open. The remaining spread in the small-cc and two-stroke buckets is a known residual from per-bike calibration work not yet completed — it is not a regression.
Methodology
Benchmark sources: Published magazine timeslips from Cycle World, Sport Rider, MCN, Motorcyclist, and BikeWale. Indian community strip records from MotoStatz, Facebook drag groups, and drag event published results. All benchmarks use professional rider / race fuel / ideal strip conditions unless explicitly noted.
Simulation conditions: All simulations run at ISA sea-level conditions (15°C, 101.325 kPa, 0% RH) to match the implicit conditions of most magazine tests. Bikes use their stock BikeConfig from the seeded database — no manual per-bike tuning of Cd or μ.
Matching: Benchmark-to-bike matching uses word-boundary regex to prevent false positives (YZF-R1 ≠ YZF-R15). Year ranges are respected — a 2009 benchmark does not match a 2015 model that shares a name but has different internals.
Known limitations: Residual small-cc and 2T bias is a calibration gap, not a physics gap. The correct fix is per-bike Cd and μ calibration using real strip data — upload a Dragy CSV to /api/calibrate/dragy to overlay your real run against the sim and quantify the gap for your specific bike; the engine's offline calibrator then fits Cd and μ from measured runs.
Regression testing: 10 dedicated regression tests intests/test_power_boost_regression.py run on every commit to catch DB drift. The test helper copies motoquant.db to a temp directory to avoid WAL journal-mode errors on mounted filesystems.