Model & performance

Results from retrospective tests — the live record grows with every season.

Retrospective test of the 2026 season (April–July, 94 days)

The model was trained exclusively on data from 2012–2025 and then tested day by day against the 2026 Torbole observations.

Ora forecasts correct

84 %

only 3 firing days missed

Peler forecasts correct

67 %

most errors were close to the 8 kt threshold

Performance across 14 years (leave-one-year-out validation)

A Peirce Skill Score of 0 means no discrimination; 1 is perfect discrimination. For each test year, the model was trained without that year’s data.

For context, Ora fires on roughly 80% of midsummer days, so a blanket “always GO” forecast would also achieve high accuracy. The skill score therefore measures actual discrimination rather than just the base rate.

Real day-ahead forecasts rather than reanalysis data

Tested on archived previous-day forecasts from 2024–2026 (513 days): switching from reanalysis data to the real forecast chain costs almost no forecast skill. The model’s decision thresholds are calibrated on exactly this data.

Honest limitations

• “Perfect setup, dead lake”
  Around 11 days in the 2026 season looked like textbook Ora days but remained calm. North foehn suppressed Ora on roughly one fifth of them; the rest remain unexplained and are being investigated.
• Borderline Peler strength
  Many Peler “errors” fall between 6.7 and 7.8 kt: the regime was identified correctly, but narrowly missed the 8 kt threshold. That is often still enough for a short session.
• One observation point
  The Torbole (Belvedere) station is the reference. Conditions in Malcesine or at the southern end of the lake may differ.

The evaluation follows the same principles as the Walchensee project (Peirce Skill Score rather than raw accuracy, yearly cross-validation, and calibrated thresholds).