Method — activity backtest

Does weather move the harvest?

Other apps put a “movement” percentage on every hour of every day, with no published accuracy. We won’t make an animal-behaviour claim we can’t check — so before any activity feature ships, the check itself ships. This page is it: 302,011 dated kills across 25 Alaska seasons, tested day by day against the weather, with the method, the effect sizes, the confounds and the failures all stated. It will be updated before any claim built on it changes.

The ground truth

Alaska is the only state that publishes true per-kill dates at scale: the Department of Fish and Game’s per-hunter harvest records carry a date of kill for 98.7% of reported harvests. We ingested regulatory years 2000–2024 for moose and caribou — 1,437 season-windows with at least 50 kills each — and joined each hunt unit to its nearest NOAA weather station within 150 km (52 units mapped; 6 were farther than that and honestly get no weather rather than a 300 km extrapolation). Kill dates are ground truth for harvest timing — which is hunter effort and animal availability together. Everything below is designed around that confound, not in denial of it.

The method, plainly

The naive analysis — “kills peak at dawn and in September” — proves only that hunters hunt mornings and Septembers. So the question we actually test is narrower: within one season, in one unit, do the days with a factor (rising pressure, wind, cold) carry more of that season’s kills than their position in the season predicts?

• Every in-season day counts, including the zero-kill days — forgetting them makes every factor look active.
• The baseline is the effort curve: the pooled day-of-season kill share for that species and hunt type. Deviations from it, not raw counts, are the signal.
• Season openers (first two days) and weekends are excluded — they are effort anomalies, not weather.
• Each season splits at its own median factor value, so a warm year or a stormy unit can’t masquerade as day-scale signal.
• Confidence intervals come from resampling whole seasons (4000 draws, seeded and reproducible) — never single days, which are dependent.

Results

The number reported is the extra share of a season’s kills landing on factor-high days, beyond the effort curve’s prediction, in percentage points. Bold rows have 95% intervals excluding zero; sixteen intervals were computed, so expect one or two to clear that bar by chance — the filter that matters is sign stability across the two regimes (fall seasons vs the January–March winter hunts, which are different weather worlds).

What survives, and what does not

• Rising pressure is real for moose, in both regimes. Post-frontal days carry one to two percentage points more of a season’s kills than their calendar position predicts. Small — and stable, which is what earns it a place.
• Wind is negative for moose in both regimes — but windy days also make hunters less effective (less glassing, harder shots), and day-scale data cannot separate suppressed movement from suppressed hunting. It is reported as a harvest effect, never as an animal claim.
• Precipitation fails. Its sign flips between regimes, so it does not meet the stability bar and will not ship as a factor.
• Caribou are not moose. Pressure shows nothing in either caribou regime; colder-than-normal winter days show more caribou kills. A model fitted on one species will not be presented as another’s.

The held-out gate

Publishing effect sizes is not the same as earning a forecast. Before any fused activity number can appear in the product, our design rule requires it to prove skill on data it has never seen: the model is trained with an entire region and an entire five-year era excluded, then scored only on that excluded region’s excluded years — repeated across every region × era cell, so every season is scored exactly once by a model that never saw its region or its era. Regions are the Department of Fish and Game’s own five administrative regions, not a partition we invented. Four criteria, all required: held-out skill above the effort baseline with an interval excluding zero; the factor’s sign holding in every region; no single region carrying more than half the skill; and no single factor’s removal flipping the result.

Leaving Alaska: the Idaho replication

A claim that only works in one state is a curiosity, not a mechanism — so the pressure finding gets retested where the data grain is completely different. Idaho publishes 95 units of general-season deer harvest with hunter-days — a true effort denominator Alaska never had — and its season windows come verbatim from the Seasons & Rules brochures. The test: within one unit and weapon, do the years whose season windows held more rising-pressure days show more harvest per hunter-day? Confidence resamples whole years, because weather within a year is correlated statewide — which is exactly why a short span of window vintages cannot fake significance.

• outcome is harvest per hunter-day: gross effort is in the denominator, but a residual effort response to weather remains possible and is named, as in Alaska.
• windows are the regulation union per take method (short-range extends all; any-weapon extends archery and muzzleloader).
• confidence resamples YEARS (weather is statewide-correlated within a year); series are never treated as independent draws.
• only 9 window years ingested so far — directional evidence, not gate input.

Named limitations

• Day-scale only. No public dataset gives hourly kill times at scale, so nothing here validates an hour-by-hour curve. Any within-day shape we ever show is a labeled crepuscular prior, not a backtested claim.
• Harvest is not movement. Every effect above is a harvest-timing effect. The design excludes the grossest effort artifacts (openers, weekends, season shape), but a residual effort response to weather cannot be ruled out at day scale.
• Alaska first. Moose and caribou, 25 years, one state. Lower-48 replication (Montana check-station weeks, Colorado season-window contrasts) is the next test, and a whitetail claim will not be borrowed from an elk state.
• Station weather is local. Units join their nearest station (median distances are in the shipped data); mountain weather varies within a unit. Within-season contrasts cancel siting bias, not distance noise.
• Reporting density changed. Electronic reporting grew the record count several-fold over the period; within-season contrasts are unaffected, but cross-era volume comparisons would be meaningless and none are made.

What this means in the product

The conditions score remains a statement about your situation — wind, scent, light. The gate above defines exactly what an activity surface has earned the right to say: for moose in Alaska, at day scale, a pressure-trend signal phrased as “conditions historically associated with more movement recorded” — nothing hourly, nothing for caribou, nothing outside Alaska until lower-48 replication passes the same gate, and always with this backtest one tap away.

That surface now exists, exactly that narrow: Alaska unit pages carry a day-scale pressure band for moose. Its construction, so you can check it: the number is today’s day-over-day change in daily-mean sea-level pressure at the same station this backtest joined to that unit, computed from live observations over paired clock hours (a running day compared hour-for-hour against yesterday, so the diurnal cycle cannot read as a front). The backtest’s within-window median split has no live equivalent, so the band classifies today against the station’s own 25-year seasonal distribution of the identical quantity — terciles per fall/winter regime, built from the committed daily interchange (scripts/build-activity-band.py). Too few paired hours and the band simply does not render. If a regenerated backtest ever fails the moose gate, the band artifact empties and the surface disappears — the component reads verdicts, not prose. The raw interchange and the harness are in the open repository (data/activity/, crates/pipeline/src/activity.rs); methodology questions and challenges are welcome — the draw odds backtests set the house standard this page follows.

Revision — v2: the shared median split silently dropped windows whose factor median tied the lower of two values, so v1 precipitation figures rest on fewer windows and do not reproduce — they were wrong, not merely superseded. The precipitation sign-flip conclusion is unchanged; every other v1 number is identical.

Backtest generated 2026-08-17T12:15:21Z · seed 20260817 · 4000 resamples · windows require ≥50 kills