Does projection ordering hold up?
Take two skaters ranked close together by their projected rate, say the 30th and the 45th: did the one ranked higher go on to produce more per game over the rest of the season? This study asks that of every such pair within seven rank regions, from the top 25 down. The order carries information well beyond the top-ranked players, though how much varies by statistic, season and region. Where a region is inconclusive, the data were too thin to tell either way, which is not the same as the order failing.
Points, Goals and Shots on Goal are three separately preregistered surfaces, reported independently. They are not combined, ranked against each other, or summarised into one score. The method and its thresholds were fixed before any of the data was computed.
Statistic
The verdicts at a glance
| Season | 1-25 | 26-50 | 51-100 | 101-200 | 201-350 | 351-550 | 551+ |
|---|---|---|---|---|---|---|---|
| 2021-22 | Inconclusive | Inconclusive | Inconclusive | Ordering evidence | Ordering evidence | No ordering evidence | Ordering evidence |
| 2022-23 | Ordering evidence | Inconclusive | Inconclusive | Ordering evidence | Ordering evidence | Ordering evidence | No ordering evidence |
| 2023-24 | Ordering evidence | Inconclusive | Inconclusive | Ordering evidence | Ordering evidence | Ordering evidence | Evidence, below threshold |
| 2024-25 | Inconclusive | Inconclusive | Inconclusive | Ordering evidence | Ordering evidence | Ordering evidence | No ordering evidence |
| 2025-26 | Ordering evidence | Ordering evidence | Inconclusive | Ordering evidence | Ordering evidence | Ordering evidence | Ordering evidence |
- Ordering evidence
- Evidence, below threshold
- Inverted ordering
- Inverted, below threshold
- No ordering evidence
- Inconclusive — the design could not have told
Full statistical results
Every cell above, with the figures behind it: s, its 95% interval, and whether the region could have resolved the preregistered effect at all. How to read a cell is below the table.
| Season | Ranks 1-25 | Ranks 26-50 | Ranks 51-100 | Ranks 101-200 | Ranks 201-350 | Ranks 351-550 | Ranks 551+ |
|---|---|---|---|---|---|---|---|
| 2021-22 | Inconclusive s 0.498 95% interval 0.410–0.586 ◇ not resolvable | Inconclusive s 0.560 95% interval 0.453–0.662 ◇ not resolvable | Inconclusive s 0.557 95% interval 0.497–0.614 ◇ not resolvable | Ordering evidence s 0.587 95% interval 0.544–0.633 ◆ resolvable | Ordering evidence s 0.598 95% interval 0.561–0.635 ◆ resolvable | No ordering evidence s 0.528 95% interval 0.499–0.558 ◆ resolvable | Ordering evidence s 0.587 95% interval 0.550–0.623 ◆ resolvable |
| 2022-23 | Ordering evidence s 0.684 95% interval 0.572–0.787 ◇ not resolvable | Inconclusive s 0.560 95% interval 0.452–0.669 ◇ not resolvable | Inconclusive s 0.513 95% interval 0.448–0.575 ◇ not resolvable | Ordering evidence s 0.627 95% interval 0.590–0.663 ◇ not resolvable | Ordering evidence s 0.572 95% interval 0.535–0.612 ◆ resolvable | Ordering evidence s 0.577 95% interval 0.547–0.607 ◆ resolvable | No ordering evidence s 0.526 95% interval 0.480–0.570 ◆ resolvable |
| 2023-24 | Ordering evidence s 0.727 95% interval 0.629–0.809 ◇ not resolvable | Inconclusive s 0.480 95% interval 0.401–0.566 ◇ not resolvable | Inconclusive s 0.565 95% interval 0.506–0.624 ◇ not resolvable | Ordering evidence s 0.592 95% interval 0.551–0.633 ◆ resolvable | Ordering evidence s 0.594 95% interval 0.556–0.630 ◆ resolvable | Ordering evidence s 0.596 95% interval 0.567–0.624 ◆ resolvable | Evidence, below threshold s 0.550 95% interval 0.510–0.589 ◆ resolvable |
| 2024-25 | Inconclusive s 0.589 95% interval 0.446–0.703 ◇ not resolvable | Inconclusive s 0.545 95% interval 0.444–0.646 ◇ not resolvable | Inconclusive s 0.551 95% interval 0.496–0.605 ◇ not resolvable | Ordering evidence s 0.594 95% interval 0.550–0.636 ◆ resolvable | Ordering evidence s 0.595 95% interval 0.565–0.625 ◆ resolvable | Ordering evidence s 0.584 95% interval 0.554–0.615 ◆ resolvable | No ordering evidence s 0.534 95% interval 0.491–0.578 ◆ resolvable |
| 2025-26 | Ordering evidence s 0.695 95% interval 0.594–0.781 ◇ not resolvable | Ordering evidence s 0.615 95% interval 0.528–0.698 ◇ not resolvable | Inconclusive s 0.544 95% interval 0.473–0.616 ◇ not resolvable | Ordering evidence s 0.605 95% interval 0.564–0.644 ◆ resolvable | Ordering evidence s 0.589 95% interval 0.555–0.624 ◆ resolvable | Ordering evidence s 0.608 95% interval 0.581–0.636 ◆ resolvable | Ordering evidence s 0.581 95% interval 0.543–0.619 ◆ resolvable |
How to read a cell
- s
- The share of correctly ordered player pairs, aggregated equally over three in-season cutoffs. 0.5 is the value a shuffled board would produce.
- 95% interval
- Uncertainty around the observed s. It says how precisely s is estimated — it is not a significance test and played no part in the verdict.
- The verdict
- Comes from comparing s against that region’s own shuffled-board range — a separate quantity from the interval above, answering is this distinguishable from no ordering at all?
- ◇ not resolvable, and why “inconclusive” is not a negative
- Some regions carry too few comparisons for the method to detect an effect of the size declared in advance, even if one existed. There, a non-finding is reported as inconclusive — meaning the design could not have told. It does not mean the ordering failed.
Where these figures come from — code version 44ddb299004f
Five seasons, three statistics and seven rank regions, all from one run of the study at this code version with its random draws fixed, so running it again at the same version gives the same verdicts.