Abstract
We consider different types of predictive intervals and ask whether they are elicitable, that is, are unique minimizers of a loss or scoring function in expectation. The equal-tailed interval is elicitable, with a rich class of suitable loss functions, though subject to translation invariance, or positive homogeneity and differentiability, the Winkler interval score becomes a unique choice. The modal interval also is elicitable, with a sole consistent scoring function, up to equivalence. However, the shortest interval fails to be elicitable relative to practically relevant classes of distributions. These results provide guidance in interval forecast evaluation and support recent choices of performance measures in forecast competitions.
Citation
Jonas R. Brehmer. Tilmann Gneiting. "Scoring interval forecasts: Equal-tailed, shortest, and modal interval." Bernoulli 27 (3) 1993 - 2010, August 2021. https://doi.org/10.3150/20-BEJ1298
Information