Minimum Description Length Revisited

,


Introduction
The Minimum Description Length (MDL) Principle [Rissanen, 1978, 1989, Barron et al., 1998, Grünwald, 2007] is a theory of inductive inference that can be applied to general problems in statistics, machine learning and pattern recognition.Broadly speaking, it states that the best explanation for a given set of data is provided by the shortest description of that data.In 2007, one of us published the book The Minimum Description Length Principle (Grünwald [2007], G07 from now on), giving a detailed account of most work in the MDL area that had been done until then.During the last 10 years, several new practical MDL methods have been designed, and there have been exciting theoretical developments as well.It therefore seemed time to present an up-to-date combined introduction and review.
Why Read this Overview?While the MDL idea has been shown to be very powerful in theory, and there have been a fair number of successful practical implementations, massive deployment has been hindered by two issues: first, in order to apply MDL, one needs to have basic knowledge of both statistics and information theory.To remedy this situation, here we present, for the first time, the MDL Principle without resorting to information theory: all the material can be understood without any knowledge of data compression, which should make it a much easier read for statisticians and machine learning researchers novel to MDL.A second issue is that many classical MDL procedures are either computationally highly intensive (for example, MDL variable selection as in Example 4 below) and hence less suited for our big data age, or they seem to require somewhat arbitrary restrictions of parameter spaces (e.g.NML with v ≡ 1 as in Section 2).Yet, over the last 10 years, there have been exciting developments -some of them very recent -which mostly resolve these issues.Incorporating these developments, MDL can be seen as a powerful extension of both penalized likelihood and Bayesian approaches, in which penalization functions and prior distributions are replaced by more general luckiness functions, average-case methodology is replaced by a more robust worst-case approach, and in which methods classically viewed as highly distinct, such as AIC vs BIC and crossvalidation vs Bayes can, to some extent, be viewed from a unified perspective; as such, this paper should also be of interest to researchers working on the foundations of statistics and machine learning.
History of the Field, Recent Advances and Overview of this Paper MDL was introduced in 1978 by Jorma Rissanen in his paper Modeling by the Shortest Data Description.The paper coined the term MDL and introduced and analyzed the two-part code for parametric models.The two-part code is the simplest instance of a universal code or, equivalently, universal probability distribution, the cornerstone concept of MDL theory.MDL theory was greatly extended in the 1980s, when Rissanen published a sequence of ground-breaking papers at a remarkable pace, several of which introduced new types of universal distributions.It came to full blossom in the 1990s, with further major contributions from, primarily, Jorma Rissanen, Andrew Barron and Bin Yu, culminating in their overview paper [Barron et al., 1998] and the collection Grünwald et al. [2005] with additional chapters by other essential contributors such as Kenji Yamanishi.The book G07 provides a more exhaustive treatment of this early work, including discussion of important precursors/alternatives to MDL such as MML [Wallace and Boulton, 1968], 'ideal', Kolmogorov complexity based MDL [Vitányi and Li, 2000] and Solomonoff's theory of induction [Sterkenburg, 2018].Universal distributions are still central to MDL.We introduce them in a concise yet self-contained way, including substantial underlying motivation, in Section 2, incorporating the extensions to and new insights into these basic building blocks that have been gathered over the last 10 years.These include more general formulations of arguably the most fundamental universal code, the Normalized Likelihood (NML) Distribution, including faster ways to calculate it as well.We devote a separate section to new universal codes, with quite pleasant properties for practical use, most notably the switch distribution (Section 3.1), which can be used for model selection combining almost the best of AIC and BIC; and the RIPr-universal code (Section 3.3) specially geared to hypothesis testing with composite null hypotheses, leading to several advantages over classical Neyman-Pearson tests.In Section 4 we review recent developments on fast calculation of NML-type distributions for model selection for graphical models (Bayesian networks and the like), leading to methods which appear to be more robust in practice than the standard Bayesian ones.Recent extensions of MDL theory and practical implementations to latent variable and irregular models are treated in Section 5.Then, in Section 6 we review developments relating to consistency and convergence properties of MDL methods.First, while originally MDL estimation was for-mulated solely in terms of discretized estimators (reflecting the fact that coding always requires discretization), it has gradually become clear that a much larger class of estimators (including maximum likelihood for 'simple' models, and, in some circumstances, the Lasso -see Example 4) can be viewed from an MDL perspective, and this becomes clearest if one investigates asymptotic convergence theorems relating to MDL.Second, it was found that MDL (and Bayes), without modification, can behave sub-optimally under misspecification, i.e. when all models under consideration are wrong, but some are useful -see Section 6.3.Third, very recently, it was shown how some of the surprising phenomena underlying the deep learning revolution in machine learning can be explained from an MDL-related perspective; we briefly review these developments in Section 6.4.Finally, we note that G07 presented many explicit open problems, most of which have been resolved -we mention throughout the text whenever a new development solved an old open problem, deferring some of the most technical issues to Appendix A.

Notational Preliminaries
We shall mainly be concerned with statistical models (families of probability distributions) of the form M = {p θ : θ ∈ Θ} parameterized by some set Θ which is usually but not always a subset of Euclidean space; and families of models {M γ : γ ∈ Γ}, where each M γ = {p θ : θ ∈ Θ γ } is a statistical model, used to model data z n := (z 1 , . . ., z n ) with each z i ∈ Z, for some outcome space Z.Each p θ represents a probability density function (pdf) or probability mass function, defined on sequences of arbitrary length.With slight abuse of notation we also denote the corresponding probability distribution by p θ (rather than the more common P θ ).In the simple case that the data are i.i.d.according to each p θ under consideration, we have p θ (z n ) = n i=1 p θ (z i ).We denote the ML (maximum likelihood) estimator given model M = {p θ : θ ∈ Θ} by θml , whenever it exists and is unique; the ML estimator relative to model M γ is denoted θml|γ .We shall, purely for simplicity, generally assume its existence and uniqueness, although nearly all results can be generalized to the case where it does not.We use θ to denote more general estimators, and θv to denote what we call the MDL estimator with luckiness function v, see (5).

The Fundamental Concept: Universal Modeling
MDL is best explained by starting with one of its prime applications, model comparisonwe will generalize to prediction and estimation later, in Section 2.3 and 2.4.Assume then that we are given a finite or countably infinite collection of statistical models M 1 , M 2 , . .., each consisting of a set of probability distributions.The fundamental idea of MDL is to associate each M γ with a single distribution pγ , often called a universal distribution relative to M γ .We call the minus-log-likelihood − log pγ (Z n ) the code length of data Z n under universal code pγ .This terminology, and how MDL is related to coding (lossless compression of data), is briefly reviewed in Section 2.3 and Section 2.4; but a crucial observation at this point is that the main MDL ideas can be understood abstractly, without resorting to the code length interpretation.We also equip the model indices Γ := {1, 2, . . ., γ max } (where we allow |Γ| = γ max = ∞) with a distribution, say π; if the number of models to be compared is small (e.g.bounded independently of n or at most a small polynomial in n), we can take π to be uniform distribution -for large (exponential in n) and infinite Γ, see Section 2.3 and Example 4. We then take, as our best explanation of the given data z n , the model M γ minimizing or, equivalently, we maximize π(γ)p γ (z n ); when π is uniform this simply amounts to picking the γ maximizing pγ (z n ).
(1) will later be generalized to π that are not distributions but rather more general 'luckiness functions' -see Section 2.3.

The Bayesian Universal Distribution
The reader may recognize this as being formally equivalent to the standard Bayesian way of model selection, the Bayes factor method [Kass and Raftery, 1995] as long as the γ are defined as Bayesian marginal distributions, i.e. for each γ, we set pγ = p bayes wγ , where for some prior probability density w γ on the parameters in Θ γ , which has to be supplied by the user.When w γ is clear from the context, we shall write pbayes γ rather than p bayes wγ .Using Bayesian marginal distributions pbayes is indeed one possible way to instantiate MDL model selection, but it is not the only way: MDL can also be based on other distributions such as pnml = p nml v (depending on a function v) , ppreq = p preq θ (depending on an estimator θ) and others; in general we add a bar to such distributions if the 'parameter' w, v or θ is clear from the context.Before we continue with these other instantiations of pγ we proceed with an example: Most standard prior distributions one encounters in the literature are beta priors, for which w(θ Note that p bayes w is not itself an element of the Bernoulli model.One could use p bayes w to compare the Bernoulli model, via (1), to, for example, a first order Markov model, with Bayesian marginal likelihoods defined analogously.We shall say a lot more about the choice of prior below.
Example 2 [Gauss and general improper priors] A second example is the Gaussian location family M gauss with fixed variance (say 1), in which Z = R, and p θ (z n ) ∝ exp( n i=1 (z i − θ) 2 /2).A standard prior for such a model is the uniform prior, w(θ) = 1, which is improper (it does not integrate, hence does not define a probability distribution).Improper priors cannot be directly used in (2), and hence they cannot be directly used for model comparison as in (1) either.Still, we can use them in an indirect manner, as long as we are guaranteed that, for all M γ under consideration, after some initial number of m observations, the Bayesian posterior w γ (θ | z m ) is proper.We can then replace We extend all these conditional universal distributions to distributions on Z n by defining p bayes wγ (z 1 , . . ., z n ) := p bayes wγ (z m+1 , . . ., z n | z m )p 0 (z m ) for some distribution p 0 on Z m that is taken to be the same for all models M γ under consideration.We can now use (1) again for model selection based on the p bayes wγ (z 1 , . . ., z n ), where we note that the choice of p 0 plays no role in the minimization, which is equivalent to minimizing − log π(γ) − log p bayes wγ (z m+1 , . . ., z n | z m ).Now comes the crux of the story, which makes MDL, in the end, quite different from Bayes: defining the pγ as in (2) is just one particular way to define an MDL universal distribution -but it is by no means the only one.There are several other ways, and some of them are sometimes preferable to the Bayesian choice.Here we list the most important ones: the NML distribution and 'MDL estimators' depend on a function v : Θ → R + 0 .The definition is then given by which is defined whenever the normalizing integral is finite.The logarithm of this integral is called the model complexity and is thus given by Here the integral is replaced by a sum for discrete data, and max is replaced by sup if necessary.This means that any function v : Θ → R + 0 such that ( 7) is finite is allowed; we call any such v a luckiness function, a terminology we explain later.Note that v is not necessarily a probability density -it does not have to be integrable.For any luckiness function v, we define the MDL estimator based on v as (5) The v-MDL estimator is a penalized ML estimator, which coincides with the Bayes MAP estimator based on prior v whenever v is a probability density.Although this has only become clear gradually over the last 10 years, estimators of form ( 5) are the prime way of using MDL for estimation; there is, however, a second, 'improper' way for estimating distributions within MDL though, see Section 2.4.In practice, we will choose v that are sufficiently smooth so that, if the number of parameters is small relative to n, θv will usually be almost indistinguishable from the ML estimator θml .comp indeed measures something one could call a 'complexity' -this is easiest to see if v ≡ 1, for then, if M contains just a single distribution, we must have comp(M, v) = 0, and the more distributions we add to M, the larger M gets -this is explored further in Section 2.2.Now suppose we have a collection of models M γ indexed by finite Γ and we have specified luckiness functions v γ on Θ γ for each γ ∈ Γ, and we pick a uniform distribution π on Γ.As can be seen from the above, if we base our model choice on NML, we pick the model minimizing over γ, where comp(M γ is given by Thus, by ( 6), MDL incorporates a trade-off between goodness-of-fit and model complexity as measured by comp.Although the n-fold integral inside comp looks daunting, Suzuki and Yamanishi [2018] show that in many cases (e.g.normal, Weibull, Laplace models) it can be evaluated explicitly with appropriate choice of v.
Originally, the NML distribution was defined by Shtarkov [1987] for the special case with v ≡ 1, leading to the rightmost definition in (3), and hence the term NML (in the modern version, perhaps 'normalized penalized ML' would be more apt).This is also the version that Rissanen [1996] advocated as embodying the purest form of the MDL Principle.However, the integral in (3) is ill-defined for just about every parametric model defined on unbounded outcome spaces (such as N, R or R + ), including the simple normal location family.Using nonuniform v allows one to deal with such cases in a principled manner after all, see Section 2.5.For finite outcome spaces though, v ≡ 1 usually 'works', and (3) is well defined, as we illustrate for the Bernoulli model (see Section 4 for more examples): Example 3 [Example 1, Cont.]For the Bernoulli model, θml (z n ) = n 1 /n and comp(M, v) as in ( 7) with v ≡ 1 can be rewritten as log , which, as we shall see in Section 2.2, is within a constant of (1/2) log n.As reviewed in that section, the resulting p nml v is asymptotically (essentially) indistinguishable from p bayes wJ where the latter is equipped with Jeffreys' prior, defined as w J (θ) ∝ |I(θ) = θ −1/2 (1 − θ) −1/2 , I(θ) being the Fisher information at θ. [Rissanen, 1978].Here one first discretizes Θ to some countable subset Θ which one equips with a probability mass function w; in contrast to the v above, this function must sum to 1.One then considers

The two-part (sub-) distribution
which is just a special case of (3).But since we can approximate p nml w by the sub-distribution p 2-p w (z n ) := max θ∈ Θ p θ (z n )w( θ).This 'distribution' adds or integrates to something smaller than 1.This can be incorporated into the general story by imagining that p 2-p w puts its remaining mass on a special outcome, say ' ', which in reality will never occur (while sub-distributions are thus 'allowed', measures that add up to something larger than 1 have no place in MDL).The two-part distribution p 2-p w is historically the oldest universal distribution.The fact that it can be considered a special case of NML has only become fully clear very recently [Grünwald and Mehta, 2019]; in that same paper, an even more general formulation of (3) is given that has all Bayesian, two-part and NML distributions as special cases.Despite its age, the two-part code is still important in practice, as we explain in Section 2.3.[Rissanen, 1984, Dawid, 1984].Here, one first takes any reasonable estimator θ for the given model M. One then defines

The prequential plug-in distribution
where for i.i.d.models, the probability inside the product simplifies to p θ(z i−1 ) (z i ).
For the normal location family, one could simply use the ML estimator: With discrete data though, the ML estimator should be avoided, since then one of the factors in (10) could easily become 0, making the product 0, so that the model for which p preq θ is defined can never 'win' the model selection contest even if most other factors in the product (10) are close to 1. Instead, one can use a slightly 'smoothed' ML estimate (a natural choice for θ is to take an MDL estimator for some v as in ( 5), but this is not required).For example, in the Bernoulli model, one might take θ(z m ) = (m 1 + (1/2))/(m + 1), where m 1 = m i=1 z i .With this particular choice, p preq θ turns out to coincide exactly with p bayes wJ with Jeffreys' prior w J .Such a precise correspondence between ppreq and pbayes is a special property of the Bernoulli and multinomial models though; with other models, the two distributions can usually be made to behave similarly, but not identically.The rationale for using ppreq is described in Section 2.4.In Section 3.2.1 we will say a bit more about hybrids between prequential plug-in and Bayes (the flattened leader distribution) and between prequential and NML (sequential NML).
Except for the just mentioned 'hybrids', these first four universal distributions were all brought into MDL theory by Rissanen; they are extensively treated by Grünwald [2007], who devoted one chapter to each, and to which we refer for details.The following two are much more recent: 5. The switch distribution pswitch [Van Erven et al., 2007].In a particular type of nested model selection, this universal distribution behaves arguably better than the other ones.It will be treated in detail in Section 3.1.
6. Universal distributions pripr based on the Reverse Information Projection (RIPr) These universal distributions [Grünwald et al., 2019] lead to improved error bounds and optional stopping behaviour in hypothesis testing and allow one to forge a connection with group invariant Bayes factor methods; see Section 3.3.

Motivation
We first give a very high-level motivation that avoids direct use of data compression arguments.For readers interested in data compression, Section 2.3 does make a highlevel connection, but for more extensive material we refer to G07.We do, in Section 2.4, give a more detailed motivation in predictive terms, and, in Section 6, we shall review mathematical results indicating that MDL methods are typically consistent and enjoy fast rates of convergence, providing an additional motivation in itself.
Consider then models M γ , where for simplicity we assume discrete data, and let θml|γ be the maximum likelihood estimator within M γ .Define 'the fit of the model to the data' in the standard way, as F γ (z n ) := p θml|γ (z n ) (z n ), the likelihood assigned to the data by the best-fitting distribution within model.Now if we enlarge the model M γ , i.e. by adding several distributions to it, F γ (z n ) can only increase; and if we make M γ big enough such that for each z n , it contains a distribution p with p(z n ) = 1, we can even have F γ (z n ) = 1 on all data.If we simply picked the γ maximizing F γ (z n ) we would be prone to severe overfitting.For example, if models are nested, then, except for very special data, we would automatically pick the largest one.
As we have seen, a central MDL idea is to instead associate each model M γ with a single corresponding distribution pγ , i.e. we set F γ (z n ) := pγ (z n ).Then the total probability mass on all potential outcomes z n cannot be larger than 1, which makes it impossible to assign overly high fit F γ (z n ) to overly many data sequences: no matter what distribution pγ we chose, we must now have z n F γ (z n ) = 1, so a good fit on some z n necessarily implies a worse fit on others, and we will not select a model simply because it accidentally contained some distribution that fitted our data very well -thus, measuring fit by a distribution pγ instead of F γ inherently prevents overfitting.This argument to measure fit relative to a model with a single p is similar to Bayesian Occam's Razor arguments [Rasmussen and Ghahramani, 2000] used to motivate the Bayes factor; the crucial difference is that we do not restrict ourselves to pγ of the form (2); inspecting the "Bayesian" Occam argument, there is, indeed nothing in there which forces us to use distributions of Bayesian form.
The next step is thus to decide which p are best associated with a given M. To this end, we define the fitness ratio for data z n as

7
A C C E P T E D M A N U S C R I P T Accepted manuscript to appear in IJMI where v : Θ → R + 0 is a nonnegative function.To get a feeling for (11), it is best to first focus on the case with v(θ) ≡ 1; it then reduces to We next postulate that a good choice for p relative to the given model is one in which FR(p, n) tends to be as large as possible.The rationale is that, overfitting having already been taken care of by picking some p that is a probability measure (integrates to 1), it makes sense to take a p whose fit to data (as measured in terms of likelihood) is proportional to the fit to data of the best-fitting distribution in M: whenever some distribution in the model M fits the data z n well, the likelihood p(z n ) should be high as well.One way to make 'FR tends to be large' precise is by requiring it to be as large as possible in the worst-case, i.e., we want to pick the p achieving where the maximum is over all probability distributions over samples of length n.It turns out that this maximin problem has a solution if and only if the complexity (4) is finite; and if it is fine, the unique solution is given by setting p = pnml , with pnml given by (3).
The NML distribution thus has a special status as the most robust choice of universal peven though p is itself a probability distribution, it meaningfully assesses fit in the worstcase over all possible distributions, and its interpretation does not require one to assume that the model M is 'true' in any sense.The nicest sub-case is the one with v(θ) ≡ 1, since then all distributions within the model M are treated on exactly the same footing; no data or distribution is intrinsically preferred over any other one.Unfortunately, for most popular models with infinite Z, when taking v(θ) ≡ 1, (13) usually has no solution since the integral p θml(z n ) (z n )dz n diverges for such models, making the complexity (4) infinite.For all sufficiently 'regular' models (curved exponential families, see below), this problem can invariably be solved by restricting Θ to a bounded subset of its own interior -one can show that the complexity (4 is finite with v ≡ 1, and thus (13) has a solution given by (3) if θml is restricted to a suitably bounded set.Yet, restricting Θ to a bounded subset of itself is not satisfactory, since it is unclear where exactly to put the boundaries.
It is more natural to introduce a nonuniform v, which can invariably be chosen so that the complexity (4) is finite and thus (13) has a solution -more on choosing v at the end of Section 2.4.A few remarks concerning this high-level motivation of MDL procedures are in order: 1.It is clear that, by requiring FR to add to 1, we will be less prone to overfitting than by setting it simply to p θml(z n ) (z n ); whether the requirement to add (at most) to 1, making FR essentially a probability density function, is a clever way to avoid overfitting (leading to good results in practice) is not clear yet.For this, we need additional arguments, which we very briefly review.First, the sum-to-1 requirement is the only choice for which the procedure can be interpreted as selecting the model which minimizes code length of the data (the original interpretation of MDL); second, it is the only choice which has a predictive interpretation, which we review in Section 2.4 below; third, it is the only choice under which time-tested Bayesian methods fit into the picture, and fourth, with this choice we get desirable frequentist statistical properties such as consistency and convergence rates, see Section 6.
them so as to be as close as possible to the NML distributions in terms of the fitness ratio they achieve.In the following subsection we show that all six of them satisfy the same MDL/BIC asymptotics, meaning that their fitness ratio is never smaller than a constant factor of the NML one, either again in the worst-case over all z n or in some weaker expectation sense.Thus, they are all 'kind of o.k.' in a rather weak sense, and in practice one would simply revert to the one that is closest to NML and still usable in practice; with the Bayesian pbayes , as we shall see, one can even get arbitrarily close to NML as n gets larger.This classical story notwithstanding, it has become more and more apparent that in practice one sometimes wants or needs properties of model selection methods that are not guaranteed by NMLsuch as near-optimal predictions of future data or strong frequentist Type-I error guarantees.This translates itself into universal codes pswitch and pripr that, for some special sequences achieve much higher fitness ratio than pnml , while for all sequences having only very slightly smaller fitness ratio.This more recent and pragmatic way of MDL is briefly reviewed in Section 3.1 and 3.3.This raises the question how we should define a universal distribution: what choices for pγ are still 'universal' (and define an MDL method) and what choices are not?Informally, every distribution pγ that for no For parametric models such as exponential families, the ' ' is partially formalized by requiring that at the very least, they should satisfy ( 14) below (G07 is much more precise on this).
3. Third, we have not yet said how one should choose the 'luckiness function' v -and one needs to make a choice to apply MDL in practice.The interpretation of v is closely tied to the predictive interpretation of MDL, and hence we postpone this issue to the end of Section 2.4.
4. Fourth, the motivation so far is incomplete -we still need to explain why and how to incorporate the distribution π on model index γ.This is done in Section 2.3 below.

Asymptotic Expansions
Now let p be defined relative to a single parametric model M. It turns out that all universal codes we mentioned have in common that, for 'sufficiently regular' k-dimensional parametric models, the log-likelihood for given data z n satisfies the following celebrated asymptotics, often called the MDL or BIC expansion: for all 'sufficiently regular' data sequences z 1 , z 2 , . .., there exists a constant C ∈ R independent of n such that for all n: For pnml and pbayes , this holds for any choice of luckiness function v and prior w that is continuous and strictly positive on the parameter space Θ.For p2-p w , this holds for clever choices of the discretization Θ and the probability mass function w; for p preq θ , this holds in a weaker expectation sense (see Section 3.2.1),as long as θ is a suitably smoothed version of the ML estimator.Essentially, 'sufficiently regular' parametric models are all exponential families (such as Bernoulli, multinomial, normal, gamma, beta, Poisson,...) and curved exponential families; corresponding results also hold for regression with (generalized) linear models.'Sufficiently regular data' are all sequences for which there is an INECCSI subset Θ 0 of the parameter space Θ such that, for all large n, the ML estimator of the sequence lies within Θ 0 .Here INECCSI stands fo a set whose I nterior is Non-Empty and whose C losure is a Compact Subset of the Interior of Θ. Essentially, this is any bounded subset of the same dimensionality as Θ that does not touch the boundaries of Θ itself; in the Bernoulli example, it would be any set of the form Accepted manuscript to appear in IJMI for > 0. For all universal distributions considered except ppreq , as long as appropriate priors/estimators/luckiness functions are used, (14) will hold uniformly for all sequences in any INECSSI subset Θ 0 , but the constant C may grow larger if we replace Θ 0 by a strictly larger INECCSI subset Θ 0 with Θ 0 Θ 0 Θ. (for ppreq see Section 3.2.1).For the first four universal distributions, the inequality is actually equality up to a constant -( 14) also holds with ≤ replaced by ≥, for a different constant.For the switch distribution pswitch however, the left-hand side will be significantly smaller for a small but important subset of possible data sequences.Finally, since ( 14) thus also holds with p = pnml and ≤ replaced by ≥, exponentiating ( 14), we see that, if one restricts the minimum in ( 13) to all such 'sufficiently regular' z n , FR(p u (z n )) is guaranteed to be within a constant (independent of n) factor of the optimal FR(p nml , z n ), for u ∈ {bayes, 2-p, preq, switch, ripr}.
The NML/comp Expansion and the Jeffreys (Fisher Information) Integral For the case that the model M = {p θ : θ ∈ Θ} is a k-dimensional exponential family and p is the NML or Bayes distribution, we can be significantly more precise and evaluate the constant C in ( 14) up to o(1): we get, under some weak additional regularity conditions on M and v: where k is the dimension of the model, |I(θ)| is the determinant of the k × k Fisher information matrix at parameter θ, the integral is over the parameter space Θ, and the remainder term o(1) vanishes as the sample size grows unbounded.This was first shown (essentially) by Rissanen [1996], for the case that Θ is restricted to an INECCSI subset of the full parameter space (so that pnml with v ≡ 1 is defined), and v ≡ 1.For this uniform v case, Myung et al. [2000] gave a differential geometric interpretation of the Fisher information term, relating it to an intrinsic 'volume' of the parameter space.The general result for nonuniform v, and without INECCSI restrictions, was very recently shown in a breakthrough paper by Suzuki and Yamanishi [2018], solving Open Problem Nr. 6 from G07.
Analogously to pnml (in fact much easier mathematically), we can expand pbayes using a classical Laplace approximation; under the same conditions as before, with now the additional restriction that there exists an arbitrary INECCSI subset Θ 0 of Θ such that for all large n, the data have ML estimator within Θ 0 , we find that 1).( 16) From ( 15) and ( 16) we see that, if the generalized Jeffreys integral v(θ) • |I(θ)|dθ is finite (see Appendix A), then there is a special choice of prior w, the generalized Jeffreys prior , with w 1), but become asymptotically indistinguishable.If v ≡ 1, this w coincides with the well-known Jeffreys' prior w J popular in Bayesian inference; the special case of this prior for the Bernoulli model was encountered in Example 1.Thus, the Bayesian universal distribution with the (generalized) Jeffreys' prior can be a very good alternative of p nml v .

Unifying Model Selection and Estimation
Suppose we are given a countable collection of models {M γ : γ ∈ Γ}.Recall that the basic idea above was to associate each individual model M γ with a single distribution 10

A C C E P T E D M A N U S C R I P T
Accepted manuscript to appear in IJMI pγ .It seems reasonable to do the same at the level of 'meta-parameters' γ: we set M := {p γ : γ ∈ Γ} and in complete analogy to (3), we define the meta-universal distribution for some nonnegative weight function π on Γ.It then makes sense to postulate that the best sub-model M γ for the given data z n is given by the γ achieving the maximum in (17).Note that for determining this maximum, the denominator in ( 17) plays no role.
Let us assume that all pγ have already been defined.Then we can use any π such that the overarching p nml π in ( 17) exists.We can now formulate a general MDL Principle for model selection: we start with a (potentially huge) set of candidate distributions M full .We next carve up M full into interesting sub-models M γ with γ ∈ Γ, so that γ∈Γ M γ = M full .We then associate each M γ with a universal distribution pγ , and we equip M as defined above with luckiness function π (note that M full , a countable union of (usually) uncountable sets, consists of all distributions under consideration, while M is a countable set).We then base the selection of a sub-model M γ on ( 17).What we earlier called the 'general MDL Principle' underneath (1) was the special case in which π(γ) = 1, i.e. π is a probability mass function.Via (9) we see that for any such probability mass function π, the denominator in ( 17) is well-defined, hence π is a valid luckiness function.
Now consider the special case in which every pγ is chosen to be an NML distribution p nml vγ for some luckiness functions v γ .We take some function π : Γ → R + 0 (which we will relate to the π above later on) and we set, for We can use p nml vfull for parameter estimation on the joint parameters (γ, θ) just as we did earlier for parametric models, by using the MDL estimator (γ, θ γ ) vfull picking the (γ, θ γ ) minimizing, over γ ∈ Γ, where again comp(M full , v full ) plays no role in the minimization.This MDL estimator really combines model selection (estimation of γ) and parametric estimation (estimation of θ γ ).If we now define π(γ) := π (γ)/ exp(comp(M γ , v γ )), we find that p nml π defined relative to model M as in ( 17) is equal to p nml vfull defined relative to the full union of models M full , and the γ achieving the maximum in (17) coincides with the γ minimizing (18).This indicates that model selection and estimation is really the same thing with MDL: if we are given a single parametric model M γ with luckiness v γ , we pick the θ minimizing the first two terms in (18) for fixed γ; if we are interested in both θ and γ, we minimize over all terms; and if we are only interested in γ, we pick the γ achieving the maximum in ( 17), which, by construction, will give us the same γ as the joint minimization over (18).Two-Part vs. One-Part Codes: the Role of Data Compression In the oldest [1978] version of MDL, only two-part codes on countable sets were used: the minimum over θ ∈ Θ γ was taken over a discretized grid Θγ and v γ was a probability mass function over this grid; then for all γ, comp(M γ , v γ ) ≤ 0 and comp(M, v) ≤ 0 (see ( 9)) and they were both approximated by 0. From the Kraft inequality [Cover and Thomas, 1991] we see where the name '2-part code' comes from: this inequality says that for every probability mass function π on a countable set A, there exists a lossless code such that for all a ∈ A, the number of bits needed to encode a, is given by − log π(a).Thus the resulting method can be interpreted as picking the ( θ, γ) minimizing the two-stage code length of 11

A C C E P T E D M A N U S C R I P T
Accepted manuscript to appear in IJMI the data, where first the parameters (θ, γ) are encoded using − log π(γ) − log v γ (θ) bits, and then z n is encoded 'with the help of γ', using − log p θγ (z n ) bits (in fact, the encoding of (γ, θ) itself has two sub-stages here so we really have a two-part code where the first part itself has two parts as well).
The discretization involved in using a probability mass function/code for continuousvalued θ makes things (unnecessarily, as was gradually discovered over the last 30 years) very complicated in general.Also, if one combines choice of θ with choice of γ, the approximation of comp(M γ , v γ ) as 0 introduces some suboptimalities.Thus, one would like to code the data in a way that avoids these two issues.It turns out that this can be achieved by replacing 2-part by 1-part codes for the data, namely, to use codes with length − log p nml v (z n ): assuming for simplicity that data are discrete, the same Kraft inequality implies that there must also be a code, directly defined on z n , which achieves code length for each z n given by − log pnml (z n ).Thus, even though for general luckiness functions v this code length cannot be decomposed into two sub-code lengths, it remains a valid code length and the name MDL for the resulting procedure remains, we feel, justified.In the past, it was sometimes thought by some MDL fans that 2-part codes on countable sets would somehow lead to inherently better estimates θw than estimators θv for general luckiness functions as in (5).However, after 30 years it turned out there is nothing either conceptually or mathematically that indicates the need for 2-part codes and countable sets: for any luckiness function v, the resulting procedure has a code length interpretation, and (Grünwald and Mehta [2019] show that all consistency and convergence results that hold for 2-part estimators also hold for general MDL estimators (Section 6.1) -thus invalidating the conjecture in Open Problem 13 of G07 that postulated a special status for luckiness functions v that are probability mass functions on countable sets.For the luckiness function π on the discrete structure Γ however, it is quite reasonable to choose a probability mass function: no probability mass is wasted (since the denominator in (17) plays no role in choosing γ), and designing π by thinking about the code lengths − log π(γ) comes very naturally, as the following example illustrates: .
Example 4 [Variable Selection -L 1 vs L 0 -penalties] Suppose that each data point Z i = (X i , Y i ) where Y i denotes the variable to be predicted and X i = (X i1 , . . ., X im ) ∈ R m is a vector of covariates or 'features' that may or may not help for predicting Y i .We consider a linear model where 1 , 2 , . . .represents 0-mean i.i.d.N (0, σ 2 ) normally distributed noise and γ = (γ 1 , . . ., γ m ) ∈ {0, 1} m is a binary vector indicating which variables are helpful for predicting the Z i .Thus, if γ has k 0-components than M γ is effectively an m−k dimensional model.Our task is to learn, from the data, the vector γ * indicating which variables are truly relevant, and/or such that predictions of new Y given new X based on M γ * are as good as possible.
In light of the above, a straightforward way to use MDL here is to pick the γ minimizing where we refer to G07, Figure 14.2 for an explanation why we can condition on x n here.v γ in ( 19) is an appropriately chosen luckiness function and π is really a probability mass function, such that L π (γ) := − log π(γ) can be interpreted as the number of bits needed to encode γ using a particular code.In terms of coding, a natural choice for such a code would be to first encode the number of nonzero components k γ in γ using a uniform code (that assigns equal code length to all possibilities).Since 0 ≤ k γ ≤ m, there are m + 1 possibilities, so this takes log 2 (m + 1) bits.In a second stage, one encodes the location

12
A C C E P T E D M A N U S C R I P T Accepted manuscript to appear in IJMI of these components.There are m kγ possibilities here, so this takes log 2 m kγ bits using a uniform code.All in all, one needs 'nits' (bits re-expressed in terms of natural logarithm log) to encode γ.Then (20) can be written as − log π(γ), with π(γ) = 1/((m + 1) • m kγ ), which, as predicted by the Kraft inequality, sums to 1 over γ ∈ {0, 1} m .
As to the left part of the code length ( 19), if the variance σ 2 is known, a natural luckiness function v γ to use is a m − k γ -dimensional Gaussian with mean 0 and variance σ 2 Σ for some (usually diagonal) covariance matrix Σ γ .This gives (see Chapter 12 of G07) where X = (X 1 , . . ., X n ) and | • | stands for determinant.We thus end up finding the γ minimizing the sum of ( 21) and ( 20).If, as here, the noise is normal with fixed variance, then for this choice of luckiness function, p nml vγ (y n | x n ) actually coincides with p bayes wγ (y n | x n ) for a particular prior w γ , thus one has a Bayesian interpretation as well (Bartlett et al. [2013] show that such a precise correspondence between NML and Bayes only holds in quite special cases, see Appendix A).If the variance σ 2 is unknown, one can treat it as a nuisance parameter and equip it with the improper Haar prior, leading to a modification of the formula above; see Example 6.Even if the noise is not known to be normally distributed, one can often still use the above method -pretending the noise to be normally distributed and accepting that one is misspecified -by varying the learning rate, as briefly explored in Section 6.3.
Note that the code/prior we used here induces sparsity: if there exists a γ with mostly 0 components that already fits the data quite well, we will tend to select it, since, for k n, log n k increases approximately linearly in k.That does not mean that we necessarily believe that the 'truth' is sparse -it just expresses that we hope that we can already make reasonably good predictions with a small number of features.
An alternative, and very popular, approach to this problem is the celebrated Lasso [Hastie et al., 2001], in which we consider only the full model M = M (1,1,...,1) and we pick the for some regularization parameter λ > 0; it is known this will tend to select β with many zero-components, thus also inducing sparsity, and it can be implemented computationally much more efficiently than the two-part approach sketched above, effectively replacing L 0 by L 1 -penalties.With our 'modern' view of MDL, this can be thought of as a form of MDL too, where we simply impose the luckiness function v(β) = exp(−(λ/σ 2 ) m j=1 |β j |) and use the estimator θv given by (5).The luckiness function v depends on λ; the optimal choice of λ is then once again related to the optimal learning rate; see Section 6.3.Finally, we note that there exists a third MDL approach one can use here: one starts out with an NML approach similar to (19) but then performs a continuous relaxation of the resulting optimization problem; the resulting 'relaxed' NML criterion is then once again tractable 13

A C C E P T E D M A N U S C R I P T
Accepted manuscript to appear in IJMI and similar to an L 1 -optimization problem such as (22); this approach has been described by Miyaguchi and Yamanishi [2018] who extend the idea to group lasso and other settings.

Log-Loss Prediction and Universal Distributions
Now consider the simple case again with a finite set of models {M γ : γ ∈ Γ} where Γ is small compared to n and we use the uniform prior π, picking the γ maximizing pγ .It was the fundamental insight of Rissanen [1984] and Dawid [1984] that such model choice by maximizing pγ (z n ) for a single distribution pγ can be motivated in a different way as well -in essence, it selects the model with the best predictive performance on unseen data.This approach shows that MDL is quite similar in spirit to cross-validation, the main difference with leave-one-out cross validation being that the cross in cross-validation is replaced by a forward and that the loss function used to measure prediction error is restricted to be the logarithmic score, also commonly known as log loss (which, however, is often used in cross-validation as well).
Formally, the log loss of predicting a single outcome z ∈ Z with a distribution p is defined as − log p(z): the larger the probability density, the smaller the loss.If one predicts a sequence of n outcomes z n = (z 1 , . . ., z n ) with n predictions p 1 , p 2 , . . ., p n , then the cumulative log loss is defined as the sum of the individual losses: Now, if we adopt a probabilistic world view and represent our beliefs about z n by a probability distribution p, then the obvious way to make sequential predictions is to set ).For arbitrary probability distributions, we have, by the formula for conditional probability: for all z n ∈ Z n , p(z n ) = n i=1 p(z i | z i−1 ).Taking logarithms gives: In words, for every possible sequence, the cumulative log loss obtained by sequentially predicting z i based on the previously observed data z i−1 is equal to the minus log-likelihood.
Conversely, if we are given an arbitrary sequential prediction strategy s which, when input a sequence z i−1 of arbitrary length i−1 outputs a prediction for the next outcome z i in the form of a probability distribution sz i−1 on Z, we can define p(z i | z i−1 ) := sz i−1 (z i ) and then further define p(z n ) := n i=1 p(z i | z i−1 ).A simple calculation shows that we must have z n ∈Z n p(z n )dz n = 1, so we have constructed a probability distribution p which once again satisfies (23).The fundamental insight here is that, when the log loss is used, every probability distribution defines a sequential prediction strategy and -perhaps more surprisingly -vice versa, every sequential prediction strategy defines a probability distribution, such that on all sequences of outcomes, the minus log likelihood is equal to the cumulative loss.
Example 5 Consider again the Bernoulli model M = {p θ : θ ∈ [0, 1]}.Each element p θ ∈ M defines a prediction strategy which, no matter what happened in the past, predicts that the probability that the next outcome Z i = 1 is equal to θ.It incurs cumulative loss, on sequence z n with n 1 ones and n 0 = n−n 1 zeros, given by n 1 (− log θ)+ n 0 (− log(1 − θ)).The Bayesian universal distribution p bayes wJ with Jeffreys' prior that we considered in Example 1 satisfies, as was already mentioned, p bayes wJ ))/(m + 1), so it 'learns' the probability of 1 based on past data and does not treat the data as i.i.d.any more.The asymptotic expansion (2) then shows that its cumulative loss is of order Accepted manuscript to appear in IJMI We may now ask: given a parametric model M, what distribution (i.e.prediction strategy) in M leads to the best predictions of data z 1 , . . ., z n ?For simplicity we will assume that data are i.i.d.according to all p θ ∈ M. We have to distinguish between the best sequential prediction strategy with hindsight and the best prediction strategy that can be formulated before actually seeing the data.The former is given by the p θ ∈ M achieving i.e. the best predictions with hindsight are given by the ML distribution θml (z n ).However, θml (z n ) is only knowable after seeing all the data z n , whereas in reality, we have, at each time i, to make a prediction p(Z i | z i−1 ) relying only on the previously seen data z i−1 .We might thus aim for a prediction strategy (distribution) p which will tend to have a small regret (additional prediction error) But what does 'tend' mean here?One strict way to implement the idea is to require (24) to be small in the worst case -one looks for the distribution p achieving where the minimum is over all probability distributions on Z n .But comparing ( 24) and ( 25) with (13), using that − log is strictly decreasing, we see that the p achieving ( 25) is just the NML distribution with v ≡ 1, which was already our 'favourite' distribution to use in MDL model comparison any way!And, just like before, if (25) has no solution, we may add a − log v luckiness term to (24) so as to regularize the problem, and then the optimal prediction strategy will be given by p nml v .We also see that with v ≡ 1, comp(M, v) is equal to the minimax regret (25); and with nonuniform v, comp(M) will become equal to the minimax luckiness regret, i.e. (24) with a − log v term added.
We now also see where the idea to use the prequential plug-in distribution ppreq instead of pnml comes from: if calculating p nml v is too difficult, or if the horizon n (which is needed to calculate p nml v ) is unknown, we might simply pick any estimator θ which we think is 'reasonable' and replace our prediction p nml v (Z i | z i−1 ) by p θ(z i−1 ) (Z i ) -if the estimator was chosen cleverly, we can expect the resulting cumulative regret to be small.Reconsidering ( 14), we see that all the universal distributions, viewed as prediction strategies, with the right choice of luckiness functions, priors and/or estimates, can be made to achieve a logarithmic (in n) worst-case regret -since the cumulative log-loss achieved by the best predictor in hindsight usually grows linearly in n, a logarithmic regret is quite satisfactory.Returning to our Bernoulli example, we see that the cumulative log loss obtained by θml , the best with hindsight, is equal to nH(n 1 /n) = nH( θml (z n )), where H(θ) is the binary entropy, H(θ) := −θ log θ − (1 − θ) log(1 − θ).Note that, in line with the above discussion, nH( θml ) is linear in n unless θml tends to 0 or 1, but the regret of pbayes with Jeffreys' prior is logarithmic in n.
We thus get a novel interpretation of MDL: it associates each model M γ with a sequential prediction strategy pγ that is designed to achieve small regret compared to the hindsight-optimal prediction strategy within M γ ; it then picks the model for which the corresponding prediction strategy achieves the smallest cumulative loss on the data.
Related Work Dawid (see [Dawid, 1984] and many subsequent works) suggests to use this prequential model choice also with respect to loss functions other than the logarithmic loss; minimax optimal cumulative prediction strategies without making stochastic assumptions about the data, with log loss but (mainly) with other loss functions, are one of the main topics in machine learning theory; see for example Cesa-Bianchi and Lugosi [2006]; but there they are generally not used for model comparison or selection.
Why the logarithmic score?Why does it make sense to minimize cumulative log loss?Outside of the MDL world, the log loss is often used for two reasons: first, it is (essentially) the only local proper scoring rule [Dawid, 2007].Second, it has an interpretation in terms of money: for every sequential prediction strategy, there is a corresponding 'sequential investment' strategy such that, the smaller the cumulative logloss, the larger the monetary gains made with this strategy ('Kelly Gambling', [Cover andThomas, 1991, Grünwald et al., 2019]).
Within the MDL field however, the use of the log loss comes from the Kraft inequality, which directly relates it to lossless data compression.As we already saw before Example 4, for any sequential prediction strategy, i.e. every distribution p on sequences of length n, there is a lossless code C such that, for all sequences of length n, − log 2 p(z n ) = nr. of bits needed to code the data z n using code C.
Conversely, for any code C, there is a corresponding distribution p such that the above holds (see Chapter 3 of G07 for a very extensive explanation).Thus, the original MDL idea to 'take the model that compresses the data most' is first made more formal by replacing it by 'associate each model with a code that compresses well whenever some distribution in this model compresses well', and this turns out to be equivalent to 'associate each model M γ with a distribution pγ that assigns high likelihood whenever some distribution in the model assigns high likelihood'.

MDL Prediction and 'Improper' Estimation
As is clear from the prequential interpretation of MDL given above, once a universal distribution pp has been fixed, one can use it to predict Z i given z i−1 by p(Z i | z i−1 ).At least for i.i.d.data, we can estimate the underlying 'true' distribution p * based on such predictions directly, by simply interpreting p(Z i | z i−1 ) as an estimate of p * !This is different from the previous form of MDL estimation described in Section 2.3, which was based on MDL (penalized ML) estimators θv .Note that this standard MDL estimator is 'in-model' or proper (to use machine learning terminology [Shalev-Shwartz and Ben-David, 2014]), whereas p(Z i | z i−1 ) is outmodel or improper : in general, there may be no p ∈ M such that p(• | z i−1 ) = p.For example, with Bayes universal distributions, p bayes w Z i |z i−1 will be a mixture of distributions in M rather than a single element; see G07 for more discussion.

The Luckiness Function
The choice of a luckiness function is somewhat akin to the choice of a prior in Bayesian statistics, yet -as explained at length in Chapter 17 of G07 -there are very important differences, both technically (luckiness functions are not always integrable) and philosophically.Basically, a luckiness function just determines for what type of data one will be "lucky" (v( θv (z n )) large) and get small cumulative regret based on small samples (and presumably, good model selection results as well), and for what data one will be less lucky and get good results only when the data set grows much largerv may thus be chosen for purely pragmatic reasons.For example, as in Example 4 (see the italicized text there), if one assigns a large value π(γ) to some model M γ within large a collection of models {M γ : γ ∈ Γ} where γ is sparse, one may do this because one hopes that this 16

A C C E P T E D M A N U S C R I P T
Accepted manuscript to appear in IJMI sub-model will already lead to reasonable predictions of future data, even though one feels that, at the same time, when more data becomes available, a model M γ with a much larger number of nonzero parameters may at some point almost certainly become better (Example 4).Such an interpretation is not possible with a Bayesian prior π, where a large value of π(γ) indicates a strong belief that M γ is true, or at least, that predictions based on acting as if it will be true will be optimal -a Bayesian with high prior on γ considers M γ likely rather than just useful -a distinction worked out in detail by Grünwald [2018].Nevertheless, just like a Bayesian prior, the luckiness function has to be chosen by the user/statistician, and often contains a subjective element.Still, in contrast to Bayesian priors, since we invariably take a worst-case log-loss stance in MDL, there often is a uniquely preferable choice of luckiness function v for parametric models M. First, if comp(M, v) < ∞ with uniform v and no clear prior knowledge or preference is available, then uniform v is usually preferable over other v, since it achieves the worst-case optimal prediction performance.Second, if comp(M γ , v) = ∞ with uniform v for some γ ∈ Γ we can often still set the first few, say m, outcomes aside and pick a luckiness function v γ (θ) := p θ (z 1 , . . ., z m ) for θ ∈ Θ γ .The corresponding estimator (for fixed γ) θvγ based on data z m+1 , . . ., z n as given by (5) will then be equal to the ML estimator based on the full data, θvγ (z n m+1 ) = θml|γ (z n ), and by choosing m large enough, one can often get that comp(M γ , v γ ) is finite after all for all γ ∈ Γ and one may then compare models by picking the γ maximizing p nml vγ (z m+1 , . . ., z n ).Thus, the luckiness function is now determined by the first few 'start-up' data points, and one uses NML based on the remaining data points with an estimator that coincides with the ML estimator based on the full data.G07 argues why this data-driven luckiness function is the best default choice available; note that it is analogous to our use of improper Bayes priors as described in Example 2.

The Switch Distribution and the AIC-BIC Dilemma
The AIC-BIC dilemma (see for example [Yang, 2005] and the many citations therein) is a classic conundrum in the area of statistical model selection: if one compares a finite number of models, the two standard benchmark methods, with (different) asymptotic justifications, are AIC and BIC.Suppose one first selects a model using AIC or BIC.One then predicts a future data point based on, e.g., maximum likelihood estimation, or by adopting the Bayes/MDL predictive distribution, within the chosen model.If one compares a finite number of models, then AIC tends to select the one which is optimal for prediction (compared to BIC, the predictions converge faster, by a factor of order log n, to the optimal ones).On the other hand, BIC is consistent: with probability 1, it selects the smallest model containing the true distribution, for all large n; the probability that AIC selects an overly large model does not go to 0 for large n.Both the predictive optimality and the consistency propertiy are desirable, but, like AIC and BIC, common methods all fail on one of the two.For example, MDL, with each of the four classical distributions, and Bayes factor model selection will behave like BIC for large n and be consistent but prediction-suboptimal; for any fixed k, leave-k-out and k-fold cross-validation will tend to behave like AIC and have the reverse behaviour.Yang [2005] shows that, in general, this dilemma cannot be solved: every consistent method has to be slightly prediction suboptimal in some situations; he also shows that prediction by model averaging cannot solve this dilemma either.
Nevertheless, as first shown by Van Erven et al. [2007] (who thereby solved Open Problem 8 and 17 of G17), one can design universal distributions that 'almost' get the best of both worlds: basing MDL model selection on them using (1) one gets a criterion which is strongly consistent while at the same time losing only an exceedingly small 17

A C C E P T E D M A N U S C R I P T
Accepted manuscript to appear in IJMI order log log n factor in terms of prediction quality compared to the AIC-type methods.Although it can be applied to arbitrarily large model collections, the idea of this so-called switch distribution pswitch is best explained by considering the simplest case with just two nested models M 0 ⊂ M 1 : one starts with two standard universal distributions (say, Bayesian or luckiness-NML) p0 for M 0 and p1 for M 1 .For every i > 0, p1 defines a conditional distribution p1 (z i , . . ., z n | z i−1 ).One now picks a 'prior' distribution π on the integers (typically one that decreases polynomially, e.g.π(i) = 1/(i(i + 1))), and one defines a new universal distribution for M 1 by: This distribution is best understood from the prequential interpretation of MDL (Section 2.4).It will satisfy In words, the cumulative log loss achieved by pswitch is 'almost' (within an order log n term) as small as that of the strategy that first predicts by p0 and then switches from p0 to p1 at the switching point i that is optimal with hindsight.By clever choice of the prior π, one can get the extra term down to order log log n.In cases where the data are actually sampled from a distribution in M 1 that is 'close' (defined suitably) to M 0 , the predictions based on pswitch will, with high probability, be substantially better than those based on p1 -a dramatic example (that makes very clear why this happens) is given in the first figure of Van Erven et al. [2012].If the data come from a distribution that is 'far' from M 0 , they will tend to be worse than those based on p1 by a negligible amount.Working out the math shows that associating M 1 with pswitch and M 0 with p0 indeed gives a strongly consistent model selection criterion that is almost (to within an O(log log n) factor) prediction optimal, thus almost solving the AIC-BIC dilemma.Van Erven et al. [2012] describes in great detail why the standard NML or Bayesian universal model p1 does not lead to the optimal cumulative log-loss if data come from a distribution close to, but not in, M 0 .
In case the number of models on the list is larger than two or even infinite, one has to associate each model with a separate switch distribution.The technique for doing so is described by Van Erven et al. [2012] who also give an efficient implementation and prove consistency and prediction optimality of the switch distribution in a weak, cumulative sense for both finite and infinite numbers of models.Van der Pas and Grünwald [2018] mathematically show the 'almost' prediction optimality for a finite number of models.

Hybrids Between NML, Bayes and Prequential Plug-In
A Problem for NML: unknown horizon Bayesian universal distributions with fixed priors have the property that the probability assigned to any initial sequence z n , where n < n, is independent of the total length of the sequence.For other universal models, such as NML, this is not always the case.Take for example the Bernoulli model extended to sequences by independence: For sequence length n = 2, the normalizing term in the NML equals 1 + (1/2) 2 + (1/2) 2 + 1 = 5/2.For sequence length n = 3, the normalizing term equals 2 + 6 × (1/3) (2/3) 2 = 78/27.For n = 2, the NML probability of 18

A C C E P T E D M A N U S R I P T
Accepted manuscript to appear in IJMI the sequence 00 is 1/(5/2) = 0.4.However, for sequence length is n = 3, the probability of the initial sequence 00 is obtained as the sum of the probabilities of the sequences 000 and 001, which becomes 1/(78/27) + (4/27)/(78/27) ≈ 0.397 < 0.4.As shown by Bartlett et al. [2013] (see also Barron et al. [2014]), there do exist cases in which NML is, like Bayes, horizon-independent, but these are very rare -see Appendix A.
The above discrepancy between the initial sequence probabilities for different sequence lengths may be a problem in situations where we need to obtain predictions without necessarily knowing the total length of the sequence, or the horizon.Another related issue is that even if the total sequence length was given, it can be computationally expensive to obtain marginal and conditional probabilities along the initial sequences.One possible solution would be to restrict to Bayesian universal distributions.However, while these solve the horizon issue, they are (a) still often computationally inefficient, and (b) they lack NML-style worst-case regret interpretations.This has spurned research into universal codes that can be calculated without knowing the horizon in advance and that behave better as regards to (a) or (b), which we now review.

Prequential Plug-In and the (k/2) log n-formula
The most straightforward way to deal with issue (a) is to use the prequential ppreq which, by construction, is horizon-independent.However, for the prequential ppreq (10) the BIC asymptotics ( 14) only hold in expectation if the data are sampled from one of the distributions in the model M.This makes the result much weaker than for the other five universal distributions considered, for which the asymptotics hold for every individual sequence in some large set, i.e. without making any stochastic assumptions at all.One might thus wonder what happens for general data.Extending earlier work by Takeuchi and Barron [1998], Kotłowski et al. [2010] show that, if data are sampled from a distribution p, and p θ is the distribution in M that is closest in KL divergence to p, then (14) holds in expectation, with a correction term involving the variances of both distributions; for 1-dimensional models, we get a formula that can be extended to multidimensional models and individual sequence settings.Solving Open Problem 2 from G07, Grünwald and Kotłowski [2010] show that, essentially, there exists no 'in-model' estimator that can achieve the standard asymptotics in general; a correction such as ( 26) is always needed, whatever estimator one tries.Here an 'in-model' estimator (or 'proper' estimator, see end of Section 2.4) is an estimator that always outputs a distribution inside the model M; the ML and Bayes MAP estimators are in-model, but the Bayes predictive distribution is not in-model, since it is a mixture distribution over all distributions in M.
Solving Open Problem nr. 3 from G07, Kotłowski et al. [2010] also provide a new universal distribution, in which for any given estimator θ, which is not in M any more (it is not an in-model estimator), but it does achieve the standard (k/2) log n asymptotics without correction.For example, in case of the normal location family with fixed variance σ 2 , it coincides with a Bayesian predictive distribution based on a standard conjugate prior, which in this case is a normal with mean μ (the Bayes MAP estimate) but variance σ 2 + O(1/n).More generally, p preq * θ (Z i+1 | z i ) becomes a hybrid between the estimator θ and a Bayes predictive distribution, but it has the advantage over the latter that it can be calculated without performing an integral over the parameter space.It thus provides an alternative to the NML distribution that is horizon-free and that is often faster to compute than pbayes .

A C C E P T E D M A N U S C R I P T
Accepted manuscript to appear IJMI Roos and Rissanen [2008] and Rissanen et al. [2010] developed other prequential, horizon-free universal codes that are non-Bayesian, yet remain more closely to NML in spirit than p preq * θ .They work out the details for discrete models including Bernoulli as well as linear regression models.For Bernoulli models, the resulting universal code coincides with the so called one-step lookahead mode proposed earlier by Takimoto and Warmuth [2000].For linear regression models the asymptotic consistency of the resulting model selection criterion was studied by Määttä et al. [2016] and Määttä and Roos [2016].Relatedly, Watanabe and Roos [2015] show that no horizon-independent strategy can be asymptotically minimax in the multinomial case and propose simple Bayesian universal models with a horizon-dependent Dirichlet prior that achieve asymptotic minimaxity and simplify earlier proposals.Among the proposed priors is Dir(α, . . ., α) with α = 1/2 − ln 2/2 ln n which converges to the Jeffreys' prior Dir(1/2, . . ., 1/2) but has a mild dependency on the horizon n.

Hypothesis Testing: Universal Distributions based on the Reverse Information Projection
Suppose we compare just two models, M 0 and M 1 , as explanations for data z n , a situation similar to classical null hypothesis testing, the standard method for evaluating new treatments in the medical sciences and scientific hypotheses in most applied sciences such as psychology, biology and the like: we can think of M 0 and M 1 as two hypotheses, where, usually, M 0 represents the status quo ('treatment not effective', 'coin unbiased').
In case1 M 0 = {P 0 } represents a simple (singleton) hypothesis, there is a strong additional motivation for using MDL as a hypothesis testing method, and in particular, for quantifying the evidence against M 0 in terms of the codelength or cumulative-log-loss difference (see Section 2.4) between encoding (or sequentially predicting) the data with p 0 and with p1 .This additional motivation is given by the no hyper-compression inequality (G07), a mathematical result stating that, no matter how p1 is defined, as long as it is a probability distribution, we have for all K > 0, and 0 ≤ α ≤ 1 This expresses, in terms of sequential log-loss prediction (compression), that, if P 0 is true, then the probability that one can predict data better, by K or more loss units, by predictions based on p1 rather than p 0 , is exponentially small in K -and this holds independently of the sequence length n.In terms of more classical quantities, it states that, no matter how we chose p1 , if P 0 holds true then the likelihood ratio is a p-value.
In fact it is a conservative p-value, giving usually somewhat less evidence against M 0 than a standard p-value, for which the rightmost inequality in ( 27) is an equality.The inequality ( 27) goes in the right direction to retain the cornerstone of classical Neyman-Pearson testing: if one sets significance level α before seeing the data and one chooses M 1 whenever D(Z n ) ≤ − log α, i.e. p 0 (Z n )/p 1 (Z n ) ≤ α, then the probability, under the null P 0 , of making a false decision is bounded by α.But the fact that the rightmost inequality in ( 27) is usually strict has pleasant practical repercussions: as explored by [Grünwald et al., 2019], the Type-I error guarantees are retained under optional continuation.This is the (common) practice to decide, on the basis of an initial sample z n , whether or not to gather new data and do a second test.One may for example decide to gather new data if the result based on z n was hopeful yet not conclusive.This is highly problematic for standard, strict p-value based hypothesis testing, but with MDL testing with a simple M 0 , one can simply multiply the likelihood ratios of the two (or more) tests performed, or equivalently, add the code length differences for each test performed.The resulting code length difference/likelihood ratio will still lead to valid Type-I error bounds [Grünwald et al., 2019].But: all this holds only for simple M 0 .Yet the tests most used in practice, such as the t-test and contingency table tests, all involve composite M 0 = {P θ : θ ∈ Θ 0 }.For composite M 0 , the no-hypercompression inequality usually only holds for some P 0 ∈ M 0 , but for Type I error guarantees and the like we would want to have it hold for all P θ with θ ∈ Θ 0 .That is, we would like to employ universal distributions p1 and p0 such that we have: In general, this will not hold for standard choices (NML, Bayes, prequential plug-in...) of p1 and p0 .However, Grünwald et al. [2019] show that, for any given (arbitrary) p1 , one can, under very mild conditions, construct a p0 such that (28) holds, thereby solving Open Problems 9 and 19 of G07.This p0 is the Reverse Information Projection (RIPr) [Li, 1999, Li andBarron, 2000] of p1 onto P bayes (M 0 ), where P bayes is the set of densities p0 for z n that can be written as Bayes marginal distributions pbayes 0 (z n ) = p θ (z n )w 0 (θ)dθ for some prior w 0 on Θ 0 -for every prior w 0 , P bayes (M 0 ) contains a separate distribution on Z n .The RIPr is defined as the density achieving min p0 ∈P0 D(p 1 p 0 ), where D(• •) is the Kullback-Leibler divergence.Thus, one constructs a p0 with the desired no-hypercompression property, and at the same time, it will minimize KL divergence to p1 , which implies that if data were sampled from p1 , it would yield optimal log-loss predictions.This, in turn, implies that the p0 constructed this way will satisfy the standard asymptotics ( 14) as long as the p1 on which it is based does.Based on the likelihood ratio between p1 and its RIPr p0 , one is also allowed to do optional continuation while retaining Type I Error guarantees.Thus, even if one is an adherent of classical, frequentist testing theory, there are strong reasons for MDL-style testing based on the RIPr universal distribution.Grünwald et al. [2019] further extend the reasoning to give guidelines how p1 can be chosen to get further good frequentist properties.
Example 6 [Right Haar Priors and the Bayesian t-test] In a series of papers (highlights include [Berger et al., 1998, Dass andBerger, 2003]), Berger and collaborators established Bayes factor testing methods for composite M 0 = {P θ : θ ∈ Θ 0 } where the only free parameters in Θ 0 are 'nuisance' parameters that are shared by Θ 1 and are governed by a group structure.A prime example is the unknown variance in the t-test.Berger uses a special type of improper prior, the so-called right-Haar prior, which can be defined for every such type of nuisance parameter.While Bayes factors usually don't combine well with improper priors, the Bayes factors for group invariance parameters equipped with the right-Haar prior behave remarkably well.Grünwald et al. [2019] show that, even though the right Haar priors are usually improper, they can also be understood from a purely MDL perspective: if pbayes 1 and pbayes 0 are equipped with the right Haar prior on the nuisance parameters, and the prior on the additional parameters in pbayes 1 satisfies some additional requirements, then both pbayes 1 and pbayes 0 can be interpreted as sequential prediction strategies, and the log of the Bayes factor can be interpreted as the code length/cumulative log loss difference.Moreover, pbayes 0 is (essentially) the RIPr for pbayes 1 and the no-hypercompression inequality (28) that is so desirable from a frequentist perspective holds uniformly for all θ 0 ∈ Θ 0 .
Let us consider the one-sample Bayesian t-test as an example.Here M 0 = {p 0,σ : σ >

21
A C C E P T E D M A N U S C R I P T Accepted manuscript to appear in IJMI 0} is the set of all normal distributions with mean 0; the variance σ 2 is a free parameter.M 1 = {p µ,σ : µ ∈ R, σ > 0} is the set of all normal distributions with as free parameters µ and σ.The question of interest is to establish whether µ = 0 or not; σ is an unknown 'nuisance' parameter -it determines the scale of the data but is not itself of intrinsic interest.In the Bayesian t-test one equips both M 0 and M 1 with the improper right-Haar prior, w(σ) = 1/σ.To complete the definition of pbayes 1 , M 1 is equipped with a conditional prior density (given σ) on the effect size δ := µ/σ.This second density has to be symmetric around 0 and proper (this is what we called the 'additional requirement on the prior on Θ 1 ', instantiated to the case where the nuisance parameter is a variance).One now proceeds by testing using the Bayes factor pbayes 0 /p bayes 1 .In this special case, the procedure was already suggested by Jeffreys [Ly et al., 2016], and the right-Haar prior coincides with Jeffreys' prior on the variance.Berger et al. extend the method to general group invariant parameter vectors such as the joint mean and variance in the two-sample t-test, testing a Weibull against a log-normal and many other scenarios.

Graphical Models
Graphical models are a framework for representing multivariate probabilistic models in a way that encompasses a wide range of well-known model families, such as Markov chains, Markov random fields, and Bayesian networks; for a comprehensive overview, see Koller and Friedman [2009].A key property of a graphical model is parsimony, which can mean, for instance, a low-order Markov chain or more generally a sparse dependency graph that encodes conditional independence assumptions.Choosing the right level of parsimony in graphical models is an ideal problem for MDL model selection.
In Bayesian network model selection the prevailing paradigm is, unsurprisingly, the Bayesian one.Especially the work of Geiger and Heckerman [1995], Heckerman et al. [1995] has been extremely influential.The main workhorse of this approach is the so called Bayesian Dirichlet (BD) family of scores which is applicable in the discrete case where the variables being modelled are categorical.Given a data sample, such scores assign a goodness value to each model structure.Exhaustive search for the highest scoring structure is possible when the problem instance (characterized by the number of random variables) is of limited size, but heuristic search techniques such as variants of local search or "hill-climbing" can be used for larger problem.
Different forms of the BD score imply different Dirichlet priors (different hyperparameters) for the local multinomial distributions that comprise the joint distribution.For example, in the commonly used BDeu score, the priors are determined by a single hyper-parameter, α.For a variable X i with r distinct values and parents Pa i that can take q possible combinations of values (configurations), the BDeu prior is Dir(α/rq, . . ., α/rq).One of the main motivations for adopting this prior is that it leads likelihood equivalence, i.e., it assigns equal scores to all network structures that encode the same conditional independence assumptions.In light of the fact that Bayesian model selection embodies a particular form/variation of MDL, these method fit, at least to some extent, in the MDL framework as well.However, there also exist more 'pure', non-Bayesian MDL methods for model selection in Bayesian networks; we mention Lam and Bacchus [1994] and Bouckaert [2005] as early representative examples.These early methods are almost invariably based on the two-part coding framework.More recently, several studies have proposed new model selection criteria that exploit the NML distribution.One approach is a continuous relaxation of NML-type complexities proposed by Miyaguchi et al. [2017] in which the model selection problem takes on a tractable Lasso-type L 1 -minimization form (see also Example 4).In other approaches, NML (or usually, approximations (but not relaxations) thereof) are used directly for encoding parts of the model; we now describe these latter approaches in a bit more detail.

A C C E P T E D M A N U S C R I P T
Accepted manuscript to appear in IJMI 4.1 Factorized NML and variants Silander et al. [2010] propose the factorized NML (fNML) score for Bayesian network model selection which was designed to be decomposable meaning that it can be expressed as a sum that includes a term for each variable in the network.This property facilitates efficient search among the super-exponential number of possible model structures; see, e.g., Heckerman et al. [1995].The fNML score factors the joint likelihood not only in terms of the variables but also in terms of distinct configurations of the parent configurations.Each factor in the product is given by a multinomial NML probability, for which a lineartime algorithm by Kontkanen and Myllymäki [2007] can be used.
A similar idea where a Bayesian network model selection criterion is constructed by piecing together multiple NML models under the multinomial model was proposed recently by Silander et al. [2018].In the proposed quotient NML (qNML) score, the local scores corresponding to each variable in the network are defined as log-quotients of the form where NML full refers to an NML distribution defined by using a fully connected network to model the variable X i and its parents Pa i in the numerator and the same thing for the parent set Pa i in the denominator.Technically, this amounts to collapsing the configurations of the variables into distinct values of a single categorical variable.Even though the resulting categorical variable may have a huge number of possible values, the linear time algorithm [Kontkanen and Myllymäki, 2007] or efficient approximations (see the next subsection) can be used to implement the computations.A notable property of the qNML score is that, unlike the fNML score, it is likelihood equivalent (see above).Eggeling et al. [2014] apply similar ideas to a different model class, namely parsimonious Markov chains.There too, the likelihood is decomposed into factors depending on the configurations of other variables, and each part in the partitioning is modelled independently using the multinomial NML formula.The authors demonstrate that the fNML-style criterion they propose leads to parsimonious models with good predictive accuracy for a wide range of different scenarios, whereas the corresponding Bayesian scores are sensitive to the choice of the prior hyperparameters, which is important in the application where parsimonious Markov chains are used to model DNA binding sites [Eggeling et al., 2015].
In all these papers, both simulated and real-world data experiments suggest that the MDL-based criteria are quite robust with respect to the parameters in the underlying data source.In particular, the commonly used Bayesian methods (such as the BDeu criterion) that are being used as benchmarks are much more sensitive and fail when the assumed prior is a poor match to the data-generating model, whereas the MDL methods are invariably very close to the Bayesian methods with the prior adapted to fit the data.This poses interesting questions concerning the proper choice of priors in the Bayesian paradigm.
In fact, the prevalence of the Bayesian paradigm and the commonly used BD scores is challenged by two recent observations: First, Silander et al. [2010] show that the Dirichlet prior with hyperparameters (1/2, . . ., 1/2), which is the invariant Jeffreys prior for the multinomial model, but not likelihood equivalent when used in the BD score, is very close to the fNML model and consequently, enjoys better robustness properties than the BDeu score which is the likelihood equivalent BD score variant.Second, Suzuki [2016] shows that the BDeu criterion is irregular, i.e., prone to extreme overfitting behavior in situations where a deterministic relationship between one variable and a set of other variables holds in the data sample.The MDL scores discussed above are regular in this respect and their robustness properties seem to be better than those of the BD scores, see [Silander et al., 2018].

A C C E P T E D M A N U S C R I P T
Accepted manuscript to appear in IJMI 4.2 Asymptotic expansions for graphical models Asymptotic results concerning MDL-based criteria in graphical models are interesting for several reasons.For one, they lead to efficient scores that can be evaluated for thousands of different model structures.Secondly, asymptotic expansions can lead to insights about the relative complexity of different model structures.
Various asymptotic forms exist for the point-wise and the expected regret depending on the model class in question.For convenience we repeat the classical expansion of the NML (as well as the Bayesian marginal likelihood with Jeffreys' prior) regret/model complexity that applies for regular model classes M = {p θ : θ ∈ Θ} for which comp(M, v) is finite with uniform v (see Section 2.2 above): where k is the dimension of the model, |I(θ)| is the determinant of the Fisher information matrix at parameter θ, the integral is over the parameter space Θ, and the remainder term o(1) vanishes as the sample size tends to infinity.
For discrete data scenarios, by far the most interesting case is the multinomial model (extension of the Bernoulli distribution to an i.i.d.sequence of r-valued categorical random variables) since it is a building block of a number of MDL-criteria such as fNML and qNML (see above).There are many asymptotic expansions for the NML regret under the multinomial model.Probably the most useful is the one proposed by Szpankowski and Weinberger [2012]: where n is the sample size, α = r n , and C α = 1 2 + 1 2 1 + 4 α .This simple formula is remarkably accurate over a wide range of finite values of n and r (see Silander et al. [2018]).Note that the leading term is proportional to n (rather than log n as usual) because the formula is derived for the regime r = Θ(n) where the alphabet size grows proportionally to the sample size.If r grows slower than n or not at all, the leading term tends to the classical form (29), where the leading term is k 2 log n.In practice, the approximation ( 30) is applicable for a wide range of r/n ratios.Roos [2008] and Zou and Roos [2017] studied the third term in the expansion of Eq. ( 29), namely the Fisher information integral, under Markov chains and Bayesian networks using Monte Carlo sampling techniques.This approach reveals systematic differences between the complexities of models even if they have the same number of parameters.

Latent Variable and Irregular Models
Although thus far we have highlighted exponential family and regression applications, NML and other universal distributions can of course be used for model selection and estimation in complete generality -and many practical applications are in fact based on highly irregular models.Often, 'classical' 2-part distributions (based on discretized models) are used, since NML distributions often pose computational difficulties.However, Yamanishi and collaborators have managed to come up with tractacble approximations of NML-type distributions for some of the most important irregular (i.e.non-exponential family) models such as hierarchical latent variable models [Wu et al., 2017], and the related Gaussian mixture models Hirai andYamanishi [2017, 2013].Suzuki et al. [2016] provides an NML approach to nonnegative matrix factorization.Two-part codes (and

24
A C C E P T E D M A N U C R I P T Accepted manuscript to appear in IJMI corresponding MDL estimators) for mixture families that come close to achieving the minimax regret were considered very recently by Myamoto et al. [2019].
When it comes to asymptotic approximations for codelengths/log-likelihoods based on NML and other universal distributions -all approximations so far (in Section 2.2 were derived essentially assuming that the model under consideration is an exponential family.Extensions to curved exponential families and generalized linear models are relatively straightforward (see G07 for details).For more irregular models, Sumio Watanabe has proposed the widely applicable information criterion (WAIC) and the widely applicable Bayesian information criterion (WBIC), see [Watanabe, 2010[Watanabe, , 2013]], where the latter can be viewed as an asymptotic expansion of the log likelihood based on a Bayesian universal distribution.It coincides with BIC when applied to regular models but is applicable even for singular (irregular) models.The asymptotic form of WBIC is where θ 0 is the parameter value minimizing the Kullback-Leibler divergence from the true model to the model p θ0 , and λ > 0 is a rational number called the real log canonical threshold (see [Watanabe, 2013]), which can be interpreted as the effective number of parameters (times two).

Frequentist Convergence of MDL and Its Implications
Rissanen first formulated the MDL Principle as -indeed -a Principle: one can simply start by assuming, as an axiom, that model ing by data compression (or, equivalently, sequential predictive log loss minimization) is the right thing to do.One can also take a more conventional, frequentist approach, and check whether MDL procedures behave desirably under standard frequentist assumptions.We now review results that show that, in general, they do -thus providing a frequentist justification of MDL ideas: with some interesting caveats, MDL model selection is typically consistent (the smallest model containing the true distribution is eventually chosen, with probability one) and MDL prediction and estimation achieves good rates of convergence (the Hellinger distance between the estimated and the true density goes to 0, with high probability, quite fast).In this section we review the most important convergence results.In particular, Section 6.1 shows that the link between data compression and consistent estimation is in fact very strong; and Section 6.4 shows that, by taking MDL as a principle, one can get useful intuitions about deep questions concerning deep learning; and the intuitions can then, as a second step, be once again validated by frequentist results.Thus, let us assume, as is standard in frequentist statistics, that data are drawn from a distribution in one of the models under M γ under consideration.We consider consistency and convergence properties of the main MDL procedures in their main applications: model selection, prediction and estimation.
Model Selection For model selection between a finite number of models, all universal codes mentioned here are consistent in wide generality; for example, this has been explicitly proven if the data are i.i.d. and all models on the list are exponential families, but results for more complex models with dependent data have also been known for a long time; see G07 for an overview of results.If the collection of models is countably infinite, then results based on associating each M γ with pbayes γ have also been known for a long time; such results typically hold for 'almost all' (suitable defined) distributions in all M γ ; again, see G07 for a discussion of the (nontrivial) 'almost all' requirement.These

25
A C C E P T E D M A N U S R I P T Accepted manuscript to appear in IJMI countable-Γ consistency results were extended to the switch distribution by Van Erven et al. [2012].
Prediction and 'Improper' Estimation As to sequential prediction (Section 2.4): rate of convergence results are very easy to show (see Chapter 15 of G07), but these typically only demonstrate that the cumulative log-loss prediction error of sequentially predicting with a universal distribution p behaves well as n increases.Thus, since the sum of prediction errors is small, say (for parametric models) of order log n, for most t the individual prediction error at the t-th sample point must be of order 1/t, since n t=1 1/t − log t = O(1).Still, it remains an open question how to prove for individual t what exactly the expected prediction error is at that specific n.Since one can view each prediction as an 'improper' estimate (end of Section 2.4), the convergence rates of the resulting estimators, which estimate the underlying distribution based on a sample of size t as p(Z t+1 | z t ), usually also behave well in a cumulative sense, but again it is very hard to say anything about individual t.The asymptotic expansions ( 15) and ( 16) imply that, for fixed parametric models M γ , pbayes γ and pnml γ achieve optimal cumulative prediction and estimation errors.If, however, they are defined relative to a full model class M = γ∈Γ M γ consisting of at least two nested models, then they may fail to achieve optimal rates by a log n factor.Van Erven et al. [2012] show that sequential prediction/estimation based on the switch distribution achieves the minimax optimal cumulative prediction/estimation error rates even in such cases.Van der Pas and Grünwald [2018] show that, if only two models are compared, then the optimal obtainable rate for individual n for any consistent procedure is achieved as well.

Frequentist Convergence of MDL Estimation
Very strong results exist concerning the convergence of MDL estimation based on an MDL estimator θv as given by (5).A first, classical result was already stated by the groundbreaking [Barron and Cover, 1991], establishing that consistency and good convergence rates can be obtained for the special case of a two-part-code estimator θw based on a probability mass function w, as long as w satisfies θ∈ Θ w(θ) η < ∞ for some η < 1 and w puts sufficient prior mass in a KL neighborhood of the true theta.These results were greatly extended by Zhang [2006a,b] and further, very recently, by Grünwald and Mehta [2019].The latter consider θv for general v. Let M = {P θ : θ ∈ Θ} be a statistical model and suppose data are i.i.d.∼ p with p = p θ * ∈ M. They find that a sufficient condition for consistency is that v(θ * ) < ∞ and that for some η < 1, the following generalized model complexity ) is bounded and o(n), i.e. it grows slower than linear (the slower it grows, the faster the MDL estimator converges to the true distribution in Hellinger distance).This condition strictly and significantly weakens the Barron-Cover requirement.The result holds without any further conditions; for example, M may be a countable union of parametric models or even a huge nonparametric model.Note that comp 1 (M, v) is the model complexity that we have encountered before in (7).Ironically, for any η < 1, slow-growth (o(n))) of comp η (M) is sufficient for consistency of θv , but for η = 1, which would be more fully in line with the MDL ideas, it is not.

A C C E P T E D M A N U S C R I P T
Accepted manuscript to appear in IJMI 6.2 From MDL to Lasso As illustrated in Example 4, when used for high-dimensional variable selection, the original MDL approach would be to use a mixed two-part/one-part code as in (1) with a − log π(γ) term to account for the model index γ ∈ Γ.In such settings, there may well be p > n variables of interest, each of which may or may not be included in the model, so that the minimization over Γ requires trying out 2 p 2 n choices -which is practically infeasible.For this reason, in practice people have strongly preferred the Lasso and related methods based on L 1 -penalties, which take linear rather than exponential time in p (note that the classic MDL essentially penalizes by an L 0 -penalty).However, Barron and Luo [2008], Barron et al. [2008] showed that, under some conditions on the true distribution (such as Gaussian noise), the Lasso method can be re-interpreted in terms of code-length minimization after all; see also [Chatterjee and Barron, 2014], and, for further extensions, Kawakita and Takeuchi [2016], Brinda and Klusowski [2018].For a different approach to unify model selection with very high-dimensional models with the luckiness NML, see Miyaguchi and Yamanishi [2018].
Although some of the details may differ, it seems that most of these works are subsumed by the aforementioned result of Grünwald and Mehta [2019] who show that general penalized estimators can be re-interpreted as minimizing a 1-part codelength as long as comp 1 (M, v) is bounded, and can be proven consistent under the (still quite weak) condition that comp η as in ( 32) is bounded for some η < 1.Thus, the connection between MDL and general (including Lasso and other L 1 , but also with entirely different penalties) penalization methods is substantially stronger than it seemed before these developments took place.
Importantly, all the works mentioned here except Grünwald and Mehta [2019] cannot show convergence under misspecification -for example, when applied to the Lasso, they would require an assumption of normal noise (corresponding to the squared error used in the Lasso fit, which is equivalent to the log loss under a normal distribution for the noise).In practice though, the Lasso (with the squared error) is often used in cases in which one cannot assume normally distributed errors.Grünwald and Mehta [2019] contains results that can still be used in such cases (although the formula for comp η (M, v) changes), based on ideas which we will now sketch.

Misspecification
As beautifully explained by Rissanen [1991], one of the main original motivations for MDL-type methods is that they have a clear interpretation independent of whether any of the models under consideration is 'true' in the sense that it generates the data: one chooses a model minimizing a code length, i.e. a prediction error on unseen data, which is meaningful and presumably might give something useful irrespective of whether the model is true (Rissanen even argues that the whole notion of a 'true model' is misguided).This model-free paradigm also leads one to define the NML distribution as minimizing prediction error in a stringent worst-case-over-all data sense (Eq.( 13)) rather than a stochastic sense.Nevertheless, it is of interest to see what happens if one samples data from a distribution for which all models under consideration are wrong, but some are quite useful in the sense that they lead to pretty good predictions.Doing this leads to rather unpleasant surprises: as first noted by Grünwald and Langford [2007], MDL (and Bayesian inference) can become inconsistent: one can give examples of {M γ : γ ∈ Γ} with countably infinite Γ and a 'true' data generating distribution P 0 such that, when data are sampled i.i.d.from P 0 , MDL will tend to select a suboptimal model for all large n -while all sub-models M γ are wrong, one of them, M γ is optimal in several intuitive respects (closest in KL divergence to P 0 , leading to best predictions under a number of loss functions), yet it will not be selected for large n.While the models considered by Grünwald and Langford [2007] were quite artificial, Grünwald and van Ommen [2017] showed that the same can happen in a more natural linear regression setting; moreover, they also showed that even if Γ is finite, although then eventually MDL will select the best sub-model, for even relatively large n it may select arbitrarily bad sub-models.De Heide [2016] shows that the problem also occurs with MDL and Bayesian regression with some real-world data sets.
It turns out that the root of the problem is related to the no-hypercompression property ( 27).If the collection of models M = γ∈Γ M γ contain the density p 0 of the 'true' distribution P 0 , then any distribution p ∈ γ∈Γ M γ will satisfy no-hypercompression relative to the true p 0 : This property the proof of all MDL consistency and rate-of-convergence results, such as those by Barron and Cover [1991], Zhang [2006a], Grünwald and Mehta [2019].
However, if the model class M does not contain the true p 0 , then, in order to prove consistency, one needs (33) to hold with the P 0 outside the brackets unchanged, but the p 0 inside the brackets replaced by p, the distribution/density in M that is closest to P 0 in KL (Kullback-Leibler) divergence (why it should be KL is explained at length by Grünwald and van Ommen [2017]).Unfortunately though, (33) does not necessarily hold with p 0 replaced by p.If it does not, MDL (and Bayesian methods, whose consistency relies on similar properties) may become inconsistent.Grünwald and van Ommen [2017], based on earlier ideas in [Grünwald, 1999, Grünwald, 2012], propose a solution that works for Bayesian universal distributions: it replaces the likelihoods p θ (z n ) for every p = p θ with p ∈ M by the generalized likelihood p η θ (z n ) for some η > 0; usually η < 1 -this η has the same mathematical function as the η appearing in (32).It turns out that with such a modification, if η is chosen small enough, a version of the no-hypercompression inequality (33) holds after all.Grünwald and van Ommen [2017], Grünwald [2012] also provide a method for learning η from the data, the 'Safe Bayesian' algorithm (note that η cannot be learned from the data by standard MDL or Bayesian methods).The recent work of Grünwald and Mehta [2019] suggests that the modification of likelihoods by exponentiating with η should work for general MDL methods as well.

PAC-MDL Bounds and Deep Learning
One of the great mysteries of modern deep learning methods in machine learning is the following [Zhou et al., 2018]: deep learning is based on neural network models which can have many millions of parameters.Although typically run on very large training samples z n , n is usally still so small that the data can be fit perfectly, with 0 error on the training error.Still, the trained models often perform very well on future test sets of data.How is this possible?At first sight this contradicts the tenet, shared by MDL and just about any other method of statistics, that good generalization requires the models to be 'small' or 'simple' (small comp(M) in MDL analyses, small VC dimension or small entropy numbers in statistical learning analyses) relative to the sample size.One of several explanations (which presumably all form a piece of the puzzle) is that the local minimum of the error function found by the training method is often very broad -if one moves around in parameter space near the minimum, the fit hardly changes.Hochreiter and Schmidhuber (1997) already observed that describing weights in sharp minima requires high precision in order to not incur nontrivial excess error on the data, whereas flat minima can be described with substantially lower precision, thus forging a connection to the MDL idea; in fact related ideas already appear in [Hinton and Van Camp, 1993].In these papers, the MDL Principle is used in a manner that is less direct than what was done thus far in this paper: we (and, usually, Barron, and Rissanen) directly hunt for the shortest description

28
A C C E P T E D M A N U S R I P T Accepted manuscript to appear in of the data.In contrast, the aforementioned authors simply note that, no matter how a vector of parameters for a model was obtained, if, with the obtained vector of parameters, the data can be compressed substantially, for example by coding first the parameters and then the data with the help of the parameters, then, if we believe the MDL principle, with these parameters the model (network) should generalize well to future data.In modern practice, neural networks are often trained with SGD (stochastic gradient descent), and it has been empirically found that networks that generalize well do tend to have parameters lying in very flat minima.
While this use of the MDL Principle seems less precise than what we reviewed earlier in this paper, it can once again be given a frequentist justification, and this justification is mathematically precise after all: the so-called PAC-Bayesian generalization bounds [McAllester, 2003] show that the generalization performance of any classifier can be directly linked to a quantity that gets smaller as soon as one needs (a) less bits to describe the parameter and as soon as one needs (b) less bits to describe the data given the parameters; both the results and their proofs are very similar to the MDL convergence results by Barron and Cover [1991], Zhang [2006a,b], Grünwald and Mehta [2019].Although in general the formulation is not as straightforward as a simple sum of the two description lengths (a) and (b), the connections between both the two-part codelength and the Bayesian codelength are quite strong, as was already noticed by Blum and Langford [2003].In particular, for discrete Θ, such PAC-Bayes bounds contain a term − log π(θ) which can be interpreted as the number of bits needed to encode θ using the codes based on some distribution π; for general, uncountable Θ, this term gets replaced by a KL divergence term that canstill be related to acodelength via a so-called 'bits back argument' pioneered by Hinton and Van Camp [1993].Dziugaite and Roy [2017], Zhou et al. [2018], inspired by earlier work by Langford and Caruana [2002], indeed show that, for some real-world data sets, one can predict nontrivial generalization using deep neural nets by looking at the number of bits needed to describe the parameters and applying PAC-Bayesian bounds.

Concluding Remarks
We have given a self-contained introduction to MDL, incorporating and highlighting recent developments.Of necessity, we had to make a choice as to what to cover in detail, and there are many things we omitted.We would like to end with briefly mentioning three additional developments.First, there has always been the question about how MDL relates to other complexity notions such as those considered in the statistical learning theory literature [Shalev-Shwartz and Ben-David, 2014]: Vapnik-Chervonkis dimension, entropy numbers, Rademacher complexity and so on.A major step towards understanding the relation was made by Grünwald and Mehta [2019] who show that for probability models with members of the form p θ (z) ∝ exp(−ηloss θ (z)), where loss is an arbitrary bounded loss function, the NML complexity can be precisely bounded in terms of the Rademacher complexity defined relative to loss.Second, we should note that Rissanen's own views and research agenda have steered in a direction somewhat different from the developments we describe: Rissanen [2007] published Information and Complexity in Statistical Modeling, which proposes foundations of statistics in which no underlying 'true model' is ever assumed to exist.As Rissanen writes, "even such a well-meaning statement as 'all models are wrong, but some are useful', is meaningless unless some model is 'true'." Rissanen expands MDL and NML ideas in the direction of the Kolmogorov structure function, taking the idea of distinguishable distributions underlying Myung et al. [2000] as fundamental; while presumably compatible with the developments we describe here, the emphasis of this work is quite different.
We end with a word about applications: since 2007, numerous applications of MDL

29
A C C E P T E D M A N U S R I P T Accepted manuscript to appear in IJMI and MDL-like techniques have been described in the literature; as discussed in Section 6.2, highly popular methods such as Lasso and Bayes factor methods can often be seen as 'MDL-like'.Even as to specific 'pure' MDL applications (such as based on NML and twopart codes), the number and scope of applications is simply too large to give a succinct representative overview.However, there is one particular area which we would like to mention specifically, since that area had hardly seen any MDL applications before 2007 whereas nowadays such applications are flourishing: this is the field of data mining.A When the original (v ≡ 1) NML is undefined: Details, Open Problems and their Solutions The original NML distribution pnml with uniform v relies on the existence of the Shtarkov integral z n ∈Z n p θml(z n ) (z n )dz n ; its asymptotic expansion (15) relies on the existence of the Jeffreys integral |I(θ)dθ being finite, the latter being equivalent to the requirement that Jeffreys' prior is proper.Both are quite strong requirements; for infinite sample spaces Y, they 'usually' -that is, in most models one considers in practice, such as normal, exponential, Poisson ... -do not hold; but once one restricts the parameter space to an INECCSI set, they generally do hold This may lead one to conjecture that the Shtarkov integral is finite if and only if the corresponding Jeffreys integral is finite.Resolving this conjecture was posed as an open problem by G07; Grunwald and Harremoës [2009], Bar-Lev et al. [2010] show that in general, the conjecture is wrong; though, for exponential families, under a very mild additional condition, it holds true.
From a more practical perspective, one would of course like to know what universal distribution to use if the standard MDL is undefined.Several proposals floated around in the early 2000s; for an overview, see Chapter 11 of G07.By now, the dominant method has become to factor in a nonuniform weight function v and calculate the luckiness NML as in (11).This method was originally called luckiness NML-2 by G07, who (among many other methods) identified several 'luckiness' versions of NML that had been proposed by various authors; luckiness NML-2 turned out both more practically useable and mathematically analyzable than other methods, and in this text we simply call it luckiness NML.In particular, Suzuki and Yamanishi [2018] show that, for exponential family models, the n-dimensional integral in the luckiness NML can be replaced by a 2k-dimensional 1, and in many cases can be performed explicitly.As we indicated in Section 2.5, one can sometimes set the first m examples aside as start-up data to define a luckiness function, leading to conditional NML.Again, G07 defined different forms of conditional NML, and again, conditional NML-2 (directly based on luckiness NML-2) turned out to be the most natural one: Bartlett et al. [2013] show that for some important classes of models, the NML distributions pnml and the Bayes marginal distributions

36
A C C E P T E D M A N U S R I P Accepted to appear in IJMI pbayes with improper Jeffreys' priorexactly, and not just asymptotically, coincide for each n.Moreover, for the case of 1-dimensional families, they completely characterize the class of models for which this holds: essentially, it holds for exponential families that are also location or scale families, i.e. the normal and gamma distributions, and monotone transformations thereof (such as e.g. the Rayleigh distributions); as well as for one curious additional family.This correspondence between objective Bayesian and conditional NML-2 approaches notwithstanding, Kojima and Komaki [2016] show that 'conditional NML-3', which G07 considered the most intuitive version, but at the same time, mathematically overly complicated for practical use, can be given a practical implementation after all, thereby solving Open Problem No. 7 of G07.

37
A C C E P E D M A N U S C R I P T Accepted manuscript to appear in IJMI Budhathoki et al. [2018]ications areVreeken et al. [2011],Koutra et al. [2015],Budhathoki et al. [2018].Most of this work centers on the use of two-part codes, but sometimes NML and other sophisticated universal distributions/codes are used as well[Tatti and Vreeken,  2008].Sumio Watanabe.A widely applicable Bayesian information criterion.Journal of Machine Learning Research, 14:867-897, Mar 2013.Tianyi Wu, Shinya Sugawara, and Kenji Yamanishi.Decomposed normalized maximum likelihood codelength criterion for selecting hierarchical latent variable models.In ACM International Conference on Knowledge Discovery and Data Mining, 2017.Y. Yang.Can the strengths of AIC and BIC be shared?A conflict between model indentification and regression estimation.Biometrica, 92(4):937-950, 2005.Tong Zhang.From -entropy to KL entropy: analysis of minimum information complexity density estimation.Annals of Statistics, 34(5):2180-2210, 2006a.Tong Zhang.Information theoretical upper and lower bounds for statistical estimation.IEEE Transactions on Information Theory, 52(4):1307-1321, 2006b.Wenda Zhou, Victor Veitch, Morgane Austern, Ryan P Adams, and Peter Orbanz.Compressibility and generalization in large-scale deep learning.arXiv preprint arXiv:1804.05862,2018.Yuan Zou and Teemu Roos.On model selection, Bayesian networks, and the Fisher information integral.New Generation Computing, 35(1):5-27, 2017.