Abstract

This paper examines the evaluation of models claimed to be relevant for policy making purposes. A number of tests are proposed to determine the usefulness of such models in the policy making process. These tests are applied to three empirical examples.

INTRODUCTION

Applied economic research produces many empirical models of various parts of the economy. The models are evaluated in a variety of ways: some will report the specification search used to reach the final model; some will employ a battery of specification tests looking at missing variables, parameter consistency or heterogeneity; some will use crossvalidation or post-sample evaluation techniques; and so forth. Discussion of these procedures and some difficulties that arise can be found in the book of readings, Granger (1990). Many applied papers, as well as some theoretical ones, will end with a section on the “policy implications” of the model. These sections rarely emphasize that strictly the policy implications only follow if the model is correct and is the actual data generating mechanism, which is an unreasonably strong assumption. It is also an assumption that cannot be true if one has two competing policy models. Of course models are built for a variety of purposes, and some are fairly easy to evaluate or to compare. For example, if two models are built to provide forecasts, they can be run in real time, after the date of the construction, and the forecasts compared using some pre-agreed cost function or criterion.