Analyzing the capabilities of the HB logit model for choice-based conjoint analysis: a simulation study

Abstract

The authors conduct an extensive simulation study to examine the capabilities of the Hierarchical Bayes (HB) logit model for choice-based conjoint (CBC) studies. The statistical performance of HB is evaluated under experimentally varying factor level settings using criteria for goodness-of-fit, parameter recovery and predictive accuracy. The results provide guidance to market researchers who are confronted with the problem that clients desire to include more and more attributes while keeping the choice task manageable. The results show that for simple CBC settings HB estimation proves to be quite robust. One of the main findings for simple CBC settings is that holding other factors at convenient levels far more attributes than previously suggested can be used in CBC studies. Further, sample size and/or the number of choice tasks per respondent can be noticeably reduced. However, for more complex CBC settings with an already high number of parameters (part-worths) but rather little information available from respondents, the HB model is starting to collapse if more than one of those factors (attributes, sample size, choice tasks) is set to an extreme level.

Appendices

Appendix 1: The HB model for choice-based conjoint analysis

The HB method can be classified as a random-effects model in which the parameters are assumed to vary across respondents according to a probability distribution. In classical random-effects models the parameters of the probability distribution can be estimated. However, it is not possible to draw inferences about individual-level parameters (e.g., Rossi and Allenby 1993; Allenby and Ginter 1995; Rossi et al. 2005). In contrast, the HB random-effects model is characterized by a hierarchical structure that allows the estimation of individual-level parameters using both information from the probability distribution of all individuals and each individual’s choice data.

At the individual level (lower level) of the hierarchical structure the probability of each individual’s choice is modeled by a multinomial logit model. The MNL model is based on the assumption that error terms are independently and identically Gumbel-distributed and can be expressed as:

$$ p_{n} (j^{\prime } ) = \frac{{\exp (\beta_{n} x_{{j^{\prime } }} )}}{{\sum\nolimits_{j = 1}^{J} {\exp (\beta_{n} x_{j} )} }}, $$

where $p_{n} (j^{\prime } )$ is the probability that the nth respondent chooses the j′-th alternative in a particular choice task, $\beta_{n}$ represents the vector of part-worths for the nth respondent, and $x_{{j^{\prime}}}$ is a dummy vector for the attribute levels of alternative $j^{\prime}$.

At the population level (upper level) the multivariate normal distribution is typically used as probability distribution (first-stage prior) to link a respondent’s individual-level preferences to the population level of the model:

$$ \beta_{n} \,\sim \,{\text{N}}\left( {\overline{\beta } ,V_{\beta } } \right), $$

where $\overline{\beta }$ represents the vector of means of the distribution of individuals’ part-worths, and $V_{\beta }$ is the covariance matrix that captures the extent of heterogeneity (as well as the correlation in the part-worths) across individuals.

To estimate the parameters $\overline{\beta }$ and $V_{\beta }$ of the prior distribution, hyperprior distributions (second-stage priors) have to be specified which are by default chosen to be very diffuse and nearly flat. It is common to assume that the vector of means follows a normal distribution and the covariance matrix is inverse Wishart distributed (cf. Rossi et al. 2005). $\beta_{n}$, $\overline{\beta }$ and $V_{\beta }$ are unknown and have to be estimated using the information from the underlying choice data. Thus the prior information is combined with the observed choice data to estimate the posterior distributions. Markov chain Monte Carlo simulation methods are used for the estimation of the posterior distributions of the parameters, as the form of the posterior distribution is not analytically tractable (e.g. Allenby et al. 1995). Using MCMC techniques random draws from the posterior distributions are generated by an iterative process where one of the three parameters is estimated at a time conditional on the values for the other two parameters (e.g. Train 2003). After a burn-in phase, the draws are finally averaged to calculate point estimates of the parameters.

Appendix 2

See Tables 6 and 7.

Table 6 F-tests of main and interaction effects w.r.t. performance measures (N = 1296; p-values in parentheses)

Table 7 Effect sizes of main effects measured by eta squared (η²)