The use of face-to-face interviews in mixed mode design: The problem of the achieved sample

Although the face-to-face mode has been losing its appeal due to declining response rates and rising costs, it nevertheless remains the most universal and effective survey mode. Therefore, it could be expected that its use in a mixed mode survey would increase the effectiveness of this design by boosting the response rate and improving the sample composition, not only in comparison with the most commonly used mix of self-administered modes, but also in comparison with unimode face-to-face surveys. The inclusion of the face-to-face mode, which is not only effective but also completely different from self-administered modes, should encourage additional categories of respondents to participate in surveys and, in this way, facilitate the achievement of objectives set for the mixed mode design. Two experiments carried out in connection with the European Social Survey (ESS) were aimed at verifying whether the face-to-face interview used in the mixed mode design did indeed produce the aforementioned benefits, compared to the use of face-to-face interviewing in unimode surveys. The idea was to boost the response rate and improve the sample composition. Both experiments were based on the sequential design with elements of the concurrent design. In the former, the face-to-face mode was used in the initial phase, whereas in the latter it was used in the follow-up phase. This mode was combined with self-administered modes, including the most effective of them, i.e. the mail mode. In the experiment aimed at verifying the effectiveness of mixed mode design with face-to-face used first, the ESS main study rounds 6 and 7 were treated as an initial phase. In the follow-up, the interviewers suggested that refusers and non-contacted persons/their proxies take part in the survey, offering a choice between the mail and the EMS (email) mode. The face-to-face phase achieved a high response rate of 74.9% in ESS 6 and 65.8% in ESS 7. The use of self-administered modes in the follow-up increased the response rate by only a fraction of a percentage point in both experiments. Relatively few nonrespondents/proxies agreed to participate in the follow-up, and the completed questionnaires were finally sent back by relatively few of them. In our second experiment, face-to-face interviewing was used in the follow-up phase, with self-administered modes used first. The self-administered modes used included regular mail, web and EMS modes. Despite using a rigorous research design in both phases, the obtained response rate was lower by nearly 10 percentage points in comparison with that achieved in the ESS 7 main study carried out in parallel. Although face-to-face proved to be the most * This article elaborates on the analyses presented in: Paweł B. Sztabiński. 2018. Does Mixed Mode Data Collection Improve the Achieved Sample? A Comparison of the ESS 7 and a Mixed Mode Experiment. In: Franciszek Sztabiński, Henryk Domański, and Paweł B. Sztabiński (eds). New Uncertainties and Anxieties in Europe. Seven Waves of the European Social Survey. Berlin: Peter Lang, pp. 247–283. Človek a spoločnosť [Individual and Society], 2019, Vol. 22, No. 4, pp. 1-28. The use of face-to-face interviews in mixed mode design: The problem of the achieved sample 2 effective mode in the experiment, the response rate achieved when using it was rather low, and exceeded that achieved in the initial self-administered phase only by a few percentage points. Analysis of the sample composition obtained in the mixed mode design using face-to-face in the initial phase and self-administered modes in the follow-up (Experiment I) seemed pointless. A negligible increase in response rate (a fraction of a percentage point) compared with the initial phase cannot significantly affect the total mixed mode sample distributions. The analysis of sample composition in the second experiment, based on the reversed sequence (Experiment II, i.e. self-administered modes first), covered six socio-demographics: respondents’ sex, age, size of domicile (no. of inhabitants), level of education, number of people in the household and the main activity. Their distributions were compared with those achieved in the ESS 7 main study and with the statistical data for the total population. The mixed mode sample did not differ from the ESS 7 main study sample in terms of five out of six characteristics included in the analysis. A difference was only found in the case of respondents’ main activity and is related to the much higher participation of permanently sick or disabled individuals in the mixed mode sample. This means that using a mix of these modes (self-administered and face-to-face, in this sequence) does not improve the sample composition in terms of socio-demographics. In the case of sex and size of domicile (no. of inhabitants), it is not reasonable to expect an improvement, as the distributions of these characteristics obtained in the ESS 7 are very close to those for the total population. However, the distributions of age and level of education differ more from the population, while the distribution of the number of people in the household differs very significantly. Therefore, both samples differ from the total population in a similar way. On the other hand, the use of such a design, despite a lower response rate, did not lead to deterioration of the sample composition. The similarity of the total mixed mode sample and the sample achieved in the ESS 7 main study results from using the face-to-face mode in the follow-up. For all six analysed characteristics, the sample composition obtained in the initial self-administered phase differs more from the sample achieved in the ESS 7 main study than does the total mixed mode sample from the ESS 7 sample. The greatest, statistically significant differences, were found in the size of domicile (no. of inhabitants) and level of education, whereas smaller differences were found in the sample composition in terms of age. This suggests that if self-administered modes are used in the initial phase, then the use of the face-to-face mode in the follow-up is conducive to selection effects and improvements in the composition of the total mixed mode sample. The findings from both experiments may undermine the sense of using a mixed mode design with the face-to-face mode used both in the initial phase, and in the follow-up. Application of the face-to-face first design only generates additional costs related to the implementation of the self-administered phase, with no clear outcomes such as an increase in the response rate. In turn, the application of self-administered modes first and face-to-face in the follow-up does not bring the effects expected from the mixed mode design, not only in terms of response rate, but also in terms of sample composition. Moreover, it fails to achieve another postulate behind the mixed mode design, i.e. reduction of survey costs. The cost of a single case in this experiment was nearly identical to that in the ESS 7, completed entirely face-to-face. It was equivalent to approx. EUR 40. However, the results of both experiments could have been affected by the use of a rigorous research design in the face-to-face phase. Both in the ESS main study (initial phase in Experiment I), and in the face-to-face follow-up in Experiment II, the interviews were carried out by experienced, highly motivated interviewers, each sampled person received an unconditional gift, and the fieldwork lasted several months. This design also drove up the costs of both surveys. However, a rigorous research design is applied rarely, even in academic surveys. For this reason, the final section of the article discusses potential consequences for the results of a mixed mode survey if a standard research design were to be used in the face-to-face phase instead of a rigorous design. Other potential reasons behind the results obtained in both experiments are also discussed.


Research problem
The increasingly more common use of mixed mode data collection is considered to be one of the major survey trends in the early 21st century ). According to Robert M. Groves (2011: 869) most survey methodologists agree that "we are moving to a future of mixedmode surveys". A mixed mode design involves the use of different modes in relation to different respondents from the same sample in order to collect the same data. 1 Its application "gives an opportunity to compensate for the weaknesses of each individual mode at an affordable cost" (De Leeuw, 2005: 235). The key weakness of contemporary surveys lies in their declining response rate (see, for instance, de Leeuw & de Heer, 2002;de Leeuw, Hox, & Luiten, 2018;Curtin, Presser, & Singer, 2005;Brick & Williams, 2013;Christian et al., 2012) and, as a result, the achieved samples in probability surveys are increasingly self-selected (Miller, 2017). Therefore, the main purpose of mixed mode data collection is to improve the response rate and reduce the nonresponse error when compared to the unimode approach (de Leeuw, 2005;de Leeuw, Hox, & Dillman, 2008;Dillman, Smyth, & Christian, 2014). Another factor of importance is the reduction of survey costs which increase as researchers attempt to reduce the nonresponse rate. These costs are related to the very widespread use of advance letters and respondent incentives, as well as the need to make repeated contact attempts with hard-to-get respondents. Apart from the declining response rate, the rising survey costs have also boosted the appeal of mixed mode data collection (Tourangeau, 2017). The belief of survey researchers that the use of mixed mode data collection allows them to accomplish the aforementioned goals is based on fairly simple premises. Since people have their preferred mode of survey participation (see, for instance, Olson, Smyth, & Wood, 2012;Groves & Khan, 1979), the use of different modes should encourage participation among additional categories of respondents who are underrepresented in unimode surveys. A mixed mode approach increases the chances that the respondents will find their preferred mode among the modes offered in the survey. As a result, the response rate should go up and the sample composition should be closer to the population (see, for instance, Dillman, West, & Clarc, 1994;Shih & Fan, 2007;Diment & Garrett-Jones, 2007;de Leeuw, Hox, & Dillman, 2008).
The mixed mode surveys based on these premises differ primarily in terms of the modes of data collection used, as well as the research design applied (concurrent vs. sequential) and, in the case of a sequential design, the sequence in which the different modes are used. However, in research practice, the most commonly used modes include the web and mail modes and, less often, telephone interviewing, i.e. the cheapest surveying modes. This is understandable given that one of the important goals of a mixed mode design is, as mentioned earlier, to reduce survey costs. However, if we limit ourselves to those modes only, this will partially challenge the assumptions behind this procedure. As self-administered modes, web and mail modes are closely akin to each other. In both these modes, the respondents are expected to complete the questionnaire by themselves. On the other hand, the use of CATI mode usually produces a very low response rate. From this point of view, it is not surprising that with their joint use the increase in response rate is not large when compared to a unimode survey (Villar & Fitzgerald, 2015Shih & Fan, Človek a spoločnosť [Individual and Society], 2019, Vol. 22, No. 4, pp. 1-28. The use of face-to-face interviews in mixed mode design: The problem of the achieved sample 4 2007; Messer & Dillman, 2011;Dillman, Smyth, & Christian, 2014), and the selection effects 2 are relatively weak, without much improvement in the sample composition Wardle & Robinson, 2007;Villar & Fitzgerald, 2015. Considering these assumptions behind the mixed mode design, one should expect that the joint use of more varied modes, which are also more effective in terms of the response rate, will enable researchers to achieve the essential goals of this design, i.e. an improvement in both the response rate and the sample composition in comparison with a unimode survey. Apart from the selfadministered modes, used normally in mixed mode designs, the idea is to use also the face-toface mode, which remains the most effective interviewer-mediated mode (de Leeuw, 1992;Holbrook, Green, & Krosnick, 2003). While the application of this design significantly raises the costs of a mixed mode survey, one can nevertheless expect that they will be lower in comparison with a face-to-face survey based on a unimode design. In this paper, based on the results of mixed mode experiments conducted in Poland in connection with the European Social Survey (ESS), I raise the following questions: (1) does the use of a face-to-face mode in a mixed mode survey, next to self-administered modes, increase the response rate, and (2) will the use of this mode improve the sample composition? However, the effectiveness of data collection modes cannot be analysed in separation from the research design applied, i.e. concurrent vs. sequential. Many studies have shown that the use of a concurrent research design, i.e. providing the respondents with different options for participation in a survey, does not generally lead to a higher response rate compared to unimode, at least for the mail and web modes (Lozar Manfreda, Vehovar, & Batagelj, 2001;Shih & Fan, 2007;Dillman, West, & Clark, 1994;Millar & Dillmanm, 2011;Smyth et al., 2010;Medway & Fulton, 2012). On the other hand, the response rate can be effectively increased when the same modes are applied sequentially, i.e. one of them is used as the initial mode and then the respondents are switched to another mode, i.e. nonrespondents are offered participation in the same survey via another mode (Shettle & Mooney, 1999;Dillman et al., 2009;Millar & Dillman, 2011). For this reason, the experiments discussed here were based on a sequential design, in order to test the use of the face-to-face mode in the initial phase and in the follow-up phase.

Previous research findings
As mentioned earlier, the face-to-face mode is used very rarely in mixed mode designs. However, it has been applied in, among others, a series of experiments conducted in connection with the ESS. 3 In an experiment conducted in the Netherlands in parallel with the ESS round 4 (main study), the web and telephone modes were also applied along with the face-to-face mode. Two designs were used, i.e. concurrent and sequential, in the following sequence: web survey, telephone interview, 2 Selection effects occur when the respondents who choose different modes differ from one another (Vannieuwenhuyze, Loosveldt, & Molenberghs, 2010). This is a key assumption when using mixed mode data collection. If this effect does not occur, then the unimode sample will be equally representative for the population, and the use of the mixed mode will no longer be justified (Vannieuwenhuyze, 2014;Revilla, 2010). 3 These studies covered a broad range of issues: apart from response rate, sample composition, and survey costs, they also covered, inter alia, mode effects (see Jäckle et al., 2006Jäckle et al., , 2008Jäckle et al., , 2010Roberts et al., 2008;Eva et al., 2010;Martin, 2011, Martin & Lynn, 2011, a summary of those studies in: Villar & Fitzgerald, 2015. Since the experiments presented later in the article were conducted in connection with the European Social Survey (ESS), this section presents mostly the results of studies connected with that project. This helps to eliminate the effect of the survey topic on the results, and makes the comparisons of results more valid. As shown by Groves, Presser, and Dipko (2004), the survey topic has a considerable impact on survey participation decisions. followed by face-to-face interviews. 4 The response rates 5 in both experimental designs were very close (45.9% and 45.0%), but lower than in comparison with the response rate obtained in the ESS main study, conducted as a unimode face-to-face study (52.0%). Regardless of the design used, web survey turned out to be the most effective mode.
The use of the mixed mode design did not improve the composition of the achieved sample in comparison with the ESS main study: both samples were very close and different in a similar way from the Dutch population overall. However, the similarity of both samples resulted from the use of telephone and face-to-face modes. The composition of the sample achieved using web only (in a sequential design) differed significantly from that obtained in the ESS main study. This seems to indicate that the use of the face-to-face mode in the follow-up step is conducive to improvements in sample composition, at least when the web mode is used in the initial phase. In the mixed mode approach, irrespective of its design (sequential or concurrent), the cost of one case was lower by about 2/3 when compared to the ESS 4 main study (Eva et al., 2010). Subsequent mixed mode experiments within the ESS mixed mode methodology programme were conducted in Estonia, the United Kingdom and Sweden. Each country implemented the design it deemed most appropriate given the country's survey environment. However, the design included the face-to-face mode in all cases. Estonia applied a sequential design with the web mode in the initial phase. The response rate achieved in the experiment was nearly identical with that in the ESS main study conducted fully on a face-to-face basis (66.2% vs. 67.8%). The web mode, used in the initial phase, turned out to be the most effective. The sample composition achieved in the experiment was very close to the ESS main study sample and to the statistical data for the population. But again, as in the Netherlands, this result was achieved by using the face-to-face mode in the follow-up phase. As for the cost per case in the experiment, it was-contrary to expectations-very similar to the cost in the face-to-face ESS main study (Ainsaar et al., 2013). Partially different results were obtained in an experiment carried out in the United Kingdom, where the design was identical to that applied in Estonia. The response rate in the mixed mode experiment was only 39% compared to 55% in the ESS main study. Nevertheless, the composition of the samples achieved were similar to each other in both surveys and generally comparable to the general population data. In the experiment carried out in Sweden, two sequential designs were used: response enhancing and cost-effective. In the first one, face-to-face was used as the initial mode, with telephone interviews as the follow-up mode. In the cost-effective design, the reverse order was applied. As in the UK, the response rate in the experiment was significantly lower than in the ESS main study carried out in parallel. In the response-enhancing design, the response rate was 38%, with 50% in the ESS main study (Villar & Fitzgerald, 2015. The results of the experiments presented here, although not entirely consistent, suggest that the use of the mixed mode design with face-to-face interviews leads to a reduced response rate compared to a unimode face-to-face survey, rather than to an increase in the response rate, as might have been expected. The sample composition is not improved, either. Such a result occurred regardless of the design used in mixed mode (concurrent vs. sequential) or the use of other modes (except face-to-face). As for the comparison between use of the face-to-face mode in the initial and the follow-up phase, the results of the aforementioned Dutch experiment indicate that the application of this mode in the follow-up phase renders slightly better results. We should bear in mind that in part of the randomly drawn sample (where it was not possible to ascertain the phone number), the sequential design was used, with the face-to-face mode in the initial phase (see footnote 4). The response rate in this group was lower by more than five percentage points in comparison with the sequential design used in the same experiment, where the face-to-face mode was used in the follow-up phase (after web and telephone interviewing). Moreover, the sample achieved using the face-to-face mode first differs much more from the sample achieved in the ESS main study than the sample achieved using the face-to-face mode in the follow-up phase. The costs of using face-to-face mode first were also significantly higher compared to using face-to-face in the follow-up phase (Eva et al., 2010;Villar & Fitzgerald, 2015). Partially different results were obtained in an experiment conducted in connection with the Raising Healthy Children (RHC) survey in the state of Washington. The participating young people, about 18 years old, were subdivided into two groups, where face-to-face and web modes were used in a different order. The response rate in both groups was similar, although slightly higher when using face-to-face first (94% vs. 90%). The distribution of answers to substantive questions (the authors do not reveal the socio-demographics) was very close in the case of both designs. However, the costs per case were much higher when using face-to-face first (McMorris et al., 2009). These experiments are not fully comparable due to the use of different sets of modes, different survey topics, different populations, as well as the longitudinal character of the RHC survey. This may be the underlying cause of the discrepancy in their results. However, they suggest that the order of using the face-to-face mode (in the initial phase vs. the follow-up phase) does not translate into significant differences in the response rate achieved, although it may translate into differences in sample composition. The results of both experiments are convergent in terms of survey costs. Using the face-to-face mode in the initial phase generates much higher costs compared to using web mode first. This is understandable because with this design the vast majority of cases were completed on the face-to-face basis.

The European Social Survey in Poland
Much like the majority of cross-national time series surveys, the European Social Survey (ESS) has been conducted using the face-to-face mode on a probabilistic, general sample of respondents aged 15+. In Poland, this is a sample of individuals. The interview takes over one hour to complete.
The experiments presented below were completed in connection with rounds 6 and 7 of the ESS. The 6th round of the ESS was carried out on a sample of 2,706 individuals during the period from the 19 th of September 2012 to the 8 th of January 2013. The sample for the 7th round comprised 2,715 individuals. The survey was conducted from the 20 th of April to the 31 st of August 2015.
The fieldwork was carried out by the Centre for Sociological Research (CSR), affiliated to the Institute of Philosophy and Sociology of the Polish Academy of Sciences. The CSR specialises in fieldwork for academic research.
The ESS survey was conducted according to a rigorous research design, aimed at maximising the response rate. The interviews were carried out by experienced and highly motivated interviewers, additionally trained in refusal avoidance and refusal conversion. The vast majority of the interviewers had also participated in the earlier ESS rounds. Their work was systematically monitored throughout the fieldwork period.
Two advance letters were sent to the sampled persons. The first one contained an insert presenting an interesting finding from the previous ESS round. This was intended to generate the respondents' interest in participating in the survey and to show that participation does not require any special competences. The purpose of the second letter was to maintain the motivation of the sampled persons to participate in the survey. Each sampled person received an unconditional gift. Also, a dedicated website for the respondents was developed, including, among other things, information on issues that the interviewers felt could be of interest to the respondents or could raise concerns. The response rate in the ESS 6 main study was 74.9%, while the response rate in ESS 7 was 65.8%.

Experiment I
The aim of the first type of experiment was to check to what extent a mixed mode design using face-to-face in the initial phase and self-administered modes in the follow-up phase would boost the response rate and improve the composition of the achieved sample. Mail and EMS were used as self-administered modes in the concurrent design. 6 Next to face-to-face interviewing, mail seems to be the most universal mode of data collection. Research has shown that regardless of their preferred survey mode, respondents are more likely to participate in mail surveys than in web and telephone surveys (Olson, Smyth, & Wood, 2012;Dillman, Smyth, & Christian, 2014). On the other hand, the need to create the possibility to participate in surveys with the use of an electronic questionnaire was suggested by the results of the aforementioned ESS experiments where the largest percentage of cases was completed with the use of the web mode. In the experiments discussed in this part of the article, EMS was used due to the higher costs of using the web mode. The use of EMS was also prompted by the high response rate achieved with this mode in a mixed mode survey among physicians, co-conducted by the CSR. 7 The experiment was carried out in connection with ESS main study in rounds 6 and 7 (face-toface). An identical design was used in both cases. If the respondent refused to take part during the ESS fieldwork, the interviewer asked the respondent to agree to participate in the survey using the mail or EMS mode. The same procedure was followed when the respondent was inaccessible, but the interviewer managed to contact a proxy. If the respondent (proxy) chose the EMS, the interviewer asked for an e-mail address to which the questionnaire would be sent. 6 The Electronic Mail Survey method (EMS) consists in sending a questionnaire attached to an e-mail message distributed among the respondents. 7 That survey was conducted by the Polish Chamber of Physicians and Dentists with the Centre for Sociological Research at the Institute of Philosophy and Sociology in 2012. The respondents who agreed to take part in the survey were first offered a choice of mail, telephone and EMS modes and then asked about their preferred mode. The largest number of the respondents opted for the EMS (58.3%), whereas the traditional mail questionnaire was selected considerably less frequently (36.5%). While the highest percentage of returned questionnaires was obtained from the mail mode (84.1%), the rate achieved from the EMS mode was also satisfactory, reaching 70.9% (Sztabiński, 2013).
The questionnaires were sent to (non)respondents successively, as information about refusals and non-contacts was received, which was intended to prevent a decline in motivation to participate in the survey. After two weeks, a reminder/thank you letter was sent to the (non)respondents.
The questionnaire used in the experiment was identical to the one used in ESS 7, except for some necessary modifications resulting from the mode of data collection.

Experiment II
The second experiment was intended to check the effects of the mixed mode design using the face-to-face mode in the follow-up phase (self-administered modes first). In the initial phase, the concurrent design was applied, i.e. the respondents were offered the option of participation in the survey via regular mail, web or e-mail (EMS). After the face-to-face follow-up phase, a persuasion letter was mailed to selected nonrespondents, along with a re-attached questionnaire (the second follow-up).
The experiment was a stand-alone survey, conducted in parallel to the ESS 7 main study, between 7 April and 30 September 2015. A sample of 800 individuals aged 15+ was drawn using the same sampling frame as in ESS 7, and in exactly the same way. In order to reduce the cost of the experiment, the individuals were drawn from the same locations as the ESS 7 sample.
There were a few reasons for using the concurrent design in the first phase instead of a sequential design (facilitating a higher response rate) with the web mode in the initial phase. First, much as in the ESS 7, we had access only to the sampled persons' names and home addresses during the experiment. As a result, the first contact with them had to be made by regular mail. According to the results of methodological research, this is not conducive to achieving a high response rate using the web mode (Shish & Fan, 2007;Millar & Dillman, 2011). The second reason for the use of the concurrent design in this phase was related to the use of the general sample, without setting the upper age limit. According to Eurostat data, access to the Internet was reported by 79% of households in Poland in 2015 (Eurostat. Statistics Explained, 2016) but, much as in other countries, the elderly and less educated are less likely to use the World Wide Web. If such persons received an invitation to participate in a web survey, some of them could decide that the survey sponsor (the Polish Academy of Sciences) and the surveying agency had completely no idea about their Internet use, and they might have even considered such an invitation as inappropriate. 8 Of course, this could translate into refusals during the application of the other modes, including the face-to-face interview in the follow-up phase. And finally, some research results (e.g. Millar & Dillman, 2011;Villar & Fitzgerald, 2015 indicate that a sequential design does not significantly boost the response rate when compared with the concurrent design. The experiment followed a rigorous research design. The first mailing contained a cover letter, the mail questionnaire, a stamped return envelope and, as in the ESS 7 main study, an insert presenting an interesting result from the previous ESS round. The cover letter invited the sampled person to participate in the survey and, apart from filling in the questionnaire, offered an opportunity to complete a web questionnaire (URL address, login and password were provided) and the opportunity to submit an electronic survey questionnaire by e-mail (EMS). However, people opting for this last mode, had to contact the research institute to provide their e-mail address. The mailing also included an incentive: a notepad with a magnet (to be placed, e.g., on a refrigerator). The results of many studies indicate that the use of incentives is particularly effective in boosting the response rate in the mail mode (for review, see Fox, Crask, & Kim, 1988;Church, 1993;Jobber, Saunders, & Mitchell, 2004;Mercer et al., 2015) as well as in the web mode (Parsons & Manierre, 2014; for review, see Göritz, 2006). A special website was developed for the respondents participating in the experiment, partly different from the website designed for the ESS 7 main survey respondents.
After one week, a reminder/thank you letter was sent to all the respondents. After another week, interviewers attempted to conduct a face-to-face interview with nonrespondents from the first phase of the experiment (follow-up 1). These were the same interviewers who carried out the ESS 7 main survey.
Once the fieldwork was completed by the interviewers, selected nonrespondents were sent a persuasion letter by mail, along with the questionnaire again and a stamped return envelope (follow-up 2). A total of 312 letters were sent altogether.
The questionnaire used in the experiment was identical to the one used in ESS 7.

Experiment I
In the ESS 6 main study, there were a total of 371 refusals (301 by respondents and 70 by proxy) and 133 non-contacts with the respondent (504 cases in total). The interviewers had the opportunity to offer participation in the survey using self-administered modes to 340 people (67.5%). No participation was offered in the case of aggressive refusals, if the respondent simply slammed the door when asked to participate in the ESS, etc., and also if no member of the respondent's household could be contacted. In total, 46 respondents/ proxies declared their willingness to participate in the survey via self-administered modes, i.e. 13.5% of those approached.
In the ESS 7 main study, the interview was not completed due to refusal in 460 cases (381 by respondent and 79 by proxy). The interview was not completed in 221 cases due to non-contacts. The interviewers offered participation in the follow-up phase to 339 people (49.8%). A total of 49 persons agreed (14.4%).
The results obtained in these experiments are shown below (Table 1) Človek a spoločnosť [Individual and Society], 2019, Vol. 22, No. 4, pp. 1-28. The use of face-to-face interviews in mixed mode design: The problem of the achieved sample In ESS 6 as a result of applying follow-up 16 completed questionnaires were obtained in all: 13 by use of mail and 3 by using EMS. In comparison with the ESS 6 main study the response rate went up by 0.6 percentage points, to 75.5%. In ESS 7 the effectiveness of follow-up was very close: 8 questionnaires were obtained. Of these 6 were gained by use of EMS and 2 by using mail. The response rate increased by 0.3 of a percentage point, to 66.1%.
The results obtained in both mixed mode experiments suggest that when the face-to-face mode is used in the initial phase, the use of self-administered modes in the follow-up is completely ineffective. Although between 10 and 20 percent of nonrespondents agree to complete the questionnaire in fact only some of them send it in. As a result, the response rate increases only by a fraction of a percentage point compared to a unimode face-to-face design. Therefore, it is pointless to analyse the impact of such design on sample composition. The aforementioned conclusion concerning the effectiveness of using additional modes after the face-to-face in the initial phase is confirmed by the results of the POLPAN Polish Panel Survey, carried out in 2013. The response rate in the face-to-face survey was 66.2%. In the follow-up, telephone interviews were used alongside self-administered modes (mail and EMS). Attempts were made to contact not only refusers and non-contacted persons, but also people who had gone abroad for a long time or who had changed their place of residence in Poland, and the interviewer had not managed to ascertain the new address. 9 Out of 203 people to whom the questionnaire was sent (by traditional or electronic mail) or who were contacted by telephone to conduct an interview, only 18 cases (8.9%) were completed. The response rate rose to 66.7%, i.e. by 0.5 of a percentage point.

Response rate
Of the sample of 800 people in the mixed mode experiment, a total of 419 cases were successfully completed. The response rate, calculated according to the rules adopted in the ESS, was therefore 55.9%. Thus, it was nearly 10 percentage points lower than the response rate achieved in ESS 7, which was 65.8%. This result suggests that using a mixed mode design with face-to-face in the follow-up and self-administered modes first, does not only fail to improve the response rate compared to a unimode face-to-face design but, in fact, leads to a significant deterioration.
The table below (Table 2) shows the percentages of completed cases, depending on the mode used. It turned out that the face-to-face interview, used in the follow-up, was the most effective mode. The response rate for this mode was 30.8%, with a share of 55.2% of the total number of cases. Even taking into account that the interviewers attempted to conduct interviews with nonrespondents from the self-administered phase, this result is far from satisfactory. The response rate achieved using the self-administered modes in the initial phase was lower by merely 5.7 percentage points and stood at 23.6%, or at 25.1%, if we include the questionnaires returned after the persuasion letter was sent.

Sample composition
The lower response rate in the above experiment compared with ESS 7 does not determine whether the composition of the sample achieved using a mixed mode design is "worse" or "better" than that obtained in the main study, i.e. whether the resulting distribution of key sociodemographics differs more or less from the distribution of these variables in the total population. As Robert Groves (2006) shows, there is no direct relationship between the response rate and the non-response bias in estimates. Non-response bias occurs when there is a covariance between a survey variable and response propensity (the likelihood of responding). While a higher response rate usually reduces the risk of non-response bias (cf. also Biemer & Lyberg, 2003: 213), it may even elevate this risk in some cases. In this part of the analysis I address the following issues: (i) Does the use of a mixed mode design with the face-to face mode in the follow-up phase (self-administered modes first) improve the sample composition in comparison with the ESS 7 main study? (ii) Does the use of the face-to-face mode in the follow-up phase improve the composition of the achieved sample versus the initial phase based on selfadministered modes? In order to assess whether the mixed mode design used in the experiment improves the sample composition compared to a unimode face-to-face survey (ESS 7), I first check whether the samples differ. Using the Chi-square test, I test a hypothesis (H0) that the total mixed mode sample (with self-administered and face-to-face modes) is similar to the sample achieved in the ESS 7 main study (face-to-face unimode). Then I compare both samples with the total population statistics using dissimilarity indices. 10 The effects of using the face-to-face mode in the follow-up phase for sample composition could be formulated as the following null hypothesis: the differences between the sample composition obtained in the initial phase of the experiment (using only self-administered modes) and that obtained in the ESS 7 main study are similar to the differences between the total mixed mode sample (also using the face-to-face mode) and ESS 7. In order to check if the face-to-face mode has improved the sample composition, I compare the initial phase sample and the total mixed mode sample (in terms of key socio-demographics) with ESS 7 using dissimilarity indices. Additionally, I use the Chi-square test to check whether the distributions of these variables differ in a statistically significant way in the samples achieved in two parts of the mixed-mode experiment: in the initial phase sample (using self-administered modes) and in the part of the sample that was completed on the face-to-face basis. A result indicating no significant differences between these samples will suggest that the use of the face-to-face mode in the follow-up phase has not improved (and has not worsened) the sample composition.
In the analyses, the persuasion letter with an attached mail questionnaire (follow-up 2see Table  2) sent to nonrespondents was treated as another reminder. The questionnaires returned as a result of this reminder were thus included in the sample achieved with the use of self-administered modes. In data analysis, 'don't know' answers and missing responses were omitted. Since the sample in the ESS 7 in Poland and in the experiment was drawn with unequal probability in terms of size of domicile, the data were weighted by this variable.
The sample composition was analysed in terms of six socio-demographics: respondents' sex, age, size of domicile (no. of inhabitants), level of education, number of people in the household and the main activity. The results of the Chi-square test for the comparison of the sample achieved in the mixed mode experiment and the ESS 7 sample, and a comparison of the sample obtained in the initial phase of the experiment (using self-administered questionnaires) with the sample achieved in the follow-up phase (face-to-face) are given in Table 3. On the other hand, the tables contained in the Annex show, for each of the analysed variables, distributions in the mixed mode experiment (in the initial phase and in the total sample, i.e. taking into account the face-to-face mode in the follow-up phase), in the ESS 7 main study and in the total population aged 15 or more. Table 3: Chi-square values for the comparison of the sample composition obtained in the mixed mode experiment with the ESS 7 main study sample, and the comparison of the sample achieved in the initial phase of the experiment (using self-administered modes) with the sample achieved in the follow-up based on the face-to-face mode. The respondent distribution by sex in the total mixed mode sample is very close to that obtained in ESS 7 (see Table 3 above, first column), and both samples differ very little from the statistics for the total population (see Table 1 in the Annex). The dissimilarity indices are 1.4 and 2.1, respectively.
In the sample achieved in the initial phase of the experiment, carried out with the use of selfadministered modes, there was a small predominance of women and an underrepresentation of men in comparison with the ESS 7 main study. The value of the dissimilarity index was 3.2. As a result of using the face-to-face mode in the follow-up, the value of this index decreased to 0.7. However, the Chi-square test indicates that the discrepancies between the gender structure in the self-administered modes and in the follow-up sample are not statistically significant (see second column in Table 3 above). This suggests that using the face-to-face mode in the follow-up phase did not improve the sample composition significantly.
A comparison of the age structure in the entire sample achieved in the experiment against the ESS 7 sample using the Chi-square test shows that they are similar. A statistically significant difference occurs in the 75+ age group, although only at a significance level of 0.10 11 (the Z test result is indicated in Table 2 in the Annex in the column entitled 'Total sample'). Both these samples are also not very different from the total population. The values of dissimilarity indices are close: 6.9 and 5.0. Therefore, the use of the mixed mode design did not improve the sample composition in terms of age in comparison to the ESS 7 main study, although it did not worsen it either. The age structure obtained using self-administered modes in the initial phase of the experiment is quite different from the structure in ESS 7 ( Table 2 in the Annex). The value of the dissimilarity index, which was 10.6, was halved to 5.3 as a result of using the face-to-face mode. The results of the Z test indicate that in the self-administered sample, compared to the total mixed mode sample, there were significantly fewer elderly people, aged over 75, and slightly fewer young people, although the latter result is only significant at the level of 0.10 (see 'Initial phase' column in Table 2 in the Annex). The Chi-square test indicates that the differences in age distribution between the self-administered modes and the face-to-face mode are not, however, significant at the level 0.05. In contrast, they are statistically significant at 0.10. This suggests that using the face-to-face mode in the follow-up phase has, to some extent, improved the age composition compared to the initial phase.
The sample composition obtained in the experiment is close to that of ESS 7, also in terms of the size of domicile (number of inhabitants). The distributions in both these samples are close to the distribution in the general population, as indicated by the relatively low and close dissimilarity indices (see Table 3 in the Annex). Therefore, also in the case of this characteristic, the use of the mixed mode design with face-to-face interviewing in the follow-up phase did not improve the sample composition.
However, the sample achieved in the initial phase (using self-administered modes) is significantly different from that obtained in the ESS 7 in terms of the size of domicile: the dissimilarity index value is 17.2. The use of face-to-face interviewing in the follow-up phase clearly improved the composition of the whole sample in the experiment, reducing this value to 3.8. As one would have expected, the sample achieved in the self-administered phase has an underrepresentation of rural dwellers (the difference exceeds 16 percentage points), while the urban population is overrepresented, especially people from the largest cities, those with over 500,000 inhabitants. Therefore, it is not surprising that the Chi-square test result indicates that the size of domicile distributions obtained in the self-administered modes and the face-to-face mode differ at a significance level of 0.000. This suggests that the assumed mixed mode selection effects occurred in terms of this characteristic, and that the aforementioned similarity between the total mixed mode sample and the sample achieved in the ESS 7 results from the use of the face-to-face mode in the follow-up phase.
A comparison between the level of education distribution in the total mixed mode sample and in the ESS 7 sample shows that they are similar. They are also fairly close to the total population. The values of dissimilarity indices are, respectively, 6.2 and 5.3 (see Table 4 in the Annex). Thus, the level of education is another characteristic where the use of mixed mode design did not improve the sample composition in comparison with the ESS main study. 12 The distribution of the level of education obtained in the initial phase of the experiment differs significantly from ESS 7. The value of the dissimilarity index is 14.9. As a result of using the face-to-face mode, this value decreased to 1.8 in the total mixed mode sample. The sample achieved using self-administered modes differs in a statistically significant way from the total mixed mode sample in nearly all categories of education, with the exception of incomplete primary education. The greatest differences are found in the category of primary or the first stage of basic education, although there are also significant differences in other categories: postsecondary, not tertiary, and tertiary education. In all these cases, the total mixed mode sample is closer to the ESS 7 sample than to the sample achieved in the initial, self-administered phase.
The result of the Chi-square test suggests that the differences in the distribution of the level of education in the initial self-administered phase and in the follow-up phase are significant at the level of 0.000. Therefore, the selection effect occurred also in the case of level of education, and the use of the face-to-face mode in the follow-up phase significantly changed the distribution of the total sample in the experiment, bringing it closer to the ESS 7 sample.
Another variable analysed is the number of people in the household. The compositions of the sample achieved in the experiment and in ESS 7 in terms of this variable are very close. However, these compositions differ significantly and to a similar extent from the general population. The dissimilarity indices are 11.2 and 13.4 respectively ( Table 5 in the Annex). 13 Therefore, using a mixed mode design with face-to-face interviewing did not improve the sample composition compared to ESS 7 also in terms of this characteristic. The sample achieved through self-administered modes in the initial phase does not differ much from the ESS 7 sample (the dissimilarity index value is 5.5). Although after the application of the face-to-face mode the total mixed mode sample became even closer, yet the Chi-square test indicates that the discrepancies between the two samples from the experiment (the self-administered phase and the face-to-face phase) are statistically insignificant even at the level of 0.10. Thus, the use of the face-to-face mode did not affect the sample composition in terms of the number of people in the household.
The last variable included in the analysis is the respondent's main activity. The Chi-square test indicates that the differences between the total mixed mode sample and the ESS 7 sample are statistically significant. This may be due to a much larger share of 'permanently sick or disabled' respondents in the former sample, which is confirmed by the result of the Z test (see Table 6 in the Annex).
Unfortunately, no data are available on the main activity for the Polish population aged 15 years and over. It is therefore difficult to determine which of the samples (the experiment sample or the ESS 7 sample) is closer to the country's total population.
The use of the face-to-face mode in the follow-up phase slightly improved the composition of the total mixed mode sample in comparison with the initial phase, if we take the ESS 7 main study as a reference point. The value of the dissimilarity index decreased from 8.1 to 4.4. As a result of adopting the face-to-face mode, the share of persons in paid work and of the permanently sick or disabled decreased (although the latter difference is significant only at the level of 0.10), while the share of the retired population went up (the difference is close to 4 points, not significant). However, the Chi-square test indicates that the differences between the two samples from the experiment (achieved in the initial phase and in the follow-up phase) are statistically insignificant. Thus, it can be concluded that the improvement of the sample composition achieved by using the face-to-face mode was not significant.
Although, as mentioned earlier, no data are available on the main activity for the Polish population, we nevertheless have some data from the Labour Force Survey (LFS) in Poland for the category described as 'In paid work'. According to these data, the share of people working during the ESS 7 fieldwork period was 51.1% (Labour Force Survey in Poland, Q1 of 2015). If we use these data as a point of reference, then the category 'In paid work' is close to the results obtained by using self-administered modes in the initial phase, while its share in the total mixed mode sample is somewhat underrepresented. The difference is 3.7 percentage points. This may suggest that the use of face-to-face interviewing in the follow-up phase led to slight deterioration of the sample composition when it comes to the participation of working people.

Summary and discussion
Although the face-to-face mode has been losing its appeal due to the declining response rate and rising costs, it nevertheless remains the most universal and effective survey mode. Therefore, it could be expected that its use in a mixed mode survey would increase the effectiveness of this design by boosting the response rate and improving the sample composition, not only in comparison with the standard mix of self-administered modes, but also in comparison with unimode face-to-face surveys. As readers may remember, the mixed mode design assumes that if the preferred mode is offered, this should encourage additional categories of respondents to participate. The inclusion of the face-to-face mode, which is not only effective but also completely different from self-administered modes, addresses these objectives well and, as such, should be conducive to the achievement of the intended objectives of mixed mode design.
The experiments carried out in connection with ESS rounds 6 and 7 were aimed at verifying whether the face-to-face interview used in the mixed mode design did indeed produce the aforementioned benefits, i.e. a higher response rate and an improved sample composition as well as reduced survey costs, compared to the use of face-to-face interviewing in unimode surveys. In the experiments, the face-to-face mode was combined with self-administered modes, including the most effective of them, i.e. the mail mode. The experiments were based on the sequential design with elements of the concurrent design.
In the experiments aimed at verifying the effectiveness of mixed mode design with face-to-face first, the ESS main study rounds 6 and 7 were treated as an initial phase. In the case of refusals and non-contacts, the interviewers asked for agreement to participate in the survey and offered a choice between the mail and the EMS mode (follow-up). The face-to-face phase achieved a high response rate of 74.9% in ESS 6 and 65.8% in ESS 7. The use of self-administered modes in the follow-up increased the response rate by only a fraction of a percentage point in both experiments. Relatively few nonrespondents/proxies agreed to participate in the follow-up, and the completed questionnaires were finally sent back by relatively few of them. Some nonrespondents also provided non-existent e-mail addresses, which suggests that they simply wanted to get rid of an intrusive interviewer. This result may undermine the sense of using mixed mode design where, after face-to-face in the initial phase, the follow-up is based on selfadministered modes. Application of this design generates only additional costs (related to the implementation of the self-administered phase), which are high anyway due to the use of the face-to-face mode in the initial phase, with no clear outcome such as an increase in the response rate.
However, it should be borne in mind that the face-to-face phase of the experiment was based on a rigorous research design in both experiments. The ESS main study (the initial phase in this experiment) was carried out by experienced, highly motivated interviewers; two advance letters were sent to the respondents; each sampled person received an unconditional gift, and the fieldwork lasted several months. In ESS 6, the average number of contact attempts needed to complete one interview was 2.13, and the share of interviews conducted during the 5th and following contact attempts was 7.6%. In ESS 7, the respective values were 2.28 and 9.6%. In both surveys, up to even three refusal conversions were also attempted. In ESS 6, at least one attempt was undertaken in the case of 43% refusals, and with 48% of refusals for ESS 7. It can therefore be assumed that the interviewers managed to persuade almost all reluctant respondents-except hard refusers-to participate in the survey, as well as to complete interviews with hard-to-reach respondents, except for inaccessibles. Therefore, it is not surprising that the follow-up phase was unsuccessful.
It is hard to speculate whether the use of a standard design instead of rigorous design in the faceto-face phase would translate into better follow-up effects and what kind of response rate would be achieved in the total mixed mode sample. The problem is important, however, because the response rate in standard academic face-to-face surveys is not very high. In Poland, this rate currently stands at below 50%. Assuming that the aforementioned mixed mode assumptions are correct, one should expect an increase in the response rate. Roger Tourangeau (2017) points out that if the sampled person declines to respond using one mode, it is less likely that the same person will agree to participate using another survey mode. Therefore, in order to maximise the response rate, this author suggests using the most effective mode in the initial phase. In this case, it would be the face-to-face first design. However, as the aforementioned results of experiments in the Netherlands and the United States show, this approach entails significantly higher costs for such surveys compared to the reversed sequence (face-to-face as follow-up). However, such costs would probably be lower versus the costs of a unimode face-to-face survey carried out in accordance with rigorous design.
The result of our second experiment, with the use of the self-administered mode followed by face-to-face interviewing, seems to confirm Tourangeau's claim. The response rate obtained was lower by nearly 10 percentage points in comparison with that achieved in the ESS 7 main study carried out in parallel. Although face-to-face proved to be the most effective mode in the experiment, the response rate achieved when using it was rather low, and exceeded that achieved in the initial self-administered phase only by a few percentage points. One could argue that it was unreasonable to expect a satisfactory result in the follow-up since the interviewers attempted to conduct interviews with nonrespondents from the initial phase. Co-operative survey respondents presumably returned a completed questionnaire in the initial phase of the experiment. On the other hand, however, the face-to-face follow-up was carried out according to a rigorous research design, and the interviews were carried out by the same experienced and highly motivated interviewers who participated in the ESS 7. As such, one could expect that they would be more effective.
Two mutually non-exclusive explanations of the low response rate obtained in the face-to-face follow-up in our experiment come to mind. The first one refers to the singularities of the ESS project. The ESS questionnaire is long and burdensome. In the experiment, the questionnaire was sent by mail in the initial, self-administered phase. As a result, when the respondents were deciding about their participation in the face-to-face phase, they were aware that participation would take considerable time. They were also aware of the topics covered in the survey, which might have discouraged some of them. In the ESS 7 main study, the sampled persons did not have this information when making their decision. As a result, they could have been more optimistic about participation. Therefore, it cannot be excluded that the respondents would have been more willing to take part in the face-to-face follow-up if the questionnaire had been shorter, easier and more interesting. 14 The second explanation refers to the self-administeredface-to-face sequence used in the experiment. According to Groves, Singer, and Corning's Leverage-Saliency Theory of Survey Participation (2000), the respondents decide to participate in a survey at the time when the survey request is presented to them. If the survey mode is of secondary importance to them, offering a different mode is unlikely to change their decision about participation. 15 Since in direct, face-toface encounters interviewers may be most flexible and immediately respond to the respondent's doubts, this mode is most effective when it comes to obtaining respondents' consent to participate in a survey. In the case of self-administered modes, however, a single, identical cover letter is usually used for all sampled persons and, in addition, the respondents tend not to read this cover letter carefully. As a result, the likelihood of a positive decision to participate in such a survey is much lower, and the interviewers' subsequent efforts to change the negative decision are less likely to be successful. Therefore, using a mixed mode design with self-administered modes in the initial phase may reduce the likelihood of the respondent participating in a face-to-face interview in the follow-up phase. As a result, the self-administeredface-to-face sequence produces a lower response rate in comparison with the reverse sequence and with unimode faceto-face surveys.
The analysis of the sample composition obtained in the mixed mode design using face-to-face in the initial phase and self-administered modes in the follow-up (Experiment I) seemed pointless. A negligible increase in response rate (a fraction of a percentage point) versus the initial phase cannot significantly affect the total mixed mode sample distributions. However, when using the reverse sequence (Experiment II, i.e. self-administered modes first), the total mixed mode sample did not differ from the ESS 7 main study sample in terms of five out of six characteristics included in the analysis. 16 This means that using a mix of these modes (self-administered and 14 A detailed discussion of other possible causes of a low response rate, related to the design of this experiment, can be found in Sztabiński (2018). 15 Research results have shown that respondents' preferences concerning the survey mode cannot be regarded as relatively stable attitudes. The preferred mode is significantly influenced by the survey mode in which the question about the preference was asked: as a rule, the mode through which the respondents are asked about their preferences is mentioned as the preferred mode (Groves & Khan, 1979;Gesell, Drain, & Sullivan, 2007;Millar, O'Neill, & Dillman, 2009;Dillman, Smyth, & Christian, 2014). The results of some studies also suggest that mode preferences are not a relatively constant disposition (Wardle & Robinson, 2007). Moreover, certain modes are "preference resistant", i.e. the obtained response rate does not depend on respondents' preferences (Olson, Smyth & Wood, 2012;Dillman, Smyth, & Christian, 2014). This seems to suggest that the issue of the mode used does not matter for some respondents when it comes to their survey participation decisions. 16 One exception is the main activity, where a statistically significant difference occurred between the total mixed mode sample and the sample achieved in ESS 7. This difference is related to the much higher participation of permanently sick or disabled individuals in the mixed mode sample in comparison with ESS 7. This is linked with their higher participation in the self-administered phase. As regards the remaining categories, the percentages are very close to one another.
face-to-face, in this sequence) does not improve the sample composition in terms of sociodemographics. In the case of sex and size of domicile (no. of inhabitants), it is not reasonable to expect an improvement, as the distributions of these characteristics obtained in the ESS 7 are very close to those for the total population. However, the distributions of age and level of education are more different from the population, while the distribution of the number of people in the household differs very significantly. Therefore, the application of self-administered modes first and face-to-face in the follow-up does not bring the effects expected from the mixed mode design, both in terms of response rate, and also in terms of sample composition. Both samples differ from the total population in a similar way. On the other hand, the use of such a design, despite a lower response rate, did not lead to deterioration of the sample composition.
The similarity of the total mixed mode sample to the sample achieved in the ESS 7 main study is the result of using the face-to-face mode in the follow-up. For all six characteristics analysed, the sample composition obtained in the initial self-administered phase differs more from the sample achieved in the ESS 7 main study than the total mixed mode sample from the ESS 7 sample. The greatest, statistically significant differences, were found in the size of domicile (no. of inhabitants) and level of education, whereas smaller differences were found in the sample composition in terms of age. This suggests that if self-administered modes are used in the initial phase, then the use of the face-to-face mode in the follow-up is conducive to selection effects and improvements in the sample composition. However, it should be noted that the face-to-face phase in the experiment was based on a rigorous research design. This had a significant impact on the final high cost of the entire mixed mode survey. The cost of a single case was nearly identical to that in the ESS 7, completed entirely face-to-face. It was equivalent to approx. EUR 40. Since the response rate was lower than in the unimode face-to-face ESS, and the sample composition did not improve, the point of using a mixed mode survey based on the design used in our experiment seems doubtful.
However, another result of this experiment is worth noting. In the sample achieved with selfadministered modes there was an overrepresentation of respondents from cities with 500,000 and more inhabitants, as well as of people with upper secondary and higher education, especially those with post-secondary, not tertiary and tertiary education. These categories are usually underrepresented in standard face-to-face surveys. On the other hand, the sample achieved using self-administered modes had an underrepresentation of categories that are usually overrepresented in standard face-to-face surveys. These include inhabitants of villages, people with primary or the first stage of basic and lower secondary education, as well as people from the oldest age group (75 years and more). This seems to suggest that using the face-to-face mode in the follow-up (after self-administered modes) may be conducive to improvements in sample composition, not only when using a rigorous design in the face-to-face phase (as in our experiment), but also when using the standard design. If this is the case, such a mixed mode design could provide an alternative to the standard unimode face-to-face surveys. The response rate obtained by using such a mixed mode may be lower than that of standard surveys while an improved sample composition could be a benefit. However, this is only a hypothesis that needs to be verified. It is certain, however, that if such a mixed mode is used, the survey costs will be reduced. And, as has been mentioned earlier, reduction of surveying costs is one of the essential goals behind mixed mode data collection.
A comparison of the total mixed mode sample with the ESS 7 main study sample and the general population, as well as the sample obtained in the initial phase of the experiment (selfadministered modes) with the total mixed mode sample and the ESS 7 main study sample.    Človek a spoločnosť [Individual and Society], 2019, Vol. 22, No. 4, pp. 1-28. The use of face-to-face interviews in mixed mode design: The problem of the achieved sample 22 Dissimilarity indices: Total mixed mode sample vs. general population: 6.2 ESS 7 vs. general population: 5.3 Initial phase (self-admin.) vs. ESS 7: 14.9 Total mixed mode sample vs. ESS 7: 1.8 Z test (comparisons of percentages for various categories): Total mixed mode sample vs. ESS 7 (result in the Total Sample column): n.s. Initial phase (self-administered modes) vs. Total mixed mode sample (result in the Initial Phase column): † p<0.10 *p<0.05 **p<0.01 ***p<0.001