Social Policy Targeting and Binary Information Transfer between Surveys

In this paper we develop a methodology for identifying a population group surveyed latently in the (target) survey relevant for further processing, for example poverty calculations, but surveyed explicitly in another (source) survey, not suitable for such processing. Identification is achieved by transferring the binary information from the source survey to the target survey by means of a logistic regression determining group affiliation in the source survey by use of variables available also in the target survey. In the proposed methodology we improve on common matching procedures by optimizing the cut-value of the probability which assigns group affiliation in the target survey. This contrasts with the commonly used Hosmer-Lemeshov cut-values for binary categorization, which equates between the sensitivity and specificity curves. Instead we improve group identification by minimizing the sum of total errors as a percent of total true outcomes. The Jewish ultra-orthodox population in Israel serves as a case study. This idiosyncratic community, committed to the observance of the Bible is only latently observed in the surveys typically used for poverty calculation. It is explicitly captured in the social survey, which is not suitable for poverty measurement. This procedure is useful for ex-post enhancement of survey data in general. --


Introduction
The Haredi (Jewish ultra-orthodox) population in Israel is an idiosyncratic community, committed to the observance of the Bible and its commandments, as interpreted by its sectarian religious leaders. Haredi poverty incidence is exceptionally high at 67.5%, with a share of 20% of all Israeli poor while its share in the total population is only half that size. Its major causes are a very high Haredi fertility (a population growth of 6% p.a.), reducing both household income per capita and the mother's earning capacity; its education system is largely independent from the national school system and neglects (particularly among boys) materially important subjects for the buildup of future earning capacity such as Mathematics, English and digital skills; a low labor-force participation of Haredi men, due to prolonged learning in religious seminars (Yeshiva), often deep into the prime working age. A further cause for the sharp increase in shortterm poverty has been the recent large cuts in child benefit payments. 1 The share of Haredi children up to age 4 is nearly 3 times higher than in the rest of the Jewish society. This, together with the empirical regularity of a negative relationship between poverty and age implies an upward-drift for Haredi and overall Israeli poverty over time. Haredi Poverty, as measured by the distribution-sensitive Sen-poverty index, nearly doubled over the last 3 years after a previous significant improvement. This deterioration stands in contrast to developments in the rest of the Israeli-Jewish society, whose poverty intensity increased only slightly over the last couple of years. 2 In 2004 Haredi male labor force participation of 37% hardly exceeded one half that of the other Jewish male population, mainly due to the Haredi high enrollment in religious seminars (Yeshiva) during their prime working age. Despite their much higher fertility the women function as the family's main providers, with a participation rate of 48%, compared to 58% of non-Haredi women. Preliminary and still statistically insignificant empirical evidence points to a recent increase in Haredi labor-market involvement, both among men and women, probably related to increased economic hardship, maybe due to the drastic cut in child allowances from 2002 to 2004.
Empirical evidence shows job training to affect labor force entry positively, particularly among Haredi men, though at low wages. 3 These schemes proved successful tools when conceived with a high sensitivity towards the particular cultural needs of the Haredi society.
Proper identification of the poor is essential for social policy targeting. When the poor belong to a specific cultural group with exceptionally high poverty incidence, the basic determinants of their poverty are typically centered on family size, educational deficiencies and labor market behavior. However, these characteristics might have more deep-seated cultural roots, reflected in collective preferences concerning fertility and gender-related differences in education and labor-force participation. Such underlying cultural determinants which are of a qualitative rather than quantitative nature should be included explicitly in the poverty analysis. Such information is usually lacking in standard income or consumption surveys typically used for poverty calculations. Such information is to be found in special social surveys, based on the same population, but not suitable for standard poverty calculations. The question arises whether an efficient procedure can be devised by which such information could be transferred ex-post from the original survey, the source-survey, to the relevant surveys for poverty calculation, i.e. the target-surveys. Such a procedure may be a useful tool for ex-post enhancement of the information content of survey data in general.
The major social and economic surveys of an economy focus typically on different aspects of the same population. While some of the questions recur in more than one survey, other information is unique to a specific survey. Since the gathering of information is not costless and some of the survey-specific information might be useful to researchers of another survey or to policy makers, we suggest an efficient method for optimal binary information transfer (BIT) from one survey (the "source" survey) to another (the "target" survey).
The optimal method for transferring the information depends crucially on three aspects of the process: (1) an overlap of the set of variables that contain explanatory power of the variable to be transferred (thus ensuring a reasonable goodness of fit of the Receiver Operating Characteristic-curve, henceforth ROC-curve), (2) the rule for determining the cutoff value, i.e. the value by which the logistic probability forecast is translated back into a binary variable and (3) a quality test of the procedure. Our quality test, while performed in the source survey, still provides a clue to the quality of the synthetic information in the target data set.
Such enhancement of socio-economic data by an ex-post information transfer is particularly useful when additional data collection by a survey is either too expensive or impossible. Our method for choosing the cutoff value of the forecasted probability is shown to improve on that suggested by Hosmer and Lemeshow, 2000. BIT has several possible applications. It can be useful in the targeting of policies to specific population groups, which is one of the purposes of poverty mapping. 4 The present results might also be used in medical research and other research employing logistic regression and cutoff values. 5 We then illustrate the application of the method to the measurement of poverty in a specific group, known for its high poverty incidence -the Israeli Jewish Ultra-orthodox ("Haredi") population. Due to the lack of information on religious affiliation in the surveys typically used for poverty calculations (the income-and expenditure surveys), and the lack of sufficiently detailed income and consumption data in the survey that does provide information on religious affiliation there arises a need for the transfer of _________________________ 4 In recent years poverty mapping has become an important tool in improving targeting. This technique utilizes information from surveys, amenable to poverty calculations, but too small for efficient targeting of the poor, by transferring information to large scale data bases such as census data, which provide less detailed information, but on a larger share of the population. Such a transfer is carried out by use of econometric tools. The purpose is to enable the calculation of policy variables, for example binary information on poverty, for small geographic areas. See for example Hentschel, J., Lanjouw, J.O., Lanjouw, P. and Poggi, J. (2000), Bigman and Srinivasan (2002) or Small Area Estimation at www.worldbank.org. 5 See for example Hadjicostas Petros and George C. Hadjinicola (2001), G. Schares et al. (2003), Schutter E.M.J. et al. (1998) and Stegeman, J.A. et al. (2006) www.economics-ejournal.org information on Haredi membership from the Social Survey to the Income and Expenditure surveys.
The paper is organized as following: In chapter 2 the model of BIT is presented. Chapter 3 describes the process of BIT in more detail. In chapter 4 we report on a case study of BIT applied to the Israeli Haredi population for the purpose of poverty calculations. 6 Concluding remarks complete the paper.

The Model
Assume a sampling of two Household surveys, one which we call the Source-survey (S), consisting of n S = 1…S households, and another survey sampled on the same population 7 , which we call the Target survey (T), n T = 1…T. Let there be a dichotomic binary group variable, say of group H, with a value of 1 for success and 0 for failure. We denote the household's probability of event H = 1 occurring, as π i and its estimate asˆi π .
The estimate is conditional, based on vector x of explanatory variables, P ( H=1| x') is a binary estimate of H for individual i in the respective sample of the source (S) or the target survey (T). Obviously, the suggested procedure requires vector x' to appear in both S and T. The logistic probability function for event H=1 is given by (1) The logit equation includes continuous (k=1…K) and categorical variables (D j l , j=1…J), such as simple dummy variables or dummy variables with more detailed coding levels (l=1…L-1): (2)

3
The BIT Process Step

1: Search for an Efficient Logistic Regression in the Source Survey
The quality of BIT depends crucially on the explanatory power (not necessarily in a causal sense) of equation (2) of group membership probability in the Source survey (ˆS i π ). The better the explanatory power, as reported in the regression's log-likelihood _________________________ 6 Gottlieb (2007), op.cit..
7 Since the households are chosen by specific mechanical processes the chances that the same household will appear in more than one survey is negligible. Of course if it does, and the researcher knows that information, then the information transfer becomes trivial.
ratio, Wald test, the z-values and additional statistical parameters, the better is the chance for a successful BIT of household i's group membership.

Step 2: A Forecast of Group Membership, Using a 'Continuous' Cutoff Value (ˆS c π )
We choose any cutoff point 0 ≤ ˆS c π ≤ 1 in the source survey, above which the forecast of household i's group membership (ˆS i H ) is either 1 or 0. We repeat this procedure,

Step 3: Assessment of the Forecast Quality in the Source Survey
The error rate or forecast quality can only be estimated in the source survey since the target survey includes only the set of explanatory variables and not the dependent variable. We characterize the forecast quality using the ROC curve as a measure.
Step 4: Searching for the Optimal Probability Cutoff Value ( ,*

S c π )
We choose the optimal cutoff value by using the outcomes of the previous step, i.e. the cutoff value that minimizes the sum of total squared errors FP and FN. Notice that Hosmer and Lemeshow (henceforth HL) suggest that the optimal cutoff value is at the level ,* S c π for which sensitivity equals specificity. In the following we show that in the present case our choice yields a significant improvement on the HL choice.
Step 5: The BIT -Calculation of the Forecast ˆT

i H in the Target-Survey
After having ascertained that we have elicited the best possible forecast we move to the target survey. As mentioned before there is no way of testing the quality of BIT, except by new data collection. We calculate ˆT i H by use of the regression equation and the optimal cutoff value as estimated in the source-survey.

A BIT Case Study: Poverty among the Jewish Ultra-Orthodox in Israel
The Israeli Ultra-Orthodox Jewish society, also called Haredi society, 8 has long been known to have an exceptionally high poverty incidence. However, since there is no indication of Haredi affiliation in the surveys used for estimating poverty, available poverty studies and in particular the official ones do not report separate poverty estimates for this population group. Some studies have attempted to estimate poverty in this population group on a national level and we shall discuss them below. Due to its idiosyncratic cultural features and a lack of their explicit inclusion in survey questions the Haredi society is an interesting example for applying the BIT process. Their heterogeneous labor market behavior leads to extreme poverty situations of many Haredi households. Consequently, there arises a need for statistical enhancement concerning Haredi group membership in the major surveys used for rational policy formulation and implementation. In order to model a logistic group membership probability of the Haredi (the first step of the BIT process) we briefly characterize them here.

The Israeli Haredi Society -Roots and Characteristics
The Israeli Haredi society is fragmented into several subgroups, each emphasizing different aspects of Judaism and obeying its own spiritual leaders. For simplicity we concentrate on three main factions: The Hassidic, the Lita'i and the Sephardic 9 groups. They all share strict observance of the Torah and the Jewish commandments and a high degree of compliance to their spiritual leaders' decisions concerning a wide range of public and Family issues. The leadership maintains a strong sense of hierarchy and issues and looks over detailed rules for individual and family behavior through its organs. When in the early 20 th century secular Jewish nationalism emerged as a rapidly growing alternative to the religious way of life, the Haredi rejected its anti-religious character strongly. Since a historical compromise in 1948 between David Ben-Gurion (then Prime Minister) and Hazon Ish 10 (then Leader of the Haredi society) male religious scholars, whose main occupation is Torah study, Haredi women and men in drafting age are by and large exempted from serving in the Israeli army. Over the years the number of exempted men from the army grew rapidly (8 to 9% p.a., reaching more than 30,000 in the early 21 st millennium from about 400 in 1948. In response to court appeals and a general public discontent over that exemption from army service and to a mounting social problem of young drop-outs from the Haredi religious seminars, an _________________________ 8 "Haredi" is the Hebrew name of the Ultra-Orthodox society. It has the meaning of a person who "trembles in awe of God". It includes distinct groups, common in their unequivocal commitment to the study and observance of the Torah and its commandments, as interpreted by their religious leaders. See also Friedman (1991). 9 Sephardic originally indicated the Judeo-Spanish origin. In the Israeli context it is sometimes used in a wider context to indicate also other Jews originating from North Africa or the Middle East. 10 Rabbi Abraham Yishayahu Karelitz (1878Karelitz ( -1953 The compromise included also an exemption of Haredi girls. In the years 1951 and 1952 the argument over drafting Haredi women developed into a government crisis, causing the Haredi party Agudat Israel to leave the government.
www.economics-ejournal.org official commission, appointed in 1999, proposed a change in government policy with the purpose of reducing the number of exemptions from the army and of improving Haredi men's labor force participation. 11

Estimates of Haredi Population Size
Several attempts to estimate the size of the Haredi population were based on the question about the "last school visited" in the household surveys of the ICBS. This education-based approach was pioneered by Berman and Klinov, 1997 and also Dahan, 1998. It was elaborated in Berman, 2000, and has since then been used for analyzing Haredi poverty and labor market behavior. 12 Other population estimates were based on election results due to typically monolithic Haredi voting-patterns. 13

The Education-based Approach
According to this approach a household is assumed to be Haredi if at least one of its male members indicates a Yeshiva (a religious seminary) 14 as the last school attended. Berman (2000) forecasted Haredi population to reach 280,000 in 1995 and 510,000 people by 2010, based on expected fertility and death rates. Such forecasts are bound to produce unsatisfactory results for a number of reasons: Yeshiva studies do not constitute a necessary condition for Haredi belief. Indeed, Yeshiva attendance among Hassidic Jews, a large group within Haredi society, is believed to be lower than in the Lita'i and Sephardic Haredi groups.

The Elections-based Approach
This approach was chosen by Gurovich and Cohen (henceforth GC), based on the 2003 elections 15 and a geographic identification of localities with a high percentage of voters for the two political parties of Haredi orientation out of the 13 party lists represented in the parliament: United Torah Judaism (UTJ, or in Hebrew "Yehadut HaTorah") and "Shas" 16 . While the voters for UTJ are supposedly mainly Haredi, many "Shas" supporters are less religious but rather traditional or ethnically oriented voters. In order to identify this subgroup, GC included Shas supporters among the Haredi only if they lived in the vicinity of areas with a high percent of UTJ support, assuming that the 14 These seminaries are not to be confounded with religious High schools (Yeshiva Tihonit), which combine religious studies with a high school curriculum. The latter are typically frequented by orthodox rather than ultra-orthodox Judaism. Orthodox Jews, distinctly from the Ultra-orthodox are fully integrated in the Israeli society, its labor market as well as in the army. 15 An earlier study by Degani and Degani (2000), was based on the 1996 elections.

www.economics-ejournal.org
Haredi like to live whitin each other's proximity. GC concluded that only 1/3 of the Shas voters are Haredi. The population estimate is calculated as following: where i = number of voters for each party, j = UTJ party/Shas party, where p j = election-participation rate of the j t h party. x j = percent of population under voting age of the j t h party supporters. In areas with a high rate of UTJ voters, the researchers report a high participation rate compared to other areas. In areas with 90% and more UTJ votes the general participation rate was 94%. In areas with 80% and more UTJ votes, the general participation rate was 85%. The study assumes a significantly higher Haredi election participation rate than that of the general public. 17 Based on fertility rates derived from the Social survey for Haredi women of Ashkenasi 18 background, GC used a fertility of 7.5 births per woman yielding an estimate of the share of people below the voting age (based on a model of stable populations) of 56% of the population. The total population of actual and potential UTJ voters is estimated to be 361,000. The Sephardic Haredi estimate amounted to 204,000 and the total Haredi population was estimated at 565,000 by the end of 2002.

The Estimate Based on the Social Surveys of the ICBS
The first Social Survey with a sample size of some 10,000 persons aged 20 or more and their household was published in 2002. The estimate of Haredi affiliation is based on question Nr. 26 of the questionnaire 19 . In order to estimate the population size including children, the weights need to be adjusted to account for the fact that in a household there may be more than one person aged 20 or more. We calculate population size by use of the following formula 20 : where Pop H = Haredi population, i = people declaring themselves as Haredi and nn i = population weight for each respondent. According to the Social Survey they were 194.9 thousand by end of 2002. Under20 i = number of people aged under 20 in the i th (Haredi) household and over20 i = number of additional people (to those questioned) _________________________ 17 The general participation rate in the 2003 elections was 67.8%. When adjusted for the very low Arab election participation rate and for the Israelis who were absent during the elections, the general participation rate is somewhat higher but still lower than the Haredi participation rate. 18 In the present context this indicates a European (including Eastern European and Russian) and Anglo-Saxon background. 19 Question 26: "Do you consider yourself (1) Haredi, (2) religious, (3) traditional-religious, (4) traditional and "not so" religious, (5) non-religious or atheist. In order to estimate the population size, the detailed data set is needed, including information on the other household members, their age and the weights attached to the interviewed person. These and more data were kindly provided by the CBS. 20 We thank Tsahi Makovki from the ICBS for providing the formula. aged 20 or more, in that household. According to this calculation, the Haredi population reached about 550,000 by end of 2002. 21 As shown in Table 2 the new estimate exceeds the commonly accepted estimate of empirical economists by 62 percent.
We find the population estimate of the Social Survey to be consistent with the calculations of GC, who calculated the size of the Haredi population based on revealed (party-) preference from the 2003 election results.

The BIT Estimate
The variable reflecting true group membership and chosen as a benchmark for the competing estimates is the sampled person's own declaration of Haredi affiliation (group membership). In the present case study this seems to be the most natural approach since religious affiliation is first of all a subjective cognition. In other examples of group membership one might be looking for a variable reflecting an objective recognition of group membership (e.g. a valid passport for citizenship, a university degree for being an academic etc.) as the benchmark that might be preferable.

Step 1: Search for an Efficient Logistic Regression in the Source Survey
Based on prior knowledge of the distinctly high Haredi fertility and fundamental changes in fertility over the last generations we decided to split the data into 3 subgroups by the mother's age in order to improve the overall empirical results. In order to identify Haredi families we analyzed differences in fertility patterns and demographic characteristics (the mother's age child ratio, country of origin of the head of household), educational characteristics (Yeshiva as the last School attended by one of the male household members or No High School or University diploma), geographic concentration (areas with high Haredi concentration), differences in social behavior (philanthropic behavior, no use of internet), living conditions (car ownership, number of children per room).
The regression results for the families, grouped by the mother's age and the variable definitions are given in Appendix Table 1. The coefficient vectors ˆ( 1,2,3) i i β = % are needed for the BIT process in order to calculate the estimated probabilities by use of the logit functions ( ) i g x with the relevant coefficients for each group respectively as mentioned in equations (1) and (2). The logistic regression model (Appendix Table 1) is statistically significant as can be seen from the log likelihood statistic and the Wald-test in Table 1. _________________________ 21 The total population should add up to 6.59 million people, but yields only 6.19 million. The discrepancy may be due to the de-facto exclusion of the Eastern-Jerusalem Arab population, Bedouins in non-recognized settlements and people staying in non-sampled institutions. Furthermore the weight adjustment reflects only an approximation of the true weight.
www.economics-ejournal.org Step 2: A Forecast for Group Membership, Conditional upon a 'Continuous' Cutpoint (ˆS c π ) In the next step we calculate a binary variable of group membership from the estimated model based probability, conditional upon the cutoff valueˆS c π . Any probabilities exceeding the cutoff value receive a value of 1. All others receive a value of 0. This step is repeated for a near-continuous number of cutoff values in small steps (say 0.01). The results can then be categorized in a classification table such as Table 2 for any specific cutoff value. Such tables measure forecast quality, as reflected in the calculation of sensitivity (the share of correctly forecasted non-members out of all true non-members), and specificity, (the correctly forecasted members as a share of all true members at any given cutoff value). They are conditional upon specific cutoff values.

Step 3: Assessment of the Forecast Quality ("goodness-of-fit") in the Source Survey
A well known indicator for the assessment of binary model forecasts is the ROC (Receiver Operating Characteristic) curve which juxtaposes sensitivity (truly identified positive response) with the percent of outcomes wrong positive responses (truly negative).  Figure 1: The ROC Curve Figure 1 indicates that the youngest age group's regression performance is best among the 3 groups when evaluated by the integral below the curve. Estimates for all 3 groups are better than the 45o line of random assignment. Figure 1 emphasizes the importance of splitting up the model estimation according to the mother's age-group, thereby allowing for age-dependent parameter coefficients in the regression. www.economics-ejournal.org Step 4: Searching for the Optimal Probability Cutoff Value ( ,* S c π ) After the ROC curves have been calculated, the optimal probability cutoff value needs to be located among all the possible cutoff values. The question how to translate logistic probabilities back into a binary variable is not conclusively dealt with in the literature. In medical research the question arises frequently and is sometimes related to the improvement or the damage in health caused by a specific treatment under review. For example, if the effect of a certain vaccine can only be known ex post and it is found to cause an important health improvement for some, while for others there is a negligible negative effect a general vaccination policy might be a reasonable course of action.
If, like in the present case, there is no a priori case for a particularly large net cost of either error (FP, FN in section 3, step 2) we opt for a cutoff value that minimizes the total sum of squared errors FP and FN.
Hosmer and Lemeshow (2000) suggest that the optimal cutoff value lies at the intersection of sensitivity and specificity. Their choice seems to hinge on the argument that we attach equal importance to each group in a relative sense. In figure 2  The optimality rule for chosing the cutoff value crucially affects the number of forecasting errors. In Table 4 we compare the incidence of errors. While the MSE-rule reduces the number of FP cases significantly, the opposite occurs in the FN cases. This pattern repeats itself in all three age-groups. However, we also observe that the deterioration in FN is more than offset by the improvement in FP. This is again the case in all three age-groups, such that we can conclude that the MSE approach meaningfully improves the forecast, reducing the sum of errors to half compared to the HL approach. Step 5: The BIT -Calculation of the ForecastˆT i H in the Target-Survey In the final step we estimate the Haredi populationˆT i H by use of the regressions in appendix Table 1. Table 5 illustrates the importance of the quality of the model to be used for forecasting group membership. In each of the observed years the downward bias of the Haredi population is much smaller in the recommended approach compared to the traditional approach as used in Berman andKlinov, 1997, or in Dahan, 1998, and elsewhere.

Conclusions
This methodology can be usefully applied to many fields, since it allows us to optimally enhance a given data base by adding a binary variable that does not exist in the relevant data base. The present study shows how our methodology can be used for poverty estimations of a population subgroup that is not sampled in the major surveys used for poverty calculations -the income survey or the expenditure survey.
The main results are reported in Table 6. Poverty incidence in the Haredi population is nearly three times higher than in the general population. It is among the poorest population groups in Israel, making it obviously highly necessary to monitor efforts of poverty reduction. Inequality among the poor, though it is lower than in the general population, does not compensate for the higher poverty incidence and income gap. Poverty intensity, as measured by the Sen-index is almost double its level for the total population. A similar conclusion can be drawn concerning child poverty. While until recently Haredi poverty has been approximated only roughly, the present methodology improves the accuracy of poverty measurement for the targeted group, a desirable feature, the more expensive and the longer the time lag of policy implementation. 23 The proposed method may also be usefully applied in the context of poverty mapping by Small Area Estimation (see for example Hentschel et al., 2000). In such an exercise we might be interested in attaching a binary forecast of poverty incidence to a household in a small area, not covered by the household surveys typically used for official poverty calculations (the Source Survey, S). Data on households in the small area, collected in an extensive but superficial large scale survey such as a survey accompanying a population census could be used as the vector x' in a Target Survey, T, as outlined in sections 2 and 3. Use of the coincidental vector x' in S and T and the choice of an optimal cutoff value would allow for the production of an estimate of poverty incidence in T.