Unit Non-Response in Household Wealth Surveys: Experience from the Eurosystem S Household

Unit non-response in household wealth surveys: experience from the Eurosystem’s Household Finance and Consumption Survey

Keywords:Sampling weights, unit non-response, response propensity, calibration, non-ignorable non-response

1.Introduction

The Household Finance and Consumption Survey (HFCS) is a recent initiative from a network of the ECB, National Central Banks, and National Statistical Institutes to collect comparable micro-data on household wealth and indebtedness among the euro area countries. The first round of the survey was successfully conducted between 2008 and 2011 and the results were published in April 2013. The second round is under way and will cover all the euro area countries.

Unit non-response is in general a concern in most household surveys. This is even more the case in the HFCS, because wealthy households might be reluctant to reveal such sensitive information to interviewers, or simply because they are usually harder to contact. Furthermore, it is essential to get the households in the upper deciles of income and wealth to cooperate in order to keep the overall sample representative of the distribution of household income and wealth in the population.

The paper first presents descriptive statistics in relation to unit non-response in the HFCS, such as response rates broken down by auxiliary characteristics available in the HFCS data files and response representativity indicatorscalculated using the RISQ methodology ([1]). It also compares several reweighting strategies for coping with unit non-response in the HFCS, in particular simple as well asgeneralised calibration methods againstthe traditional approaches based on the estimation of response propensities. The methods are assessed with respect to their impact on the main HFCS-based estimates and their statistical properties in terms of bias and variance.

2.Methods

2.1.The R-indicator

Key indicators of survey representativity have been defined in the framework of the “RISQ” project (representativity indicators for survey quality - Such quality indicators, called the R-indicators, may serve as counterparts to survey response rates and are primarily directed at evaluating the non-response bias ([1]).

The indicator is given by:

is the standard deviation of the response propensity within the population. In practice the value of the indicator is unknown and has to be estimated using auxiliary information available both for the responding and the non-responding units.

2.2.Simple calibration

Originally developed by Deville and Särndal ([2]), calibration has grown today into a widely used technique in official statistics. The principle is to adjust the sampling weights using population benchmarks so the new weights reproduce exactly the population totals (case of quantitative variables) or the population distributions (case of categorical variables) for a predefined set of auxiliary variables. In addition, the calibration technique can be used as a standard method to treat unit non-response bias ([3]), assuming the calibration variables are both correlated with the probability of response to the survey and the variables of interest. This approach is less stringent than the traditional one based on the direct estimation of response propensities, as adjustment variables are no longer required to be available for the non-respondents. However, population benchmarks need to be known, which can still be a hurdle. Survey estimates based on large samples (e.g. the Labour Force Survey) can be used for calibration. However, this solution is not doable when the probability of response depends on the topic of the survey itself (non-ignorable non-response), as this might be the case with wealth surveys.

2.3.Generalised calibration

Generalised calibration ([3]) can produce non-response adjusted weights which reflect characteristics that are observed only for the respondents and for which no benchmark totals are available. This is a major improvement compared to simple calibration. In particular, survey variables of interest can themselves be used for non-response correction. Therefore, the technique ought to be well suited for coping with non-ignorable non-response. The generalised calibration approach is implemented in the new version of the SAS macro CALMAR, developed by France’s INSEE and in the R package 'Sampling'.

3.Results

3.1.Descriptive results

Household response rates to the HFCS ranges from less than 20% in Germany to around 70% in Finland. In general, all other things being equal, the HFCS achieves lower response than other European-wide household surveys, such as the European Statistics on Income and Living Conditions (EU-SILC) and the Household Budget Surveys (HBS)

Figure 1: Household response rates (%) in the HFCS, by country

Source: European Central Bank, HFCS methodological report for the first wave

Further analysis shows that non-response is not uniform across households, but rather depends on household and dwelling characteristics. Some auxiliary information for non-response analysis is already collected in the HFCS, for instance, the type and the rating of the dwelling, the rating of the neighbourhood or the presence of security measures in the dwelling (dog, watchman…). There is also nationally available auxiliary information which can be powerful in explaining non-response.

3.2.Impact of reweighting strategies on the estimates

In this section, we calculate the main HFCS indicators (median household wealth, share of assets in total wealth…) using different reweighting strategies, namely the traditional strategy based on the estimation of the response propensity and the calibration-based approaches (simple and generalised calibration).

3.3.A simulation study

In order to compare re-weighting strategies for coping with unit non-response, a simulation study based on the first wave of the HFCS has been conducted. The calculations were made using the second version of the SAS macro CALMAR. Replications were drawn according to a non-ignorable mechanism such that the probability of response depends on the total household wealth: the wealthier the household, the lower its probability to participate in the HFCS. For each strategy, the bias, variance and mean square error were computed.

4.Conclusions

Unit non-response is an important issue in the HFCS. The complexity as well as the sensitivity of the core wealth information in the survey leads to a high proportion of the households not participating in the survey at all. Even though all sample surveys are liable to unit non-response, non-response rates are generally higher in wealth surveys than in other household surveys, all other things equal.

The generalised calibration approach can be an option to deal with non-response bias, particularly when the response propensity depends on the characteristics of interest in the survey. The method is easy and flexible enough to implement using standard software tools. Yet, the price to pay for bias reduction is an increase in sampling variance. Therefore, caution is needed when using this approach, in particular with regard to the choice of the calibration variables ([4]).

References

[1] B. Schouten, F. Cobben and J. Bethlehem, Indicators for the representativeness of survey response, Survey Methodology, vol. 35, No. 1 (2009), 101-113.

[2] J-C. Deville and C-E.Särndal, Calibration estimators in survey sampling, Journal of the American Statistical Association, No. 87 (1992), 376-382.

[3] C-E. Särndal and S. Lundström, Estimation in Surveys with Nonresponse, Wiley Series in Survey Methodology (2005).

[4] D. Haziza and E. Lesage (2013), On the problem of bias amplification of the instrumental calibration estimator with missing survey data, Graybill Conference (2013).

1