Response burden, quality and communication

Summing up Simply2010 in a few quotes

Stephan Moens

Statistics Belgium

Belgian Presidency Team eu2010

I am not a statistician, even if I have been working for many years in Statistics Belgium and I still am working there. I am more a man of communication, and I have also done some journalism.

Therefore, I will look at what has been said at this Conference a bit as a journalist would do, asking a bit naïvely some questions that occurred to me while listening to the interventions and putting them into headlines.

I will not go into technical and methodological matters. I think you have amply had the opportunity to discuss these bilaterally in the coffee breaks and on other moments. So let’s go.

I think the introductory speakers, and more in particular Mr. Radermacher, did already raise some of the main issues. I will give you some quotes, which from my point of view summarize rather well what is at stake.

  1. “Our first objective is not reducing response burden; it is making high-quality statistics.” This seems obvious but sometimes we forget.
  2. “We always get new requests, and it will not stop. These requests are horizontal,” crossing many fields of statistics.
  3. “We should also use private sources.” I would never have thought of it, but why not?
  4. “We must go from a technocratic approach to a strategy-driven approach”, i.e. act as entrepreneurs.
  5. “We must go from maximising happiness to minimising unhappiness.”

There was also bad news:

  1. “Official statistics lose relevance.”
  2. From our Dutch colleagues: the only thing the new Dutch government as to say about statistics: “There should be a cut in statistics.”

This is where we come to the issue of response burden.

  1. “Perceived response burden is more important than real response burden”.
  2. “There is a burden caused by the research into burden”. Most of the burden comes from surveys. And what do we do to measure it? A survey! This looks like a statistician’s approach to a problem caused by a statistician. In my opinion, a survey should be the last idea when it comes to measuring response burden, not the first.

Another issue: survey redesign processes. It was said that often there is no research done before and after such a process, leaving the effect of it unmeasured. I could then state hypothetically that things go worse – worse response, worse quality - afterwards and nobody would be able to contradict.

Many speakers raised the issue of communication. One of the nicest quotes: “It is not a business who responds but a person.” Are we really aware of that?

Or, on a websurvey: “It is not so easy to go from one page to another.” Do we think of that when we are now planning censuses by internet? Do all respondents find the right button?

On business statistics: “There is a mismatch between what we ask and what they have.”

And finally: “”Sometimes you have to increase response burden to raise quality.” This is certainly something statisticians love to hear, but I would ask: “When exactly is sometimes?”

There came good news from DESTATIS: there are more than one hundred persons working full-time on the measurement of all kinds of burden caused by the German administration. Now I look at my own director-general and ask myself: how many do we have? We probably hire consultants.

Another speaker made an important point: “We need different staff profiles” after all the reengineering of our processes: not only less junior staff and more ICT people but perhaps also less ‘pure’ statisticians and more research and communication staff.

And, as a last quote: “All NSIs should constantly run quality analyses”. How many among us really do?

To sum up – and this is something that is said on all conferences, but it remains true – there is work to do.

  • We have to restore and monitor constantly the balance between reducing burden and maintaining quality. The last years, the focus was on reducing the burden. I have the impression that the pendulum now goes in the direction of quality. This is an opportunity we should not miss.
  • We will have to bring the workload from citizens and businesses to the NSI. This will bring a lot of work, only to be solved by increasing in-house efficiency.
  • Share experiences! If one NSI has elaborated a good practice, it should immediately share it with all other NSIs that can usefully work with it. The ESS seems to be the ideal forum where to share good practices.
  • From the moment that the issue of response burden is solved or at least secured - and we have heard many possibilities to do so – we can go further. More in particular, we will have to address many new, horizontal requests.
  • We will need continuous communication about the benefits of statistics, to motivate respondents but also to convince politicians.

If I may suggest something, a next conference should address this issue of communication. And there should be not only statisticians in the room, but also businessman, citizens and politicians.