Report for the World Bank Trust Fund Project
Organisational Development and Governance of the National Statistical System
for
Statistics South Africa
By Mary Strode
Senior Statistician
Oxford Policy Management
June 2005
Preface/Acknowledgements
I would like to thank the Statistician-General for supporting me in this mission, and for the personal support and understanding given to myself and my family during my son’s serious illness which caused the mission to be interrupted. I would also like to give special thanks to Dr. Letsebe for her advice and support during the mission.
Professor Kahimbaara and his team made special efforts to ensure this mission was fruitful and productive and I thank them for their time, hospitality and comradeship.
My thanks also go to all the staff at Statistics South Africa and the many people in government who gave up their time to assist me.
Executive Summary
Background
1. The development of the National Statistical System (NSS)[1] has made considerable progress over the life of the project particularly in raising awareness of the NSS among all parts of government, and forming partnerships with producer and user government bodies. Several consultants have worked on plans for the NSS, including a business plan, departmental audits and training. A step change in data quality in the wider statistical system of government departments will require an increase in resources and greater priority to be given to improving information systems.Additional resources will be required in the producing departments outside Statistics South Africa (Stats SA). This report looks at the governance and management systems required to establish the National Statistical System, and is the result of two missions to South Africa by the consultant.
Progress
2. The NSS development is part of the Government’s monitoring and evaluation (M & E) strategy which extends through all three tiers of government in the country. This M & E strategy is focused on improving service delivery and improving public sector effectiveness. It rests at the very heart of government. While good progress has been made in increasing awareness of the NSS and developing partnerships with departments, the Cabinet Office was not aware that little real progress had been reached with the NSS in improving data quality, indeed the Programme of Action on the Government website states that the NSS has been established and this gives the appearance that the task is completed or near to completion. The latest public posting[2] (7 March 2005) of the Government Programme of Action does not fully recognize the scope of the task required to put the NSS in place. Although the progress to date is commendable, more focus is required on improving the productive capacity of the NSS. Progress in productive capacity has been made by Stats SA with using Home Affairs data to produce a ‘causes of death’ series. The development of a productiveNSSwill require more than setting up a NSS Division in StatsSA and signing a number of Memoranda of Understanding, there is a major technical task ahead to achieve usable output from the NSS.
3. There is commitment to the development of the NSS at the highest level in government, and the Presidency reiterated this at two meetings during the mission. This commitment extends as far as the Director General level, but it was recognized that there were often blockages lower down in the Government service. The will to improve departmental information is lacking in some cases, and it will require interdepartmental decisions and actions to raise the priority of the NSS in departments to ensure action is taken. It is unlikely that Stats SA will be able to do this unaided. There is a danger that the launch of the NSS will provide a temporary smokescreen which masks the truth that little additional capacity to produce quality statistics from the wider NSScan be developed with the current level of resources and prioritisation.
Resources Needed to Develop Capacity in the NSS
4. While StatsSA has concerns about the quality of its own statistics, the very poor quality of other key government data should be of even greater concern to users. For example, a meeting with the Department of Education suggested that if their school survey data reached a quality level of 40% accuracy, then they would have made significant progress. Given the huge public expenditure programmes in health, education, housing and social development, the case to improve their reporting systems would seem to be overwhelming. As the majority of the information required is a by-product of management systems, the producing department has to be responsible for improving and maintaining data quality. Little can be done to replace poor quality or missing data after collection. The ‘rubbish in, rubbish out’ principles hold, if the data held by departments are incomplete, full of errors and incompatible with other statistics collected in the country, there is little thatStats SA can do to convert them to high quality statistics.
5. Previous reports in this series of missions have recommended the secondment of staff from StatsSA to other producing departments to improve data quality. Given the human resource constraints affecting South Africa and the priority given to producing and improving Stats SA’s own statistics, this proposition seems unlikely to be acted on. Furthermore, producing departments in the wider statistical system would benefit from taking more responsibility for the production and quality of their own statistics, they should be accountable for reporting on their ownperformance. Stats SA has the skills and the mandate to provide coordination, harmonization and coherence in the system. Stats SA coulduse its professional skills to assess and advise on improving quality which would enable the Statistician-General to certify statistics as having reached quality standards; and it couldalso provide archiving and data access to users.
Departmental Resource Needs
6. Ultimately departmental statistics from the NSS will be used by the monitoring and evaluation (M&E) system, concentrating first on the key indicators used for the 10-year Review. Departmental internal registers and MIS systems will need to be improved to enable them to provide, as a by-product, key statistics of a quality standard which enables them to be declared official statistics. The improved data will enable departments, the government and other stakeholders to monitor output and performance, and the series selected for improvement should ideally be chosen with monitoring needs in mind. This quality improvement will involve changes in the working methods and practices of departmental staff and those from the provinces and cannot be accomplished by an external agency such as Stats SA. The work can be guided by Stats SA which will need to take a major role in harmonising classifications and definitions, and in quality assurance. The bulk of the work of improving the NSS must therefore rest with the producing departments themselves, who must be adequately resourced to undertake the task.
7. The majority of the resource requirementswill be needed in the departments themselves to enable the reengineering of information systems to provide usable output and to improve coherence in the statistical system. The resources requiredhave not yet been assessed, and depend largely on NSS priorities and the quality of existing statistics. This assessment is required in order to agree resources for NSSdevelopment. The requirements are likely to be financial support to replace information systems, redesign data collection activities, and training of staff and professional statistical input. Senior statistical human resource capacitywill be required to improve or redesign data collection systems, maintain quality standards and to analyse the data.These skills are in short supply in the country and it is unlikely that StatsSA would want to second staff to departments for any significant period. In any case it is preferable for there to be sustainable statistical capacity in departments, and for statistical staff to be included on their establishments. The requirement for professional statisticians in departments will be on a continuing basis, although the bulk of the work needed to improvedata quality and systems will be at the start of the action on each series.
8. Some consultants and short-term assignments may be required at the early stages of capacity building, when the workload is heaviest and the human resources most limited, although a permanent statistical capacity will be required in all key producer departments. It would be desirable for the Statistician-General to head this professional cadre, in order to maintain professional standards (such as ensuring respondent confidentiality), and to ensure statistical coordination. The UK model utilizes a common statistical service and each department has a head of statistics. This head is responsible for statistical standards and answerable to the Government Statistician in this respect, while working to a work schedule set by the departmental head. Data standards and policies are governed by a series of protocols.
9. Training for the wider statistical system and the production of newly trained professional statisticians for the whole of government needs to be a priority for NSS development and should be considered as one of the NSS resource needs. The Guteland report (February 2005) summarises the needs and options.
Role of Stats SA in the NSS
10. The consultant observed a common misunderstanding about the role of Stats SA in the NSS, right across government. Most people understood that Stats SA would take responsibility for improving NSS statistics and that the NSSD – a division of Stats SA - will provide the information for the NSS or at least unveil a ‘system’, which will provide the information. Producer departments are looking to Stats SA to provide the solution to their reporting problems. Stats SA in doing so would be shouldering a burden that it is unable to properly address. The consultant stressed to the Presidency the need to resource the NSS, both in Stats SA and in Departments. This must be clarified in order to make progress with the NSS. The main work needs to be carried out in departments, by departments themselves. The departments need to improve their registers and management information systems to produce quality assured information, with common concepts and definitions used across government.
11. The Statistician-General is empowered by law to harmonise definitions andstandards for statistics produced by government. Stats SA can provide the professional knowledge input to improve and maintain data collection instruments, data systems, archiving and quality standards. StatsSA has neither the capacity to collect the data itself, nor to reengineer systems. The data in departments is typically of too poor a quality for StatsSA to make use of it at present. Work with education data has shown many inconsistencies and incompatibilities between provincial data sets. Home Affairs data has also been problematic; the approach that StatsSA has used to ‘lift’ Home Affairs data to create causes of death series has shown the need to improve collection and processing modalities in Home Affairs. Home Affairs is currently engaged on improving its data systems, in 2004 an invitation was extended to Stats SA to become involved in a working group to develop systems. This invitation has not yet been taken up, even though it is this sort of input to departmental work that could ensure that sustainable improvements are made in departmental statistical work. It is recommended that Stats SA participates in this work.
12. The Statistician-General is also responsible for certifying statistics as official statistics once acceptable standards have been met. In doing this he will be guided by the Statistics Council and assisted by staff from Stats SA. Ultimately Stats SA will be responsible for assessing data quality, and a quality review mechanism for NSS statistics will be required.
Priorities for the NSSD in Stats SA
13. The NSSD in StatsSA cannot achieve much progress without departments being willing to make major improvements in their own data systems, and without the active participation of other divisions in Stats SA. NSSD has only two qualified statisticians available, much of their time is spent making departmental statistics available to users or developing indicators with external users. While this is a very useful exercise in developing a results-based culture, in terms of improving productive capacity it is merely ‘tinkering at the edges’ of the problem. Attempts to address monitoring needs are too often of an ad hoc nature; this is inevitable when sustainable capacity is absent as urgent demands have to be met. However the transaction costs are high. NSSD has too few resources to attempt extensive ad hoc activities, and to foster sustainable capacity. Until NSSD input is combined with departmental support and resources it is unlikely that the NSS will produce much fruit.
14. The Deputy Director General responsible for the NSSD asked that priorities for the NSSD be considered in this mission. It does seem that too few resources are too widely spread, and that the development of the NSS would benefit from a concerted effort on one particular activity. While existing client contact must be maintained, it is recommended that the NSSD concentrate on holding an Indicators Forum to establish priorities, on supportingaworking party arising from the Forum, to assess the resources necessary to develop key series in the NSS; and in drafting the necessary protocols and guidance to establish a working NSS.
15. The NSSD would benefit from a ‘quick win’ but this will be difficult to achieve in the context of improving productive capacity in the NSS. Improvement of entire series will take a considerable period of time and will require resources in departments and expertise from other parts of Stats SA. Initially NSSD should work with Policy Coordination and Advisory Services (PCAS) on agreeing priorities, assessing resource needs and receiving endorsement for these recommendations at the highest level. More information and publicity about the roles and responsibilities of the NSS partners is urgently required by potential NSS partners. This must be followed up with protocols and procedures for statistical integration, data confidentiality and quality assurance.
16. The current Memoranda of Understanding are a major step forward, but additional protocols will be needed as data production moves forward. Once work turns to improvement of priority series it would be advisable for Stats SA to work with a single department on a key series which figures prominently nationally and internationally. The department should also have shown willingness to participate in NSS activities and have an understanding of the requirement to report on its activities to users beyond the department. The Department of Education would satisfy these criteria and has already worked with NSSD staff. NSSD staff are familiar with the available data and some of its weaknesses. Education staff have also recognized the need to revise the instruments, and to ensure universal coverage of schools. The Annual Survey of Schools would be an ideal series for early activity in improving the productive capacity of the NSS. Initially some Stats SA staff would need to work with departmental staff, but the department should take steps early on in the process to recruit a statistician to take responsibility for the task.
Governance of the NSS – Priority Setting
17. The allocation of resources and the setting of NSS priorities is a central function of government and are beyond the remit of the Statistician-General (SG) alone. The driver for NSS development needs to come from a higher, cross-cutting authority in government and this would seem to be the Governance and Administration Cluster, or the Presidency itself. PCAS has a major role to play in determining priorities and reviewing progress in the Government Programme of Action. Statistical priorities for the M & E system should ultimately be in the hands those responsible for coordinating the monitoring systems within government.
18. Governance of the NSS is required to prioritise activities, give political impetus, and to facilitate the required resources. Given that the Presidency and PCAS are responsible for coordinating policies across government and monitoring their performance, ultimately governance of the monitoring system rests there. As Goldstein points out, Stats SA reports to the Minister of Finance whose primary interest is in economic statistics; setting priorities in the wider NSS requires guidance from Cabinet or Presidential level. Clear recommendations, based on evidence will be needed by the top decision makers; these should be researched and developed in a consensual manner which is a prerequisite for working across departments and levels of government in the country. The Governance and Administrative Cluster is responsible for most government functions and would be an appropriate governance structure to guide and monitor the development of the NSS and the indicator system. The responsibility for investigating monitoring needs and priorities rests with PCAS, while a review of the technical requirements for producing statistics of appropriate quality rests with Stats SA and ultimately the SG.