Inventory of the Openness of Official Statistical Data in Russia. ODIN.Russia

Inventory of the Openness of Official Statistical Data in Russia. ODIN.Russia

Inventory of the Openness of Official Statistical Data in Russia GuideODIN.Russia

Nationallevel

2017

Table of Contents

Overview

1Project Management

1.1.Role of Russian Association of Statisticians (RASt) in project

1.2The team

2.Inventory Process

2.1The Inventory Sequence

2.2Locating the websites of the NSS agencies

2.3Scoring the data coverage and aggregation

2.4Scoring the data openness

2.5Website evaluation

2.6National evaluation

2.7Comparison of the ODIN.Russia scores with the ODIN results

3.Contacts

Annexes

Annex 1a: Data coverage and disaggregation guidelines for social statistics

Annex 1b: Data coverage and disaggregation guidelines for economic statistics

Annex 1c: Data coverage and disaggregation guidelines for environmental statistics

Annex 2: Scoring guidelines for data coverage (elements 2-5)

Annex 3: Scoring guidelines for data openness (elements 1-5)

Overview

The project on the inventory of the openness of Russia's official statistical data (ODIN.Russia) is a national Russian project that is methodologically and organizationally linked to the global ODIN project conducted by public non-commercial organization Open Data Watch (ODW, USA) ( ODIN (Open Data Inventory) was launched in 2015 and includes an increasing number of countries. Both projects aim to identify gaps in official statistics, maintain open data policy, improve access to official data and encourage dialogue between government agencies - members of the National Statistical System (NSS) and users of data. It is assumed that, based on the results of the inventory, recommendations will be prepared for the government agencies - members of NSS to increase the coverage and openness of statistical data.

The organizer and the main performer of the ODIN.Russia project is the All-Russian Public Organization "Russian Association of Statisticians" (RASt). In order to provide methodological comparability of the ODIN.Russia with the global results of ODIN, RASt consults with ODW. At the same time, RASt conducts all practical works on inventory of official statistics within the ODIN.Russia project and it is fully responsible for the results.

Inventory is an assessment of the coverage and openness of official statistical data available on the websites of Rosstat and other agencies of the NSS. Coverage is assessed by the availability of the standard set of key statistical indicators presented on the websites of Russian official agencies. This set includes 20 categories of social and economic statistics, as well as environmental statistics. In order to ensure international comparability of results, the set of indicators used in ODIN.Russia is the same to the global ODIN set. Openness is assessed according to the format and conditions for using data. Each data category is assessed according to five elements of coverage and five openness elements. Total score is summarized for each of the data categories and for each element of coverage and openness.

Currently, the inventory is carried out only at the national / federal level. In the future, the project will be expanded to the regional level.

The methodology used in ODIN.Russia is very close to ODIN methodology, but Russian version allows to achieve more complete and objective results, because in ODIN only web-site of the national statistical office and few sites of agencies directly linked to it are examined. ODIN.Russia surveys sites of all agencies of NSS.

The Guide details the organization of the project and the process of assessing the coverage and openness of the official statistical data.

1Project Management

1.1.Role of Russian Association of Statisticians (RASt) in project

The project is implemented by the All-Russian Public Organization "Russian Association of Statisticians" (RASt, The RASt is a voluntary and self-governing association of statisticians, which is funded by membership fees and private donations. The RASt does not receive any funds from any of the Russian official body or from abroad and is completely independent. Thus, the results of the ODIN.Russia project are also independent.

RASt is the initiator and main organizer of this project. During the project, RASt cooperates with other organizations, in particular - with the National Research University "Higher School of Economics" (HSE), the Novosibirsk State University of Economics and Management and the Orenburg State University. RASt also cooperates with the public non-profit organization Open Data Watch (ODW, USA). The purpose of this cooperation is to ensure the methodological and organizational comparability of ODIN.Russia with a similar project ODIN, which is carried out by ODW since 2015 at the global level. To ensure this comparability, the RASt and ODW are consulted on the timing of project, agree a methodology, and also verify the results, since ODW, regardless of the RASt, is also carrying out an inventory of Russian official statistics. Cross checking of results helps to avoid mistakes and inaccuracies, but each organization produces and publishes its results independently. This can be explained, among other things, by the fact that when carrying out an inventory in the framework of ODIN.Russia, the RASt evaluates official statistics published on the websites of all Russian agencies included to the National Statistical System (NSS), regardless of whether these data are being developed in according to the Federal plan of statistical works or other official plans. ODW take into account during the inventory only the data published on the website of the official statistical office (Rosstat) or few other agencies, directly linked to the Rosstat website. Thus, the results of the PAC are by definition broader than the results of ODW, so they may differ.

The RASt does not coordinate the results with the state agencies - members of the NSS, carrying out the inventory completely independently. However, agencies are informed of the results and any changes that are made to their sites to improve the openness are welcome.

1.2The team

Currently, all works under this project are carried out by volunteers on a free basis.

The Scientific leader of the ODIN.Russia project is Alexey Ponomarenko (RASt, HSE); the project’s manager is Kirill Zaikov (Novosibirsk State University of Economics and Management).

Practical work is carried out by a group of volunteers from Moscow, Novosibirsk and Orenburg.

1. Inventory Process

2.1The Inventory Sequence

The inventory process involves the sequential implementation of the following steps:

1. Locating the websites of the NSS agencies, which produce the indicators are included to ODIN global set
2. Scoring the data coverage and aggregation
3. Scoring the data openness
4. Website evaluation
5. National evaluation
6. Comparison of the ODIN.Russia scores with global ODIN results

2.2Locating the websites of the NSS agencies

To carry out an inventory of Russian official statistics, the ODIN.Russia project uses a list of indicators similar to one used by ODW to conduct a global inventory. Thus, the comparability of the results of Russian inventory with the results obtained for other countries is achieved and there is the possibility of rating, that is, determining the place of the NSS of Russia among the NSSs of other countries by the criteria of data coverage and openness.

All indicators are grouped to three categories (social statistics and demography, economic and financial statistics, environmental statistics) and 20 sub-categories. Each of the categories includes one to six specific indicators. A complete list of indicators for each of the categories and sub-categories is provided in annexes 1a-1c.

Volunteers determine the exact name of the Russian indicators on the basis of international analogues on the ODW list. Then, according to the Federal Plan of Statistical Works and according to the plans of statistical work of NSS agencies, a list of indicators is drawn up, indicating the agency responsible for calculating and publishing the relevant indicators. In case of problems with the search for related indicators, volunteers ask the experts in the field (economists, sociologists, demographers, environmentalists) to decide whether the data are actually available and published, and if so published, by which agency.

The list does not include any data that is not available on the websites of agencies. For example, data that can be obtained on request (including for a fee) or by telephone are not included. The data received as a result of additional calculations performed by scientific or public organizations on the basis of official or indirect information are not taken into account. Data from paid business sources are not taken into account. Data not taken into account, if it is necessary to install special software to get it. Since in practice a sufficiently large amount of information can be distributed for a fee or through the additional requests, this restriction can lead to a significant reduction in coverage.

A list of e-mail addresses of websites of agencies responsible for the development and publication of relevant data is compiled.

2.3Scoring the data coverage and aggregation

The coverage is assessed for each of the indicators included to the 20 sub-categories and for each data category listed in Appendix 1a-1c, regardless of which agency's website published the data.

The coverage is assessed by the following criteria:

Time Coverage

•Data availability for the last 5 years

•Data availability for the last 10years

Geographic Coverage

•First administrativelevel

•Second administrativelevel

Disaggregation

•Recommended aggregations as described

The general approach to evaluation is as follows: if the data published on the NSS website fully meets the established criteria, one ball is awarded; if only partially meet the established criteria, then each element can be awarded a half point. Each indicator and each rating category have equal weight. The final evaluation is the sum of the total points awarded for each of the categories and for each evaluation element

2.4Scoring the data openness

Openness is also assessed for each of the indicators, for each of the 20 sub-categories and for each data category listed in Appendix 1a-1c. Additionally, openness is assessed for each of the websites of the agencies participating in the inventory.

The principles for assessing openness are the same as for coverage estimates. That is, if the data published on the website of the NSS fully meets the established criteria, one ball is awarded; if only partially meet the established criteria, then each element can be awarded a half point. Each indicator and each rating category have equal weight. The final evaluation is the sum of the total points awarded for each of the categories and for each evaluation element.

The openness is assessed by the following criteria:

Download Format

•Machinereadable

•Non-proprietary

•User selectable/API or bulk download

Metadata Available

•Metadata available

Licensing Terms

• Terms of use stated/ CC BY 4.0 or similar

More detailed instructions are given in Annex 3

2.5Website evaluation

The website of each agency participating in the inventory is assessed only by the criterion of openness, that is, the format of the data published on the site is evaluated. If the format completely satisfies the requirements listed in paragraph 2.4 for all categories of data, then the site exhibits one point for this category, if the site meets the requirements in part, or a part of the data is published in a format that meets the requirements, and the other part does not satisfy, then a half-point . The overall score is obtained as the sum of the estimates for each of the categories of openness.

2.6National evaluation

The national score is obtained by summing the relevant scores by coverage and by openness, without weighing. In other words, the national score is the sum of all the points scored in assessing all factors of coverage and openness for each of the statistical indicators included in the list.

2.7Comparison of the ODIN.Russia scores with the ODIN results

Both assessments (Russian and global) are conducted independently of each other by different performers, who do not consult each other during the work. Only methodology is agreed. Comparison of Russian and global assessments is carried out in order to avoid mistakes and to receive a balanced final evaluation in terms established and satisfying both parties. In case of disagreement, the experts of the RASt and ODW consult among themselves and find out their cause. If the discrepancy is caused by the difference in approaches, that is, the Russian side additionally took into account the data that is located on the agency's website, which was not considered by ODW, then the discrepancy remains. If errors are detected by any of the partners, they are eliminated.

Thus, the final evaluations of both projects are agreed upon methodologically and are verified by the experts of the partner organization. However, the probability of discrepancy persists, because both organizations collected primary information in different ways.

1. Contacts

Regarding the methodology and results of the ODIN.Russia project, please contact with the mark ODIN_Russia

Annexes

Annex 1a: Data coverage and disaggregation guidelines for social statistics

Social Statistics
Data category / Representative indicators / Recommended disaggregation / Coverage and disaggregation scoring guidelines / Additional notes for scoring coverage and disaggregation
Population & Vital Statistics / (1.1) Population by 5 year age groups (see additional notes for exceptions);
(1.2) Crude birth rate;
(1.3) Crude death rate. / (1.1) Sex; marital status;
(1.2) Sex; marital status;
(1.3) Sex. / To receive full point: Must have all indicators disaggregated as follows: (1.1) population data by 5 year age groups and sex (0.5 point); and (1.2)crude birth and (1.3) death rates by sex (0.5 point).
To receive half point: Must have at least population by 5 year age groups with one disaggregation, or crude birth and death rates by one disaggregation each. / Because it is not always clear whether or notprojections and estimates have been included, include any population data that are available between 2006 and 2015.
Point for first and second administrative levels can be given for population data not disaggregated by 5-year age groups.
Birth and death registration figures do not qualify, but number or births/deaths do qualify.
Education Facilities / (2.1) Number of schools and/or classrooms;
(2.2) Teaching staff;
(2.3) Annual budget. / (2.1) Student age group; school stage; school type.
(2.2) Student age group; school stage; school type;
(2.3) School stage; school year; functional categories (such as teachers, supplies, administration, etc.). / To receive full point: Must haveall three indicators disaggregated as follows: (2.1) number of schools and/or classrooms and(2.2) teaching staff by multiple school stages (primary, lower secondary, secondary, tertiary) and one other disaggregation such as age groups and/or school types (technical training; apprenticeship programs, public/private and so forth); as well as (2.3) annual budget by year and one other disaggregation.
To receive half point:Must have at least one indicator with two disaggregationsor two indicators with one disaggregation each. / School stage at a minimum refers to primary, secondary, and post-secondary or tertiary.
School type may refer to technical schools, public, private, etc.
Education budget must reference school year. GDP contribution to education does not qualify. Budgetary figures for the Ministry of Education does qualify.
Education Outcomes / (3.1) Enrollment rates;
(3.2) Completion and/or
graduation rates;
(3.3) Competency exam results. / (3.1-3.3) Sex; school stage; age groups; school type. / To receive full point: Must have all indicators disaggregated as follows:(3.1) enrollment and (3.2) completion and/or graduation rates by sex and either school stage, school type, or age groups (0.5 point); as well as (3.3) exam results by sex and either school stage, school type, or age groups (0.5 point).
To receive half point:
Must have at least one indicator disaggregated by both sex and either school stage, school type, or age groups. Two indicators with only one disaggregation each is not enough. / Attainment, attendance, and literacy rates are not counted.
The number of students qualifies as indicator (3.1).
Health Facilities / (4.1) Number of health facilities;
(4.2) Hospital capacity (number of beds
and/or budget);
(4.3) Health care staff (doctors, nurses, midwives). / (4.1) Facility type;
(4.2) Facility type;
(4.3) Facility type; staff type. / To receive full point:Must havetwo of three indicators disaggregated by facility type.
To receive half point:
Must have at least one indicator disaggregated by facility type. / Facility types include clinics, hospitals, pharmacies, dental clinics, public/private hospitals, etc.
Health Outcomes / (5.1)Immunization rates (must indicate specific type).
(5.2) Incidence and prevalence major communicable diseases (must indicate specific disease type). / (5.1) Age; sex;
(5.2) Age; sex. / To receive full point: Must have all indicators disaggregated as follows: (5.1)immunization rates by sex and/or age (0.5 point); and (5.2) disease incidence/prevalence by type and sex and/or age (0.5 point).
To receive half point:
Must have both indicators without disaggregation orone indicator with one disaggregation. / HIV incidence/prevalence qualifies as indicator (5.2).
Immunization rates will likely be for children/infants. This is enough to qualify as indicator (5.1) provided rates are disaggregated by immunization type.
Reproductive Health / (6.1)Maternalmortality ratio;
(6.2) infant mortality rate;
(6.3) under-5 mortality rate;
(6.4)fertility rate;
(6.5) contraceptive prevalence rate;
(6.6)adolescent birth rate / (6.1) Disaggregation optional;
(6.2-6.3) Sex;
(6.4-6.6) Other indicator disaggregation optional. / To receive full point: Must have at least five indicators with one disaggregation each.
To receive half point:
Must have at least two disaggregated indicators (unless disaggregation is optional). One must be a mortality rate.
Gender Statistics / (7.1) Data on violence against women;
(7.2) Proportion of women in parliament and/or management/
senior positions; / (7.1-7.2) Disaggregation optional. / To receive full point:Must have
(7.1) data on violence against women (0.5 point); and(7.2) data on women in management or political office (0.5 point).
To receive half point:
Must have at least one indicator. / Share of women in employment in the non-agricultural sector does not qualify as indicator (7.2).
Poverty & Income Statistics / (8.1) Number/
percentage of poor and/or poverty rate/incidence;
(8.2) Distribution of income by deciles or finer. / (8.1-8.2) Disaggregation optional. / To receive full point:
Must have (8.1) number and/or percentage of poor and/or poverty rate/incidence (0.5 point);as well as (8.2) income distribution by deciles or finer (0.5 point).
To receive half point:
Must have at least one indicator. / Poverty (8.1) is a broad indicator that can include international or national poverty lines.
Income distribution presented in sets larger than deciles does not qualify as indicator (8.2).

1