EUROPEAN COMMISSION
***
EUROSTAT
G-2 / Luxembourg, 17/02/2009
Doc MET 1033
INTERNAL Document

Net mass estimation;
methodology to establish EU conversion factors

Data source

There are two options for setting conversion factors to estimate net mass: basing whether on data on value per kilogram or on unit mass (kg per supplementary unit). Unit mass was selected as the primary source for estimations due to the assumption that value per kilogram is more volatile and dependent on the market situation. However, data on value per kilogram are not ignored and can be used as a supplementary data source for some codes or countries where data on value are considered to be of significantly better quality.

For the calculation of conversion factors, data on dispatches and exports as well as mirror arrivals and dispatches of years 2005 and 2007 July – 2008 June were extractedfrom theEurostat referencedatabase. High volume codes (representing ~ 80% of total affected net mass) based on dispatches of 2007 July – 2008 June were selected for thorough manual analysis (only data of the Member States not applying the net mass simplificationor partially collecting information on net mass). Groups of ‘mandatory’ and ‘partially collecting net mass data’ MSs were defined based on the 2009 net mass collection policy for the tables of year 2005 and on the previous collection policy for the tablesof July 2007 – June 2008.

Confidential data were used for the calculation of the indicators (unit mass and value per kilogram). Adjustments were not included, only collected data.

Data filtering

At the beginning, possible outliers were removed from the detailed unit mass and value per kilogram data sets. To obtain normalized data sets for the estimation of outliers, logarithmic transformation on unit mass and value per kilogram data was performed. Further, the first and the third quartiles of the ordered data sets were calculated for the data filtering and the lower (LL) and upper (UL) thresholds determined:

(1)

(2)

Here:

, –the first quartile, – the third quartile (3)

An observation was considered as an outlier if it fell outside the calculated range. Filtering process was performed twice: once the outliers were removed from the raw data sets, the filtered data sets were rechecked for outliers once again.After outliers were removed, exponential transformation was performed on unit mass and value per kilogram data.

Estimation methodology

R squared values were derived for each CN8 code / flow / reporter to evaluate the quality of the relationship between supplementary quantity and net mass (for unit mass tables),or value and net mass (for value per kilogram tables)to have information on the quality of the data during further steps of analysis:

(4)

Here– an average,n – number of observations, – prognosis of function, where – supplementary quantity (or value) and –net mass function.

Average unit mass and value per kilogram were calculated over a 12 month period for each CN8 code / flow / reporter.Subsequently the tables were aggregated to a yearly basis. For the selected high volume CN8 codes, medians and means of average unit mass (or value per kilogram) over Member States weighted by public net mass totals were calculated for each table (see Annex). The medians and means were compared, and the reasons for significant differences were analysed. (Despite filtering outliers during previous steps, not all of them could be identified. Median is robust since it is not so much influenced by outlying observations as the mean. Otherwise, the 'outlying' observations in some cases may be real values.).Thenthe medians in different flowswere compared code by code, and the reasons for significant differences between the medians wereexamined (if the reason was a big discrepancy in the data of one or a fewindividual countries, such medians were excluded from further calculations).A conversion factor was defined as a rough average of the medians(see example below).

For each code, the list of countries, for which an individual conversion factor could be advantageous, was defined. It was considered that a MemberStatecould benefit froma customised conversion factor for the code if the unit mass in the MS in question was consistently significantly higher or lower than the proposed EU conversion factor. However, for some codes technicalinformation on the commodity was used for making a decision on the conversion factor (for example, it is known that the net mass of a liter of beverage equals 1 kg).

Example[*]

CN code / 44071091 / 44039930 / 22029010
Description / Spruce "Picea abies Karst." or silver fir "Abies alba Mill.", sawn or chipped lengthwise, sliced or peeled, of a thickness of > 6 mm (excl. planed, sanded or end-jointed; boards for the manufacture of pencils; wood with a length of <= 125 mm and a thickness of < 12,5 mm) / Eucalyptus wood in the rough, whether or not stripped of bark or sapwood, or roughly squared (excl. rough-cut wood for walking sticks, umbrellas, tool shafts and the like; wood cut into boards or beams, etc.; wood treated with paint, stains, creosote or other preservatives) / Non-alcoholic beverages, not containing milk, milk products and fats derived therefrom (excl. water, fruit or vegetable juices)
Supplementary unit / m3 / m3 / l
Flows: / Medians:
Exports 2005 (EU-25) / 570 / 920 / 0.5
Exports July 2007 – June 2008 (EU-27) / 540 / 940 / 1
Dispatches 2005 (EU-25) / 547 / 1160 / 0.5
Dispatches July 2007 – June 2008 (EU-27) / 583 / 1195 / 1
2005, arrivals from 'non-mandatory' MSs (based on 2009 net mass collection policy of MSs) EU-25 / 520 / 1030 / 1
July 2007 – June 2008, arrivals from 'non-mandatory' MSs (based on 2007, 2008 net mass collection policy of MSs) EU-27 / 476 / 980 / 0.75
2005, dispatches data to 'mandatory' and partially collecting net mass MSs (based on 2009 net mass collection policy of MSs) EU-25 / 543 / 1.1[**] / 1
Proposed conversion factor / 540 / 1040 / 1

The conversion factor for the rest of theCN8 codes (the lower volume ones) is an average unit mass of weighted medians over the Member States of all the selected flows.

It was as well foreseen to calculate a confidence interval for the conversion factors on the basis of July 2007 – June 2008 dispatches:

(5)

Here – estimated conversion factor

– Student’s value

– standard deviation of distribution

– critical value

– sample size

For this reason the sample size for each CN8 code in dispatches July 2007-June 2008 data was calculated.

Annex

The following tables were produced for the final analysis (separately for unit mass and value per kilogram):

  • Exports 2005
  • Exports July 2007 – June 2008
  • Dispatches 2005
  • Dispatches July 2007 – June 2008
  • 2005, arrivals from ‘non-mandatory’ MSs (based on 2009 net mass collection policy of MSs) EU-25
  • July 2007 – June 2008, arrivals from ‘non-mandatory’ MSs (based on 2007, 2008 net mass collection policy of MSs) EU-27
  • 2005, dispatches data to ‘mandatory’ and partially collecting net mass MSs (based on 2009 net mass collection policy of MSs) EU-25

In the mirror flow tables for the year 2005 the following MSs were considered as 'mandatory' and partially collecting net mass data:

  • 'Mandatory' MSs: CZ, EE, ES, GR, IE, LT, LV, MT, PL, SI, SK
  • MSs applying partially the simplification (to ~ 500 codes): DE, DK, FI, HU, PT, SE

In the tables for the period of July 2007 – June 2008 the following split was used:

  • 'Mandatory' MSs: BG, EE, ES, GR, IE, LT, MT, PL, RO, SI, SK
  • MSs applying partially the simplification (to ~ 500 codes): DE, DK, FI, HU, PT, SE

Fields of the produced tables:

Unit mass:

Exports 2005

CN code
Exemption flag
SU
EU-25 – aggregate, EU-25 by reporting MS / Public net mass (tonnes)
Average unit mass
R squared

Exports July 2007 – June 2008

CN code
Exemption flag
SU
EU-27 – aggregate, EU-27 by reporting MS / Public net mass (tonnes)
Average unit mass
R squared

Dispatches 2005

CN code
Exemption flag
SU
EU-25 – aggregate, EU-25 by reporting MS / Public net mass (tonnes)
Average unit mass
R squared
Mandatory MSs (based on 2009 data) – aggregate / Public net mass (tonnes)
Average unit mass
R squared
Dispatches of non-mandatory MSs (based on 2009 data) – aggregate / Public net mass (tonnes)
Average unit mass
R squared
MSs collecting partial information on net mass (based on 2009 data) – aggregate / Public net mass (tonnes)
Average unit mass
R squared

Dispatches July 2007 – June 2008

CN code
Exemption flag
SU
EU-27– aggregate, EU-27 by reporting MSs (only mandatory and collecting partial data MSs) / Public net mass (tonnes)
Average unit mass
R squared
N – sample size
Mandatory MSs – aggregate / Public net mass (tonnes)
Average unit mass
R squared
N – sample size
MSs collecting partial information on net mass – aggregate / Public net mass (tonnes)
Average unit mass
R squared
N – sample size

2005, arrivals from non-mandatory MSs (based on 2009 net mass collection policy of MSs) EU-25

CN code
Exemption flag
SU
EU-25 arrivals from non-mandatory MSs to mandatory MSs and MSs collecting partial information on net mass – by partner / Public net mass (tonnes)
Average unit mass
R squared
EU-25 arrivals from non-mandatory MSs to mandatory MSs – aggregate / Public net mass (tonnes)
Average unit mass
R squared
EU-25 arrivals from non-mandatory MSs to MSs collecting partial information on net mass – aggregate / Public net mass (tonnes)
Average unit mass
R squared

July 2007 – June 2008, arrivals from non-mandatory MSs (based on 2007, 2008 net mass collection policy of MSs) EU-27

CN code
Exemption flag
SU
EU-27– arrivals from non-mandatory MSs to mandatory MSs and MSs collecting partial information on net mass – by partner / Public net mass (tonnes)
Average unit mass
R squared
EU-27 arrivals from non-mandatory MSs to mandatory MSs – aggregate / Public net mass (tonnes)
Average unit mass
R squared
EU-27 arrivals from non-mandatory MSs to MSs collecting partial information on net mass – aggregate / Public net mass (tonnes)
Average unit mass
R squared

2005, dispatches data to mandatory and partially collecting net mass MSs (based on 2009 net mass collection policy of MSs) EU-25

CN code
Exemption flag
SU
EU-25 dispatches from non-mandatory MSs to mandatory MSs and MSs partially collecting partial information on net mass – by reporting MS / Public net mass (tonnes)
Average unit mass
R squared
EU-25 dispatches from non-mandatory to mandatory MSs – aggregate / Public net mass (tonnes)
Average unit mass
R squared
EU-25 dispatches from non-mandatory MSs to MSs partially collecting net mass data – aggregate / Public net mass (tonnes)
Average unit mass
R squared

Value per kilogram:

Exports 2005

CN code
Exemption flag
SU
EU-25 – aggregate, EU-25 by reporting MS / Public trade value (thous. EUR)
Average value per kilogram
R squared

Exports July 2007 – June 2008

CN code
Exemption flag
SU
EU-27 – aggregate, EU-27 by reporting MS / Public trade value (thous. EUR)
Average value per kilogram
R squared

Dispatches 2005

CN code
Exemption flag
SU
EU-25 – aggregate, EU-25 by reporting MS / Public trade value (thous. EUR)
Average value per kilogram
R squared
Mandatory MSs (based on 2009 data) – aggregate / Public trade value (thous. EUR)
Average value per kilogram
R squared
Non-mandatory MSs (based on 2009 data) – aggregate / Public trade value (thous. EUR)
Average value per kilogram
R squared
MSs collecting partial information on net mass (based on 2009 data) – aggregate / Public trade value
Average value per kilogram
R squared

Dispatches July 2007 – June 2008

CN code
Exemption flag
SU
EU-27 – aggregate, EU-27 by reporting MS (only mandatory and collecting partial data MSs) / Public trade value (thous. EUR)
Average value per kilogram
R squared
Mandatory MSs – aggregate / Public trade value (thous. EUR)
Average value per kilogram
R squared
MSs collecting partial information on net mass – aggregate / Public trade value (thous. EUR)
Average value per kilogram
R squared

2005, arrivals from non-mandatory (based on 2009 net mass collection policy of MSs) EU-25

CN code
Exemption flag
SU
EU-25 arrivals from non-mandatory MSs to mandatory MSs and MSs collecting partial information on net mass – by reporting MS / Public trade value (thous. EUR)
Average value per kilogram
R squared
EU-25 arrivals from non-mandatory MSs to mandatory MSs – aggregate / Public trade value (thous. EUR)
Average value per kilogram
R squared
EU-25 arrivals from non-mandatory MSs to MSs collecting partial information on net mass – aggregate / Public trade value (thous. EUR)
Average value per kilogram
R squared

July 2007 – June 2008, arrivals from non-mandatory (based on 2007, 2008 net mass collection policy of MSs) EU-27

CN code
Exemption flag
SU
EU-27 arrivals from non-mandatory MSs to mandatory MSs and MSs collecting partial information on net mass – by reporting MS / Public trade value (thous. EUR)
Average value per kilogram
R squared
EU-27 arrivals to MSs from non-mandatory MSs to mandatory MSs – aggregate / Public trade value (thous. EUR)
Average value per kilogram
R squared
EU-27 arrivals from non-mandatory MSs to MSs collecting partial information on net mass – aggregate / Public trade value (thous. EUR)
Average value per kilogram
R squared

2005, dispatches data to mandatory and partially collecting net mass MSs (based on 2009 net mass collection policy of MSs) EU-25

CN code
Exemption flag
SU
EU-25 dispatches from non-mandatory MSs to mandatory MSs and MSs collecting partial information on net mass – by reporting MS / Public trade value (thous. EUR)
Average value per kilogram
R squared
EU-25 dispatches from non-mandatory MSs to mandatory MSs – aggregate / Public trade value (thous. EUR)
Average value per kilogram
R squared
EU-25 dispatches from non-mandatory MSs to MSs collecting partial information on net mass data – aggregate / Public trade value (thous. EUR)
Average value per kilogram
R squared

Exemption flag refers to whether the CN code had been on the list of ~500 exempted codes.

1

[*]The figures in this table are hypothetical and given just to show examples of possible situations.

[**]This figure is an outlier and is not taken into consideration when defining a conversion factor. The reason for such an outlier is the data available only for one MS for this code. It was not removed during the filtering because of a very small sample size.