Monitoring Learning Achievement MONGOLIA
Joint UNESCO-UNICEF project
Monitoring Learning Achievement
(A Study Report)
Ministry of Science, Technology,
Education and Culture
Ulaanbaatar, Mongolia 2000
Preface
“Monitoring Learning Achievement” (MLA) Project which is launched in more than 20 countries around the world, is one of the specific attempts in achieving “Education For All” goals. The MLA project is initiated, guided and assisted by UNESCO/UNICEF.
The UNESCO/UNICEF Monitoring Learning Achievement Project (MLA) is based on the premise that a systematic mechanism for monitoring student learning is essential to the improvement of the quality of basic education.
Tests and questionnaires are developed to monitor the basic learning competencies (BLCs) of learners and the factors, conditions influencing their learning outcomes in the area of Mathematics, Literacy (mongolian) and Life skills.
The Project had begun in May 1999 a survey to study the basic competence level of children of grade 4.
In Mongolia, the MLA project aims to establish and sustain a monitoring culture through capacity building to improve the performance of the educational system. The main importance of this study is to create the evaluation indicators of Education for all.
Mongolian secondary education system has the structure of 10 year (4+2+2).
The secondary education system structure in Mongolia is divided into the following:
1. Primary education - 1 - 4 grades
2. Basic education -5 - 8 grades
3. Complete secondary education - 9 - 10 grades
In our country, from the 4th grade the exams of elementary education are held by the schools, from the 8th grade the exams of the basic education by the local administration and from the 10th the exams of the complete-secondary education by the Ministry of Science, Technology, Education and Culture.
Learning achievement is not stable, and therefore we need monitoring learning achievement. This MLA is of great importance and newness in our country.
In the future, MOSTEC is interested in widening and implementing this study in the pre-primary and non-formal education.
Thank you for all support and promotion from international organizations, which made the “Monitoring Learning Achievement Project” possible in Mongolia, and everybody who participated in the activity.
D.Munkhjargal
National Coordinator for Monitoring Learning Achievement Project
Director of Policy Implementation and Coordination Department
Ministry of Science, Technology, Education and Culture
Contents
Preface
Chapter 1Research purpose and method 5
1.1. About research work
1.2.General structure of research and its sphere
1.3. Project objectives
1.4. Expected Outcomes of research
1.5. Sample Design
1.6. Research methods
Chapter 2Analysis of Tests 11
2.1. Distribution of Pupils’ Scores
1. General Distribution of Pupils’ Scores
2. Gender
3. Location
4. Region
5. Subject
2.2. Achievement Mean scores by percent
1. Achievement Mean Score by percent
2. Correlation between 3 subjects and the mean score
3. Gender differences
4. School Location
5. Analysis of Mean-Score Differences
Conclusion
Chapter 3Analysis of research on comparison
of Mean score and questionnaire19
Part 1. Correlation coefficients between achievement in BLCs and selected variables
Part 2. Characteristics of the school and of pupil
1. –Type of school
2. -Total Enrolment of school
3. -Year of birth
4. -Reasons for starting primary school at the age of 7
5. -How many years did you spend in kindergarten?
6. -Subject
7. -Which subject is the hardest for you?
Conclusion
Part 3. Teacher and Headmaster’s questionnaire 29
3.1 Teacher’s personal and professional characteristics
1. -Teacher’s Sex
2. -Teacher’s Age by interval
3. -Teacher’s Qualification
4. -Teacher’s Qualification level
5. -Teacher’s working experience in education
3.2School Headmaster’s personal and professional characteristics
1. -School Headmaster’s Sex
2. -School Headmaster’s Age
3. -School Headmaster’s Qualification
4. -Headmaster’s working experience in education
3.3Class Teacher’s personal opinions
- Indicate the extent to which these are responsible for the poor performance of pupils.
Conclusion
Part 4. Learning environment39
4.1 Home environment
1. -Place of residence during the study
2. -Do you feel hungry when you are at home?
3. -Family helps with homework
4. -Do you have any of the following where you live?
4.2 Pupil related factors, Free time
1. -Do you have adequate amount of supplies needed for studying?
2. -How do you spend your free time?
3. -How do you arrive at school?
4. –How much time do you spend on your way to school?
4.3 School environment, infrastructure
1. -Classroom by shift
2. -School heating
3. -Electrification
4. -School development fund
5. -Auxiliary farm
4.4 Relationship between teacher and parents
1. -Do you collaborate with parents of pupils?
Conclusion
Chapter 4General Conclusion50
References
Appendix I - List of districts and schools selected for the study
Appendix II - Basic learning competencies (BLCs) tests
Appendix III - MLA Questionnaires
Chapter 1Research purpose and method
1.1. About research work
The World Conference on “Education for all” in Jomtien, Thailand (March 1990) has agreed that improvement in the quality of basic education and the results of the educational system of any country is necessary. Within the provisions of this program, a research on improvement in the quality of basic education has been proposed.
Many studies on education systems in developing countries show the urgent need for these countries to increase the internal efficiency of their primary education system, which is the foundation of children’s future learning acquisition.
The Joint UNESCO-UNICEF Monitoring Learning Achievement (MLA) Project focuses on strengthening the national capacity to monitor the quality of basic educational programs and learning achievement.
Within the provisions of the “Monitoring Learning Achievement” UNESCO-UNICEF Joint project on primary education in Mongolia, standards of basic knowledge and learning achievement of 4th-grade primary school pupils will be examined through means of tests and researches. In conclusion, evaluations will be made on how much of learning achievement depends on the learning environment.
During the research process, pupils’ skills and knowledge of maths, mongolian language and life skills were examined using three tests on maths, mongolian language and life skills set by the International standards.
1.2.General structure of research and its sphere
The assessment covered a sample of 6 from 22 provinces, 85 from 660 primary and secondary schools and 2545 from 56400 pupils of 4th grade.
Tests and questionnaires are made according to international standard. Some of the aspects are adjusted for the country.
Contents:
- 3 tests: Mathematics, Mongolian language and Life skills
- 5 questionnaires: pupils, parents, teachers, school heads, district governors
Table 1.1 General statistics by Province
Province / school / Pupil / Girl / % girl / Mean score % / Teacher1. ARHANGAI / 9 / 267 / 150 / 56.2% / 59.7% / 10
2. DUNDGOVI / 10 / 261 / 143 / 54.8% / 64.2% / 10
3. SELENGE / 13 / 373 / 192 / 51.5% / 68.2% / 13
4. HOVD / 10 / 262 / 143 / 54.6% / 65.5% / 10
5. HENTII / 12 / 347 / 199 / 57.3% / 64.6% / 12
6.ULAANBAATAR / 31 / 1035 / 545 / 52.7% / 72.2% / 32
Total / 85 / 2545 / 1372 / 53.90% / 67.8% / 87
1.3. Project objectives
The main aim of the project is to strengthen the national capacity in monitoring the quality of basic education.
The project will focus on areas where from the standpoint of, and perspective for, quality improvement meeting Basic Learning Competencies (BLC’s) in literacy, numeracy and life skills.
-to provide an information-base for policy interventions for reducing disparities between and within schools in learning outcomes that arise due to such factors as regional, gender and socio-economic differences.
-to establish common levels of learning outcomes.
-to develop methods and indicators for long-term monitoring in order to assist national policies for an Education for All
-to evaluate the basic knowledge and learning achievement among fourth grade pupils of primary education.
-To improve learning achievement
-to establish the main reasons for decreases and increases in learning achievement, and to examine ways in which learning achievement can be improved by studying the environments the pupils are in, which are:
- The pupils’ environment
- family and parental environment
- school and teacher environment.
1.4. Expected Outcomes
The objective of this research is to evaluate the basic education standards and to establish means of improving the standards of primary and basic education, which lays the foundation for the future development of our country.
The project is expected to achieve the following outcomes:
- To study the relationship between learning achievement and learning environment.
- To identify whether the learning achievement satisfies the requirement and level of the education standard
- To strengthen the national capacity building
1.5. Sample Design
Research has involved 6 regions, 2545 pupils of 85 schools, 87 headmasters, 87 teachers, chairman and parents in total. 5% of all pupils involved in the research were 4-th grade pupils. Total number of female pupils involved in the research, 1372 (53.9%), and represents the gender proportion fully.
The sampling was made using UNESCO’s example.
The sample design involves both non-probability (purposive) and probability (random) sample. The purposive sample used for selection of provinces, regions.
Selection of schools, pupils
The pupil selected randomly from each school using UNESCO’s standart.
Selections of schools were made based upon the following factors
-Location and specific features
-School capacity and total number of pupils
1.6. Analysis of Data
A statistical software package (SPSS for Windows) and Excel were used for the purpose.
The results of the achievement test were analyzed in the form of mean score by percent and compared on the basis of province, location and gender. In interpreting data not only the mean score by percent, but also the levels of mastery, minimum and non-mastery were taken into consideration. In the analysis, the pupil was categorized as follows.
80-100% - mastery level
50-79%- minimum level
0-49%- non-mastery level
The correlation of independent variables with dependent variables will be worked out. The dependent variable is the pupils’ achievement mean scores in math, mongolian and life skills. The independent variables include gender, age, and location of school, school-related information, the teacher’s academic and professional qualifications, and experience, class size i.e.
Chapter 2Analysis of Tests
Table 1 The mean score by percent by 3 subjects
% / Math / Language / Life skills / TotalFull marks / 35 / 38 / 59 / 132
Mean score / 59.96% / 74.05% / 69.46% / 67.82%
The highest mean score was 74.05% and this was in the subject of Language (mongolian). Mathematics has become the most difficult area for them and the achievement mean score was poor (60%). (see Table 1)
Overall, pupils’ learning achievement in Maths is considerably lower than their achievement in other subjects.
Table 2 The mean score by percent by 3 subjects
Score / Math / Language / Life skills / Total50-100% / 71.2% / 93.8% / 90.6% / 91.0%
Table 3 Distribution of Pupil’s Score
Score Group (%) / Pupil / %80-100% (mastery level) / 433 / 17%
50-79% (minimum level) / 1884 / 74%
0-49% (non-mastery level) / 228 / 9%
Total / 2545 / 100%
An analysis result reveals that a large proportion of children (91%) obtained more than 50% of marks. In the examinations, 433 (17%) of children scored 80-100% marks (mastery level) and 228 (9%) of children scored 0-49% marks. (see Table 3) This shows that most pupils (74%) have satisfied the minimum requirements.
2.1. Distribution of Pupil’s Score
1. -General Distribution of Pupil’s Score
Table 4
Score Group (%) / Total / % / Mean Score / 80-100% / % / 50-79% / % / 0-49% / 50-100%Total / 2545 / 100% / 67.8% / 433 / 17% / 1884 / 74% / 9% / 91%
Boy / 1173 / 46.1% / 68.2% / 182 / 15.5% / 880 / 75% / 9.5% / 90.5%
Girl / 1372 / 53.9% / 67.3% / 251 / 18.3% / 1004 / 73.2% / 8.5% / 91.5%
Urban / 529 / 20.8% / 75.0% / 179 / 33.8% / 344 / 65.0% / 1.1% / 98.9%
Semi-urban / 815 / 32.0% / 70.8% / 174 / 21.3% / 600 / 73.6% / 5% / 95.0%
Rural / 1201 / 47.2% / 62.5% / 80 / 6.7% / 940 / 78.3% / 15.1% / 84.9%
ARKHANGAI / 267 / 10.5% / 59.7% / 11 / 4.1% / 192 / 71.9% / 24.0% / 76.0%
DUNDGOBI / 261 / 10.3% / 64.2% / 32 / 12.3% / 191 / 73.2% / 14.6% / 85.4%
SELENGE / 373 / 14.7% / 68.2% / 58 / 15.5% / 298 / 79.9% / 4.6% / 95.4%
KHOVD / 262 / 10.3% / 65.5% / 36 / 13.7% / 196 / 74.8% / 11.5% / 88.5%
KHENTII / 347 / 13.6% / 64.6% / 33 / 9.5% / 272 / 78.4% / 12.1% / 87.9%
ULAANBAATAR / 1035 / 40.7% / 72.2% / 263 / 25.4% / 735 / 71.0% / 3.6% / 96.4%
Mathematics / 59.9% / 397 / 15.6% / 1414 / 55.6% / 28.8% / 71.2%
Language / 74.0% / 991 / 38.9% / 1396 / 54.9% / 6.2% / 93.8%
Life skills / 69.46% / 710 / 27.9% / 1595 / 62.7% / 9.4% / 90.6%
An analysis result reveals that a large proportion of children (91%) obtained more than 50% of marks.
2. -Gender
Table 5 Gender Distribution of Pupil’s Score
Score Group (%) / Boy / % / Girl / %80-100% (mastery level) / 182 / 15.5% / 251 / 18.3%
50-79% (minimum level) / 880 / 75.0% / 1004 / 73.2%
0-49% (non-mastery level) / 111 / 9.5% / 117 / 8.5%
Total / 1173 / 100% / 1372 / 100%
When analyzed on the basis of gender, it shows that the proportion of girls who scored 80-100% marks was a bit higher than of the boys. (see Table 5)
3. -Location
Table 6 Location Distribution of Pupil’s Score
Score Group (%) / urban / % / Semi-urban / % / Rural / %80-100% / 179 / 33.8% / 174 / 21.3% / 80 / 6.7%
50-79% / 344 / 65.0% / 600 / 73.6% / 940 / 78.3%
0-49% / 6 / 1.1% / 41 / 5.0% / 181 / 15.1%
Total / 529 / 100% / 815 / 100% / 1201 / 100%
Examination scores evaluated based on location show that children living in urban areas had fairly better results. The proportion of urban children who obtained 80-100% marks was considerably higher than that of children living in semi-urban and rural areas. (see Table 6)
Most of rural children have been able to obtain minimum level (50-79%).
4. Region
Table 7 Region Distribution of Pupil’s Score
The proportion of children living in the capital city, who obtained 80-100% marks, was higher than that of the children of any other regions. (see Table 7)
5. -Subject
Table 8 Subject Distribution of Pupil’s Score
Score Group (%) / Math / % / Language / % / Life skills / %80-100% / 397 / 15.6% / 991 / 38.9% / 710 / 27.9%
50-79% / 1414 / 55.6% / 1396 / 54.9% / 1595 / 62.7%
0-49% / 734 / 28.8% / 158 / 6.2% / 240 / 9.4%
Total / 2545 / 100% / 2545 / 100% / 2545 / 100%
The proportion of children (38.9%) who obtained 80-100% marks, in Mongolian language, was higher than of mathematics and life skills.
In Mathematics, the achievement grades of children (28.8%) were slightly lower than that of other 2 subjects. (see Table 8)
In life skills, most of children (62.7%) obtained minimum level (50-79%).
2.2. Achievement Mean scores by percent
1. General mean score by percent
Table 9 General Mean Score by percent
Total / % / Math / Language / Life skills / MeanTotal / 2545 / 100% / 59.96% / 74.05% / 69.46% / 67.82%
Boy / 1173 / 46.1% / 59.87% / 75.14% / 69.65% / 68.22%
Girl / 1372 / 53.9% / 60.06% / 72.78% / 69.23% / 67.36%
Urban / 529 / 20.8% / 69.81% / 79.24% / 75.94% / 75.00%
Semi-urban / 815 / 32.0% / 62.15% / 77.14% / 73.38% / 70.89%
Rural / 1201 / 47.2% / 54.13% / 69.67% / 63.95% / 62.58%
ARKHANGAI / 267 / 10.5% / 49.90% / 70.22% / 59.25% / 59.79%
DUNDGOBI / 261 / 10.3% / 58.90% / 68.88% / 64.84% / 64.20%
SELENGE / 373 / 14.7% / 58.50% / 73.78% / 72.33% / 68.20%
KHOVD / 262 / 10.3% / 60.58% / 75.46% / 60.64% / 65.56%
KHENTII / 347 / 13.6% / 57.00% / 69.08% / 68.00% / 64.69%
ULAANBAATAR / 1035 / 40.7% / 64.18% / 77.75% / 74.94% / 72.29%
Capital city (UB) / 1035 / 40.7% / 64.18% / 77.75% / 74.94% / 72.29%
Other regions (5 aimags) / 1510 / 59.3% / 57.06% / 71.51% / 65.70% / 64.76%
Pupil Age=12 / 1810 / 71.1% / 59.85% / 74.34% / 69.87% / 68.02%
Pupil Age=11 / 380 / 14.9% / 64.31% / 76.41% / 72.07% / 70.93%
UB=Ulaanbaatar (capital city)
The highest mean score was 74.05%, in the subject of Language (mongolian). Mathematics has become the most difficult area for them and the achievement mean score was poor (60%).
2. -Correlation between 3 subjects and the mean score
Table 10 Correlation between 3 subjects and the mean score
Subject / Math / Language / Life skillsMathematics / 1.000 / 0.459 / 0.471
Language (mongolian) / 0.459 / 1.000 / 0.505
Mean score / 0.827 / 0.797 / 0.795
The correlation between 3 subjects and mean score was highly positive.
On average, the coefficient of correlation appeared to be 0.80. It indicates that the relationship between the 3 tests was around 80 percent and shows that it is sufficient to consider the Total Mean score. (See Table 10)
3. -Gender differences
Table 11 Mean Score by percent (%) by sex
Subject / Boy(n=1173) / Girl
(n=1372) / Total
(n=2545)
Mathematics / 60.06% / 59.87% / 59.96%
Language (mongolian) / 72.78% / 75.14% / 74.05%
Life skills / 69.23% / 69.65% / 69.46%
Mean score / 67.36% / 68.22% / 67.82%
n= number of pupil
Both boys and girls showed almost similar levels of achievement. However the girls showed slightly better results in Language than mathematics. The grades of boys were slightly higher than of the girls in Mathematics. (see Table 11)
4. School location
Table 12 The mean score by percent across 3 subjects
Correlation / Math / Language / Life skills / Total(urban, semi urban, rural) / -0.349 / -0.276 / -0.359 / -0.407
Table 13 Mean Score by percent (%) by Location
Subject / Urban(n=529) / Semi-urban
(n=815) / Rural
(n=1201) / Total
(n=2545)
Mathematics / 69.81% / 62.15% / 54.13% / 59.96%
Language (mongolian) / 79.24% / 77.14% / 69.67% / 74.05%
Life skills / 75.94% / 73.38% / 63.95% / 69.46%
Mean score / 75.00% / 70.89% / 62.58% / 67.82%
A location analysis of the achievement mean score of children shows that performance of the pupils living in urban areas was better than that of the children living in semi-urban and rural areas. The urban children achieved better in mathematics. In this particular area, the performance of the urban children was almost the same as that of the semi-rural children. (see Table 13)
Table 14 Mean Score by percent (%) by Region
Subject / Arkhangai(n=267) / Dundgobi (n=261) / Selenge (n=373) / Khovd
(n=262) / Khentii
(n=347) / Ubaatar (1035) / Total
(n=2545)
Mathematics / 49.90% / 58.90% / 58.50% / 60.58% / 57.00% / 64.18% / 59.96%
Language (mongolian) / 70.22% / 68.88% / 73.78% / 75.46% / 69.08% / 77.75% / 74.05%
Life skills / 59.25% / 64.84% / 72.33% / 60.64% / 68.00% / 74.94% / 69.46%
Mean score / 59.79% / 64.20% / 68.20% / 65.56% / 64.69% / 72.29% / 67.82%
n= number of pupil
When the examinations mean scores were analyzed on the region basis, it was found that the performance of Selenge was better than that of the children of other semi-urban regions.
It is connected with the development of the infrastructure of a certain region located along the railroad.
Learning achievement depends on substructure of the particular region. To improve the overall quality of education, the quality of teaching in rural areas must be improved.
5. Analysis of Mean Score Differences
Mean score-Score Differences are used to account for between-group disparities.
Table 15 General Mean Score-differences
Mean Score Differences / Math / Language / Life skills / MeanGirl/ Boys / -0.19 / 2.36 / 0.42 / 0.86
Urban/ semi-urban / 7.66 / 2.10 / 2.56 / 4.11
Semi urban/ rural / 8.02 / 7.47 / 9.43 / 8.31
Urban/ rural / 15.68 / 9.57 / 11.99 / 12.42
Capital city/ other regions / 7.12 / 6.24 / 9.24 / 7.53
Pupil Age=12/ Pupil Age=11 / -4.46 / -2.07 / -2.20 / -2.91
Mathematics / 0 / -14.09 / -9.50 / -7.86
Mongolian language / 14.09 / 0 / 4.59 / 6.23
Conclusion:
-All pupils performed well in the subject of language but showed insufficient knowledge in mathematics. Therefore improving teaching methods for Maths will also help with the improvement in other subjects. (see Table 15)
-As the results show, learning achievement among girls is higher than among boys. Boys show slightly better results in Maths, while girls tend to dominate languages. Female pupils of primary schools are considered to be more active and energetic, and to possess better learning achievement especially in areas of language.
-The learning achievement gap between pupils living in urban areas and those living in rural areas can clearly be seen from Table 15. Such gap exists as a result of the differences between sub-structures of the areas and the qualities of education. Pupils in rural schools have received lower scores on achievement tests. (see Table 15) These results indicate the need for improvement of teaching methods, learning environment and quality of education in rural areas.
-In addition, learning achievement in Maths among pupils who have started school at the age of 7 is high. This also indicates that starting primary education at a younger age can have positive effects on the learning achievement of pupils.
Chapter 3 Analysis of research on comparison
of Mean score and questionnaire
The following section includes results of tests on three subjects taken from 2545 pupils of 85 schools and questionnaires taken from pupils, teachers and headmasters.
Part 1. Correlation coefficients between achievement in BLCs and
selected variables
Table 16 Correlation coefficients between achievement in BLCs and selected variables
Correlation / Math / Language / Life skills / MeanMean achievement / 0.827 / 0.797 / 0.795
Region (urban, semi urban, rural) / -0.349 / -0.276 / -0.359 / -0.407
Sex / -0.05 / 0.081 / 0.15 / 0.035
Year of birth / 0.134 / 0.105 / 0.125 / 0.157
Month of birth / -.012 / 0.000 / 0.003 / -0.004
How many years did you go to kindergarten? (0-4) / 0.105 / 0.087 / 0.121 / 0.129
Place of residence ( home, relative family..) 1,2 … / 0.092 / -0.113 / -0.175 / -0.154
Teacher’s qualification level / 0.258 / 0.153 / 0.170 / 0.245
Total enrolment of school / 0.155 / 0.209 / 0.218 / 0.236
Headmaster’s age / 0.107 / -0.27 / 0.173 / 0.105
Headmaster’s working experience in education / 0.109 / 0.028 / 0.156 / 0.121
Learning achievement depends much on the location of the school. Also it depends on the year of birth, but it doesn’t depend on month of birth as it is spread equally. Successful results of study depend on teacher’s ability and the number of pupils in a class.