Academic Profile Examination Results
Fall 1992 through Spring 2006
Revised by
Laura B. Taylor
Previous Reports Prepared by
Shauna L. Stewart
Ann M. Nakamura
Tony Y. Lam
David W. Murphy
Center for Assessment and Instructional Support
MissouriStateUniversity
Martha Kirker, Ph. D.
Director
Report Number: LOCSUR07-01
TABLE OF CONTENTS
Abstract...... 1
Highlights for Fiscal Year 2006...... 2
Highlights for Fall 1992 Through Spring 2006...... 6
Introduction...... 8
Data and Methods...... 9
Instrument...... 9
Participants...... 9
Data...... 11
Results...... 13
Norm-Referenced Data...... 13
Mean Total Scores for Baccalaureate Students...... 13
College-Level Reading Scores...... 17
College-Level Writing Scores...... 17
Critical Thinking Scores...... 18
College-Level Mathematics Scores...... 18
College-Level Humanities Scores...... 19
College-Level Social Science Scores...... 19
College-Level Natural Science Scores...... 20
Student-Reported Data……………………………………………………….…..21
Mean Total Score by Student-Reported Class Level...... 21
Mean Total Score by Student-Reported GPA...... 21
Mean Total Score by Student-Reported Portion of General
Education/Core Curriculum Requirements Successfully Completed...22
Mean Total Score by Student-Reported Hours Worked...... 23
Mean Total Score by Student-Reported Number of Credits
Transferred...... 24
Criterion-Referenced Data...... 25
Writing Scores...... 26
Fall 2001 – Spring 2006 Writing Summary...... 27
Mathematics Scores...... 29
Fall 2001 – Spring 2006 Mathematics Summary...... 30
Reading/Critical Thinking Scores...... 32
Fall 2001 - Spring 2006 Reading Summary...... 33
Concluding Discussion...... 35
Issues that Pertain to Norm-Referenced Data...... 35
Issues that Pertain to Total Scores for Subgroups of
Student-Reported Data...... 36
Issues that Pertain to Criterion-Referenced Data...... 36
Conclusion...... 38
Appendix A: Specific Materials Covered in the Humanities,
Social Sciences, and Natural Sciences Areas...... 40
Appendix B.1: Proficiency Level Definitions for Writing...... 41
Appendix B.2: Proficiency Level Definitions for Mathematics...... 42
Appendix B.3: Proficiency Level Definitions for Reading/Critical Thinking...... 43
Appendix B.4: Proficiency Level Classifications for Reading/Critical Thinking,
Writing, and Mathematics...... 44
Appendix C.1: Colleges and Universities Included in Comparison Group for
Summer 2003 through Spring 2006...... 45
This report summarizes the results of the Academic Profile examination administered to seniors and juniorsat Southwest Missouri State University (now Missouri StateUniversity)between fall 1992 and spring 2006. The purpose of this examination process was to evaluate the general academic knowledge and skills of the students receiving a baccalaureate degree, to assess the outcomes of the general education program, and to improve the quality of instruction and learning. Future testing will apply the Measure of Academic Proficiency and Progress (MAPP).
For the past fourteen years this campushas consistently assessed baccalaureate graduates. Over 98% of baccalaureate graduates participated with the exception of the initial two years when fiscal year 1995 included 95% and fiscal year 1994 included 90% participation. Seniors represented 88% of the 32,856 students taking the exit exam between fall 1992 and spring 2006.
This report provides an overview of the data collected over the past 14 years. The most recent data from the 2006 fiscal year is highlighted briefly and included in the aggregate summary. Normative data is illustrated by semester ranging from fall 1992 through spring 2006. Student-reported data is based on questions and response options at the time of examination. Due torevisions of the test by Educational Testing Service (ETS), some response options were adjusted. Criterion-referenced data is illustrated to include data from the past 14 years and national comparison scores. In 2001, ETS recalculated criterion for proficiency definitions and comparisons. Years were categorized based on ETS criterion for proficiency definitions and include: fall 1992-summer 1997; fall 1997-summer 2001; fall 2001-spring 2006; and the Educational Testing Service national comparison scores established in the 2001-2002 academic year.
Since fall 1992, this Universityannual mean total score has been at or above the national comparison for every fiscal year.
The report concludes the administration of the Academic Profile edition of the general education examination produced by the Educational Testing Service. This edition is being discontinued and replaced by the Measure of Academic Proficiency and Progress (MAPP). According to the Educational Testing Service, this new general education examination will produce scores that are comparable to the scores produced by the Academic Profile
Norm-Referenced Data
Missouri State University mean scores for the 2006 fiscal year were above the Master’s (Comprehensive) Colleges and Universities I and II-Senior comparative means for all seven of the norm-referenced test dimensions: Reading, Writing, Critical Thinking, Math, Humanities, Social Science, and Natural Science. The subscores are reported on a scale that extends from 100 to 130. Although the subscores are all reported on the same scale, they are not comparable between dimensions.
Norm-Referenced Test / MissouriStateUniversity FY06 Senior Mean / National Comparative MeanReading / 120.79 / 119.88
Writing / 116.06 / 115.42
Critical Thinking / 112.81 / 112.16
Mathematics / 115.66 / 114.60
Humanities / 116.42 / 115.78
Social Science / 115.12 / 114.60
Natural Science / 116.69 / 116.04
Criterion-Referenced Data
For thesummer 2005 through spring 2006 semesters, 78% ofour seniors were proficient at Level 1-writing, the lowest level.
Writing Average Percentage of Seniors (Fiscal Year 2006)Level / Not Proficient / Marginal / Proficient
Level 1- lowest / 4.7% / 17.0% / 78.3%
Level 2 / 29.0% / 46.0% / 24.3%
Level 3- highest / 50.7% / 38.7% / 10.7%
For the summer 2005 through spring 2006 semesters, 67%of the seniorswere proficient at Level 1-mathematics, the lowest level.
Mathematics Average Percentage of Seniors (Fiscal Year 2006)Level / Not Proficient / Marginal / Proficient
Level 1- lowest / 7.3% / 25.7% / 67.0%
Level 2 / 30.0% / 30.3% / 39.7%
Level 3- highest / 67.0% / 24.3% / 8.7%
For the summer 2005 through spring 2006 semesters, 77% of the seniorswere proficient at Level 1-reading, the lowest level.
Reading Average Percentage of Seniors (Fiscal Year 2006)Level / Not Proficient / Marginal / Proficient
Level 1- lowest / 7.3% / 16.0% / 77.0%
Level 2 / 31.0% / 20.3% / 48.3%
Level 3- Critical Thinking / 73.7% / 19.3% / 6.7%
Student-Reported Data
The total scores are reported on a scale that extends from 400 to 500.
Mean total score for Missouri StateUniversity baccalaureate graduates (451.00) was above the national comparative mean total score (448.0).
Average total score was higher for seniors (452.40) than for juniors (446.37).
Mean total scores increased relative to increases in self-reported grade point averages.
Students who took the examination after successfully completing 100% of the general education/core curriculum distribution requirementshad a higher mean total score (452.31) than those students who had completed about 75% of the requirements (445.22) before taking the examination.
Those students who reported working 16 to 30 hours per week had the highest mean total score (452.94).
Students who reported that they had transferred one to 15 credits to MissouriStateUniversity received a higher mean total score (453.73) than students in the other three categories.
During the 14 years, the Universityvirtually met the goal set for baccalaureate students: One hundred percent of baccalaureate graduates will be assessed by a standardized test of general education and the resulting institutional mean will meet or exceed the national comparative mean.
For each fiscal year, the University average score for graduating seniors consistently met or exceeded the national comparative mean.
General education scores for students near graduation followed the national trend of declining several points over the fourteen year time period.
Although students generally met or exceeded the comparative mean for reading, writing, mathematics, and critical thinking, the number of students rising above the minimal proficiency level relative to specified criteria have room for considerable improvement when compared to specific competencies.
Scores in the humanities, social sciences, and natural sciences generally exceeded slightly or met the national mean, and followed the direction of the national trend.
Seniors consistently obtained higher scores than juniors during the fourteen year time period. The average senior scores ranged from 449 to 456 and the average junior scores ranged from 437 to 456.
General education scores increased relative to increases in student reported grade point averages.
Students who took the examination after completing their required general education courses had higher total scores than those students who reported they had completed less than 75% of the general education requirements.
Writing: Recent Senior Proficiency
In the most recent five years, 78% of seniors indicated proficiency at the basic level of writing; they could recognize agreement among basic grammatical elements and incorrect word choice, ordered sentences in a paragraph and elements in an outline.
Twenty-five percent of seniors were able to incorporate new material in a passage, recognize agreement among basic grammatical elements when complicated by intervening words, and combine simple clauses into single, more complex combinations.
Twelve percent of seniors met the highest level of writing and in addition to basic competencies, discriminated between appropriate and inappropriate uses of parallelism, and between appropriate and inappropriate uses of idiomatic language.
Conversely, five percent of the seniors did not meet the most basic level of writing on the examination.
Mathematics: Recent Senior Proficiency
In the most recent five years, 68% of seniors indicated proficiency at the basic level. They could work problems that would most likely be solved by arithmetic and did not involve conversion of units or proportionality, but did involve the informal properties of numbers and operations, often involving the number line, including positive and negative numbers, whole numbers and fractions and square roots and squares of numbers.
Thirty-nine percent of seniorscould also solve problems with some complications, such as complex wording, maximizing or minimizing, and embedded ratios.Those included algebra problems that can be solved by arithmetic. Seniors were able to simplify algebraic expressions, perform basic translation, and draw conclusions from algebraic equations and inequalities. They could interpret a trend represented in a graph or choose a graph that reflects a trend. These seniors also solved problems involving sets with numeric answer choices.
Nine percent of seniors met the highest level of mathematics in addition to basic competencies. They could solve word problems unlikely to be solved by arithmetic. These problems involved difficult arithmetic concepts such as exponents and roots other than squares and percent of increase or decrease. Seniors were able to generalize about numbers, solve problems requiring an understanding of the properties of integers, rational numbers, and interpret a graph in which trends are to be expressed algebraically or require insight or logical reasoning.
Eight percent of seniors did not meet the most basic level of mathematics.
Reading: Recent Senior Proficiency
In the most recent five years, 77% of seniors indicated proficiency at the basic level; recognized factual material explicitly presented in a reading passage and understood the meaning of words or phrases in the context of a reading passage.
Forty-seven percent of seniors could also synthesize material from different sections of a passage. They could recognize valid inferences derived from material in the passage, identify accurate summaries of a passage, understand and interpret figurative language and discern the main idea, purpose, or focus of a passage or a significant portion of a passage.
Critical Thinking: Recent Senior Proficiency
Almost seven percent of seniors were proficient at the highest level of reading which was critical thinking. They could evaluate competing casual explanations, and hypotheses for consistency with known facts, as well as determine the relevance of information for evaluating an argument or conclusion. Seniors could determine whether an artistic interpretation is supported by evidence, and recognize the salient features or themes in a work of art. This level of proficiency also required seniors to evaluate the appropriateness of procedures for investigating a question of causation, evaluate data for inconsistency with known facts, hypotheses, or methods, and recognize flaws and inconsistencies in an argument.
Introduction
The primary purpose of this report is to provide an aggregate overview of the data collected overthe past 14 years. This report provides the results of the Academic Profile examination, which was given to undergraduate studentsbefore graduation. The report provides information collected by the Center for Assessment and Instructional Support from the fall semester of 1992 through the spring semester of 2006.
TheUniversity philosophy of developing educated persons places the student at the center of its efforts. The general education program is the academic foundation that provides the shared knowledge and intellectual tools with which students explore their disciplines. Administering a standardized test of general education provides a means of assessing the general academic knowledge and skills of the students in order to evaluate the outcomes of the general education near graduation. By assessing student learning in this way, steps can be taken to improve the quality of instruction and learning.
The College Board and Educational Testing Service developed the Academic Profile. These organizations define the examination as “…a test of general academic knowledge and skills. It is intended for use by colleges and universities in assessing the outcomes of their general education programs to improve the quality of instruction and learning. The test focuses on the academic skills developed through general education courses rather than on knowledge acquired about the subjects taught in these courses. It does this by testing college-level reading, college-level writing, critical thinking, and mathematics in the context of humanities, social sciences, and natural sciences.”
During a pilot assessment project in the fall semester of 1992, the Academic Profile was an elective general education exit examination given to baccalaureate degree seeking students prior to graduation. The examination continued to be elective until the fall semester of 1993. At that time, a standardized test of general education became a requirement to be fulfilled prior to graduation, according to the undergraduate catalog. For those students who entered a degree program prior to fall 1993, the examination remainedelective. However, students who have entered a degree program since the initiation of the requirement are required to take the examination to receive a baccalaureate degree.
Students are given the option of taking the examination during their senior year or junior year, if leaving campus, prior to graduation. Students should complete their general education requirements before taking the examination. There are two reasons for this. First, this examination is administered to assess the general education program. To obtain an accurate reflection of the general education program, students need to have completed the general education requirements. Second, as stated by The College Board and Educational Testing Service, “a student who has taken general education courses in humanities, social sciences, and natural sciences will be better able to read the materials in the test and answer the questions than a student who has not taken such courses.” MissouriStateUniversity considers students’ experiences during the entire educational career while enrolled as contributing to the general education of graduates.
The examination results are used for a number of purposes. The combined results from the examination aid in the evaluation of the effectiveness of the general education curriculum at the University. The information gained provides a basis for improving the quality of instruction and student learning. The results also represent the performance of students to the public and are incorporated in legislative funding decisions, which influence the general budget of the University. Finally, students can use the examination results as an assessment of their individual abilities relative to the graduating class for that semester and include the information in resumes or portfolios.
Data and Methods
Instrument
This University adopted the short form of the Academic Profile examination for assessment of general education resulting from a four-year baccalaureate degree program. The Academic Profile consists of 36 multiple-choice questions to be answered in 40 minutes.Studentsare allowed to use standard, non-graphing calculators to complete the examination. The materials included in the Academic Profile address issues, themes, and ideas from the humanities, social sciences, and natural sciences. Detailed descriptions of these three academic areas are in Appendix A. The questions on the Academic Profile examination assess college-level reading, college-level writing, critical thinking and mathematics in the context of humanities, social sciences, and natural sciences. Mathematical data are presented not only within the context of the three academic areas, but in a numerical form as well.
Participants
A total of 32,856undergraduates have taken the Academic Profile from fall 1992 through spring 2006. Seniors represented 88% of the total. The following table shows the number of students who took the Academic Profile during each of the41 semesters as well as their self-reported class standing.
Number ofMissouriState Students Who Took the General Education Exam During Each SemesterSemester / Juniors / Seniors / Total*
Fall 1992 / 8 / 278 / 288
Spring 1993 / 96 / 1003 / 1119
Summer 1993 / 25 / 151 / 179
Fall 1993 / 183 / 1054 / 1249
Spring 1994 / 173 / 1219 / 1401
Summer 1994 / 24 / 160 / 190
Fall 1994 / 114 / 824 / 948
Spring 1995 / 169 / 1124 / 1300
Summer 1995 / 18 / 138 / 156
Fall 1995 / 111 / 709 / 823
Spring 1996 / 143 / 1087 / 1238
Summer 1996 / 31 / 141 / 174
Fall 1996 / 125 / 752 / 883
Spring 1997 / 141 / 988 / 1142
Summer 1997 / 23 / 140 / 164
Fall 1997 / 107 / 772 / 882
Spring 1998 / 135 / 1000 / 1145
Summer 1998 / 21 / 125 / 149
Fall 1998 / 115 / 675 / 798
Spring 1999 / 133 / 990 / 1138
Summer 1999 / 21 / 156 / 176
Fall 1999 / 91 / 749 / 844
Spring 2000 / 143 / 990 / 1141
Summer 2000 / 29 / 110 / 141
Fall 2000 / 118 / 696 / 821
Spring 2001 / 140 / 1108 / 1261
Summer 2001 / 26 / 167 / 195
Fall 2001 / 72 / 842 / 922
Spring 2002 / 102 / 1351 / 1469
Summer 2002 / 7 / 166 / 177
Fall 2002 / 60 / 860 / 922
Spring 2003 / 105 / 1408 / 1520
Summer 2003 / 16 / 154 / 173
Fall 2003 / 81 / 811 / 897
Spring 2004 / 100 / 1380 / 1491
Summer 2004 / 9 / 154 / 165
Fall 2004 / 86 / 840 / 928
Spring 2005 / 117 / 1420 / 1544
Summer 2005 / 18 / 139 / 160
Fall 2005 / 87 / 865 / 959
Spring 2006 / 138 / 1430 / 1584
TOTAL / 3,461 / 29,126 / 32,856
*The data presented in this table shows that the majority of the students who took the examination reported being of junior or senior class standing. However, a total of 130 students reported being of sophomore standing and 38 reported being of freshmen standing during the 14 years. The remaining 101 students did not choose to report class standing.