Predicting Academic Success and Retention of Engineering Students Using One, Non-technical Assignment
Laura W. Lackey[1] and W. Jack Lackey[2]
Abstract
Numerous papers have been written on the topic of student retention in higher education. A variety of multi-variable models have been developed to predict student success. This paper evaluates the efficacy of using freshman student scores from one, non-technical assignment to correlate with academic success as measured by grade point average. The predictor assignment is keeping a course notebook and corresponds to the student’s attitude, persistence, and organizational skills rather than math and science preparedness. Statistical analysis, at the 99 percent confidence level, indicated that there was a strong relationship between the student notebook scores and grade point average. Although there was scatter in the data, this one variable does provide insight into student success in the Mercer Engineering program.
Introduction
Predicting the success of students engaged in higher education is important to both university faculty and administrators. Many models have been developed to predict student success and retention in chosen fields of study as well as at the chosen college or university. Many models incorporate scores from standard achievement tests (SAT) and the high school GPA with additional predictive variables.
Success in engineering programs has been linked to a variety of intellectual and non-cognitive characteristics. A variety of researchers have historically used scores from the math portion of the SAT to distinguish persisters from those that either drop out of the engineering curriculum and enter into an alternate field of study and those that no longer pursue any type of secondary education (Elkins and Luetkemeyer, 1974; Dickenson, 1969; Levin and Wycokoff, 1991). Additional intellectual success factors include the high school GPA and university math placement scores (Levin and Wycokoff; 1991; Levin and Wycokoff, 1988; Dorio et al., 1980). Levin and Wycokoff (1988 and 1991) also incorporated a variety of non-intellective factors such as attitude toward high school mathematics and physics into a model used to predict success. Additional non-academic factors such as personality and motivation have all been shown as predictive retention and success variables (Felder et al., 1995; Brown and Cross, 1993; Elkins and Luetkemeyer, 1971; Levin and Wycokoff, 1991).
For admittance to Mercer University School of Engineering (MUSE), potential freshman are required to meet the following three criteria:
- SAT score ³ 1000
- SAT score on math portion (SAT-M) of the exam ³ 550, and
- Academic GPA ³ 3.0 (continuous scale from 0 to 4).
The academic GPA includes only grades from core academic classes such as math, science, and English. Grades from courses such as physical education and band are not included. If students fail to meet one of the above listed criteria, students may still be admitted if their MUSE defined academic preparedness index (API) score is ³ 85, where API is defined as follows:
Pedagogy began to reform in the mid 1980’s as the importance of writing across the curriculum (WAC) was recognized for technical (non-English) courses (Kurfiss, 1985). The importance of writing in science, engineering, and technology coursework is benchmarked by the work of J.A. McLellan and John Dewey (McLellan and Dewey, 1889) and has been successfully applied by a variety of researchers (Boyd and Hassett, 2000; Randolph, 2000; Agrawal, 1997; Sharp, 1997) and integrated into numerous colleges and universities (McLeod and Maimon, 2000). In brief, the WAC system encourages instructors to minimize traditional classroom lecture activities and permit students to learn by writing (Maimon et al., 1989; McLeod and Maimon, 2000). The purpose of this type of active learning (journals, inclass writing, etc) is to provoke students to engage in critical thinking and problem solving within their specific discipline. WAC has been shown to improve student learning by embracing all four learning styles as described by Kolb, 1984 and student’s achieve higher levels of comprehension as described by Bloom’s taxonomy (Bloom, 1956).
MUSE faculty have adopted the WAC concept and it is initially introduced to the students during their freshman year in a Professional Practices course (EGR 108). Student based outcomes for EGR 108 are listed below.
Upon successful completion of this course, the students will have achieved the following learning goals:
(1) To demonstrate the ability to read critically for content, implications, and communication strategies by keeping a critical notebook and writing short essays
(2) To develop an understanding of the history of engineering and its impact upon society by writing several short essays and a comprehensive research paper
(3) To develop and apply methods for solving moral and ethical engineering problems by analyzing and presenting several case studies
(4) To communicate successfully in formal and informal, individual and group presentations.
EGR 108 course content is divided into two distinct modules. Module 1 introduces students to the engineering innovations that have caused paradigm shifts in society. Three books are read: Five Equations that Changed the World, Beyond Engineering, and Science and Technology Today, which document the social, political, and global forces that shape engineering and scientific developments. Students also select a specific engineering innovation, research its development, assess its impact on society, and present their findings in writing and orally. This module is designed to foster critical reading, thinking, writing, and speaking skills. Students are introduced to a variety of active reading strategies and to a variety of rhetorical devices for both written and oral communication. Module 2 introduces the student to personal and professional ethics that govern the actions of engineers. Using a multimedia case study of the Challenger, as well as case studies from the book Engineering Ethics, students identify ethical problems/issues and develop a means for solving them. The engineering codes of ethics serve as a framework for discussing issues of professional conduct. Students focus on what it means to be a responsible engineer and how the actions of engineers can affect the well being of others. Working in small groups, students develop and resolve an ethical case study to present to the class.
Throughout EGR 108, students are required to take critical notes on all reading assignments. Furthermore, class time is often devoted to thoughtful freewrite activities associated with assigned readings. These and all other assignments are kept by the student and organized in a three ring binder as instructed by the professor. The student grade from this three-ring binder is the independent variable used in this study to predict success and persistence of MUSE undergraduate engineering students.
This research relied on the premise that all students meeting the MUSE admittance criteria were academically prepared and capable of succeeding in the engineering curriculum. But unfortunately on a national scale, a large percentage (attrition of 55-60%) of entering engineering students do not complete the degree requirements (Astin 1993a, 1993b; Moller-Wong and Eide, 1997). This paper details the effectiveness of using one non-technical assignment given during the freshman year to predict the success of engineering students.
Methods
Participants
All Mercer Engineering students are required to take an engineering professional practices course that is focused on technical writing, history, and ethics. Students enrolled in the course were primarily freshman and were unaware of the study. The participants, a total of 109 students, were from 6 different EGR 108 sections over a three year period. All study participants were MUSE students. As previously discussed, prior to admittance into MUSE all students must achieve an acceptable level of academic success as measured by their high school grade point average as well as their SAT score. It was therefore assumed that all students participating in the study were well prepared academically and were properly equipped from their K-12 experience to succeed in the engineering curriculum.
Procedure
As an EGR 108 requirement, students keep all course work organized in a three-ring binder that is graded at the end of the semester. Both in writing and through oral instruction, students were advised to organize their notebooks in three separate sections that included class notes and quizzes, both in and out of class writing assignments, and critical notes associated with reading assignments. Notebook grades were dependent on a variety of non-technical issues including notebook completeness, neatness and organization. All tests, quizzes, and writing assignments had been previously graded, and therefore minimal effort was made by the instructor to review the quality of class notes or critical notes taken during the semester. The total grading time per notebook was less than 5 minutes. The study includes students in six sections of the course taught over a three-year period. Possible scores were integers within the range from 0 to 4 with 4 being the highest possible grade.
Analysis
The primary interest was to determine if there was a correlation between individual student notebook scores and grade point averages (GPA; 0 – 4). To this end, least squares linear regression was performed and the confidence intervals for the fitted line as well as the predicted interval were obtained with the statistical package Statistix7.
Results
A strict admittance criterion is used to select students that have an aptitude toward mathematics and science such that they will succeed in the rigorous engineering curriculum. As a result of this criterion, it is assumed that each student admitted has the ability to succeed.
The SAT score is a frequently used predictor of academic success. The relationship between SAT score and MUSE GPA is shown in Figure 1. As expected, it is apparent that as the SAT score increases, the MUSE GPA increases. The R-squared value equals 0.06 while the correlation coefficient (R) equals 0.24, indicating a relatively weak relationship between the variables. Although there was appreciable scatter in the data, the p-value was less than 0.05 indicating a statistically significant relationship between SAT and GPA at the 95% confidence level (i.e., the slope could not be zero). The inside curved lines represent the 95% confidence interval for the fitted line. Unfortunately, SAT scores cannot be used as an accurate predictor of success for individual students as the outside curve marks the 95% predicted interval. For example, statistics predict that student’s with an SAT score of 1200 can have a GPA between approximately 1.1 and 4 with the average being between 2.4 to 2.8. Similar results were observed when the SAT-M scores where analyzed with respect to MUSE GPA. A relationship is noted between the variables, but the relationship is weak having an R-squared value equal to 0.08.
All student notebook scores, current MUSE GPAs, high school GPAs, and SAT scores are shown in Table 1. Of the students that scored a 0 or 1 on their notebook, 67% are either currently on academic warning, suspension, or have been placed on academic probation. But, of the students that scored 2 or better, only 7% are currently struggling academically. The relationship between MUSE GPA and the notebook score is graphically shown in Figure 2. The R2 value equals 0.36 while the correlation coefficient equals 0.6, indicating a moderately strong relationship between the variables. Although there was scatter in the data, the p-value was less than 0.01 indicating a statistically significant relationship between notebook score and GPA at the 99% confidence level.
Figure 3 shows the relationship between the SAT and notebook scores. Note that although statistically a slope of zero could be possible, there is a trend that indicates that as SAT scores increase the predicted notebook score decreases. It is anticipated that students with superior academic preparedness recognize they can skimp on the notebook activity (i.e., everyday readiness) and still obtain the grade desired.
ASEE Southeast Section Conference
10
Table 1. Academic data from all study participants.
ASEE Southeast Section Conference
10
ASEE Southeast Section Conference
10
notebook grade (0-4) / SAT Score / MUSE GPA / HS GPA / StatusASEE Southeast Section Conference
10
0 / 1060 / 0.000 / 2.5 / Suspension0 / 1100 / 0.364 / 3.1 / Probation
0 / 1120 / 1.600 / 3.56 / Warning
0 / 1130 / 1.340 / 3.7 / Probation
0 / 1130 / 2.033 / 3.5 / Warning
0 / 1150 / 2.976 / 3.42
0 / 1230 / 1.661 / 3.84 / Suspension
0 / 1240 / 0.635 / Suspension
0 / 1350 / 1.308 / 3.29 / Probation
1 / 1110 / 2.460 / 3.15
1 / 1120 / 3.534 / 3.92
1 / 1120 / 3.141 / 3.86
1 / 1190 / 1.948 / 3.7 / Warning
1 / 1230 / 1.667 / 3.75 / Probation
1 / 1260 / 1.284 / 3.47 / Suspension
1 / 1290 / 2.080 / 3.72 / Probation
1 / 1290 / 2.620 / 3.68 / Warning
1 / 1290 / 1.727 / 3.5 / Warning
1 / 1290 / 2.859 / 3.95
1 / 1320 / 3.310 / 3.97
1 / 2.519 / Probation
1 / 3.050 / 3.92
1 / 2.269
1 / 1.717 / Warning
2 / 970 / 2.571
2 / 1020 / 2.548 / 3.68
2 / 1030 / 2.600 / 3.5
2 / 1090 / 2.966 / 3.35
2 / 1110 / 2.664 / 3.92
2 / 1120 / 1.229 / 3.23
2 / 1140 / 2.521 / 3.42
2 / 1180 / 2.946 / 3.4
2 / 1180 / 2.720 / 3.25
2 / 1190 / 1.875 / 2.92 / Warning
2 / 1220 / 3.484 / 3.89
2 / 1270 / 3.455 / 3.42
2 / 1290 / 2.336 / 3.19
2 / 1330 / 3.452 / 4
2 / 1360 / 4.000 / 4
2 / 1510 / 3.859 / 3.7
2 / 2.625
2 / 2.553
3 / 940 / 2.554 / 3.69
3 / 980 / 3.250 / 3
3 / 1050 / 2.647 / 3.6
3 / 1080 / 2.891 / 3.5
3 / 1090 / 2.734 / 3.6
3 / 1100 / 3.017 / 3.93
3 / 1110 / 2.447 / 3.29
3 / 1110 / 3.142 / 3.25
3 / 1120 / 2.232 / 3.56
3 / 1140 / 2.621 / 3.8
3 / 1140 / 3.000 / 4
3 / 1150 / 2.708 / 3.28
3 / 1150 / 3.208 / 3.7
3 / 1160 / 1.688 / 3 / Warning
3 / 1160 / 3.036 / 3.5
3 / 1160 / 3.133 / 3.9
3 / 1180 / 3.735
3 / 1190 / 2.871 / 2.72
3 / 1200 / 3.016 / 4
Notebook grade (0 – 4) / SAT Score / MUSE GPA / HS GPA / Status
3 / 1200 / 2.576 / 3.83
3 / 1210 / 3.719 / 3.8
3 / 1230 / 3.476 / 4
3 / 1250 / 3.266 / 3.5
3 / 1250 / 2.404 / 3.42
3 / 1250 / 2.773 / 3.66
3 / 1270 / 3.346 / 3.1
3 / 1280 / 3.406 / 4
3 / 1280 / 4.000 / 3.88
3 / 1290 / 3.786 / 3.96
3 / 1500 / 3.275 / 4
3 / 2.382
3 / 3.589
3 / 3.089
4 / 1420 / 3.880 / 3.98
4 / 920 / 2.464 / 3.79 / Probation
4 / 980 / 2.796 / 3.92
4 / 1000 / 3.488 / 3.5
4 / 1010 / 2.494 / 4
4 / 1020 / 3.591 / 3.1
4 / 1020 / 3.010 / 4
4 / 1020 / 2.488 / 3.43 / Warning
4 / 1030 / 3.466 / 3.6
4 / 1050 / 3.368 / 3.67
4 / 1080 / 2.423 / 4 / Warning
4 / 1090 / 2.773 / 3.75
4 / 1100 / 3.038 / 4
4 / 1120 / 1.264 / 3.22
4 / 1130 / 3.352 / 3.47
4 / 1140 / 2.467 / 3.41
4 / 1150 / 3.258 / 4
4 / 1180 / 3.750
4 / 1230 / 3.984 / 4
4 / 1240 / 2.500 / 3.3 / Warning
4 / 1260 / 3.792 / 4
4 / 1280 / 3.975 / 4
4 / 1310 / 3.630 / 4
4 / 1320 / 3.697 / 3.9
4 / 1340 / 3.820 / 3.19
4 / 1350 / 3.321
4 / 1400 / 3.148 / 3.83
4 / 3.214
4 / 3.639
4 / 3.500 / 3.8
4 / 4.000
4 / 3.285
4 / 3.532
4 / 3.596
ASEE Southeast Section Conference