Automotive Technology Certificate
Educational Effectiveness
Assessment Plan
Version 1.0
Adopted by
The Automotive Technology faculty: November 30, 2007
Submitted to
The Dean/Director of the Community and Technical College: November 30, 2007
The Office of Academic Affairs: December 7, 2007
Table of Contents
Mission Statement 3
Program Introduction 3
Assessment Process Introduction 4
Program Outcomes 5
Table 1: Association of Assessment Measures to Program Outcomes 6
Assessment Measures 7
Table 2: Program Outcomes Assessment Measures and Administration 7
Assessment Implementation & Analysis for Program Improvement 8
General Implementation Strategy 8
Method of Data Analysis and Formulation of Recommendations for Program Improvement 8
Modification of the Assessment Plan 8
Appendix A: NATEF Exit Exams 9
Measure Description: 9
Factors that affect the collected data: 9
How to interpret the data: 9
Appendix B: Employer Interview 11
Measure Description: 11
Factors that affect the collected data: 12
How to interpret the data: 12
Appendix C: ASE Exams 13
Measure Description: 13
Factors that affect the collected data: 13
How to interpret the data: 13
Appendix D: Practicum Reports 14
Measure Description: 14
Factors that affect the collected data: 15
How to interpret the data: 15
Mission Statement
It is the mission of the Automotive Technology program at the University of Alaska Anchorage to prepare students for high paying, high demand careers in the transportation repair and maintenance field, to assist working automotive technicians in continuing and professional education, to assist other relevant programs (both secondary and post-secondary throughout Alaska) and to advance the image of the Automotive and Technology industry.
Program Introduction
The Automotive Technology Certificate program is designed to prepare students for careers in the automotive maintenance and repair industry. Curriculum design is based on standards developed by the National Institute for Automotive Excellence (ASE), and each of the options are accredited by ASE through the National Automobile Technicians Education Foundation (NATEF) Classroom and lab activities are designed to prepare students with specific knowledge and skill sets to assess, maintain and repair current technology, and to teach strategies required to adapt to new technologies. Graduates from the Automotive Technology Certificate program may continue through the Automotive Technology career path through the Automotive Technology AAS.
The Automotive Technology certificate program was successfully reviewed by NATEF and recertified in 2006 as part of the General Automotive certification process. The next full recertification is due by April 2011, with a mid-term review in 2009. The recertification process requires a self-study by the faculty and local industry partners serving as an advisory committee, followed by a review by a NATEF certified evaluation team leader working with industry volunteers from the local area. The complete program standards can be viewed in pdf at http://natef.org/documents/Auto_standards_2005.pdf.
Assessment Process Introduction
This document defines the expected student learning outcomes for the Automotive Technology program and outlines a plan for assessing the achievement of the stated outcomes. This assessment plan represents a major change from previous plans in that we have elected to use fewer more general outcomes. We will continue to assess the specific NATEF technical areas as embedded items within the assessment tools.
The development of the outcomes consisted of a review of NATEF Standards, Ford Motor Company standards, and General Motors standards. In addition, the Automotive Technology advisory committee reviews the curriculum at least twice during the 5-year certification cycle, and program effectiveness at least one time per year.
One recent addition to the plan is the addition of external national exit tests taken by students in key courses. These tests are provided by NATEF and proctored by the faculty responsible for the courses. Additionally, student and mentor reports are being standardized and upgraded for the mandatory practicum. The faculty met and accepted the outcomes and assessment processes on November 30, 2007.
Faculty from the UAS and UAF Automotive Technology programs are scheduled to meet with the UAA faculty in 2008 to review curriculum and assessment issues to ensure that our curriculum remains consistent between the MAU’s.
Because the Automotive Technology AAS and Certificate programs are based on the same core curriculum and the same national standards, the assessment plans for the 2 programs are nearly identical. The ASE/NATEF certification of the general automotive option covers both the AAS and certificate programs.
Program Outcomes
At the completion of this program, students are able to:
§ Demonstrate technical knowledge and skills necessary for success in the automotive maintenance and repair industry.
§ Demonstrate academic proficiency necessary to pass national examinations.
§ Demonstrate proficiency in performing occupationally related tasks in a professional setting.
§ Integrate knowledge from diverse areas to develop effective diagnostic and repair strategies involving complex systems.
§ Demonstrate effective oral and written communication skills necessary for success in the workplace.
Table 1: Association of Assessment Measures to Program Outcomes
Outcomes / NATEF Exit Exams / Employer Interview / ASE Exams / Practicum ReportsDemonstrate technical knowledge and skills necessary for success in the automotive maintenance and repair industry. / 1 / 0 / 1 / 1
Demonstrate academic proficiency necessary to pass national examinations. / 1 / 0 / 1 / 0
Demonstrate proficiency in performing occupationally related tasks in a professional setting. / 0 / 0 / 0 / 1
Integrate knowledge from diverse areas to develop effective diagnostic and repair strategies involving complex systems. / 1 / 0 / 1 / 1
Demonstrate effective oral and written communication skills necessary for success in the workplace. / 0 / 1 / 0 / 1
0 = Measure is not used to measure the associated outcome.
1 = Measure is used to measure the associated outcome.
Assessment Measures
A description of the measures used in the assessment of the program outcomes and their implementation are summarized in Table 2 below. The measures and their relationships to the program outcomes are listed in Table 1, above.
There is a separate appendix for each measure that shows the measure itself and describes its use and the factors that affect the results.
Table 2: Program Outcomes Assessment Measures and Administration
Measure / Description / Frequency/ Start Date / Collection Method / Administered byNATEF Exit Exams / The NATEF exit exams are a series of 8 national exams, corresponding to the 8 automotive areas recognized by the National Institute for Automotive Service Excellence. / Tests will be administered during the semester that the relevant course was taken. / Tests are taken on line. / Faculty or program assistant
Employer Interview / Employer interviews are required in the ADT A102 course and as part of the practicum process. Interviews are conducted by faculty, by students, or by employers brought in from industry for a career day. Interviews are conducted and critiqued. / Fall and Spring Semester / Faculty debriefs employers and an evaluation sheet is collected. / Faculty
ASE Exams / ASE exams are the recognized national certification for the industry / Annual / ASE Exams are administered at official test centers. Our program submits information on graduates and receives a report on those who successfully pass the tests. / Program assistant
Practicum Reports / All students in the program are required to complete a paid practicum. Reports are gathered from students and supervisors during the practicum. / Each semester / Faculty communicate directly with students and mentors / Faculty
Assessment Implementation & Analysis for Program Improvement
General Implementation Strategy
NATEF exit exams were implemented during the 2006-2007 academic year. All 8 tests were administered to students in the final semester of the program. Due to constructive feedback from students and faculty, the faculty elected to have relevant tests administered to students each semester, based on the courses taken during that semester. (e.g. students will take the exit exam for the electrical area during the electrical 3 course.)
Employer interviews have been a part of the program for several years but will be included in the formal assessment process for the first time in Spring 2008. The General Automotive option will pilot the data collection through in conjunction with the career day.
Method of Data Analysis and Formulation of Recommendations for Program Improvement
The program faculty will meet at least once a year with the NATEF advisory committee to review the data collected using the assessment measures. This meeting should result in recommendations for program changes that are designed to enhance performance relative to the program’s outcomes. The results of the data collection, an interpretation of the results, and the recommended programmatic changes will be forwarded to the Office of Academic Affairs (in the required format) by June 15th each year. A plan for implementing the recommended changes, including of advertising the changes to all the program’s stakeholders, will also be completed at this meeting.
The proposed programmatic changes may be any action or change in policy that the faculty deems as being necessary to improve performance relative to program outcomes. Recommended changes should also consider workload (faculty, staff, and students), budgetary, facilities, and other relevant constraints. A few examples of changes made by programs at UAA include:
o changes in course content, scheduling, sequencing, prerequisites, delivery methods, etc.
o changes in faculty/staff assignments
o changes in advising methods and requirements
o addition and/or replacement of equipment
o changes to facilities
Modification of the Assessment Plan
The faculty, after reviewing the collected data and the processes used to collect it, may decide to alter the assessment plan. Changes may be made to any component of the plan, including the outcomes, assessment measures, or any other aspect of the plan. Faculty will approve any changes to the program. The modified assessment plan will be forwarded to the dean/director’s office and the Office of Academic Affairs.
Appendix A: NATEF Exit Exams
Measure Description:
The NATEF End of Program test series is designed to measure students' knowledge of course content and learning outcomes within an automotive training program. The tests include a combination of theory and vehicle service related questions, and are developed using the NATEF task list.
The End of Program Tests provide instructors and administrators with impartial feedback on how well students are learning and comprehending the materials presented throughout the curriculum. The invaluable information provided to students and school personnel makes this test beneficial to any school. It is available to any entry-level Automobile or Collision Repair and Refinish training program regardless of ASE/NATEF certification status.
These tests are appropriate for students who have nearly completed an individual automotive training course or who are preparing to graduate from a technician-training program. Schools choose which tests in the series to administer to each student. Testing is available in both fall and spring to ensure maximum usefulness.
AUTOMOBILE
· Suspension and Steering
· Brakes
· Electrical/Electronic Systems
· Engine Performance
· Engine Repair
· Automatic Transmission/Transaxle
· Manual Drive Train and Axles
· Heating and Air Conditioning
All students who complete an End of Program Test receive a detailed score report and pass/fail determination. Students who pass the test areas that correspond to the NATEF core areas for program certification will receive an ASE Student Recognition document suitable for framing as well as attractive insignia.
This information was retrieved from the NATEF Website (http://www.natef.org/end_of_program_test.cfm)
Factors that affect the collected data:
These tests must be administered during a specific time window as scheduled by NATEF.
How to interpret the data:
The tests provide an outside review of the student’s academic proficiency with the body of knowledge necessary to be successful in the automotive maintenance and repair industry. In addition, a passing score on the tests indicate the ability to read and interpret data relating to the specific area relating to the test.
The rate of passing scores on the tests indicates the effectiveness of instruction and the student’s ability to retain and apply the subject material.
The tests are proprietary in nature and cannot be included in this document. The data will be tracked as a percentage of students achieving a passing grade for each area tested.
Criteria / Satisfactory (3) / Needs Improvement (2) / Not Satisfactory (1)Students are well prepared to take the standardized NATEF exit exams. / 90% of students taking the NATEF exit exams receive a passing score. / 80% of students taking the NATEF exit exams receive a passing score. / Less than 80% of students taking the NATEF exit exams receive a passing score.
Appendix B: Employer Interview
Measure Description:
This measure consists of an interview by an either an instructor, a panel of students, or an employer in the automotive repair industry. Students are interviewed and critiqued. The purpose of this measure is to ensure that students learn the skills necessary to successfully interview for a position in the automotive repair industry. A sample assessment form is shown below.
Factors that affect the collected data:
This assessment is made toward the end of the first course in the program. It is expected that the results may not be as positive as a similar exercise, however the information gathered during this assessment can be used to prepare students for interviews to achieve a practicum placement.
How to interpret the data:
Criteria / Satisfactory (3) / Needs Improvement (2) / Not Satisfactory (1)The student demonstrates clear and concise written skills in preparation of resume / The resume is consistent in formatting. Format enhances the reader’s ability to locate key information. Grammar and punctuation are correct. / There are few problems in format, grammar and layout, but readability is not dramatically affected. / The format, grammar or layout interfere with the reader’s ability to find key information
The student projects a professional appearance, appropriate for the position for which employment is sought. / Grooming and hygiene are excellent. Hair is worn in an appropriate manner for the shop environment. The student is not wearing excessive jewelry. / Appropriate clothing is worn but is wrinkled, soiled, or excessively worn. Personal hygiene is satisfactory. / Clothing is inappropriate for the work environment. Poor hygiene or grooming creates a distraction. .
The student projects a confident, outgoing, and positive attitude. / The student is on time for the interview and projects a level of confidence consistent with the applicant’s level of expertise. The student expresses a willingness to learn and a positive attitude toward employment in the field and specific to the employer. / The student is positive but does not project enthusiasm for the career field or the specific employer. The interviewer is required to work at getting information from the applicant. / The student projects a negative attitude toward the industry, the employer, or toward work in general. The student is late for the interview.
The student demonstrates a knowledge of the importance of education and professional certification for success in the automotive industry, / The student clearly and effectively articulates the nature of the program in which she/he is currently enrolled, goals for achieving national certifications, and the importance of career-long professional development to success in the field. / The student is able to articulate basic information regarding the AAS program and expresses a desire to become ASE certified. / The student is not knowledgeable about the program in which he/she is enrolled.
The student communicates in a clear and concise manner. / Responses to questions indicate good listening skills. Reponses are clear and concise. Student makes eye contact with interviewer. Communication indicates enthusiasm for the position. / Responses are respectful and on subject but may not be complete. Listening skills may not be evident through applicant’s answers. Communication does not indicate enthusiasm. / The student’s lack of communication skills is distracting to the interviewer.
Appendix C: ASE Exams