Goal 4Evaluation of Project Services

To develop and maintain a comprehensive system of objective performance measures that clearly relates to the intended outcomes of the project and will produce quantitative and qualitative data.

Important Note:This Evaluation Goal is one of two components of the project’s evaluation plan. Within Goal 3 – Synthesizing Research and Information several comprehensive evaluation initiatives are detailed specifically in regards to determining the efficacy of the major project training initiatives. Therefore, it is critical to note that the Comprehensive Evaluation Plan is strategically imbedded within the Management Plan to assure an extensive review of results and effects.

Activity/Methodology / Evaluation Question Addressed / Timelines
4.1.1 Satisfaction: Conduct survey (written & telephone
interviews) to determine the satisfaction of the participating faculty, administrators and students.
4.1.2 Faculty Results: Conduct study noted in Goal 3 (Objective
3.3) to obtain information regarding the effects of the Faculty Ambassador Model of Professional Development on faculty actions and student technology use.
4.1.3 Administration Results: Conduct student meta-analysis of the self-assessment results (Faculty Knowledge & Skills) conducted by TCNJ, its partner institutions and any other IHE that showed interest in using the self-assessment protocols developed in Goal.
4.1.4 Administration Results: Conduct a meta-analysis of the self-assessment results (student’s knowledge of, access to, and satisfaction with assistive technology on campus) conducted by TCNJ, its partner institutions and any other IHE that showed interest in using the self-assessment protocols developed in Goal 1 of the project. Compare this analysis with the results of students that participated in this project.
4.1.5Student Achievement: Conduct studies noted in Goal 3
(Objectives 3.1 & 3.3) to determine the effects of the Faculty Ambassador Program on student achievement. / To what degree are the faculty, administrators, and students satisfied with their project participation?
What effect did the Faculty Ambassador Model of Professional Development have on the actions of faculty and student use of enabling technology?
What is the overall knowledge and skills of faculty in regards to equity, reasonable accommodations, and instructional/assistive technology for students with disabilities?
What is the current knowledge of, access to, and satisfaction with assistive technology of students with disabilities? How has this project positively influenced it?
Has student achievement been positively influenced by participation in the Faculty Ambassador Program? / Annually
Spanning Project Period
Conclusion of Project
Conclusion of Project
Spanning Project Period
Activity/Methodology / Evaluation Question Addressed / Timelines
4.2.1 Satisfaction: Conduct survey (written & telephone
interviews) to determine the interest and satisfaction of the participating New Jersey Higher Education Regional Training Centers.
4.2.2 Faculty Results: Conduct study noted in Goal 3 (Objective
3.3) to obtain information regarding the effects of the Faculty Ambassador Model of Professional Development on faculty actions and student technology use.
4.2.3 Administration Results: Conduct follow-up survey with the administrators who attended training sessions.
4.2.4 Faculty Ambassador Program Manual: The training manual will be disseminated to 10 nonparticipating New Jersey Colleges and Universities to obtain a comprehensive review of the content and usability.
4.2.5 Faculty Ambassador Program Training for Regional Training Center Staff: Measure outcomes of workshop through post-session interviews and follow-up surveys to obtain information regarding results and measure the extent to which the program is being used.
4.2.6 Enabling Technology Team: Measure training and technical assistance efforts of the team to obtain information that quantifies the type and nature of services, and qualifies the extent to which their services impacted students and faculty.
4.2.7 Project Web Site: Analyze effectiveness and use by number of hits and email responses/requests from the project’s web site. / To what degree are the New Jersey Higher Education Regional Training Center staff interested in promoting the project activities and satisfied with the project services?
What effect did the Faculty Ambassador Model of Professional Development have on the actions of faculty and student use of enabling technology?
To what extent did the participating administrators increase their knowledge, skills and abilities?
How did they utilize the information within the context of their specific institutions?
To what extent does the Faculty Ambassador Program Training Manual contain accurate and useful information? What would be its usability across institutions?
To what extent will the Regional Training Center Staff promote the use of Faculty Ambassador Program? To what extent is the Faculty Ambassador Program being used and what are the preliminary results?
How often and in what ways have the Enabling Technology Team been used within the project?
How has the training and technical assistance impacted faculty ambassadors and the participating students?
Is the project web site being used and in what ways? / Annually
Spanning Project Period
Annually
Conclusion of Project
Conclusion of Project
On-Going
On-Going
Activity/Methodology / Evaluation Question Addressed / Timelines
4.3.1 Conduct a review of the management plan to assure that all project activities are being accomplished within the time and manner it which they are indicated.
4.3.2 Review project activities to assure that objectives are meeting the intended objectives
4.3.3 Have an outside evaluator assess the project to assure that project goals are being met. / Are the project activities being accomplished?
Are the project activities accomplishing the
stated performance objectives?
Have the project goals been met? / On-Going
On-going
Summer 2003 and
Summer 2005

Page 1 of 40