Systematic Plan for Program Evaluation and Assessment of Outcomes

SYSTEMATIC PLAN FOR PROGRAM EVALUATION AND ASSESSMENT OF OUTCOMES

Standard VI The program has an identified plan for systematic program evaluation and assessment of educational outcomes.

CRITERION 18: Written planning for systematic program evaluation and assessment of outcomes includes the following elements:

definitions of criteria and required and elective outcomes;
defined levels of achievement (decision rules for action);
time frames from assessment of all plan components;
person(s) responsible for each component of the plan;
methods and/or tools to assess each criterion and outcome;
reliability, validity, and trustworthiness of methods and tools used;
data collected, analyzed, aggregated, and trended; and,
verification that findings are used for decision making in program development, maintenance, and revision.

Operational Definition
Evaluation Plan—document outlining the components of the Nursing Program and including all the NLNAC required elements for evaluation.
Expected Outcomes:

A written plan is in place that meets the criteria and is used to guide systematic evaluation of the program.

Graduates exhibit critical thinking.
Students complete required Nursing Process Papers (NPPs) satisfactorily at each level in the Program in order to progress.
Graduates show a positive disposition toward critical thinking on the CCTDI.
Graduates score above the 50th percentile on the steps of the nursing process as shown on NCLEX RN Program Reports.
Graduates rate #2, #3, & #5 on the Graduate Self-Assessment Survey (GSS) as 3.5 or better on a scale of 1-5 upon graduation.
Graduates demonstrate communication abilities.
100% of students passing are rated as "satisfactory" on communication skills on their clinical evaluation tools each quarter.
90% of students passing complete satisfactorily (grade 2.0 or above) a communication project at each level (paper, verbal presentation, etc.)
Students rate #7 on the GSS as 3.5 or better on a scale of 1-5 upon graduation.
Graduates demonstrate therapeutic nursing intervention skills.
Students are rated as satisfactory in all clinical outcomes at each level in the Program in order to progress.
Graduates score at or above the 50th percentile for all graduates of all nursing programs in meeting client needs for: safe, effective care environment, health promotion and health maintenance, psychosocial integrity, and physiological integrity as shown in NCLEX RN Program Reports.
Students rate #4 on the GSS as 3.5 or better on a scale of 1-5 upon graduation.

4.Graduates have the knowledge and skills to pass the NCLEX RN and become licensed RNs.

Graduates pass the NCLEX RN at or above the mean rate for the United States.

Patterns and rates of employment are tracked.
At least 90% of graduates are employed as RN's within six months of graduation.
Employment patterns reflect that 80% of those employed are practicing in roles for which they have been prepared (i.e. direct care in hospitals, nursing homes, ambulatory clinics, and home care.)

Graduates are satisfied with the preparation the Program has provided for the RN role.
Graduating students rate program satisfaction as "3.5" or better on a scale of 1 – 5 upon graduation.
Graduate surveys show that 90% of graduates were satisfied with Program preparation.
Students accepted into the program are successful in achieving their goals.
At least 60% of those admitted to the Nursing Program graduate with an AAAS degree within the standard six quarters (two academic years) 70% within 9 quarters (three academic years.)

Component / Where Documentation is Found / Responsible person / Time/ Frequency of Assessment / Assessment method (and rationale) / Reliability, Validity, Trustworthiness of Assessment method / Report of the data to: / Implementation
Assessment / Action
1. Critical Thinking / 1. Nursing course summary evaluations / 1. Program Director, Evaluation Committee / 1a.b. At the end of each quarter: fall, winter, spring / 1a. Nursing Process Papers (NPPs) / Interrater reliability is .467 / 1abcd. Nurs-ing faculty, Advisory Committee / Greater reliability is needed for this to be an effective measure
1a. Faculty will continue to evaluate NPPs using course criteria as a guide. / Our Institutional Effectiveness Dept. will work with us on evaluating after a new rubric is developed.
1b. Scoring data provided by Institutional Effectiveness office / Program Director in cooperation with Institutional Effectiveness office. / Quarterly / 1b. California Critical Thinking Dispositions Inventory (CCTDI) will demonstrate a positive disposition to critical thinking. (Instrument has been evaluated for its ability to measure this attribute.) / 1b. CCTDI is an established instrument that has been statistically evaluated for reliability and validity. / Evaluation Committee / 1b. The overall mean score for each disposition is above 40 (the recommended cut score for each scale) & the overall total mean score is above 280,
Students are demonstrating that they are disposed toward critical thinking and the Program is assisting them to maintain this disposition / 1b. Continue use
1c. NCLEX Program Reports purchased by Program / Program Director / 1c. Twice Annually / 1c. NCLEX feedback information is purchased and evaluated for patterns of change by the Evaluation Committee. (Provides a reliable source of information about outcomes measured by NCLEX.) / 1c. NCLEX-RN is a valid and reliable instrument for measuring entry level RN competency. / Evaluation Committee / 1.c. Mean percentile rankings have varied but without a specific pattern. Most have remained above the 50th percentile. / 1c. Continue current curricular plan for teaching nursing process and monitor.
1d. Graduate Self Assessment Survey / 1d. Program Director / 1d. Quarterly. Students are asked to complete the GSS just prior to graduation. / The survey matches the Program Educational Outcomes
(Students approaching graduation are most able to assess their own abilities with regard to the components of the curriculum.) / Comparative analysis of tool and outcomes shows consistency. The Evaluation Committee constitute a "panel of experts" who are qualified to evaluate for trends which are revealed in the data. / 1d. Nursing Faculty, Advisory Committee / 1d. Students rate #2, #3, & #5 on the GSS 3.5 or greater. They believe they possess critical thinking skills. / No action needed.
2. Interpersonal and communication skills / 2a. Course Evaluation summary annually / 2a.b. Program Director,
Team Coordinators, Clinical Instructors / 2abc. At the end of each quarter: fall, winter, spring / 2ab. Course evaluation forms are filled out each quarter at each level of the program to collect data about success rates on student papers, clinical grades, and theory grades. (Provides a way to quickly monitor for trends in internal measures of success within the program and at each level.) / 2a.b. Course evaluation forms are designed to collect raw "objective" data that is unbiased in nature. The Evaluation Committee constitute a "panel of experts" who are trustworthy to evaluate for trends in the data. / 2a. Nursing faculty, Evaluation Committee, Advisory Committee / 2a. 100% of Students who pass are rated satisfactory. All clinical evaluations include communication abilities. / 2a. No change
2b. Course Evaluation summary annually / See 2a. above / See 2a. above / See 2a. above / See 2a. above / See 2a. above / 2b. 90-100% of passing students at each level have a 2.0 or greater on communication project. / 2b. Continue working with students to improve writing and interaction skills.
2c. Graduate Self Assessment Survey / 2c. Program Director, Evaluation Committee, Advisory Committee / See 2a. above / 2c. See 1d. above / 2c.See 1d above / See 2a. above / 2c. Graduates for last 3 years rated number 7 on GSS as 3.5 or above. / 2c. Continue working with students to improve communication skills.
3. Therapeutic nursing intervention skills / 3a. Syllabi passing requirements and annual course evaluation summary / 3a. Program Director, Team Coordinator, Clinical Instructors / 3a. At the end of each quarter: fall, winter, spring / 3a. See 2a. above
All clinical evaluation tools clearly require satisfactory on all therapeutic intervention outcomes. / 3a.See 2a. above. / 3ab. Nursing Faculty, Evaluation Committee, Advisory Committee / 3a. All graduates are satisfactory in clinical evaluations / No change
3b. NCLEX Program Reports / 3b. Program Director, Evaluation Committee / 3b. Twice Annually / 3b. See 1c. above / 3b. See 1c above / 3b. Evaluation Committee / 3b. In the last 4 years only 3 instances of below the 50th percentile occurred. These were in different measures and do not constitute a trend / Continue to monitor .to determine if trend begins
3c. Graduate Self Assessment Survey / 3c. Program Director,
Evaluation Committee / 3c. At the end of each quarter: fall, winter, spring / 3c. See 1d. above / 3c .See 1d. above / 3c. Evaluation Committee, Nursing faculty, Advisory Committee / 3c. All measures have been above 3.5 for the last 3 years. / No change
4. Performance on NCLEX-RN / 4a. Nursing Commission Reports on NCLEX RN results / 4a. Program Director, Nursing faculty, Evaluation Committee, Advisory Committee / 4a. Annually / 4a. NCLEX-RN / 4a. NCLEX-RN is a valid, reliable, instrument for measuring entry level RN competency. / 4a. Evaluation Committee, Nursing Faculty, Advisory Committee / 4. Pass rates have remained above both state and national or there is some decrease in pass rate. / 4. Monitor closely. Plan for diagnostic readiness exam to assist graduates in studying for exam.
5.Patterns and Rates of Employment / 5ab. Graduate Survey
Advisory committee
Communication with facility administrative personnel / 5ab. Program Director, Nursing Faculty, Evaluation Committee, Advisory Committee / 5. a & b. Survey sent one year after graduation
Other--ongoing / 5. a & b. The survey asks for specific factual data regarding employment.. / 5. Factual data is trustworthy. / 5. Evaluation Committee
Nursing Faculty
Advisory Committee / 5a. All graduates responding are employed.
Informal data indicates all students desiring employment are employed.
b. All graduates responding working in roles for which prepared. / No change
6. Graduate satisfaction / 6a. Graduating Student Self Assessment Survey
b. Graduate surveys one year after graduation / 6ab. Program Director, Evaluation Committee / 6a .At the end of each quarter: fall, winter, spring
6b. One year after graduation for each class / 6. Graduate survey matches the program educational outcomes. / 6. Comparative analysis of tool and outcomes shows consistency. The Evaluation Committee constitute a "panel of experts" who are qualified to evaluate for trends which are revealed in the data. / Evaluation Committee
Nursing Faculty,
Advisory Committee / 6a. Survey shows satisfaction ranging from 3.9 to 4.5 on a 5 point scale with highest ratings most recent. This meets the desired outcome.
6b. Satisfaction has ranged from 3.9-4.27, which meets the desired outcome. / 6a. & b. Continue processes in place
7. Graduation/ attrition / 7. Nursing Program Office
& Institutional Effectiveness Office / 7. Program Director, Evaluation Committee / Quarterly / 7. Graduation/ completion data is collected by the Program Director and is used to evaluate trends in these areas by the / 7. The Evaluation Committee constitute a "panel of experts" who are qualified to evaluate the graduation/ completion data for trends. / Evaluation Committee
Nursing Faculty,
Advisory Committee / 7. The goal of 60% in 6 quarters was not met although there is improvement in the class after support services instituted.
The goals of 70% in 9 quarters was met and actually exceeded. This is a brief gain and we must sustain efforts.. / 7. Continue with supports for student success and continue to implement additional ways to assist students.

S:\X_ALL_DEPTS\Nursing\EVALUATION\EVAL STANDARD VI Rev.doc