2015-16
This document provides an overview of the evaluation plan of the King University School of Nursing DNP Program. The plan includes a description of the plan, sources of data, process and timeline. Following are the most current data from the identified sources and example of metric-driven process improvement and changes. / DNP Evaluation Plan
Evaluation Plan
Doctor of Nursing Practice
Introduction:
The overall evaluation plan for the School of Nursing Doctor of Nursing Practice Degree Program at King University assesses both student learning outcomes and program effectiveness. Additionally, the evaluation plan and process contain both formative and summative assessment methods, and methods which render both quantitative and qualitative data. The evaluation plan for the DNP program is comprehensive across the entire program of study as well as individual courses and incremental time intervals. The evaluation process is performed systematically across the program to deliver consistency and to strengthen the validity of results. Ultimately the purpose of program evaluation is to drive improvements, both in process and outcomes. Deliberate attention is given to:
1) What is going well?
2) What can be improved?
3) How and when to measure in order to know when success is reached and when revisions are indicated?
The DNP evaluation plan is reviewed for needed updates and revisions annually during the annual DNP evaluation day at the end of each spring semester.
Sources of Assessment Data and Information:
A variety of sources provide data and information for the evaluation of DNP student learning and the program effectiveness. These sources are multi-level and offer diversity of formative, summative, quantitative, qualitative, student and program assessment information. Information and data are sought from the following sources:
- IDEA Surveys
- Preceptor evaluations
- Student evaluations
- Compliance Assist Data (Program Level Assessment)
- Final Competency Translational Research Project
- Attrition Rates
- Alumni Accomplishments/Activities: Professional ,Scholarly, Service, Educational
Even though some overlap exists with formative and summative assessment methods, the table below seeks to categorize sources of evaluation data and information as to formative/summative and quantitative/qualitative.
Formative / Summative / Quantitative / QualitativeIDEA Surveys / ü / ü / ü
Online Course Evaluations / ü / ü
Preceptor Evaluations / ü / ü / ü / ü
Student Evaluations / ü / ü / ü
Compliance Assist / ü / ü / ü
Final Competency / ü / ü / ü
Attrition Rates / ü / ü
Alumni Information / ü / ü
Process:
The aforementioned measures of formative, summative, quantitative and qualitative information are collected, analyzed, presented and reviewed by faculty and leaders in the SON Doctor of Nursing Practice program. After which, opportunities for improvement are identified, refined and a plan put in place for accomplishment of identified goals. The aforementioned sources of evaluation information and data are delegated to specific faulty for oversight and maintenance of currency, and for reporting at full DNP faculty meetings for discussion, action planning and goal setting. The process of evaluation occurs in a continuum, however, certain activities occur during the academic year at specific times as a result of timeliness of survey instruments, availability of results and formulation of action plans for identified opportunities for improvement.
Timeline:
The timeline below illustrates assessment instruments and activities though the academic year. The timeline is constructed as a loop, as the evaluation process is the continual practice of analyzing, planning, acting, and checking of results in an ongoing fashion. The continual improvement process is employed for the purpose of studying the most current data for metric-driven decision-making and acting on indicated revisions in a timely fashion, in order to produce formative results that optimize student learning outcomes and enhance overall program effectiveness.
Timeline:
The graphic timeline below illustrates assessment instruments and activities throughout the academic year. All assessment activities are foremost grounded in the program mission and guided by the steps of planning for goals and outcomes, evaluating outcomes and processes, and identifying improvement opportunities.
Review IDEA/Online course surveys from spring
Student/Preceptor Evaluations
Final competency spring graduates
Update on improvement processes identified
• Review IDEA/Online course surveys from fall semester
Student/Preceptor Evaluations
Ongoing/new improvement processes
Review course evaluations
Preceptor, student evaluations
Update, evaluate process improvements
• Annual DNP Program Evaluation Day
• Compliance Assist Data: Program Level Outcomes
• IDEA Survey Aggregate Summary
• Attrition/graduation rates
• Alumni information
• New improvement areas identified, goals set, action plans made
• DNP Evaluation Plan review and revision
Program Outcomes and Student Outcomes:
The Doctor of Nursing Practice Program is built on the American Association of Colleges of Nursing Essentials for Baccalaureate and Graduate Nursing Education (2011). The AACN DNP Essentials are found in the appendices as appendix A.
Both the graduate program outcomes (DNP program outcomes) and the expected student leaning outcomes are available to all students and faculty and are published in the DNP Handbook and on the King University website. Both are listed below:
DNP Program Outcomes:
1. Provide advanced educational, leadership, and practice awareness in the expansion of nursing knowledge and actions to translate evidence into practice for outcomes improvement.
2. Prepare the nurse for advanced roles in leadership and practice, with the skills necessary to lead and participate in inter-disciplinary teams to improve outcomes among populations and across systems.
3. Create an environment of scholarly inquiry that provides self-directed learning and advancement, employing the tools of theory, research application, outcomes measures and quality management.
4. Undergird students in the philosophy of Christian principles for personal and professional growth that will serve to enhance both the student and society.
Expected DNP Student Learning Outcomes:
1. Integrate knowledge from the fields of nursing, biological sciences, psychosocial studies, public health and organizational management to drive improvements in nursing care across diverse populations and settings.
2. Plan, implement and lead change strategies across systems to meet the current and future health care needs of patient populations in producing optimal outcomes with a quality focus and fiscal stewardship.
3. Translate research findings and theoretical concepts into practice to resolve problems, promote desired change and disseminate new knowledge for the integration of best practices and performance improvement.
4. Employ information systems to implement quality initiatives, support best practices, substantiate administrative decision-making and monitor outcomes on the individual, system and aggregate levels.
5. Intercede at the systems and governing levels to collaboratively advocate and develop policies to improve health care access, resource allocation, delivery, equity and outcomes.
6. Facilitate and/or lead inter-professional teams in the analysis of complex practice and organizational issues to collaboratively create change for improvements.
7. Integrate concepts, including cultural diversity, psychosocial dimensions, ethical and Christian standards to promote disease prevention and improve the health of patient populations.
8. Demonstrate advanced nursing knowledge and related sciences through the practice of applied interventions, clinical judgment, systems thinking and accountability to improve patient and organizational outcomes.
Program Level Effectiveness
Program level effectiveness is evaluated in the university-wide compliance assist program. The DNP faculty and leaders have identified the following three intended educational outcomes for evaluation and tracking in compliance assist. The identified student learning outcomes to demonstrate program effectiveness are: Students will be able to:
1) Translate research findings to improve health care outcomes
2) Lead inter-professional teams to change health care delivery to meet current and future health care needs.
3) Influence systems and communities through Christian principles, service, scholarship and population-based care.
Each student learning outcome has three assessment measures by which effectiveness is determined. Two of the measures are direct measures, providing evidence of student learning, and one measure for each outcome is indirect, providing qualitative information of the student’s perception, confidence or comfort level with the identified outcome. For each assessment measure, criteria of success - which are the levels of student performance expected to demonstrate mastery or competence of the student learning outcome are delineated. Finally, rationale is provided for each selected criteria level of success.
These outcomes address what students should know or be able to do upon completion of the academic program. Following are the three program level student learning outcomes with the associated assessment measures, (2 direct and 1 indirect), purpose of measure and lastly, criteria for success for each measure.
Student Learning Outcome #1
Translate research findings to improved healthcare outcomes
Assessment Measure #1
[Direct Measure] Translational Research Project Proposal
The intent of this paper is to establish the student’s ability to analyze a problem or issue and formulate a research question with supportive evidence from scholarly literature with a statement of purpose. [Paper]
Criteria for Success
95% of the students will achieve an 85 or greater on this paper
Assessment Measure #2
[Direct measure] Population Health Service Learning Paper
The population health service learning project paper examines a health care topic of interest in a select population of interest, and describes the implementation of a population-focused intervention based on evidence to improve health care outcomes. [Paper]
Criteria for Success
95% of the students will achieve an 85 or great on this assignment
Assessment Measure #3
[Indirect measure] Student Survey
A Likert scale questionnaire will be completed after one year of study and at the end of the program to evaluate the students’ confidence in their ability to translate research as a means of improving healthcare outcomes.
Criteria for Success
A total group mean score of 4 or greater on a 5 point Likert scale
Student Learning Outcome #2
Lead inter-professional teams to change health care delivery to meet current and future health care needs.
Assessment Measure #1
[Direct measure] Leadership Team Assessment [individual rubric-guided presentation]
The student will conduct a team assessment of a permanent leadership team and present to class. The leadership team assessment is an individual rubric-guided assignment
Criteria for Success
95% of the students will achieve an 85 or greater on this assignment
Assessment Measure #2
[Direct Measure] Healthcare Disparity Case Study [group rubric-guided presentation]
Student group presentations will focus on a defined health care disparity case study in which the group will identify inter-disciplinary needs, roles and desired outcomes. The health care disparity case study is a rubric-guided team assignment.
Criteria for Success
95% of the students will achieve an 85 or greater on this assignment
Assessment Measure #3
[Indirect measure] Student Survey
A Likert scale questionnaire completed after 1 year of study and at the end of the program of study will evaluate the students’ perceptions of their ability to perform in various roles on an inter-disciplinary health care team.
Criteria for Success
A total group mean score of 4 or greater on a 5 point Likert scale
Student Learning Outcome #3
Influence systems and communities through Christian principles, service, scholarship and population-based care.
Assessment Measure #1
[Direct Measure] Personal Moral Statement
Personal moral statement paper in which the student reflects upon his or her own moral and ethical foundations and their origins; then applies those beliefs to advanced nursing practice [paper]
Criteria for Success
95% of the students will achieve an 85 or greater on this assignment
Assessment Measure #2
[Direct Measure] Concept Analysis Paper
Concept analysis paper that requires the student to examine a single concept and expound upon its interpretation, meaningfulness and application of that concept to his or her professional practice
Criteria for Success
95% of the students will achieve an 85 or greater on this assignment
Assessment Measure #3
[Indirect measure] Student Survey
A Likert scale questionnaire completed after 1 year of study and at the end of the program of study will evaluate the students’ perceptions of their ability to influence culture in communities and systems through Christian principles, service, scholarship and population-focused care
Criteria for Success
A total group mean score of 4 or greater on a 5 point Likert scale.
Curriculum Mapping
The DNP curriculum map illustrates the 3 identified program level outcomes: 1) translate research findings, 2) lead inter-professional teams, and 3) influence systems and communities through Christian principles , and delineates the level of integration of these concepts based on introduction, reinforcement, and emphasis within the context of individual courses in the DNP program. The curriculum map is found in the appendices as appendix B.
Crosswalks:
Appendix C: CROSSWALK SUMMARY DNP PROGRAM - ALLIGNMENT OF DNP ESSENTIALS WITH STUDENT LEARNING OUTCOMES
Appendix D: DNP PROGRAM – DETAILED: ALIGNMENT OF COURSE OUTCOMES, LEARNING OUTCOMES, PROGRAM OUTCOMES AND AACN DNP ESSENTIALS
Closing the Loop:
- Process Improvement Activities/Projects – Quality management methods and techniques are employed to identify opportunities for improvement, recognize when revisions are indicated, and understand best strategies for sustaining successful processes. The DNP evaluation process is updated annually during the DNP Evaluation Day at the end of each spring semester. It is during this time that new improvement opportunities are identified from the data. Faculty then prioritizes the improvement projects identified to be undertaken, assign responsibility and set goals and timelines. The evaluation of ongoing improvement processes is continual throughout the academic year, with some projects or improvements being retired as goals are met, and others surfacing for intervention, action and measurement.
Appendix A
The Essentials of Doctoral Education for Advanced Nursing Practice
I. Scientific Underpinnings for Practice
II. Organizational and Systems Leadership for Quality Improvement and Systems Thinking
III. Clinical Scholarship and Analytical Methods for Evidence-Based Practice
IV. Information Systems/Technology and Patient Care Technology for the Improvement and Transformation of Health Care