SOE Assessment Plan 10/22/12

COMPREHENSIVE ASSESSMENT

PLAN SYSTEM (CAS)

Fall 2012; Revised 2015

Contents

Introduction

Understanding Assessment

Why Assessment?

What Is Assessment?

The School of Education Conceptual Framework

Johns Hopkins University Mission Statement

Core Values of the School of Education

Vision of the School of Education

Mission of the School of Education

The Comprehensive Assessment Plan

Key Components

Annual Assessment Activity Cycle

Standards and Frameworks Addressed

Appendices

Program Improvement Plan...... 19

Program Level Assessment Activities

Introduction

This document, in alignment with the conceptual framework, presents our Johns Hopkins School of Education Assessment Plan (CAS) for all programs. This plan was developed through faculty and staff cooperation, collaboration, reflective practice, and by reviewing performance based assessment measures that ultimately drive program improvement and increase positive graduate outcomes. The members of the School of Education (SOE) advisory groups reviewed the plan and offered suggestions to enhance its effectiveness. The implementation of the plan is an iterative process of continuous improvement.

Understanding Assessment

Why Assessment?

Assessment is a culture of continuous improvement that parallels the School’s focus on scholarship and research.

Assessment ensures that students learn

Assessment results provide evidence of the achievement of learning goals of each individual student when measured against established acceptable and target benchmarks. Each student’s performance is measured through the assessment and benchmarked against either a standardized or assessment-specific scoring guide. These assessments are used to monitor student learning achievement and to provide indicators to guide support and learning experiences.

Assessment ensures that courses and programs are effective

The analyses of assessments results provide indicators of courses’ effectiveness and guidance for courses’ development. The courses assessments are designed to provide evidence of student learning in relation to the objectives and outcomes associated with each specific course. Similarly, evidence across all the courses in a program provides indicators of the strengths and needs of programs to guide the effectiveness and relevance of courses. On a holistic level, these indicators of program effectiveness with data collected providing indicators of student learning are evidence of unit effectiveness.

Another important function for measures of learning is to assist the school administration and individual faculty members to improve programs and courses. By measuring learning the school faculty can evaluate its students’ success at achieving learning goals, can use the measures to plan improvement efforts, and (depending on the type of measures) can provide feedback and guidance for individual students and individual programs.

Assessment ensures that professional standards are met

Assessment results provide evidence that professional standards such as, NCATE, and discipline specific standards are effectively addressed and that students demonstrate achievement of the standards relative to their learning goals.

Assessment to demonstrate accountability (such as in accreditation) is an important reason to assess learning accomplishments. Measures of learning can assure external constituents such as potential students, trustees, public officials, supporters, and accreditors, that the organization meets its goals.

Assessment ensures that the school’s mission is addressed

Assessment results analyzed across the school provide indicators to ensure the effectiveness of learning activities in supporting the achievement of the school’s mission. Program assessments aligned to the school mission provide evidence of the individual programs and students achieving the mission as outlined.

Assessment evaluates how well a school accomplishes the educational aims at the core of its activities. The learning process is separate from the demonstration that students achieve learning goals. Do students achieve learning appropriate to the programs in which they participate? Do they have the knowledge and skills appropriate to their earned degrees? Few characteristics of the school are as important to stakeholders as knowing the accomplishment levels of the school's students when compared against the school's learning goals.

What Is Assessment?

Assessment is deciding what knowledge, skills, and attitudes/dispositions students need to learn and then ensuring that they learn them and are effective in applying them. Assessment is a process of continuous improvement that involves the following steps:

Assessment Steps

  • Set learning goals, objectives, and outcomes
  • Design assessments with meaningful rubrics to measure student learning achievement
  • Provide authentic learning opportunities
  • Assess student learning
  • Collect and analyze results
  • Use findings to inform decisions regarding candidates, programs, and SOE operations
  • Implement evidence based changes
  • Assess again
  • Make modifications as needed
  • Repeat continuously as a systemic process

This process closes the loop by measuring the results of the changes being driven by the data collected.

Assessment Levels

Student Level

  • The goal is to ensure that individual students achieve the designated outcomes for a course.
  • Students must clearly understand the expectations and the benchmarks for their learning
  • Varied opportunities for learning and assessment exist throughout the course
  • Achievement is summarized in grades
  • This is an individual focus

Course Level

  • The intent is to ensure that the students as a whole reach designated course outcomes
  • Varied assignments and assessments from across the entire course
  • Assignments and assessments are aggregated across students and course sections
  • Focus is on both students individually and the course as a whole

Program Level

  • Ensure that students as a whole achieve program level outcomes
  • Key assessments are captured at key points in the program
  • Assessments consist of summative internships, portfolios, capstones or comprehensive exams
  • Focus is on both the students and the programs

Unit Level

  • Ensure that students as a whole achieve unit level outcomes
  • Identified key assessments are captured at key points across all programs
  • Assessments consist of common measures such as entry exams and post-graduation surveys
  • Focus is on students and unit levels

Assessment Types

Critical assessments include measures that are direct and indirect, course embedded and stand-alone, and formative and summative. Direct assessments measure actual performance such as an individual student’s score on a specific assessment, a score on a standardized test such as the Praxis I test, and university supervisor intern observations. Indirect measures include course evaluations, surveys, meeting notes, and stakeholders’ feedback.

Course-embedded measures are assessments within required courses that expose students to systematic learning experiences designed to produce graduates with the particular knowledge or abilities designed to address identified student outcomes. These include test questions or assignments that are often discipline specific. Stand-alone testing or performance measures require students to demonstrate certain knowledge or skills as a requirement for graduation or at some other specific point in their degree program. These include such assessments such as portfolios, comprehensive exams, or standardized tests.

Formative assessments provide feedback and guidance to candidates as they progress in their development toward achievement of mission-based student outcomes. Summative assessments capture performance at the five key points and are measured against expected levels of progress at each of those points. Analysis of both assessments informs changes in courses, programs, and unit operations.

The School of Education Conceptual Framework

Johns Hopkins University Mission Statement

The mission of Johns Hopkins University is to educate its students and cultivate their capacity for life-long learning, to foster independent and original research, and to bring the benefits of discovery to the world.

The School of Education (SOE) addresses this mission through its core values, vision, mission, and conceptual framework graduate outcomes.

Core Values of the School of Education

The School of Education’s core values motivate are reflected in its members’ actions, its initiatives, and its impact.

  • Innovation in discovery, scholarship, leadership, and application to create new knowledge and approaches, and to apply both new technologies and innovative use of technologies addresses the difficult challenges facing education and communities.
  • Excellence as reflective practitioners ensure that faculty and students engage in the process of reflection in and on action for the purpose of continuous improvement and hypotheses based teaching.
  • Collaboration and Partnership in multidisciplinary and inter-institutional teams to address issues facing schools and communities.
  • Evidence-Based Practice informs practice, policy formulation and advocacy, and future research to discover new knowledge and approaches, disseminate it, and apply it to address difficult complex and difficult problems.
  • Integrity to maintain our reputation as a reliable and effective partner in our schools, communities, government agencies, corporations, and professional communities.
  • Civility and Diversity to both learn from and provide support to the array of individuals, families, and communities

Vision of the School of Education

The Johns Hopkins University School of Education will lead the world in attracting the most talented and diverse individuals into the fields of education, counseling, and public safety. We will guarantee educational improvement and community well-being by assuring that our students, and others in the profession, have the most innovative tools and effective approaches to advance learning.

The SOE attractsthe most innovative and progressive scholars without disciplinary boundaries to solve complex educational problems through discovery and dissemination of new knowledge. Grounded in the Johns Hopkins tradition of research and innovation, its researchers and faculty explore the intersection of scientific research and how it can impact PK to higher education. In addition, school partnerships serve as living laboratories to test new and effective approaches to practice and leadership. Many of these partnerships have been recognized and adopted as national models.

Mission of the School of Education

The Johns Hopkins School of Education's (SOE) mission is to support and advance the quality of education and human services for the continuous development of children, youth and adults. This addresses the Johns Hopkins University Mission to educate its students and cultivate their capacity for life-long learning, to foster independent and original research, and to bring the benefits of discovery to the world.

The SOE prepares leaders and change agents who are READY to address complex changes in education and communities.

We accomplish this mission through the following initiatives:

  • Research
  • Teaching
  • Program Development
  • Leadership and Service
  • Partnership Programs

Efforts to address the mission and vision are visible in the conceptual framework proficiencies of our graduates’ who are:

Knowledgeable in their respective content area/disciplines:

  • Demonstrate content knowledge at a high conceptual level
  • Integrate appropriate national, state, and local standards in their work
  • Demonstrate the potential to be an innovative content leader in their classrooms, organizations, and/or professional associations (new)

Reflective practitioners:

  • Modify, differentiate, and analyze effective instruction or intervention within the context of a particular classroom or school
  • Engage in and benefit from research that leads to improved student outcomes

Commitment to diversity:

  • Respect differences among learners (cultural, ethnic, racial, gender, sexual orientation, ability, socio-economic, etc.) in their classroom/school
  • Commit to and advocate for (new) the development of all students
  • Commit to working in schools characterized by diversity of student
  • Articulate different learning needs and differentiate instruction appropriately.

Data-based decision makers:

  • Describe formal and informal assessments by which progress can be measured at the individual student, classroom, and school levels
  • Use data and other evidence to plan for instruction and/or development
  • Utilize an evidence-based approach to teaching and intervention

Integrators of applied technology:

  • Advance instruction through the appropriate integration of technology
  • Seek technologies appropriate to school-based instruction

The Comprehensive Assessment Plan

The Comprehensive Assessment Plan (CAS) builds on the conceptual framework and addresses the professional and state standards that are appropriate to each program. In keeping with the SOE mission to support the continuous development of children, youth, and adults, our assessment plan provides evidence to guide decisions about candidate performance, program quality, and SOE operations in an effort to create effective models for educator preparation.

The CAS was developed by and for the entire School of Education. Its assessment cycle reflects an ongoing systemic approach to continuous improvement (Figure 1). Its logic map illustrates the inputs, outputs, and outcomes/impact of the conceptual framework (Figure 2). It incorporates assessment as a critical driver in addressing the vision for the SOE. It includes key assessments at each of the five transition points: admission, midpoint/pre-clinical experience, program completion/clinical experience, post-graduation (1 years), and post-graduation (3 years). The transition points include both internal and external assessments. The transition points address the conceptual framework graduate outcomes (proficiencies).

The CAS is reviewed annually by all faculty and administrators at the assessment retreat and biannually by the Teaching and Learning Advisory Committee and the National Advisory Council. Tk20 is the assessment technology platform for the collection of SOE assessment data. The CAS is a work in progress and elements of it continue to be developed and implemented.

Formative assessment provides students with ongoing feedback regarding their performance from their instructors, advisors, and clinical supervisors. Summative assessment captures student performance on key assessments at transition points. Faculty share data with individual students to guide and inform their performance. Program faculty with members of their professional community analyze this data to improve their courses and programs. These results are reported for each program in a Program Improvement Plan (PIP).

The SOE faculty and administrators meet to review the PIP plans and aggregate data to identify patterns and trends that are important to the continuous improvement of programs and SOE operations. They compile recommendations for immediate implementation, to forward to the Faculty Senate for further study/deliberations, or to share with the Dean for the planning and budget process.

Figure 1. The Johns Hopkins University School of Education Assessment Cycle


In addition, each program is aligned with national, state, and professional standards. These standards are addressed through assessments for each program. The aim is to demonstrate that SOE graduates are knowledgeable in content, reflective practitioners, committed to diversity, data-based decision makers, and effective integrators of technology as evidenced in their knowledge, skills, and professional dispositions.

To ensure this, student learning achievement is assessed at transition points - admissions, midpoint/pre-internship, post internship/program completion and post-graduation both one and three years out. The goal is to ensure that graduates are ready to address complex, challenging issues to improve education and community well-being.

Table 1

Alignment of Conceptual Framework to Transition Points and Sample Data Points

Conceptual Framework Graduate Outcomes / Transition Points / Sample Data Points
Knowledgeable in their respective content area/discipline / Admission
Midpoint
Program Completion
Post-Graduation / Content analysis
GPA
Internship evaluations & exit survey
Alumni and employer surveys
Reflective practitioners / Midpoint
Program completion
Post-graduation / GPA
Portfolio & internship evaluations
Alumni and employer surveys
Committed to diversity / Admissions
Midpoint
Program completion
Post-graduation / Disposition Survey
GPA
Internship evaluations & exit survey
Alumni and employer surveys
Data-based decision-makers / Midpoint
Program completion
Post-graduation / GPA
Internship evaluations & exit survey
Alumni and employer surveys
Integrators of applied technology / Midpoint
Program completion
Post-graduation / GPA
Internship evaluations & exit survey
Alumni and employer surveys

1

SOE Assessment Plan 10/22/12

Figure 2. Johns Hopkins University Logic Model for the Conceptual Framework

The Johns Hopkins University School of Education
Vision
Mission
Conceptual Framework
Inputs / Assessment Transition Points / Outputs / Outcomes
Initiatives / Domains / Admit / Midpoint or pre-internship / Internship or end of program / Post Grad / Graduates / Impact
1 year / 3 years
Attract talented and diverse students
Provide effective teaching and
innovative tool / Excellent Professional Preparation
High Quality
Research
Innovative
Outreach / Knowledge
Skills
Professional Dispositions / GPA
Content analysis
Essay
Disposition survey / GPA
Pre-internship review
Course based assessments
Field experience evaluation / GPA
Performance in internship, clinical practice, or capstone
Clinical supervisor evaluations
Exit survey / Alumni survey
Employer survey / Alumni survey
Employer survey / Content experts
Reflective practitioners
Committed to diversity
Data based decision makers
Integrators of applied
technology / Graduate leaders and change agents who are ready for challenging schools and communities
Improve education and community well-being
rev 2-15

1