Overview of theCenter for Application and Innovation

Research in Education (CAIRE)

Background

In 2010, Maryland was awarded a $250 million, 4-year grant under the Race to the Top (RTTT) initiative developed by the Obama administration to support K-12 education reforms proposed by states. Maryland used $125 million of its funding to develop 54 projects that are managed by the Maryland State Department of Education (MSDE). One of these projects was established solely to evaluate the success and impacts of the other 53 State–managed projects. MSDE reached an agreement with the University System of Maryland (USM) to conduct these project evaluations and, in order to do so, the USM created the Center for Application and Innovation Research in Education (CAIRE). To date, CAIRE’s primary work has been funded by RTTT but its role is envisioned to be much broader, as described below.

Mission Statement

CAIRE’s overall mission is:

  • To provide Maryland with the overall program evaluation promised in its Race to the Top application
  • To deliver an embedded evaluation system with sustainable evaluation capacity that will be ready and able to continue tracking desired educational outcomes beyond Race to the Top, contingent on the availability of necessary funding.
  • To establish a sustainable network of educational evaluation capacity involving:
  • Institutes of Higher Education (IHEs),
  • Local Education Agencies/School Systems (LEAs), and
  • Maryland State Department of Education (MSDE).

RTTT Evaluation Model

In compliance with the State’s RTTT grant application, CAIRE has followed an evaluation model that focuses on 3 different phases of each project:

  • Phase 1 (years 1-2): “Process and Product”
  • Concerns the creation and implementation of the software systems, the staff development efforts, and any of the many new “products” that will be developed and delivered to the educators in the State of Maryland.
  • Phase 2 (years 2-3): “Utilization”
  • Focused on the use of materials by various stakeholders with the understanding that teachers, principals, and other educators actually utilize materials that have been created.
  • Phase 3 (years 3-4): “Impact”
  • The ultimate reason for the creation of the process, materials, and the eventual product, is to answer the question: Did the processes/products/materials make a difference?

Research Activities

To successfully conduct the evaluations, CAIRE partnered with MSDE and LEAs to align initiatives and assessments that will broaden and deepen Maryland’s evaluation capacity locally, regionally and statewide. Further, CAIRE has recruited educational and evaluation scientists from across USM institutions to provide expertise and aid in the design and implementation of research models and studies. These partnerships and resources have enabled CAIRE to:

  • Develop operational definitions of “success” for each project
  • Conduct surveys of teachers, principals, and other stakeholders affected by the RTTT projects to obtain information about their experiences and perspectives
  • Analyze quantitative data to examine patterns and trends in project outcomes using methods such as time series analysis, path analysis, cross tabulation, regression analysis, etc.
  • Interview staff, project managers, and others involved in projects to obtain relevant information and develop an understanding of how diverse stakeholders understand the goals and scope of RTTT projects
  • Conduct case studies
  • Collect and analyze qualitative data
  • Organize field exercises with users of products/services in various projects to measure utilization and impact
  • Arrange site visits to schools in various LEAs for observational studies of student learning and teacher instruction
  • Test online systems and IT structures to measure accessibility, usability, network load, etc.

Current Status

CAIRE completed its final assessment of the “process and product” phase in September 2012. It is currently collaborating with MSDE to simultaneously measure both the utilization and impact of products/services for the RTTT projects; the evaluations are expected to be complete in September 2014.