Recommended Outline for TAACCCT Final Report Executive Summary
The recommended outline for your final evaluation report’s executive summary helps ensure that high-level findings from the TAACCCT grants are effectively conveyed to a wide audience, especially non-experts. Using this outline will also allow for findings across TAACCCT evaluations to be compared and to build a body of evidence on particular approaches to building capacity for workforce education and training at community colleges. The executive summary should focus on providing clearly written information, condensed to highlight the most important things readers should know about your evaluation and the findings from it.
The outline highlights the topics that may be included in your executive summary. The topics and suggested content cover much of what was required in the grant solicitation for the third-party evaluations. However, we know that many evaluations go beyond the requirements and we encourage evaluators to include findings that reflect their entire evaluation, not just what is noted below. We also provide a suggested length for each section but again, this depends on what is important to convey about the TAACCCT project and evaluation for a wide audience to understand the project and design, the evaluation findings, and their implications.
We also recognize that you may have covered topics in this outline such as the evaluation design and implementation findings in greater detail in an interim report. We encourage you to include a short summary of those components in your final evaluation report and the executive summary to ensure readers have an overview of these pieces, especially as they relate to the components on which you focus in this report.
I. TAACCCT Program/Intervention Description and Activities [0.5 page]
A. Briefly describe your TAACCCT project and purpose
B. Describe each program/intervention that was evaluated (if the grant is funding more than one) and how it was supposed to effect change for the target population
1. Describe each component of the program/intervention including components such instructional design, prior learning assessments, college and career coaching/navigation, job placement assistance, and tutoring
2. Discuss the population served
3. Summarize the evidence-based or promising model the funded program/intervention used for its design, citing appropriate literature (if relevant)
II. Evaluation Design Summary [0.5-1 page]
A. Describe the goals of the evaluation
B. Discuss implementation study design
1. Identify the research questions the study will address
2. Describe the conceptual framework for the implementation study (e.g., theory of change, logic model)
3. Describe how the conceptual framework was used to guide the implementation analysis
4. Implementation data and methods
5. Describe how capacity building was measured, include a description of the indicators that were used
C. Discuss outcomes/ impact study design
1. Identify the research questions the study will address
2. For the outcomes analysis (quantitative descriptive) and impact analysis (causal analysis) (see clear.dol.gov for more information on this distinction)
1. State the overall methodology used (e.g., randomized controlled trial, propensity score matching, regression discontinuity, pre-post testing) and whether or not causal inferences (internal validity) can be made from the analysis conducted
2. Briefly describe the data used and their reliability
3. Describe the outcomes and impacts measured
III. Implementation Findings [1-1.5 pages] [Please use bullet points with 1-2 sentences each]
Issues to consider in this section:
1. Describe how the grant was used to build institutional capacity
2. Provide a summary of the key steps taken by the institution (or institutions if part of a consortium) to create and run the training program
3. Highlight any important partnerships
4. Discuss whether grantee was able to implement the program/intervention with fidelity to the original design/model and, if not, why
5. Describe the operational strengths and weaknesses of the program
[Note: Please use as a basis for this section (1) the four required research questions as articulated in the Solicitation for Grant Applications; (2) questions regarding efforts to expand institutional capacity; and (3) any additional research questions]
IV. Participant Impacts Outcomes [1-1.5 pages] [Please use bullet points with 1-2 sentences each, and a chart that summarizes key outcomes and impacts]
Issues to consider in this section:
1. Highlight the key outcomes impacts estimated, including the nine (9) outcomes articulated in the Solicitation for Grant Applications
2. Include any additional outcome and impact findings, expanding on or refining what was discussed in your evaluation plan
3. Note any important limitations to interpreting the findings (e.g., internal and external validity)
V. Conclusions [0.5 pages] [Please use bullet points with 2-3 sentences each]
A. Highlight any key lessons from your evaluation that would help others who want to replicate your TAACCCT project – helping them to understand what worked well and where challenges may lie
B. Consider the main implications for future workforce and education research– what are the next steps to rigorously studying the types of approaches and strategies tested under your TAACCCT project?
2