/ AVID Evaluation Report

Results 2011-13For more information contact

Melody Jacobs-Cassuto, Research Evaluation and Assessment

Maria Roberts, District AVID Coordinator, Office of College and Career Readiness

Introduction

AVID is an acronym that stands for Advancement via Individual Determination. It is an elective class for students in grades 6-12, devoted to increasing post-secondary readiness and success of first generation college students (for more information at With support from a Cargill Foundation grant to AchieveMpls, Minneapolis Public Schools (MPS) has been increasing the quality and reach of AVID programs since 2008. In fall of 2012, 1698 MPS students earned credit in AVID.

Three main outcomes are looked at in this study to see if “AVID works":

1.Students will improve academic performance and growth, withdecreases documented in achievement gaps.

  1. Students are prepared for and succeed in advanced classes and post-secondary education.
  2. Young people are engaged in learning.

Methodology: Propensity Matching

The evaluation of the AVID program(Advancement via Individual Determination)includes a propensity-matched component, in order to better compare outcomes among AVID and non-AVID students. The objective is to create a comparison group of non-participants that is as similar as possible to the group of students selecting AVID.This type of study is viewed as more accurate in regard to program impact because the participants and non-participants have similar characteristics.

To perform these studies, AVID and non-AVID students attending grades 6-10 in 2011-12 were matched on: prior reading MCA scores (Spring 2011), grades, English language learner (ELL) status, Free or Reduced-Price Lunch (FRL) status, ethnicity, and number of AVID sections offered at theparticipating sites. Students included in the AVID group were enrolled in AVID for the entire school year 2011-12. Matched non-AVID students were not enrolled in AVID during that year.

Student Demographics Fall 2012

One of the goals of the AVID program is to encourage participation from underrepresented college enrollees. This means that AVID is encouraging participation of students of color, low income, immigrants or children of immigrants. AVID appears successful in reaching underrepresented groups such as African Americans, Hispanic and qualifying free/reduced lunch students. These graphs below demonstrate a higher representation of these groups in AVID than the district as a whole.

Findings

  • AVID is successful in reaching students who are under-represented among college enrollees, according to race and income.
  • Asian students and male students are slightly underrepresented

Measures of Academic Progress (MAP) Growth Fall 2011 to Fall 2012

Thisindicator focuses on the percentage of students meeting or exceeding the goal of making one year of academic growth as measured by the MAP.

AVID students outperformed matched non-participants on MAP growth, in both reading and math. The differences are more pronounced for students of color.

Matched Groups / Reading % making 1 yr growth / Math % making 1 yrgrowth / #Reading / #Math
All Ethnic Groups
non-AVID / 52.63% / 48.43% / 228 / 223
AVID / 61.68% / 52.65% / 321 / 302
Students of Color
non-AVID SOC / 47.50% / 49.37% / 160 / 158
AVID SOC / 60.99% / 55.77% / 223 / 208

Findings

  • AVID students exceeded target of 56% for Reading both overall and students of color
  • AVID students did not meet the target of 55% for Math overall but did meet for students of color

*For students of color, the Reading difference was statistically significant at the .05 level for grades 6-8 and statistically significant at the .01 level for grade 6

*Results for 2012-13 will be forthcoming once the MAP is administered in Fall of 2013

See Technical Report for further details.

Minnesota Comprehensive Assessment (MCA) Growth Spring 2011 to Spring 2012

Thisindicator focuses on the percentage of students making high growth (1/2 standard deviation above the mean) as measured by the MCA.

AVID students performed similarly tomatched non-participants on MCA growth, in both reading and math. The same was true for students of color. This is different than the MAP results despite the high correlation between the two tests. MCA is a grade level and measures more precisely where students are performing at grade level. MAP measures equally precisely across grade levels which may account for the discrepancy.

Matched Groups / MCA Reading % making High growth / MCA Math % making High growth / # Reading / # Math
All Ethnic Groups
non-AVID / 29.10% / 32.22% / 299 / 295
AVID / 27.95% / 32.31% / 390 / 390
Students of Color
non-AVID SOC / 25.58% / 28.73% / 215 / 212
AVID SOC / 24.29% / 30.00% / 280 / 280

Findings

  • AVID students performed similarly to non-AVID students in terms of high growth on the MCA both overall and for students of color
  • No statistical significance was found for MCA growth overall or for students of color
  • Results for 2012-13 will be forthcoming once results are available for Spring MCA

See Technical Report for further details.

Courses of Rigor

AVID studentsare required to successfully complete at least one advanced HS class or rigor (class with college equivalency) sometime during their high school career. Some examples of rigorous courses are any of the Advanced Placement (AP) courses or Interbaccalaureate (IB) courses. Many AVID students take several courses of rigor. The majority of AP and IB courses aren’t offered until grade 11, so AVID students need to wait to complete this requirement. Below is the number of courses of rigor completed so far by the class of 2014 who were enrolled in AVID in 2011-2012 in grade 10.

Number of Courses of Rigor Completed Successfully to date (Class of 2014) / Frequency / Percent / Cumulative Percent
0 courses / 18 / 13.1 / 13.1
1 course / 32 / 23.4 / 36.5
2 courses / 28 / 20.4 / 56.9
3 courses / 22 / 16.1 / 73.0
4 courses / 18 / 13.1 / 86.1
5 courses / 10 / 7.3 / 93.4
6 courses / 5 / 3.6 / 97.1
7 courses / 4 / 2.9 / 100.0
AVID students who fulfilled the requirement 119 / 137

Findings

  • 87% of grade 10 AVID students successfully complete at least one course of rigor by the end of first semester in grade 11
  • 64% of grade 10 AVID students successfully complete two or more courses of rigor by the end of first semester in grade 11

On Track to Graduate

High school students are identified as “on-track to graduate” if they have passed GRAD tests andearned credits appropriate to their grade level. For grade 9 that means passing the GRAD writing test, and for grade 10 that means passing boththe GRAD writing and reading tests.

Findings

  • AVID students met target rate of 60%
  • AVID students outperformed matched non- participants at a significantly higher rate for being on track to graduate

Performance on EXPLORE, PLAN and ACT* (2012-2013)

EXPLORE and PLAN are pre-ACT tests that are correlated with the ACT and like the ACT, predict college readiness, or the probability for success in credit-bearing college courses in four areas: English, Reading, Math and Science. EXPLORE and PLAN were administered in the Fall of 2012 to grade 8 and grade 10 respectively. Scores were compared only after students had received a full year of AVID. EXPLORE compared results of students who were in AVID in grade 7 during the 2011-2012 academic year. PLAN compared results of students who were in grade 9 during the 2011-2012 academic year.

Matched AVID students performed slightly higher than their non-AVID counterparts on the subtests and the composite of the Grade 8 EXPLORE test. For students of color, the differences were more pronouncedandit was statistically significant. For Grade 10, AVID students did not perform better than their non-AVID counterparts. At the high school level, there are many college preparation programs being implemented and this may have minimized the differences. Also schools do not seem to put as much emphasis on EXPLORE and PLAN as they do MAP and MCA. They are not used for district and state accountability. They are also timed test whereas MAP and MCA are not. This may also have contributed to the lack of positive, significant differences for AVID students. On the other hand, the differences for students of color were statistically significant indicating that the curriculum is helpful for those students.

Matched AVID students performed slightly lower than their non-AVID counterparts on the subtests and the composite of the Grade 10 PLAN test both overall and for Students of Color.

Findings

  • AVID students performed slightly better than non-AVID students on the EXPLORE
  • Differences for students of color on the EXPLORE were statistically significant
  • AVID students performed slightly worse than non-AVID students on PLAN
  • PLAN differences were not statistically significant

*Results for the ACT for all will be available sometime in September

See Technical Report for further details.

Post-Secondary Transition and AVID

One of the goals of AVID is to increase enrollment and success in post-secondary programs. National Student Clearinghouse is used by post-secondary institutions nationwide to verify enrollment and degree attainment. The nation-wide database is called the Student Tracker. The Student Tracker follows post-secondary enrollment and matriculation of high school graduates. Below are the results for the classes of 2010, 2011 and 2012 of who went on to a post-secondary institution and where. The graph below summarizes three different cohorts of graduates.

Findings

  • All three cohorts show AVID graduates enroll in post-secondary institutions at a much higher rate (80s vs. 60s) than district graduates
  • As the number of high schools that graduate AVID seniors increases, AVID enrollments in post-secondary institutions remain well above the 65% target rate

Class of 2010 / Class of 2011 / Class of 2012
AVID Graduates / 86% / 85% / 80%
Non-AVID Graduates / 62% / 63% / 69%

Class of 2010: Grade 10 2007- 2008:

  • 86% of AVID students enrolled in college after graduation
  • 62% of MPS students enrolled in college after graduation
Colleges of 2008 AVID Students (class of 2010) / Frequency / Percent / Cumulat. Percent
AUGSBURG COLLEGE / 3 / 6.0 / 6.0
GUSTAVUS ADOLPHUS COLLEGE / 3 / 6.0 / 12.0
HENNEPIN TECHNICAL COLLEGE / 1 / 2.0 / 14.0
MINNEAPOLIS COMMUNITY AND TECHNICAL COLL / 17 / 34.0 / 48.0
MINNESOTA STATE UNIVERSITY - MANKATO / 1 / 2.0 / 50.0
NORMANDALE COMMUNITY COLLEGE / 5 / 10.0 / 60.0
NORTH DAKOTA STATE COLLEGE OF SCIENCE / 1 / 2.0 / 62.0
NORTHLAND COLLEGE / 1 / 2.0 / 64.0
PRATT COMMUNITY COLLEGE / 1 / 2.0 / 66.0
RIDGEWATER COLLEGE - WILLMAR CAMPUS / 1 / 2.0 / 68.0
SOUTHWEST MINNESOTA STATE UNIVERSITY / 1 / 2.0 / 70.0
ST. CATHERINE UNIVERSITY / 3 / 6.0 / 76.0
UNIVERSITY OF IOWA / 1 / 2.0 / 78.0
UNIVERSITY OF KANSAS / 1 / 2.0 / 80.0
UNIVERSITY OF MINNESOTA - MORRIS / 1 / 2.0 / 82.0
UNIVERSITY OF MINNESOTA-TWIN CITIES / 6 / 12.0 / 94.0
UNIVERSITY OF ST THOMAS / 1 / 2.0 / 96.0
UNIVERSITY OF WISCONSIN - STOUT / 1 / 2.0 / 98.0
WILEY COLLEGE / 1 / 2.0 / 100.0
Total 2010 AVID Grads in College out of 58 / 50

Class of 2011: Grade 10 2008- 2009:

  • 85% of AVID students enrolled in college after graduation
  • 63% of MPS students enrolled in college after graduation
Colleges of 2009 AVID Students (class of 2011) / Frequency / Percent / Cumulat. Percent
AUGSBURG COLLEGE / 3 / 6.7 / 6.7
BETHUNE-COOKMAN UNIVERSITY / 1 / 2.2 / 8.9
CONCORDIA COLLEGE / 1 / 2.2 / 11.1
CONCORDIA UNIVERSITY / 1 / 2.2 / 13.3
CONCORDIA UNIVERSITY AT ST PAUL / 1 / 2.2 / 15.6
HAMLINE UNIVERSITY / 1 / 2.2 / 17.8
MINNEAPOLIS COMMUNITY AND TECHNICAL COLL / 16 / 35.6 / 53.3
MINNESOTA STATE UNIVERSITY - MANKATO / 1 / 2.2 / 55.6
NORMANDALE COMMUNITY COLLEGE / 4 / 8.9 / 64.4
NORTH HENNEPIN COMMUNITY COLLEGE / 1 / 2.2 / 66.7
OKLAHOMA CITY COMMUNITY COLLEGE / 1 / 2.2 / 68.9
SAINT PAUL COLLEGE / 3 / 6.7 / 75.6
ST CLOUD STATE UNIVERSITY / 2 / 4.4 / 80.0
ST OLAF COLLEGE / 1 / 2.2 / 82.2
ST. CATHERINE UNIVERSITY / 2 / 4.4 / 86.7
STANFORD UNIVERSITY / 1 / 2.2 / 88.9
UNIVERSITY OF MINNESOTA-TWIN CITIES / 1 / 2.2 / 91.1
UNIVERSITY OF NEVADA LAS VEGAS / 1 / 2.2 / 93.3
Total 2011 AVID Grads in College out of 53 / 45

Class of 2012: Grade 10 2009- 2010:

  • 80% of AVID students enrolled in college after graduation
  • 69% of MPS students enrolled in college after graduation
Colleges of 2009 AVID Students (class of 2012) / Frequency / Percent / Cumulat. Percent
ANOKA RAMSEY COMMUNITY COLLEGE / 1 / 1.4 / 1.4
AUGSBURG COLLEGE / 6 / 8.5 / 9.9
CARTHAGE COLLEGE / 1 / 1.4 / 11.3
CENTURY COLLEGE / 1 / 1.4 / 12.7
CLARK ATLANTA UNIVERSITY / 1 / 1.4 / 14.1
DUNWOODY COLLEGE OF TECHNOLOGY / 1 / 1.4 / 15.5
HAMLINE UNIVERSITY / 4 / 5.6 / 21.1
METROPOLITAN STATE UNIVERSITY / 1 / 1.4 / 22.5
MINNEAPOLIS COMMUNITY AND TECHNICAL COLL / 23 / 32.4 / 54.9
MINNESOTA STATE UNIVERSITY - MANKATO / 3 / 4.2 / 59.2
NORMANDALE COMMUNITY COLLEGE / 7 / 9.9 / 69.0
NORTH HENNEPIN COMMUNITY COLLEGE / 1 / 1.4 / 70.4
NORTH IOWA AREA COMMUNITY COLLEGE / 1 / 1.4 / 71.8
NORTHLAND COLLEGE / 1 / 1.4 / 73.2
NORTHWESTERN COLLEGE / 1 / 1.4 / 74.6
ST CLOUD STATE UNIVERSITY / 1 / 1.4 / 76.1
ST. CATHERINE UNIVERSITY / 1 / 1.4 / 77.5
UNIVERSITY OF MINNESOTA-TWIN CITIES / 8 / 11.3 / 88.7
UNIVERSITY OF ST THOMAS / 5 / 7.0 / 95.8
UNIVERSITY OF WISCONSIN - EAU CLAIRE / 1 / 1.4 / 97.2
UNIVERSITY OF WISCONSIN - RIVER FALLS / 2 / 2.8 / 100.0
Total 2011 AVID Grads in College out of 89 / 71

Findings

  • AVID students choose equally among 2-yr and 4-yr colleges
  • At least 70% choose public institutions

Attendance

Students in AVID are required to keep their attendance up. This indicator looks at percent of days attending (days present/days membership). Attendance of AVID students was compared with matched non-AVID students who were members of the district for a minimum of 94 days.

Findings

  • AVID students attend at a higher rate than non-AVID students
  • The difference in attendance is statistically significant at the .01 level

Student Engagement Instrument (SEI)

This set of indicators, collected by surveying students at AVID sites, focuses onself-reported student engagement levels, and the changes in these ratingsover time. The SEI consists of 35 school engagement statements grouped under 6 scales:

  • Teacher-Student Relationships
  • Control & Relevance of School work
  • Peer Support for Learning, Future Aspirations & Goals
  • Family Support for Learning
  • Intrinsic Motivation.

The evaluationfollowed two cohorts of students who were in 6th and 9thgrades in school year 2011-12. Grade 6 was chosen because this is a student’s first exposure to AVID. Grade 9 was selected, because students are entering high school for the first time. It was suggested that we look at Fall to Fall growth since school engagement tends to drop at the end of the school year. School engagement ratings and growth by AVID and non-AVID students in each cohortwere compared.

Below is the mean change in score for the overall SEI as well as each of the scales for grade 6 students. For AVID students there was a drop in engagement on every scale. The SEI was administered in Spring 2012 and we did not see this trend. Because of this difference, it may have been due to some differences in administration protocol. It is still being investigated. The SEI will be administered to the same cohort again in Fall 2013. Growth from Fall 2011-Fall 2013 will be examined in addition to Fall 2012-Fall 2013 to see if this was an anomaly.

Below is the mean change in score for the overall SEI as well as each of the scales for grade 9 students. With the exception of School Work, all growth is positive for AVID students and negative for non-AVID students indicating that AVID students demonstrate a higher level of school engagement after participating in AVID.

Findings

  • Grade 6 AVID students showed less growth than non-AVID students. This is not clear why since the same AVID cohort demonstrated better growth when examining Fall to Spring SEI growth
  • The SEI will be administered one more time Fall 2013 to follow both cohorts across 2 years of time; the grade 6 cohort will be scrutinized very carefully
  • Grade 9 AVID students showed positive growth on the overall SEI as well as 5 of the 6 scales
  • Grade 9 overall SEI and Aspirations were statistically significant in favor of the AVID students at the .05 level
  • Grade 9 School Work and Family Support were statistically significant at the .01 level

See Technical Report for further details.

Conclusions

In sum, AVID appears to be working. The study demonstrates that the vast majority of indicators met or exceeded the targets in the areas of academic growth and performance, preparation for post-secondary success and student engagement. One of the exceptions was the EPAS system. In the 2009-10, propensity matching, EXPLORE results were statistically significant, but only for the AVID students of color in 2011-12. In both propensity matching analyses, PLAN did not yield positive results for AVID students. Since there has been an emphasis on college preparation for all students, perhaps that has narrowed the gap between AVID and non-AVID. Another reason may be a reduced emphasis on those tests. For a college preparatory course like AVID, there may need to be a shift in emphasis. EXPLORE and PLAN are timed tests whereas MCA and MAP are not. More exploration needs to be done in that area. This fall we will have the results of the ACT and see if AVID students perform better than their matched non-AVID counterparts. This will be especially interesting, since in the past, ACT participation was voluntary but in 2013 was administered to all grade 11 students.

Based on the results of the study, AVID students demonstrate college readiness in other ways. AVID students are enrolling in courses of rigor and the rate of enrollment in post-secondary institutions immediately following graduation is much higher than the district as a whole. All of this has been accomplished with increased enrollment in AVID of traditionally underrepresented student populations.

Additional Resources

Visit the MPS AVID webpage for profiles of AVID schools and students, evaluation information and more.

Maria Roberts, District AVID Coordinator, Office of College and Career Readiness

Minneapolis Public SchoolsPage 1 Reporting assistance fromResource

Development & Innovation