DRAFT

LAST Student Learning Outcome Assessment Report

AY 04-05

William Loker

Interim Associate Dean

School of Graduate, International

and Interdisciplinary Studies

Jacquelyn Chase

Department of Geography and Planning and Latin American Studies Program

Charlotte Ekland

Latin American Studies Program

Steve Lewis

Department of History and Latin American Studies Program

In AY 04-05 the Latin American Studies Program (LAST) carried out the following activities in program assessment:

  • Review of LAST mission statement, goals and the definition of student learning outcomes (SLO) based on program mission and goals (see attached).
  • Ten SLOs were defined based on discussion among faculty.
  • An initial assessment plan was undertaken to evaluate student performance on two of the SLOs:
  • SLO 1: Students can describe key elements of two or more of these domains (geography, environment, politics, economics, social and cultural systems and/or literature and arts) with broad regional and temporal scope.
  • SLO 4: Students can write a coherent essay on two or more of the domains mentioned in SLO 1 and/or make an effective oral presentation of this material.
  • Assessment of these SLOs was carried out by evaluating final papers submitted in the capstone class, LAST 200 (LAST 495 under the new numbering system). Using a common rubric (see attached) papers were read for content and clarity and mechanics of writing by three LAST faculty members: Jacqueline Chase, the instructor of LAST 200 in Spring 05, Steve Lewis and William Loker.
  • The current report is an attempt to distill some conclusions from that process and communicate these results to the LAST faculty for their consideration and action.

Assessment Process and Methodology

LAST 200/495 was chosen as the “site” for gathering student work for direct assessment of student performance on LAST SLOs because it is the capstone seminar that is required of all LAST majors and is usually taken near completion of the program. Hence students should be capable of demonstrating achievement of most LAST SLOs in this course. It is also the Writing Proficiency course for the LAST major. At the outset, it bears mentioning that neither the students nor the instructor were informed beforehand that assessment activities would be carried out in this course or on these student papers. The decision to do so was made mid-semester in Spring 05 and Dr. Chase graciously agreed to work with other LAST faculty on this process. While students were supplied the scoring rubric used to assess the papers early in the semester, they were not specifically informed that their papers would be used for program assessment purposes.

It is important to reiterate that the purpose of the work reported here is program assessmentnot for grading individual students nor for evaluating the instructor of this course. Papers were read separately by the three faculty members. Chase read and graded the papers as part of the regular grading process for students enrolled in the class. Lewis and Loker read the papers specifically for program assessment purposes, working together to establish a common standard for evaluating the papers. Both read a set of four of these papers, evaluated them separately, then compared evaluations on the two SLOs indicated to try to establish a common standard for SLO evaluation (“inter-rater reliability”). The papers were also read anonymously, that is students’ names were blacked out on the papers that Lewis and Loker read. Chase read the papers separately (not in collaboration with Lewis and Loker) in the course of grading them for evaluating student performance in LAST 200/495. The three faculty members then systematically compared their evaluations and jointly produced therecommendations in this report.

Outcomes

Twenty-five students were enrolled in LAST 200/495 in Spring 05. A total of 19 papers were submitted for assessment by the instructor from self-identified LAST majors and minors. One paper was pulled as problematic. It is not clear why there are 5 students who are not LAST majors/minors in LAST 200/495, the capstone seminar for LAST students. Most likely these are students who consider themselves LAST majors/minors, but have not yet filed official declarations of major/minor forms. It is possible that there are a few students who are in the class simply to advance their knowledge of the region without intending to declare LAST majors or minors. According to data supplied by the Provost’s office, nine students in LAST 200 are declared LAST majors and four are LAST minors. According to this data, the class was composed of seventeen seniors, five juniors, and three sophomores.

Final papers covered a wide variety of topics in the social sciences and humanities with broad geographic scope from Mexico to Chile and Argentina. Papers tended to focus on events and people from either the 20th C or contemporary social issues (drug trafficking, social inequality, gender roles, political violence). Topics were shaped by the theme of LAST 200: Love and violence in Latin America. The table that follows presents the consensus assessment of the papers by professors Chase, Lewis and Loker in terms of student performance on the SLOs outlined above. Papers were evaluated separately for content and writing and sorted into three groups: High Achieving, Achieving or Not Achieving on these two SLOs. Numbers in the table refer to arbitrary codes applied to each paper in lieu of names or topics. (A list of topics keyed to numbers is provided below.) As can be seen in the table, three papers were assessed as “High Achieving” on both content and writing (SLO 1 and 4) and one paper was High Achieving on content, but Achieving on writing. Eleven papers were Achieving on both content and writing. One paper was Achieving on content, but Not Achieving on writing. Three papers were Not Achieving on both content and writing.

SLO 1: Content / SLO 2: Writing
High Achieving / 2,3,4,17 / 2,3,17
Achieving / 5,6,7,8,9,11,12,14,
15,16,18,19 / 4,5,7,8,9,11,12,14,
15,16,18,19
Not Achieving / 1,10,13 / 1,6,10,13

Discussion and Recommendations

Assessment results show some cause for concern regarding the writing skills of the students represented in this sample. Without more informationit is difficult to know if there is any pattern in the performance on these measures between LAST majors and LAST minors and others. It is useful to keep in mind that LAST students (majors and minors) tend to come to the program relatively late in their academic careers. There are two main entry points into the LAST major/minor: the Spanish major and Upper Division GE Theme I, Mexico and Central America. In the case of Spanish majors, many of these students are “heritage learners” who grew up in either Spanish speaking or bilingual families. Because of their academic focus on the Spanish language at the university-level (many LAST majors are dual Spanish-LAST majors) their writing skills in English may be somewhat weak. It is a difficult, but necessary, task to bring these students up to speed on their English-language writing skills in the few LAST classes in which they enroll.

Assessment results on content are more encouraging. The vast majority of students produced papers that were acceptable in terms of their knowledge of the subject.

Based on this admittedly preliminary and less-than-ideal assessment exercise, we recommend the following:

  1. Improving students’ writing skills: This includes language mechanics, organization, effective argumentation and adequate referencing. We would recommend that many LAST classes include submission of multiple drafts of written work to provide developmental feedback to students on their writing. This is admittedly a difficult task in GE classes (such as Upper Division Theme courses) that may have enrollments of 45-50 students, only a few of which are LAST majors/minors.
  2. Rigor: We need to convey high standards to LAST students and hold them to these standards. LAST majors need to be held to the same standards of performance as majors in other disciplines on campus.
  3. Role of LAST 200/495: This course plays an enormously important role in the LAST curriculum. A preliminary attempt to align curriculum with SLOs reveals that LAST 200/495 is the only required course in the LAST major where students are expected to demonstrate mastery of most SLOs (see attached course alignment matrix). LAST faculty need to discuss ways to structure this course in order to provide students with the opportunity to demonstrate mastery of these learning outcomes. It has generally been the privilege of the faculty teaching this course to determine the topic or theme of the seminar. This is entirely appropriate. However, if we are going to take a more intentional approach to helping students master the outcomes we view as essential in the major, the LAST faculty need to engage in an in-depth discussion of the organization and pedagogy of this course to be sure that students get the maximum benefit from their enrollment and participation in the class.
  4. Research and critical thinking skills: The depth of knowledge demonstrated by our students in the papers submitted is encouraging. Our students appear to have a firm grasp of important and controversial issues facing Latin America. Our role as faculty to thoughtfully guide our students to the critical intellectual resources available for a through exploration of these topics and to encourage independent, critical thinking among our students. While we are doing a good job in this respect, it is evident from the papers we read that students tend to rely on few sources and may lack familiarity with important journals and monographs in focused on Latin America. Given the multi-disciplinary nature of the field, it is incumbent on the faculty to more purposefully guide our students in finding and using these sources. We also need to provide examples and be role models for critical thinking so essential to good scholarship as well as effective functioning in society.

Next Steps

A copy of this report will be circulated to all LAST faculty. Discussion of SLOs, alignment of courses with SLOs and efforts to coordinate organization and delivery of curricular materials will be a primary topic of discussion at a retreat planned for early in Fall 05 semester as part of the Five Year review process. It is hoped that the SLO assessment process described in this report will lead to thoughtful efforts at curricular change that will result in enhanced student learning in the LAST major and minor.

Paper topics:

Paper #Topic

1Violence and street children in Brazil

2Afro-Colombian political struggles

3Effects of the US embargo on Cuba

4Frida Kahlo’s life and art

5Domestic violence in the Caribbean

6Frida Kahlo and Diego Rivera

7Argentina’s Dirty War

8Soldaderas in Mexico

9Sendero Luminoso in Peru

10Women in Chile

11Student movements in Mexico in the 1960s

12Street children in Brazil

13Soldaderas in Mexico

14Argentina’s Dirty War

15Machismo and gender roles in Latin America

16Indigenismo in Mexico

17Coca production in Peru

18Feminism in Chile

19Drugs and violence in Colombia