2013-2014Annual Program Assessment Report(Final 10/21/2014)

2013-2014Annual Program Assessment Report(Final 10/21/2014)

Please submit report to your department chair or program coordinator, the Associate Dean of your College, and to , director of assessment and program review, by Tuesday, September 30, 2014. You may submit a separate report for each program which conducted assessment activities.

College: Humanities

Department: Chicana/o Studies

Program: BA Undergraduate

Assessment liaison: Rosa RiVera Furumoto

1. Overview of Annual Assessment Project(s). Provide a brief overview of this year’s assessment plan and process.

RESPONSE:This year’s focus was on collecting, processing, analyzing and interpretingexit writing data from the capstone course taken by all Chicana/os Studies majors and double majors-CHS 497 Senior Seminar in Chicana/o Studies. The idea was to then compare the findings from the exit data to the entry point data that was collected, processed, and analyzed last year in the following CHS stretch writing courses- CHS 113, CHS 114A, CHS 114B, and CHS 115 (see 2012-2013 CHS Dept. Annual Program Assessment Report).

Goal: Our goal was to then consider what the data were showing regarding the writing of CHS undergraduate majors and double majors at the exit point.

1. Assessment Buy-In.Describe how your chair and faculty were involved in assessment related activities. Did department meetings include discussion of student learning assessment in a manner that included the department faculty as a whole?

RESPONSE: The chair of the department, Mary Pardo, is highly involved in the assessment process in the CHS Department. Faculty are also engaged through information provided at faculty meetings and the regular Assessment Committee Meetings held throughout the academic year. The Assessment Committee is composed of faculty and the department chair. Every effort is made to include faculty whose courses are the focus of program assessment processes.

Faculty engage in the assessment process at many levels. For example, writing program faculty, met in the summer (July) of 2013 to review and once again revise the rubric that is used to assess themajorwriting requirement in Approaches to University Writing. The writing faculty are also interested in another retreat or training workshop to discuss how we teach(Second Language Learners) L2 students in the writing program?A better question:Do we evaluate and gradeL2 students in the writing program the same as other students? Is it time for another retreat?

1. Student Learning Outcome Assessment Project.Answer items a-f for each SLO assessed this year. If you assessed an additional SLO, copy and paste items a-f below, BEFORE you answer them here, to provide additional reporting space.

3a. Which Student Learning Outcome was measured this year?

RESPONSE: SLO 2: Demonstrate competency in oral, written, and research skills

3b. Does this learning outcome align with one or more of the university’sBig 5 Competencies?(Delete any which do not apply)

• Critical Thinking
• Oral Communication
• Written Communication
• Information Literacy

3c. What direct and/or indirect instrument(s) were used to measure this SLO?

RESPONSE: The Chicana/o Studies Portfolio Evaluation Rubricwas used to score the CHS 497 capstone course research papers. This rubric has four categories- High Pass, Pass, Low Pass, and No Pass.Each of the categories was assigned a numerical value as follows: High Pass = 4; Pass = 3; Low Pass =2; No Pass =1.

3d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points)or was a cross-sectional comparison used (Comparing freshmen with seniors)? If so, describe the assessment points used.

RESPONSE: A cross sectional comparison was made of the rubric scores on student research papers from the entry point CHS Stretch courses (CHS 113, CHS 114A, CHS 114B, and CHS 115) to the rubric scores on research papers from the exit point CHS 497 Senior Seminar in Chicana/o Studies capstone course.

Sample: Final research papers from one section of the CHS 497 capstone course were read. Only papers of CHS majors and double majors were included in the sample, N=14.

Methods: Three faculty members served as raters of the CHS 497 final research papers. They met to review the rubric, discuss the categories, and calibrate themselves using sample research papers. All of the papers were read and scored by at least two readers and 50% of the papers were read and scored by all three raters. Inter-rater reliability was 12/14 or 86% with 14 opportunities to agree or disagree on the final overall score of High Pass, Pass, Low Pass, and No Pass.

3e. Assessment Results & Analysis of this SLO:Provide a summary of how the results wereanalyzed and highlight findings from thecollected evidence.

RESPONSE:

Results: N=14; 6/14 or 43% of the CHS 497 research papers were scored as High Pass or 4.0 on a scale of 1-4; 8/14 or 57% scored a Pass or 3.0. The average exit score was 3.42 (between a Pass and a High Pass). In comparison the rubric scores on the research papers at the entry point were in the range of 2.75-2.92 with an average score of 2.8 (between Low Pass and Pass).

Analysis and Discussion:This data collection is not analyzing the change in writing scores in a particular group of students. Instead we are examining what we can learn from looking at an entry population and a separate exit population of CHS majors and double majors. The results indicate that students in the exit population CHS 497score at an average rate of 3.42 (Pass – High Pass) range that is higher than the average rate of 2.8 (between Low Pass and Pass) of the entry population.

Whereas, this is a positive indicator, we noted particular trends in the exit writing samples that were not necessarily reflected in the rubric scores. Among the observations we made were that some students in the exit population continue to make errors associated with Second Language (L2) learners. For example, there were lexical, morphological and syntactic errors that in some cases interfered with the comprehensibility of their ideas. Although there is some controversy about providing feedback to Second Language L2 writers, we strongly believe that CHS Department faculty need to know more about strategies, pedagogy, and resources for supporting L2 writing. We also assert that more can and should be done at the institutional level to provide adequate support to L2 writers. For example, focused training for faculty, writing tutors, and teacher assistants on identifying and addressing the unique needs of L2 writers.

In addition, we noted that CHS 497 students scoring at the ”Pass” level generally needed to strengthen their abilities to critique the scholarly sources they were using and to argue more effectively for their perspective.

Limitations: The data collected at the entry point in CHS 113, 114 and 115 courses does not necessarily reflect only CHS majors because the students taking these courses may not all be CHS majors or double majors at the time they took the course(s). At the present time, we do not have an entry-level writing course that enrolls only CHS majors. At the exit point, we were able to examine only CHS majors and double majors research papers writing samples by examining the papers from one of the CHS 497 sections.

3f. Use of Assessment Results of this SLO:Describe how assessment results were used to improve student learning. Were assessmentresults from previous years or from this year used to make program changes in this reporting year? (Possible changes include:changes to course content/topics covered, changes to course sequence, additions/deletions of courses in program, changes in pedagogy, changes to student advisement, changes to student support services, revisions to program SLOs, new or revised assessment instruments, other academic programmatic changes, and changes to the assessment plan.)

RESPONSE:

1. The CHS Dept. Writing Retreat held May 11, 2013 provided an opportunity for faculty to come together to engage in focused discussion about student writing and how to support it. Among the salient outcomes:

1. Students need more guided instruction on how to do critical reading of their sources, scholarly and otherwise.
2. Students need more support in evaluating and critically analyzing their sources.
3. Students need to better understand the context (social, cultural, historical, political, etc) of the materials they are examining.
4. Students need support in critically examining and analyzing primary and secondary source documents.
5. Faculty have started to examine the major writing/reading assignments addressed in each CHS course in order to ascertain the gaps and overlap that may be occurring.
6. A commitment to continuing with the e-portfolio development process.
7. Students need support for developing a scholar activist identity.

1. Resources to support student writing are more easily accessible to faculty via a Resource Moodle Page.
2. Writing lab tutors continue to provide workshops and support to students on MLA style, grammar, and syntax (among other topics).
3. A revised writing rubric is now in place.

4. Assessment of Previous Changes: Present documentation that demonstrates how theprevious changes in the program resulted in improved student learning.

RESPONSE: See 2013-14 results noted above describing a cross-sectional comparison. Average rubric scores on research papers at entry and exit points indicates that research papers in the CHS 497 (exit point) have higher average scores 3.42 (Pass-High Pass) than those at the entry point 2.8 (between Low Pass and Pass).

The CHS Dept. offered two Selected Topics courses in Spring 2013 to explore creative ways to enhance students’ communication and critical reading skills. Assessment data were collected in CHS495 C -"Developing Vocabulary through Latin/Greek Etymologies." Theobjective in teaching the course was to improve comprehension of Latin/Greek based vocabulary used in academic disciplinesand to develop a deeper insight into the meaning of commonly occurring words of Greek and Latin origin. Students were taught about stems (noun, verb, adjective), prefixes, and suffixes*.

Instrument and Methods: A pre and post test were administered to students enrolled in the one section of CHS495 C, N=26 to collect data on students’ recognition of the meaning ofGreek and Latin roots.

Findings: The findings indicatedthat thestudents’recognition of the meaning of Greek and Latin roots weresignificantly enhanced by the end of the course.

Analysis and Discussion: As would be expected, the students’ knowledge of Greek and Latin roots grew significantly. More importantly, their knowledge of these elements enabled them to unravel the meaning of unfamiliaracademic terms and also to grasp more fully the meaning of already familiar ones. They also discovered the connection between Latin and the Spanish language and learned to take advantage of their familiarity with Spanish vocabulary to get at the meaning of academic language which is very much rooted in Latin, thereby building on students’ strengths. Indeed, this course supports many L2 Spanish-speaking learners to build on their strengths. Based on the assessment findings from the course, the CHS Department intends to make CHS495 C -"Developing Vocabulary through Latin/Greek Etymologies”part of our regular curriculum by submitting a course proposal in Spring 2015.

5. Changes to SLOs?Please attach an updated course alignment matrix if any changes were made. (Refer to the Curriculum Alignment Matrix Template,

RESPONSE: Not Applicable

6. Assessment Plan: Evaluate the effectiveness of your 5 year assessment plan. How well did it inform and guide your assessment work this academic year? What process is used to develop/update the 5 year assessment plan? Please attach an updated 5 year assessment plan for 2013-2018. (Refer to Five Year Planning Template, plan B or C,

RESPONSE: The 5-year Assessment Plan has been effective in guiding our assessment efforts to date. However, it is our intention to revise the 5-year plan this year to focus our efforts on the development of an e-portfolio process for CHS majors, double majors, and minors.

7. Has someone in your program completed, submitted or published a manuscript that uses or describes assessment activities in your program? Please provide citation or discuss.

RESPONSE: No.

8. Other information, assessment or reflective activities or processes not captured above.

RESPONSE:

Developing an E-Portfolio as an exit project for Majors is part of an ongoing dialogue in the CHS Department. To date, we have identified some courses that can support this process and possible signature assignments. We are also identifying and refining rubrics and other reflective tools for use in the e-portfolio process.

1