1)Call to Order
Announced that Jeanne-Marie Draper has regretfully resigned her role on the committee. Carol also clarified that Amy Dwelle is no longer working for COS. Joni will take those two openings to Senate for a call for new members.
2)Comments/Questions
a)Regarding Items on the Agenda
b)Regarding Items Not on the Agenda– Dialogue Day planning!
Dialogue Day is March 4. An announcement has been sent out and the chairs were informed at yesterday’s IC meeting. Joni will send out a request for resources to the faculty next week. Discussion about the logistics of getting snacks to the division meetings. Jennifer has offered her staff to assist in this, so that should work well for us. Discussion about a theme around which to organize this day resulted in a focus on the ILO connections between course and program outcomes to the ILOs, so that the mapping processes and the identification of students for assessment will be possible.
3)Approval of Minutes
Review/approval of minutes from December 4. Turner/D’Agostino M/S/C
4)Unfinished Business
a)ILO Assessment planand assignments
Committee members shared updates on where they were with their assignments. Joni has met with Kristen Foster and is developing the roll out campaign, which will coincide with Dialogue Day. Kristen has some ideas about the posters. We decided that putting them in classrooms and major conference rooms would be the best. The posters will have the ILO of the year in bold with the others in grayed out font. Also liked Kristen’s suggestion of having a statement at the bottom about the purpose of the ILOs. The raffle postcards will be created and distributed with an event (TBD) for drawing the prize.
Carol shared the amazing work she’s done with Priscilla in creating the survey instrument (see below). Noted that we need a list of the courses that are connected to the ILOs, which Priscilla said she’d have for our next meeting. We discussed that the Dialogue Day focus could be on mapping the ILO to courses and programs. Joni will create a tutorial page for this and send to divisions. Further discussion involved how to select sections of courses for the student surveys. If we do a random selection, can we also include those faculty who want to participate? Priscilla will check with Dali about this.
Also talked about how to roll this out. Is it better to have admin assts do the administering, rather than leaving it to faculty? Carol referred us to a memo by Dr. Scroggins for a similar event when the CSSE surveys went out. This will warrant further consideration.
There is concern over how to get the comments section of the survey transcribed. Joni will talk to Jennifer about this.
It was suggested that an introduction and some context be added to the survey, including the outcome statement and thank you to students for taking the time for doing the survey.
At our next meeting, we’ll decide what faculty info will go out about the survey. Huge kudos to Carol for pulling together this part of the project!
b)Transfer Studies outcomes project
Joni noted that everyone sent word that they had completed their projects. We should be complete for the year.
c)Website changes
Kristen Foster has some ideas about how to do the revision of the website. Joni will meet with her about this before the next OA meeting.
5)New Business
a)Report on SLO Coordinators conference (Carol)
Carol noted that the meeting was very productive and urged others to go next year. See attached notes for the highlights.
b)ACCJC workshop: Taking Assessment to the Program Level
Jennifer has asked for those interested from the committee to let her know if they would like to attend this conference. None of the committee members present is currently able to go. Joni will further discuss with Jennifer and may end up attending.
c)Adjunct faculty compensation—moved to March meeting.
d)OH! Schedule and themes
Joni noted that the OH! Sessions would be in Tulare for this semester, as there is greater need there and several of the adjunct faculty are requesting help. She is working with the deans and faculty there to schedule the sessions.
6)Adjournment
Spring 2016 – Institutional Learning Outcome Assessment Survey
Research Plan
Background:
This survey assesses learning for the Institutional Outcome for Communication. The questions are based on the affective domain of Bloom’s taxonomy. Bloom’s is widely referenced as the source for “measurable verbs” used widely in outcome statements. The affective domain of Bloom’s taxonomyrefers to people’s reaction to learning with respect to feelings, values, appreciation, enthusiasms, motivations and attitudes (grades tend to focus more on the cognitive and psychomotor domains.) This focus allows the college and students to assess the commitment to practicing effective writing and communication. In addition, by asking the questions, students can consider whether they should change their attitude toward communication skills learned and practiced in various courses. Categories in the affective domain are arranged in this survey with the simplest behavior in the first question and the most complex in the last question.
Bloom identifies three “domains of learning” including the affective (emotive-based), psychomotor (action-based) and cognitive (knowledge-based) domains. There is some overlap between the affective and psychomotor domains in that the student “reacts” to the teaching; the psycho motor domain descries ability to physically manipulate a tool or instrument. The cognitive domain for “communication” may be assessed at a later date at COS. Students have a wide range of abilities at COS to analyze, evaluate and create ideas. Future assessment of the cognitive domain might be based upon a COS common rubric that is flexible enough to be adapted for a variety of courses (e.g. CSUF has a campus-wide generic rubric.)
Research design:
Course sections are selected randomly according to those courses that address the “Communication” Institutional Learning Outcome as recorded in TracDat by faculty members. Surveys are passed out in the selected classes by survey administrators or faculty. The questions on the survey allow for any one of several accomplishments such are awareness of the importance of precise writing or precise speaking since the survey results are compiled to assess institution-wide learning and each course identified may contribute to the outcome in a variety of ways. The survey includes only 3 questions to avoid having too much survey data and to focus student attention on the essential question about changed attitudes toward Communication. In addition, only one comment section is provided, to allow students to independently prioritize their learning report rather than being forced to comment for each question, for example.
Scenarios for use of data:
- Scenario #1
The data might show that 85% of students are aware and care about writing better but only 30% change the priorities in school to allow for more time to write or speak better. Faculty might then focus classes on reinforcing the idea that taking time to write will make for more effective results, sooner. - Scenario #2
Disaggregation of datum into male and female categories might show that males don’t care about the writing and speaking precisely, although they are more aware of the need and have spent time getting help. Faculty might then speak more often to the value of showing care in writing to enhance the depth and power of communication. - Scenario #3
In the comment section, students might share examples of their increased commitment to better communication awareness and attitudes; yet, they may not identify the change in the survey questions. In this case, faculty may want to highlight with students that the activities in the class are having an effect on the student’s valuing of writing and speaking well. - Scenario #4
The data from the survey would give faculty in a department a baseline measure from which to improve, so that the teaching emphasizes the importance of attitude toward success. These faculty members might repeat the survey at various intervals to measure change in student report.
To the Instructor:
- Please have your students complete this survey. The information from the survey will give COS faculty information regarding student awareness, attitudes, behaviors, and values regarding verbal and written communication in this course. The student ID # allows us to evaluate responses by age, ethnicity and gender.
- Please return these surveys to the Research Office.
- The surveys will be tabulated and the comments recorded for review by the Outcomes Assessment Committee members, Instructional Council and each Division. Courses are selected on a random basis according to faculty indications (on TracDat) that the course contributes to the Communication outcome. Future surveys will be conducted for additional Institutional Outcomes.
- Questions may be addressed to your Division Chairs, Outcomes Assessment Committee members or the Research Office.
Student Survey on Institutional Learning Outcome
Communication 12/23/18 draft
Course CRN: / Student ID#: / Date:
Did this class change your awareness, attitudes, or behavior regarding speaking or writing? Darken with a pen or pencil the box that best describes your learning is this class.
Change in Awareness: As a result of this class, I pay more attention to my writing or speaking.
[ Yes ] (Have you done any one of the following as a result of this class?)
- I pay more attention to saying exactly what I mean rather than just quickly writing for an assignment.
- I pay more attention to using words that fit the person to whom I am writing or speaking.
- I pay more attention to relating my ideas to the people I am writing about or to my audience.
- I pay more attention to those who have opinions different than mine.
[No change] I don’t pay attention to my writing and speaking.
Change in Attitude: As a result of this class, I care more about my writing or speaking.
[Yes] (Have you done one of the following as a result of this class?)
- I want to be more effective when I communicate with others.
- It bothers me when I haven’t been careful in writing or when I say things that I do not mean.
[No change] I don’t pay attention to whether my writing and speaking is effective.
Change in Priorities: As a result of this class, I take more time to make sure my writing or speaking
is better.
[Yes] (Have you done any one of the following as a result of this class?)
- I met with a COS tutor, a friend or someone else to get some good ideas to improve my writing or my spoken presentation in this class.
- I spend extra time to make sure my assignments have correct grammar, spelling and organization.
- I work hard to make my writing proper for college writing.
[No change] I am too busy to add time to be a better write or speaker.
Comments: Please explain your answer and give an example of your new commitment to better writing and speaking as a result of this class. (Use the back of the paper for further comments, if needed.)
Thank you for completing this survey. We will use your views to improve teaching at COS.
Notes: 3rdAnnual SLO Symposium
Carol EnnsJanuary 29, 2016
Breakout #1:TYPES OF RESEARCH AND EXPERTS
Dr. Janice Chadwick, Fullerton College.
- Challenge, Accreditation Standard 1.B.6. Disaggregated data
- Trailblazers: Linda Suskie (Assessing Learning: A Common Sense Guide, Wiley 2010), Linda Panky sp. ?, Paul Dressel sp.?)
- See Student Learning Assessment Handbook Middle States (not sure why?)
- Science versus Action Research
- Science research starts with hypothesis, uses research design, statistical tests, control groups, sampling techniques
- Action research is to inform one’s own practice; less rigor but findings “good enough” to initiate dialogue
- See Valencia College, Florida, web site
- Stephen KemmisMcTaggert (The Action Researcher)
- 70% of the class achieving a LO is likely too low
- Use at least one direct and one indirect measure in your SLOs
- Key Performance Indicator (KPI) and “Achievement Data” (administrator interests)
David Wood, San Antonio College, San Antonio, Texas
- I.B.2, II.A.3, II.A.9, IIA 11, 12, 13, & 16
- Incorporate SLOs into curriculum and use for improvement.
- 50% faculty buy-in to assessment & improvement (better than 5 years ago.)
- How is your class focused?
- Miss most important outcome but learn a lot?
- Target grades to outcomes?
- Do those that “pass” succeed at the next level?
- Maybe a “B” or 90% should be the expected outcome?
- A “C” in a foundation course may mean only 50/50 chance future success.
- Do those that “pass” succeed at the next level?
- Do your students ask, “How am I doing on that outcome?”
- Could you answer that question?
- Be explicit with students regarding learning/assessing outcomes
- Outcome should INSPIRE students to achieve.
- Ask students to rank themselves on whether they are learning the outcomes?
- Write an email in week 2, “How is it going? What could I tweak to improve your learning?” – just having contact improves your outcomes.
- Do not use final exam for outcomes assessment?
- Too much pressure on students in the end.
- Tweak one outcome to reduce busy work.
- Embed SLO assessment in existing exams.
- Use multiple measures, qualitative and quantitative
- Avoid binary assessment of “yes” or “no”. Consider using progressive standards.
- Some students reach basic outcome, others reach higher outcome measured over time can show improvement in teaching.
- May notice success in music rehearsal and not in performance.
- Does it matter if a student has “F”, “C”, “A” over time versus “A”, “C”, “D” on three writing assessments?
Jillian Kinzie, Ph.D. National Institute for Learning Outcomes Assessment (NILOA)
Using Evidence of Student Learning to Improve Higher Education. George D. Kuh, Stanley O. Ikenberry, Natasha A. Jankowski, Timothy Reese Cain, Peter T. Ewell, Pat Hutchings, Jillian Kinzie. ISBN: 978-1-118-90339-1. 304 pages. January 2015, Jossey-Bass.
- Focus assessment practices on USE of data rather than data collection
- Not perfect research but reasonable basis for action (Barbara Woodward?)
- What is intended USE for institutional level research, program level, etc.?
- Normalize assessment – make it easy and part of teaching
Jillian Kinzie, Ph. D. Indiana University
- Use NILOA web page, National Institute for Learning Outcomes Assessment
- Most of the book information is on the web site
- “Assignments” –
- DQP (Degree Qualifications Profile) assignment library
- Search by outcome for assignments (AA level)
- “Catalyze…” walks through scaffolding
- library.org
- See also Lumina Foundaiton website, David Marshall, C.S.
- “Resources”
- “Occasional Papers”
- Pat Hutchings, “Alignment…”
- Cliff Adelman, “To Imagine a Verb…”!!!
- fine tunes measurable verbs
- pages 15-18 especially
- Viewpoints – see James Madison University
- SLO Web pages examples - Long Beach CC (also note transparency on the web site) , Rio Salado College(recognized by NILOA)
- SLO Stars –James Madison University, La Guardia Community College, Rio Salado College
- FLEX and OAC need to work together
- Focus on “how improved” not just on doing assessment
- Give stipend to use what learned
- See “Roadmap to DQP” on NILOA web site for a FLEX idea
- See “Assignment Charrette” on NILOA web site
- “Daring Greatly” – being vulnerable in assessment dialogue
- Peter Ewell writes extensively on informing trustees and public
- See NILOA web site and a book in the book featured in this presentation
- New version allows
- Overview page on a table of all aspects of assessment
- Disaggregation of data using Banner
- Allows faculty input for individual students
- Allows for drawing a sample (e.g. every 5th student?)
- Data can be moved to Excel for further manipulation
- Creates bar graphs
- Compares last year’s bar graph (if saved) with this year’s bar graph