PENN STATE
School of Graduate Professional Studies(Great Valley)
Instructional Systems - INSYS 522 (section 101)
Analyzing Learners and Outcomes
Spring 2010 (1/12-2/25)
______
Course Instructor
Dr. Roy B. Clarianae-mail:
Office: CC212J, Conference Center Building
Phone: 610.648.3253 (office)610.725.5250 (admin) skype: rclariana
Office Hours: Tuesday and Thursday, 5-6 pm and by appointment (preferred)
Adobe Connect Class URL:
Ustream channel online:
Skype: rclariana
Penn State CMAP server site:
Course Texts
- Nikto & Brookhart (2007).Educational assessment of students (5th edition) ISBN 0-13-171925-4
- Tufte (1983).The visual display of quantitative information.
Course Overview
Determining what learners know and can do is essential in instructional design. But analyzing learners and outcomes also must necessarily be about how people think and learn.This course will increase your expertise in analyzing and improving learning as you think about and apply principles of measurement and assessment. An emphasis will be placed on helping you complete the following:
- Apply principles of measurement and assessment
- Classify learning outcomes using several taxonomies
- Write goals and objectives
- Develop various kinds of test items for various learning outcomes
- Develop assessment of a complex performance (process) or tangible artifact (product)
- Assess and describe the uses of individual learner characteristics
- Conduct complex subject matter analysis and cognitive task analysis
- Examine less traditional measures of learning
- Display and report your findings and results
Course Structure and Expectations
Overview - This course is applied adult learning theory.Every session will include a mix of lectures, discussions, group activities, demonstrations, hands-on guided practice, and hands-on independent work.[Add online discussion groups and activities plus commitment to reading chapters here]. You will be provided with a substantial amount of in-class work time and with direct assistance and written guidance from the instructor for all exercises and projects.However, you will be expected to spend out-of-class time preparing for classes, working independently with the software tools, and completing your projects.
Participation - A good class requires both an effective teachers and prepared students.Read the assignments.You should attend every class meeting.If you are unable to attend, contact me as soon as possible, and make a plan to make-up missed work.Excessive absence (i.e., more than 2 classes) will impact your final course grade. You are encouraged to talk to me at any time about anything.I will try to return all assignments by the next class period.For class cancellation due to inclement weather or other emergency, I will send a class email but also check web site or listen to local news media and/or call the Great Valley Campus 610-648-3300 for recorded information.
Academic Integrity–“Academic integrity is the pursuit of scholarly activity free from fraud and deception and is an educational objective of this institution.Academic dishonesty includes, but is not limited to, cheating, plagiarizing, fabricating of information or citations, facilitating acts of academic dishonesty by others, having unauthorized possession of examinations, submitting work of another person or work previously used without informing the instructor, or tampering with the academic work of other students.”At the beginning of each course it is the responsibility of the instructor to provide a statement clarifying the application of academic integrity to that course(from 1989-1000 Policies and Rules for Students, p.25).Violationsof academic integrity will likely result in a failing grade for the assignment or course.Even though academic dishonesty, plagiarism, or any form of cheating is not tolerated, but collaboration and sharing information among students is recommended and encouraged.
Special Services - Special services are available to students with disabilities.“PennState encourages academically qualified students with disabilities to take advantage of its educational programs. It is University policy not to discriminate against persons with disabilities in its admission policies or procedures or its educational programs, services, and activities. The University is responsible for making all of its programs and services available to its students. When auxiliary aids are needed for students with disabilities, PennState is responsible for making reasonable accommodations to individuals with documented disabilities.” If you anticipate needing any type of accommodation, please contact, please contact Kathy Mingioni n advance of your participation or visitation.”
Grading - There are several scored components.Your final project carries the most weight (see table below).Final letter grades will follow the convention:
A / A- / B+ / B / B- / C+ / C / D / F100 - 92 / 91 -- 90 / 89 -- 88 / 87 -- 82 / 81 -- 80 / 79 -- 78 / 77 -- 70 / 69 -- 60 / 59 -- 0
Emergency Evacuation Exercises or Actual Emergency Events
Periodic fire/evacuation exercises are conducted in all occupied PSUGreatValley buildings. Every PSUGreatValley faculty, staff, and student is expected to exit the building and remain outside until the drill or actual event is completed. Drills are a safe opportunity to test the building emergency plan, insure that the fire alarm is working properly, and allows every employee a chance to experience the procedures.
Guidelines in the Event of a Drill or Emergency
(1) Familiarize yourself with the emergency evacuation plan posted in each classroom.(2) Do not use elevators. Use the stairs. (3) Fire doors close but do not lock or trap a person in an area. (4) Should evacuation be necessary, go to the nearest exit or stairway and proceed to a pre-designated evacuation meeting area (150 feet away from building) outside the building. Stay with your instructor and class. (5)_Do not leave the meeting area, return to your classroom, or go to your vehicle unless instructed to do so.
Assignments
Create a paper-based survey / Create a brief survey that includes measures of both ‘importance’ and ‘satisfaction’ for each item. Forma working group in class, agree on a “Level 1” survey purpose and audience (e.g., evaluate a course, poll an audience for their training needs). First create acollaborative mind map of the conceptual qualities and categories and then turn that into a survey. Submit a word document using the Angel drop box that includes: your names, the purpose of your survey, the audience, the mind map, and how the information gathered would be used, and the actual survey (pairs or teams)
Project 2 (5%)
Put a Survey Online / Research how/where to create/deliver your survey online, and then put your survey online (i.e., usingZoomerang or other). Submit using the Angel drop box, be sure to tell me your name(s) and the URL of your survey. (pairs, or teams)
Project 3 (15%)
Primary responder (plus graphic map summary)
(i.e., a concept map)add the voice of challenge here / Sign up for one class to create a one page graphic mapsummary of a chapter (i.e., use CMAP, PowerPoint, Bubbl.com or other) before that class session and also initiate and closely follow the discussion board posts for your chapter (see Project 4). Submit by e-mail to all with the name and URL location of your map ASAP. (individual or pairs)
Project 4 (15%)
Discussion posts / Ask provocative questions,respond to others, and list a resource on our Angel discussion board for each chapter - 2, 3, 4, 7, 8, 9, 10, 11, and 12 (individual)
Project 5 (15%)
Apply measurement concepts / Analyze posttest data.Your score will be based on the accuracy of your analysis. Submit as a word document using the Angel drop box. (work individually or in pairs)
Project 6 (15%)
Assess a complex product / Develop a quality scale with rubrics for evaluating a complex visual display.Submit using the Angel drop box: team member names, your mind map and the quality scale along with a groupreflection on the process of developing and of using the scale. (3 teams of 4)
Final Project (25%) / Analyze posttest data and write a report that contains judgments you have made based on the data.The class will complete a CBT lesson (see below) to produce ‘mock' data for the analysis, or you may use real data from work (preferred). Convert several of the posttest questions into another format, i.e., essay, short answer, etc. Your final project score will be based on a rubric; note that the decisions you make based on the data and your ability to communicate those decisions carries as much weight as the accuracy of the data analysis. Submit using the Angel drop box. (individual, pairs, or teams)
Preparation and participation (5%) / Prepare for classes and actively participate in activities. Ask questions, challenge and respond to others in class. Q: How will we measure this? (individual)
Course Schedule
Date / Scheme /Assignments
/ Readings1 / 1/12/10
T / Course Goals & syllabus, ANGEL, the importance of vocabulary, why assess? Kirkpatrick's 4-levels (intro)
Reporting Level 1 data, funnels / Form teams to
(Prjs. 1 & 2 work) / * means paper-based handout
2 / 1/14/10
Th
(online) / Surveys (Level 1)
Paper, interview, internet?
Internet sites to post your surveys / Meet online:
Teams work on
(Prjs. 1 & 2 work) / * Nicholson et al (2006), Chpt. 9 – Kirkpatrick’s four levels
*Hill et al. (1999), Chpt. 8 – Rating Scales
3 / 1/19/10
T / Taxonomies of learning
& assessment congruency;
Teams work on Projects 1 & 2 / Prj. 3 & 4 are due on depending on your Chapter / NitkoChpt. 2
* Krathwohl (2002)
4 / 1/21/10
Th
(online) / Validity(and Reliability??) / Prjs. 1 & 2 due / Nitko Chpt. 3(and 4?)
5 / 1/26/10
T / Julie Meyers – Excel 2007 lab / Julie’s handout
6 / 1/28/10
Th
(in class) / Beyond Level 1 & Reliability
Data tools (Excel lab)
Test Item analysis overview
Measurement concepts t-tests /
Excel lab and work on Prj. 5
/ Nitko Chpt. 4* Item Analysis – and any online source
7 / 2/02/10
T / Charts and Graphs in Excel
(Excel lab and work on Project 5) / Prj. 5 work
8 / 2/04/10
Th
(online) / Objectives (learning targets)
Writing test items: Declarative knowledge / Prj. 5 due / Nitko Chpts. 7-8
Start reading Tufte
9 / 2/09/10
T / Essays, extended writing
Tufte book club / Nitko Chpts. 9-10
Tufte Chpts. 1-3
10 / 2/11/10
Th
(online) / Quality reports
Quality data display
Work on final project in class / Nitko Chpts. 11-12
11 / 2/16/10
T / Assessing Process & Product
Teams work on Project 6 in class to complete a graphic excellence rubric (Tufte) / Teams work on
Do Prj. 6 in class / Tufte (all Chapters)
12 / 2/18/10
Th
(online) / Learner analysis & Employee Qual/certification processes (i.e., PRAXIS) don’t forget SRTEs / Prj.6 due
2/20/10
Sat. / 10am – 2pm (by appointment)
catch-up/makeup
13 / 2/23/10
T / Work on final project
Looking at Excel 2007 (optional)
14 / 2/25/10
Th / Closing remarks and project work / ALLPrjs. due
Delivery: Tuesday in class and (**)Thursday online as appropriate, except Thursday Jan 28 in class.
Kirkpatrick readings
Item analysis
INSYS 522Clarianapage 1 of 4