Closing the Loop: Assessment of Introduction to Criminal Justice CJI 101
SLOAT REPORT
Table of Contents
Results of Assessment Plan
I.Introduction:
II.Methodology:
a.Population Sample Used:
b.Administration:
1.Multiple-choice questions blueprinted to MPOs:
2.“It’s the Law” Assignment
3.The Crash Paper
4.Reaction Paper
5.Essay Questions
6.Journal Entry (Court) Paper
c.Instrumentation:
1.Multiple-Choice Blueprinted Midterm Exam Questions
2.Multiple-Choice Blueprinted Final Exam Questions
3.Essay Questions
4.“It’s the Law” Checklist Rubric
5.Reaction Paper Rubric
6.Crash Paper
7.Journal Entry (Court) Paper Assignment Rubric
d.Data Collection:
1.Multiple-choice questions blueprinted to MPOs:
2.“It’s the Law” Assignment
3.Crash Paper
4.Reaction Paper
5.Essay Questions
6.Journal Entry (Court) Paper
e.Data Processing:
III.Results Obtained from the Assessment Instruments Used to Gather Data for this Study:
a. Findings
SUMMARY OF ALL FINDINGS
Summary of all of the Blueprinted Exam Results
Evaluation of Findings by Course Goal
COURSE GOAL #1: Describe the functions of the various components of the criminal justice system (law enforcement, courts and corrections) and explain the interrelationship of these components.
COURSE GOAL #2: Demonstrate basic knowledge of criminal law and the rights of individual citizens.
COURSE GOAL #3: Describe the role of the criminal justice professional in the community and the organization and administration of the various entities in the criminal justice system.
SUMMARY OF SLO FINDINGS FOR COURSE GOAL #3
COURSE GOAL #4: Explain the social, political, economic, and cultural factors within society that influence the development of criminological theory, laws and criminal justice practices and their application to criminal behaviors and sanctions.
COURSE GOAL #5: Demonstrate critical thinking skills within the context of evaluating the complexity of criminal justice issues.
COURSE GOAL #6: Communicate effectively with accurate ‘criminal justice’ terminology in written and/or oral form.
b. Sharing the Results
c. Closing the Loop
d. Recommendations for Spring 2012
APPENDIX A – Criminal Justice Program Curriculum Mapping
SLO Assessment Plan
Detailed Goal (SLO) / Assessment Method / Introduction (I) orMastery (M)
of SLO
Course Goals / Describe the functions of the various components of the criminal justice system (law enforcement, courts and corrections) and explain the interrelationship of these components. / Blueprinted examination questions distributed to all sections and included in midterm and final; “It’s the Law”; Journal entry (Court) paper evaluated using a rubric / N/A
Demonstrate basic knowledge of criminal law and the rights of individual citizens. / Blueprinted examination questions distributed to all sections and included in midterm and final
Describe the role of the criminal justice professional in the community and the organization and administration of the various entities in the criminal justice system. / Blueprinted examination questions distributed to all sections and included in midterm and final
Explain the social, political, economic, and cultural factors within society that influence the development of criminological theory, laws and criminal justice practices and their application to criminal behaviors and sanctions / Homework assignment evaluated using a checklist rubric – Crash
Demonstrate critical thinking skills within the context of evaluating the complexity of criminal justice issues / Reaction Paper assignment evaluated using a rubric
Communicate effectively with accurate ‘criminal justice’ terminology in written and/or oral form. / Essay questions evaluated using a rubric; Homework assignment evaluated using a checklist rubric
Program Goals*
(if course is a major requirement) / Demonstrate basic knowledge of criminal law and the rights of individual citizens. / Blueprinted examination questions distributed to all sections and included in midterm and final / I
Describe of the role of the criminal justice officer in the community, and the organization and administration of the various entities in the criminal justice system. / Blueprinted examination questions distributed to all sections and included in midterm and final / I
Demonstrate critical thinking skills within the context of evaluating the complexity of criminal justice issues. / Reaction Paper assignment evaluated using a rubric / I
Demonstrate effective communication skills. / Essay questions evaluated using a rubric; homework assignments evaluated using a checklist rubric / I
Explain the theory, structure, and function of courts. / Blueprinted examination questions distributed to all sections and included in midterm and final; Journal entry (Court) paper evaluated using a rubric / I
Gen Ed Goals* (if course is a Gen Ed course) / N/A / N/A / N/A
Results of Assessment Plan
Course Goal / MPO / Sample Questions and Instruments / SPECIFIC RESULTS / AVERAGE / COMBINED (midterm final)[1]1. Describe the functions of the various components of the criminal justice system (law enforcement, courts and corrections) and explain the interrelationship of these components. / 1.2 identify and describe the procedural, cultural, tactical, and constitutional impediments involved in the adversarial system / M2. Which Supreme Court case incorporated the right to a lawyer in all criminal felony cases? / 73.53%
Met / 71.24%
Met / 81.58%
Met
M4. Miranda is required______. / 74.51%
Met / 76.32%
Met
M5. The Fourth Amendment requires______. / 65.69% / 51.32%
1.5 explain the theory, structure, and function of courts / F5. The term dual court system refers to ______. / 80.00%
Met / 72.93%
Met
“It’s the Law” Rubric / 31.58% / not included
Court Paper Rubric / 65.85%
2. Demonstrate basic knowledge of criminal law and the rights of individual citizens. / 2.10 distinguish between substantive and procedural law / M1. In criminal law, ______prevents unfair practices such as forced confessions, denial of counsel, or unreasonable searches. / 79.41%
Met / 76.96%
Met
M4. Miranda is required______. / 74.51%
Met / 76.32%
Met
2.11 explain the importance of the due process clause in the criminal justice system / M3. Which Supreme Court case applied the exclusionary rule to the states? / 72.55%
Met / 73.53%
Met / 65.79%
M4. Miranda is required______. / 74.51%
Met / 76.32%
Met
M5. The Fourth Amendment requires______. / 65.69% / 51.32%
3. Describe the role of the criminal justice professional in the community and the organization and administration of the various entities in the criminal justice system. / 3.2 list the four basic responsibilities of the police / F2. The most controversial responsibility of the police involves_____. / 80.00%
Met / 80.00%
Met / 81.58%
Met
3.6 explain community policing and its strategies / F3. A return to foot patrol in order to “connect” with members of the community is a tactic consistent with ____. / 93.00%
Met / 93.00%
Met
3.7 describe the procedures involved in a constitutionally sound criminal investigation / M5. The Fourth Amendment requires______. / 65.69% / 70.10%
Met / 51.32%
M4. Miranda is required______. / 74.51%
Met / 76.32%
Met
3.8 explain the ethical responsibilities of criminal justice professionals / F1. Which of the following is not listed in the text as a justification for court recognition of police discretionary power? / 79.00%
Met / 75.43%
Met
F4. In which case did the U.S. Supreme Court set the limits for the use of deadly force by law enforcement officers? / 75.00%
Met
Crash paper scored using a rubric / 72.29%
Met
4. Explain the social, political, economic, and cultural factors within society that influence the development of criminological theory, laws and criminal justice practices and their application to criminal behaviors and sanctions. / 4.6 analyze the nature of crime and criminal behavior based on major theories and current issues and the relationship of the theories to research, policies, and practices in the field of criminal justice / Crashpaper scored using a rubric / 75.44%
Met / 75.44%
Met
4.9 analyze the impact of societal diversity on the criminal justice system / Crashpaper scored using a rubric / 75.44%
Met / 75.44%
Met
5. Demonstrate critical thinking skills within the context of evaluating the complexity of criminal justice issues. / 5.12 use critical thinking and problem solving; focusing on the criminal justice system and analyzing information from multiple sources, including the print and video media / Reaction paper scored using a rubric / 73.56%
Met / 73.56%
Met
6. Communicate effectively with accurate ‘criminal justice’ terminology in written and/or oral form. / 6.1 utilize effective and persuasive communications skills in written and/or oral form / Reaction paper scored using a rubric / 73.56%
Met / 73.56%
Met
6.2 use accurate ‘criminal justice’ terminology in writings and oral presentations / “It’s the Law” assignment scored using a rubric / 31.58% / not included
Essay questions scored using a rubric / 67.90% / 66.88%
Court paper scored using a rubric / 65.85%
- Introduction:
An assessment study of the CJI 101 Introduction to Criminal Justice course at Essex County College was conducted in Fall 2011 as part of the SLOAT (Student Learning Outcomes Assessment Team) initiative. The study was conducted by Professor Patrice Davis, of the Social Sciences Division, who also is the Criminal Justice Program Coordinator. This study continued the evaluation that originated during Spring 2011 of course-level student learning outcomes on one measurable course performance objective (MPO) in CJI 101. The results of the Spring 2011one-MPO SLO assessment study appear in the table below. Results showed that this MPO was met, with 81.34% of students demonstrating proficiency.
Spring 2011 MIDTERM E X A MSection / 001 / % / 002 / % / 003 / % / Total Day / % / LS (included in Final & Administered separately) / % / CW1 (Administered separately) / % / All sections / %
Total # of Students / 29 / 30 / 25 / 84 / 13 / 37 / 134
CORRECT#=5 / 3 / 10.34% / 3 / 10.00% / 5 / 20.00% / 11 / 13.10% / 4 / 30.77% / 0 / 0.00% / 15 / 11.19%
CORRECT#=4 / 10 / 34.48% / 12 / 40.00% / 11 / 44.00% / 33 / 39.29% / 5 / 38.46% / 25 / 67.57% / 63 / 47.01%
CORRECT#=3 / 8 / 27.59% / 9 / 30.00% / 4 / 16.00% / 21 / 25.00% / 3 / 23.08% / 7 / 18.92% / 31 / 23.13%
CORRECT#=2 / 6 / 20.69% / 4 / 13.33% / 2 / 8.00% / 12 / 14.29% / 1 / 7.69% / 3 / 8.11% / 16 / 11.94%
CORRECT#=1 / 2 / 6.90% / 2 / 6.67% / 3 / 12.00% / 7 / 8.33% / 0 / 0.00% / 1 / 2.70% / 8 / 5.97%
CORRECT#=0 / 0 / 0.00% / 0 / 0.00% / 0 / 0.00% / 0 / 0.00% / 0 / 0.00% / 1 / 2.70% / 1 / 0.75%
Total MIDTERM / 29 / 100.00% / 30 / 100.00% / 25 / 100.00% / 84 / 100.00% / 13 / 100.00% / 37 / 100.00% / 134 / 100%
MET MPO / 21 / 72.41% / 24 / 80.00% / 20 / 80.00% / 65 / 77.38% / 12 / 92.31% / 32 / 86.49% / 109 / 81.34%
PARTIALLY MET / 6 / 20.69% / 4 / 13.33% / 2 / 8.00% / 12 / 14.29% / 1 / 7.69% / 3 / 8.11% / 16 / 11.94%
DID NOT MEET / 2 / 6.90% / 2 / 6.67% / 3 / 12.00% / 7 / 8.33% / 0 / 0.00% / 2 / 5.41% / 9 / 6.72%
Purpose:
The purpose of this Fall 2011 SLOAT assessment study is to determine whether CJI 101 students are learning thirteen (13) of the MPOs (Measurable Performance Objectives) related tothe six (6) Course Goals established by the Criminal Justice Department for this course. The six CJI 101 course goals and their related MPOs that were measured are as follows:
Course Goal / MPO / Sample Questions and Instruments1. Describe the functions of the various components of the criminal justice system (law enforcement, courts and corrections) and explain the interrelationship of these components. / 1.2 identify and describe the procedural, cultural, tactical, and constitutional impediments involved in the adversarial system /
- M4. Miranda is required______.
- M2. Which Supreme Court case incorporated the right to a lawyer in all criminal felony cases?
- M5. The Fourth Amendment requires______.
1.5 explain the theory, structure, and function of courts /
- F5. The term dual court system refers to __.
- “It’s the Law”assignment scored using a rubric
- Court paper scored using a rubric
2. Demonstrate basic knowledge of criminal law and the rights of individual citizens. / 2.10 distinguish between substantive and procedural law /
- M1. In criminal law, ______prevents unfair practices such as forced confessions, denial of counsel, or unreasonable searches.
2.11 explain the importance of the due process clause in the criminal justice system /
- M3. Which Supreme Court case applied the exclusionary rule to the states?
- M4. Miranda is required______.
- M5. The Fourth Amendment requires______.
3. Describe the role of the criminal justice professional in the community and the organization and administration of the various entities in the criminal justice system. / 3.2 list the four basic responsibilities of the police /
- F2. The most controversial responsibility of the police involves______.
3.6explain community policing and its strategies /
- F3. A return to foot patrol in order to “connect” with members of the community is a tactic consistent with ______.
3.7describe the procedures involved in a constitutionally sound criminal investigation /
- M5. The Fourth Amendment requires______.
- M4. Miranda is required ______.
3.8explain the ethical responsibilities of criminal justice professionals /
- F1. Which of the following is not listed in the text as a justification for court recognition of police discretionary power?
- F4. In which case did the U.S. Supreme Court set the limits for the use of deadly force by law enforcement officers?
- Crash paper scored using a rubric
4. Explain the social, political, economic, and cultural factors within society that influence the development of criminological theory, laws and criminal justice practices and their application to criminal behaviors and sanctions. / 4.6 analyze the nature of crime and criminal behavior based on major theories and current issues and the relationship of the theories to research, policies, and practices in the field of criminal justice /
- Crash paper scored using a rubric
4.9analyze the impact of societal diversity on the criminal justice system /
- Crashpaperscored using a rubric
Course Goal / MPO / Sample Questions and Instruments
5. Demonstrate critical thinking skills within the context of evaluating the complexity of criminal justice issues. / 5.12use critical thinking and problem solving; focusing on the criminal justice system and analyzing information from multiple sources, including the print and video media /
- Reaction paper scored using a rubric
6. Communicate effectively with accurate ‘criminal justice’ terminology in written and/or oral form. / 6.1 utilize effective and persuasive communications skills in written and/or oral form /
- Reaction paper scored using a rubric
6.2 use accurate ‘criminal justice’ terminology in writings and oral presentations /
- “It’s the Law”assignment scored using a rubric
- Essay questionsscored using a rubric
- Court paperscored using a rubric
- Methodology:
- Population Sample Used:
All sections and instructors of CJI 101 were selected to participate in this study conducted in Fall 2011 encompassing a total student population of 188 students. Based on information provided by Academic Affairs, the sections consisted of the following distribution of students:
Course / Section / CRN / Title / Enrollment / Withdrawal / InstructorCJI 101 / 001 / 20551 / Intro to Criminal Justice / 35 / 1 / Davis, Patrice
CJI 101 / 002 / 21776 / Intro to Criminal Justice / 33 / 2 / Davis, Patrice
CJI 101 / 003 / 26598 / Intro to Criminal Justice / 36 / 3 / Davis, Patrice
CJI 101 / OAC / 20552 / Intro to Criminal Justice / 29 / 2 / Rutherford, Marilyn
CJI 101 / CW1 / 21512 / Intro to Criminal Justice / 33 / 3 / Blaettler, Joseph
CJI 101 / LS1 / 24239 / Intro to Criminal Justice / 22 / 1 / Troiano, Matthew
Total (All) / 188 / 12
All six of the CJI 101, Introduction to Criminal Justice,sections were selected to participate in this study. The total population of CJI 101 students was 188 students. However, one instructor did not provide data. Hence, only five out of six sections of CJI 101 returned the SLO data collected on any of the blueprinted exams.
Only 4 out of 6 sections took the final and 4 out of 6 sections took the midterm exam. The sample size for the midterm exam was 102, and the sample size for the final exam was 100. Two of the sections administered only one exam; one of these sections administered the midterm exam near the end of the semester. Accordingly, only 76 students within the study cohort took both exams. Note that these 76 students were administered a final exam that included some of the questions from the midterm exam to test their retention of the material covered in the first half of the course and tested on the midterm exam. Furthermore, the midterm and final exams takenby the 76 students each contained 100 multiple-choice questions.
Instructor Response Rate: 5/6 = 83 percent
Student Response Rate: 76/188 = 40 percent for both exams
Student Response Rate: 100/188 = 53 percent for the final exam
Student Response Rate: 102/188 = 54 percent for the midterm exam
The total population of CJI 101 students contained 188 students. The following tables indicatethe confidence interval of the 76-student sample size as well as the 100- and 102-student samples in order to ensure a confidence level of 95%, which was establishedfor this study.
According to the sample size calculator, available at a sample size of 75 students is necessary to be 95 percent confident[2] that the sample size accurately reflects the entire CJI 101 student population with 8.75 percent confidence interval[3]. This means that the population of students taking this course would score either 8.75 points higher or 8.75 points lower than the sample group. The sample of 76 satisfies these parameters.
Using the same sample size calculator, the samples of 100 for the final exam questions and 102 for the midterm exam questions are large enough to accurately predict the CJI 101 student population response with a 95% percent confidence level with a 6.75 margin of error. This means that the population of students taking this course would score either 6.75 points higher or 6.75 points lower than the assessed sample group.
- Administration:
An assessment plan was developed to evaluate the students currently enrolled in CJI 101. The evaluation was multifaceted and employed several different assessment instruments to determine if all of the six course goals were achieved by the students. Assessment designed to include all of the CJI 101 sections employed blueprinted exams in order to facilitate evaluation of sections currently taught by adjunct instructors. The blueprinted exams focused on course goals#1, #2 and #3. The Criminal Justice Program Coordinator, a full-time faculty member, also usedappropriate rubrics to evaluate written assignments that assessed course goals #4, #5 and #6. All of the written assignments have been included in the CJI 101 course for several semesters; i.e., no assignments were added or changed in anticipation of this assessment study.
1.Multiple-choice questions blueprinted to MPOs:
There were two exams administered,which included five questions drawn from the textbook’s test bank and blueprinted to CJI 101 MPOs. These test bank questions were nationally validated, objective multiple-choice questions. There were two different sets of questions for the midterm and final exams. These blueprinted questions were distributed to all of the instructors via email for inclusion in the midterm and final exams, but two of the instructors administered the blueprinted exam questions separately. Two of the sections administered only one exam; one of these sections administered the midterm exam near the end of the semester.
Day Students
The day-student cohort of 76 students within the assessment study sample took both exams. This 76-studentcohort was also administered a final exam that included some of the questions from the midterm exam to test their retention of material covered during the first half of the semester and tested on the midterm exam. The midterm and final exams taken by these 76 students consisted of 100 multiple-choice questions. The blueprinted questions were randomly distributed within different versions of the exams at each administration. All of the day sections received different versions of the same exam. A scantron sheet was used to administer the exams for the day students. A summary scantron report was generated for each class to evaluate for each question the number of correct and incorrect responses. The number of correct answers for each individual student was manually calculated.
Late Start Students
The late start CJI 101 class completed the exam without a scantron sheet. The midterm exam was administered near the end of the semester. The completed exams were scanned and forwarded to the Criminal Justice Program Coordinator. The number of correct answers given for each question and for each individual student was manually calculated by the Criminal Justice Program Coordinator.