PADM 6750: Program Evaluation
Instructor: Wesley L. Meares
Office Email:
Office Location: E 309Allgood Hall
Class Room: N240 Allgood Hall
Class Time: 6:00-8:45
Program evaluation focuses on determining the “success” of a“solution”. Of course one major problem is the definition of success. Program evaluation is important to all scholars who conduct applied research and all practitioners who participate in and monitor public programs; they need the tools of program evaluation. As any researcher will tell you, one of the hardest part of research is formulating a hypothesis, select a design and operationalizing the variables. This is the same with program evaluation. With this in mind particular attention will be paid to designing evaluation and recognizing the validity threats associated with common designs.
We will begin by understanding the different frames or perspectives for program success. Then we will work on developing a program theory (our theory of a hypothesis) and measures of program outcomes. Next, we will review experimental and quasi-experimental research designs and apply them to evaluation. Finally,we will conclude with the standards and ethics associated with the professional practice of program evaluation.
Learning Outcomes: This course is designed to assist you in mastering specific competencies identified by our accrediting body, the Network of Schools of Public Policy, Affairs, and Administration and specific learning outcomes we have identified as being essential to your success. This course addresses:
NASPAA Competency / Student Learning Outcome / Assessment MethodTo lead and manage in public governance / Students will be able to develop a logic model for a program along with an appropriate data collection strategy / Midterm/Final/Exercise
To analyze, synthesize, think critically, solve problems and make decisions / Students will be able to
1. select proper data collection techniques for particular situations
2. present the results of analysis orally and in writing / Assessment of completed program evaluation
To analyze, synthesize, think critically, solve problems and make decisions / Students will demonstrate the ability to effectively communicate and interact with others in a team project. / Peer evaluations
Required Texts:
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. Evaluation: A systematic approach. Sage publications.
All other readings will be provided by the instructor through email or Desire to Learn (D2L).
Suggested Reading:
Bobrow, D. B., & Dryzek, J. S. (1987). Policy analysis by design. University of Pittsburgh Press. Here is a link for the full book at no charge:
Grading:
Grades will be determined by performance on quizzes, a team research project, attendance and participation in class. The breakdown is as follows:
Test 1:20%
Test 2:20%
Presentation: 15%
Program Evaluation Project:30%
Participation: 15%
Grades: The grades are assigned as such:
A = 93-100A- = 90-92B+ = 87-89B = 83-86B- = 80-82C+ = 77-79
C = 73-76C- = 70-72D+ = 67-69D = 63-66D- = 60-62F = 00-59
Course Requirements and Evaluation:
Please note that all assignments and papers should be submitted on paper and on time. No assignments will be accepted via e-mail, disc, or other electronic format. Late assignments will be accepted with a letter grade reduction per every class late.
Program Evaluation Project:
The single most important aspect of this class is the program evaluation project. This project will test your ability to use what you learn in this class by evaluating a real program. The class will be divided into teams working on a project for a real customer. The details of the assignments will be covered on the first night of class, and teams will be assigned that night as well. The evaluation of the project will have four components. Three of the components will be group grades—the applied research report, the oral presentation of your results, and your customer’s evaluation. The fourth part will be peer evaluations.
- Faculty evaluation of written report and oral presentation to the client and the classusing rubrics to be posted on D2L. Please be prepared come prepared to ask questions and actively participate during the presentations.
- Peer evaluations. Each team member will rate his/her teammates using the standardized rubric available on D2L. The completed evaluations will be submitted to the instructor by the due date he/she sets. The instructor averages the assessments to create the final peer evaluation score entered into the grade computation.
- Client evaluation. The client will be given a standardized rubric for use in evaluating the project. That rubric will also be posted on D2L for students to see. That completed evaluation will be submitted to the instructor by the due date he/she sets.
Tests:
You will be given two tests that cover important concepts. The test can contain multiple choice, fill in the blank, matching and essays. It is strongly encourage that everyone stays after test are complete and use the time to work on the group project.
Course Participation:
Student participation is arguably the most valuable component of this course.Students should attend class prepared to discuss the readings and engage in conversations about the subject matter. It is important to note that students will not automatically receive a full grade for simply showing up to class or asking an occasional question. Rather, students are expected to earn their participation grade by acquainting themselves with the course material ahead of time and discussing it with the instructor and other students.
Attendance: (From the AugustaUniversity Student Manual)
Regular, punctual attendance is expected of students in all classes at Augusta University and is counted from the first class meeting each term. Professors are required to monitor student attendance or ongoing participation in courses. Students who incur an excessive number of absences are subject to academic penalty. Additional attendance requirements may be established by the individual schools or programs at Augusta University as well as by the faculty for distance learning courses.
Professors will be responsible for counseling with their students regarding the academic consequences of absences from their classes or laboratories. Students are obligated to adhere to the requirements of each course and each course professor.
To assist the University in complying with federal regulations pertaining to financial aid, faculty members are also required to maintain a record of and report student non-attendance at the start of each academic term. The Vice Provost is responsible for informing faculty of the duration of the nonattendance verification period and appropriate reporting method at the beginning of each academic term. In accordance with this policy, a student who does not attend a class or begin participation in an online course during the non-attendance verification period will be dropped from the course by the professor unless they have contacted their professor and notified them of their reason for non-attendance. In the event a student is dropped for non-attendance during this designated time period, the effect is the same as if the student never registered for the class and the course will not appear on the student’s transcript.
Professors will be flexible enough in their attendance and grading policies to allow students a reasonable number of absences without penalty for extraordinary personal reasons or for officially representing the university. However, if the student has been absent for more than the equivalent of 10 percent of class time, regardless of cause, then the professor may withdraw the student from the class for excessive absences.
It is important to note that the instructor may—or may not—withdraw a student from class based upon attendance. No student should assume that the instructor has initiated a withdrawal. A student not withdrawn from a course who stops attending class (or who never attends class) is subject to receiving a grade of WF or F for the course.
Online Component:
This course may at some point entail an online components. Students will need to complete online class assignments on their respective due dates in order to receive the maximum amount of points. Also, students will need to know the course material covered through online course content (e.g., online lectures, notes, discussions, exercises and activities) for each exam. Furthermore, announcements regarding online class sessions, assignments and discussions will be announced in class as well as through email or in person. Therefore, students must check their email regularly and come to class to stay up to date regarding online course content and activities.
Class Decorum:
Every person in class is expected to behave professionally. Each person is responsible for all of the readings and material covered in class even if you do not attend class. Also, please arrive to class on time. Students showing up more than five minutes late to class will be counted absent. Students that arrive late to class more than two times will be dropped from the course. Assignments should be turned in when they are due. Late assignments will not be accepted.[1]
Each class member is expected to treat their fellow class members and the professor with respect at all times throughout the course (this includes during in-person class sessions as well as during on-line class sessions/online class activities and assignments). Personal attacks, rudeness and acts of disrespect will not be tolerated under any circumstances. Personal attacks, rudeness and acts of disrespect will result in a student being immediately dropped from the course. Also, students disrupting class will be immediately dropped from the course. Inappropriate behavior that will result in a student being immediately dropped from the course includes but is not limited to: insulting another student or the professor in the course; talking while another student or the professor is talking; displaying rude or disrespectful behavior that is directed at another student or the professor in the course during in-person class sessions; displaying rude or disrespectful behavior that is directed at another student or the professor during online class discussions, online debates and online exercises and assignments.
ADA Compliance and Disability Statement:
The University will make reasonable academic accommodations for students with documented disabilities. Students should contact Testing and Disability Services (Galloway Hall; 706-737-1469; as soon as possible for more information and/or to initiate the process for accessing academic accommodations. If you are registered with Testing and Disability Services and have not met with me yet. Please see me as soon as possible to discuss your accommodations and how I may be of assistance to you throughout the course.
Augusta University abides by the Americans with Disabilities Act (equal and timely access) and Section 504 of the Rehabilitation Act of 1973 (non-discrimination on the basis of disability). If you have a disability and are in need of academic accommodations but have not yet registered with Testing and Disability Services (TDS) (Galloway Hall; 706-737-1469; please contact the office as soon as possible for more information and/or to initiate the process for accessing academic accommodations. I also encourage students with disabilities receiving accommodations through TDS to discuss these with me, after class or during my office hours, so that I may be better informed on how to assist you during the semester.
Any student with a documented disability receiving academic accommodations through the Testing and Disability Services is requested to speak with me as soon as possible. All discussions will remain confidential and are intended to assist me with ensuring your accommodations are appropriately implemented throughout the course.
Students with disabilities who believe they may need academic accommodations are encourage to speak with me after class and will need to contact Testing and Disability Services (TDS) (Galloway Hall; 706-737-1469; as soon as possible for more information and/or to initiate the process for accessing academic accommodations.
Academic Integrity:
It is expected that a student will refrain from plagiarism and cheating. Plagiarism and cheating are serious breaches of academic conduct and may result in permanent dismissal. Each student is advised to become familiar with the various forms of academic dishonesty as explained in the Code of Student Rights and Responsibilities in either the Undergraduate or Graduate Catalog on the Augusta University website. A plea of ignorance is not acceptable as a defense against the charge of academic dishonesty.
Instructors may use a range of strategies (including plagiarism-prevention software at the university) to compare student works with private and public information resources in order to identify possible plagiarism and academic dishonesty. Comparisons of student works may require submitting a copy of the original work to the plagiarism-prevention service. The service may retain that copy in some circumstances. Academic units or programs may establish a more rigorous standard of review or consent, which will be noted in the relevant guidelines.
Plagiarism: To adopt and reproduce as one’s own, to appropriate to one’s own use and incorporate in one’s own work without acknowledgement, the ideas of othersor passages from their writings and works.
Collusion: To obtain from another party, without specific approval in advance by the professor, assistance in the production of work offered for credit to the extentthat the work reflects the ideas or skills of the party consulted rather than those of the person in whose name the work is submitted.
Duplicity: To offer for credit identical or substantially unchanged work in two or more courses, without specific advance approval of the professors involved.
Other Course Policies:
All other aspects of this course will follow the Pamplin Course Policies posted online at this URL: Please read them carefully, as they include important information about Academic Honesty, Disruptive Behavior, Accommodations for Disabilities, Withdrawals, and other topics. By remaining in this course, you agree to abide by these policies.
Course Schedule:
Please note that the dates for each topic are tentative. Some topics will require greater discussion and more time than others. Students are expected to attend all class sessions and to be prepared for the next topic by completing readings ahead of time.
The course syllabus is subject to change – any changes will be announced during class sessions or through email.
Date / Topic / Preparation / Deliverable1/10 / First Day: Introduction and expectations / Syllabus and Project Rubric. Discuss Expectations and Set up Drop box
1/17 / Introduction to Program Evaluation / Rossi, Chapter 1; Fitzpatrick pp. 1-17 and Chapter 2
1/24 / Planning for Evaluation / Rossi, Chapters 2-4; Fitzpatrick, Chapter 5
1/31 / Project Work Day 1: / Mapping out the Project and Meeting with the Sponsor (PERT and Design). Work on Designing the Survey for Sponsor. (A light reading on surveys and survey design TBD)
2/7 / Program Theory / Posavac, Chapter 3; Rossi, Chapter 5; Learning from Logic Models; Fitzpatrick, Chapter 6
2/14 / Measuring and Monitoring / Rossi, Chapters 6 and 7; Fitzpatrick, Chapter 7
2/21 / Descriptive Designs / Designs for Description;Balbach on Case Studies; Fitzpatrick Chapter 8
2/28 / Project Work Day 2: / Crash Course in Qualtrics and Survey Design. Discuss possible sources for information needed in the survey (BLS, etc.)
3/7 / Test 1 / Test
3/14 / Research Design I and II / Trochim's Design and Internal Validity, Experimental Design, Quasi-Experimental Design; Fitzpatrick Chapter 9
3/21 / Project Work Day 3: / Refresher for SPSS and Working on the Analysis for the paper.
3/28 / Cost Benefit Analysis / CBA Case, Chapter 14 Gupta. Fitzpatrick Chapter 12
4/3-4/7 / SPRING BREAK! ENJOY THE MASTERS.
4/11 / Evaluation Ethics Getting Results and Moving Forward / Rossi, Chapter 12; Fitzpatrick 14 and 15. Review for Test
4/18 / Test 2 / Test
4/25 / Project Work Day 4 / Putting on the Finishing Touches: Project Work Day
5/2 / Presentation: Project Due / The Final Project is Due this Day
PEER EVALUATION FORM
Peer Being Assessed: ______
Assessor: ______
Assessing a peer is an important responsibility. I am relying on each of you to give me an honest appraisal of the contributions this peer made to your group project. For each dimension below, assign your peer a score.
Criteria / Scale / Examples / PointsInvolvement in the planning process / 0-10 points / Suggested deadlines, ways to structure tasks, or accomplish group goals
Time management / 0-10 points / Turned in required materials/assignments on time, prepared for team meetings, responded to communications in a timely manner
Contribution to the research project / 0-15 points / Suggested solutions to problems the group encountered, found/gathered/analyzed data, shared/taught others skills important to project completion, found/gathered/summarized source materials, designed materials for a presentation such as a handout, slides, or volunteered to do the group’s oral presentation
Teamwork / 0-15 points / Raised problems or issues to the team as they occurred, listened well to concerns/issues raised by others on the team, encouraged other team members, demonstrated courtesy and respect for the views or needs of others
Provide two specific examples of positive things this person contributed to the group, either in terms of the work itself or how the group interacted as a team.
Suggest at least one way this person could improve, either in terms of the work itself or how he or she interacts with others.
1
[1] Late assignments will not be accepted under most circumstances. In the event of an unforeseen family or medical emergency students may turn in late assignments if the appropriate documentation is provided. The instructor retains sole discretion in determining whether to accept late assignments. Also, grade deductions may be applied to assignments that are turned in late.