BYU-Idaho Program Review Template and Instructions as of September 27, 2017
Program Review Purpose:
1. to raise quality of education by instilling a culture of outcomes and assessments at all levels
2. to ensure that what is happening in the program is fulfilling our mission, meeting the program outcomes, and aligning to BYU-Idaho Core Themes.
A dashboard that summarizes information from each program in your department will be provided to you before your scheduled review. Your time in Curriculum Council will be spent discussing how well your outcomes are being achieved and areas that need attention and/or improvement. In addition to the dashboard, the material asked for below is critical to understanding your programs. Much of the data provided below will be pulled from this report and used to populate the Outcomes and Assessment site for accreditation. We expect a separate packet of information for each program within your department.
Introduction - What overarching goal do you have for your students upon completion of this program of study?
You may include:
1. A short paragraph description of the degree offered.
2. Maybe include a short paragraph discussing numbers specific to your majors such as detailing the number of majors (use the number from the dashboard), % face to face students, % online students, number of FT faculty and number of PT faculty and other numbers from your dashboard.
3. A short paragraph about employment trends in the major, where your graduates are getting work, and how the information is being gathered.
4. How are you incorporating the Purdue/Gallup survey findings into this program?
Program Outcomes and Assessment Plan
1. Have you verified the Program Outcomes listed on the BYU-Idaho Outcomes & Assessments website? Please send changes or updates to Jacob Adams ()
The program outcomes page of the Outcomes & Assessments website is linked to the Catalog as the official statement of program-level outcomes. As program outcomes are modified on this page, remember that these changes should be approved by the Curriculum Council, and if the changes are deemed significant, they may require submission to the Northwest Commission on Colleges and Universities.
2. List the program outcome statements and map each to one (or more) of the six Student Outcomes: Disciples & Leaders, Lifelong Learners, Creative & Critical Thinkers, Effective Communicators, Skilled Professionals, Engaged Citizens
3. Describe the assessment plan for each program outcome including, at minimum, one direct measure from key supporting courses and one indirect measure for each outcome (more would be better)
Assessment Calendar
The Assessment Calendar should indicate the timeline for measuring how well the program outcomes are being achieved. Show on the calendar where analysis and review of the assessment data is taking place.
Assessment Data
1. Summarize the assessment data that your plan has generated over the past two years using the Distinguished, Proficient, Developing, and Unsatisfactory levels where appropriate in either Percentages of students who score in each category or raw numbers of students who score in each category.
2. Have you sent this data to Jacob Adams () to upload onto the BYU-Idaho OA website?
Analysis and Improvement Plan
1. Establish Target Thresholds for each outcome.
2. Summarize conclusions that have been drawn from your analysis of the assessment data.
3. How do you analyze and make decisions based on your assessment data?
4. Who are the people that need this information and how do you provide it to them?
5. Describe program improvement efforts that have been made based on the assessment data?
6. Describe any improvements in student learning based on your data?
Rubrics – Score your program on the Program Outcomes &Assessment Rubric and the Institutional Alignment Rubric. Your self-evaluation based on these rubrics should be discussed in a department faculty meeting as you review each program.
1. When were the department faculty meetings held to discuss these rubrics?
Course-Level Outcomes and Assessments
1. Have you verified the Course Outcomes listed on the BYU-Idaho Outcomes & Assessments website? Please send changes or updates to Jacob Adams ().
2. If the course outcome changes are for a course with online course sections, remember that you must also communicate with the Online Course council about the changes so that course outcomes remain the same across all sections and all modes of delivery. The online course outcomes are set from the Outcomes & Assessment website.
3. How do you ensure that course-level outcomes are clearly articulated for each course in your department?
4. What are you doing to make sure each course has at least one key assessment and that all outcomes and key assessments are common across all sections and modes of delivery?
5. Describe your process for regular course-level reviews. Give an example of a course-level review of outcomes and key assessments that has been done on a multi-section or multi-teacher course within the last year.
6. Does this review compare student learning in campus courses to student learning in online courses? Please share some examples of such comparisons.
7. What are you doing to ensure that course outcomes are included in all syllabi? Has the Department Chair reviewed the course outcomes?
8. What are you doing to ensure that your courses are meeting the credit-hour policy outlined by BYU-Idaho? This is the hours of contact for the credit given policy.
Resources
1. Are there any bottlenecks in your program that slow your students down?
2. What efforts are being made to reduce the number of credits and semesters on campus to graduation?
3. What efforts are you making as a department to ensure the wise use of faculty and other resources? Be specific about scheduling, minimizing low enrolled sections, and other issues.
4. Give a detailed review of course fees in your department. What is the total average cost, in course fees, for a student to complete this program? What is the cost range for this program?
5. Have the Department Chair and Program Lead reviewed Program Sheets, Catalog Information, and Program Review Dashboard in preparation for this program review? What strengths or weaknesses have been identified in the review?
Changes anticipated or needed?
What curriculum changes are anticipated or needed, based on your data, for the coming year? Provide as complete a list as possible.
Recommended page length is 10-12 pages.
Sample Program Outcomes and Assessment Plan
Program Outcome 1 – Students will demonstrate entry-level competency in financial and managerial accounting, accounting systems, tax, and auditing. (Skilled Professionals)
Direct Measure: Final Project Grades from ACCTG 460
Indirect Measures: Alumni and Capstone Student Surveys
Program Outcome 2 - Students will prepare and analyze accounting information in accordance with current professional standards. (Skilled Professionals, Disciples & Leaders)
Direct Measures: Grades on Tax Returns from ACCTG 321 and ACCTG 322
Indirect Measures: Alumni and Capstone Student Surveys
Program Outcome 3 - Students will employ decision-making abilities by identifying issues, analyzing alternatives, and choosing reasoned solutions to problems. (Creative & Critical Thinkers)
Direct Measures: Simulation Exam from ACCTG 312, Group Consulting Project from ACCTG 322, and Final Case from ACCTG 499
Indirect Measures: Alumni and Capstone Student Surveys
Program Outcome 4 - Students will use research skills to access, understand, and apply relevant professional accounting guidance. (Effective Communicators)
Direct Measures: Research Paper #2 from ACCTG 301, SEC Website Project from ACCTG 302 and ACCTG 403, and Research Paper from ACCTG 344
Indirect Measures: Alumni and Capstone Student Surveys
Program Outcome 5 - Students will demonstrate effective use of technology to support accounting information. (Lifelong Learners)
Direct Measures: Major Projects from ACCTG 333 and ACCTG 456
Indirect Measures: Alumni and Capstone Student Surveys
Sample Assessment Calendar
Fall / Win / Spr / Fall / Win / Spr / Fall / Win / Spr
1. Entry Level Competency / Acct 460 Final Proj. / A / RD / A / RD / A / RD
Exit Interview / Q1-3 / A / RD / A / RD / A
5 Yr Alumni Survey / Q1-10 / A / RD / A / RD
2. Prep and Analyzing Tax Information / Acct 321 Exam Score / RD / A / RD / A / A
Acct 322 Research Project / RD / A / RD / A
5 Yr. Alumni Survey / Q11-12 / A / RD / A / RD
3. Decision-making abilities / CPAP Test / A / RD / A / RD / A / RD
5 Yr. Alumni Survey / Q13-17 / A / RD / A / RD
4. Research Skills / Paper #2 / A / RD / A / RD
Alumni Survey / Q. 18-25 / A / RD / A / RD
5. Effective Use of Technology / 333 Project / A / RD / A / RD
456 Project / A / A / RD / A / RD
Alumni and Capstone Survey / A / A / A
RD / A / A / A
RD / A / A / A
RD
A = Assess the Outcome
RD = Review the Data
Sample Assessment Data Table
1. Entry Level Competency / Acct 460 Final Proj. / % / 15 / 60 / 15 / 10 / 70 / 75
Exit Interview / Q1-3 / % / 30 / 65 / 5 / 0 / 80 / 95
5 Yr Alumni Survey / Q1-10 / % / 20 / 31 / 10 / 39 / 70 / 51
2. Prep and Analyzing Tax Information / Acct 321 Exam Score / % / 28 / 21 / 51 / 0
Acct 322 Research Project / % / 35 / 35 / 25 / 5
3. Decision-making abilities / CPAP Test / n / 123 / 1143 / 77 / 3
5 Yr. Alumni Survey / Q13-17 / n / 117 / 1200 / 55 / 15
4. Research Skills / Paper #2 / % / 10 / 70 / 15 / 5
Alumni Survey / Q. 17-25 / n / 130 / 1120 / 86 / 11
5. Effective Use of Technology / 333 Project / n / 227 / 1025 / 124 / 29
456 Project / % / 28 / 48 / 12 / 12
Alumni and Capstone Survey / n / 17 / 1233 / 10 / 0
Distinguished: Excellent level of achievement; exceeds expectations
Proficient: Good level of achievement; meets expectations
Developing: Fair level of achievement; minimally meets expectations
Unsatisfactory: Low level of achievement; fails to meet expectations
Program Outcomes & Assessment Rubric
Program Outcomes / Clear & Complete / Outcomes clearly define expected student learning and cover all areas that are significant to the program / Most outcomes define expected student learning, but may lack clarity or neglect some areas that are significant to the program / Outcomes do not define expected student learning or do not cover most of the areas that are significant to the program / No outcomes are defined
Aligned / All program outcomes are clearly mapped to key supporting courses and to the Student Outcomes / Most program outcomes are mapped to key supporting courses or to the Student Outcomes / Some program outcomes are mapped to key supporting courses or to the Student Outcomes / Outcomes are not mapped to courses or to Student Outcomes
Assessment Plan / Specific Plan / All outcomes have a clear assessment plan, with specific key assessments and a clear data collection schedule / Most outcomes have specific key assessments, but no data collection schedule is established / Some outcomes have specific key assessments, but no data collection schedule is established / No key assessments or data collection schedule have been established
Quality Measures / Each outcome has at least one direct measure and one indirect measure that is targeted specifically to that outcome / Most outcomes have multiple measures, but not all measures are targeted specifically to one particular outcome / Some outcomes have multiple measures, but the measures are not targeted specifically to one particular outcome / No outcomes have key assessments or measures of achievement
Assessment Data / Collected & Reported / All assessment data has been collected per the assessment plan, and applicable data is reported in the 4-level format* / Most assessment data has been collected per the assessment plan, but all applicable data is not reported in the 4-level format* / Little assessment data has been collected per the assessment plan, and applicable data is not reported in the 4-level format* / No assessment data has been collected
Reviewed / All collected assessment data is published or uploaded annually and reviewed by a department committee / Most collected assessment data is published or uploaded annually, but it is not reviewed by a department committee / Little of the collected assessment data is published or uploaded annually, and it is not reviewed by a department committee / No assessment data is published annually or reviewed
Analysis and Improvement Plan / Program Changes / Program changes since the last review are described, along with a clear analysis of the corresponding impact on student performance / Program changes since the last review are described, but with little or no analysis of the corresponding impact on student performance / Some program changes since the last review are described, but with no analysis of the corresponding impact on student performance / No program changes are described
Analysis & Plans / Program strengths and improvement plans are described clearly and are based on evidence from collected data / Program strengths and improvement plans are described, but are not based on evidence from collected data / Program strengths and improvement plans lack clarity and supporting evidence from collected data / No program strengths or improvement plans are described
* The 4-level format refers to reporting assessment data in terms of the percentage of students who achieved Distinguished, Proficient, Developing, and Unsatisfactory levels of performance on the program outcome.