BYU-Idaho Program Review Template and Instructions As of Dec. 31, 2017

BYU-Idaho Program Review Template and Instructions as of Dec. 31, 2017

Program Review Purpose:

1. to raise quality of education by instilling a culture of outcomes and assessments at all levels
2. to ensure that what is happening in the program is fulfilling our mission, meeting the program outcomes, and aligning to BYU-Idaho Core Themes.

A dashboard that summarizes information from each program in your department will be provided to you before your scheduled review. Your time in Curriculum Council will be spent discussing how well your outcomes are being achieved and areas that need attention and/or improvement. In addition to the dashboard, the material asked for below is critical to understanding your programs. Much of the data provided below will be pulled from this report and used to populate the Outcomes and Assessment site for accreditation. We expect a separate packet of information for each program within your department.

Introduction - What overarching goal do you have for your students upon completion of this program of study?

You may include:

1. A short paragraph description of the degree offered.
2. Maybe include a short paragraph discussing numbers specific to your majors such as detailing the number of majors(use the number from the dashboard), % face to face students, % online students, number of FT faculty and number of PT faculty and other numbers from your dashboard.
3. A short paragraph about employment trends in the major, where your graduates are getting work, and how the information is being gathered.
4. How are you incorporating the Purdue/Gallup survey findings into this program?

Program Outcomes and Assessment Plan

1. Have you verified the Program Outcomes listed on the BYU-Idaho Outcomes & Assessments website?Please send changes or updates to Jacob Adams ()

The program outcomes page of the Outcomes & Assessments website is linked to the Catalog as the official statement of program-level outcomes. As program outcomes are modified on this page, remember that these changes should be approved by the Curriculum Council, and if the changes are deemed significant, they may require submission to the Northwest Commission on Colleges and Universities.

1. List the program outcome statements and map each to one of the six University Student Learning Outcomes: Disciples & Leaders, Lifelong Learners, Creative & Critical Thinkers, Effective Communicators, Skilled Professionals, Engaged Citizens
2. Describe the assessment plan for each program outcome including, at minimum, one direct measure from key supporting courses and one indirect measure for each outcome (more would be better)

Assessment Calendar

The Assessment Calendar should indicate the timelinefor measuring how well the program outcomes are being achieved. Show on the calendar where analysis and review of the assessment data is taking place.

Assessment Data

1. Summarize the assessment data that your plan has generated since the last Program Review using the Distinguished, Proficient, Developing, and Unsatisfactory levels where appropriate in either Percentages of students who score in each category or raw numbers of students who score in each category.

1. Have you sent this data to Jacob Adams () to upload onto the BYU-Idaho OA website?

Analysis and Improvement Plan

1. Establish Target Thresholds for each outcome.
2. Summarize conclusions that have been drawn from your analysis of the assessment data.
3. How do you analyze and make decisions based on your assessment data?
4. Who are the people that need this information and how do you provide it to them?
5. Describe program improvement efforts that have been made based on the assessment data?
6. Describe any improvements in student learning based on your data?

Rubrics – Score your program on the Program Outcomes &Assessment Rubric and the Institutional Alignment Rubric. Your self-evaluation based on these rubrics should be discussed in a department faculty meeting as you review each program.

1. When were the department faculty meetings held to discuss these rubrics?

Outside Advisory Council

1. Do you have an outside advisory council for your program?
2. If so, when was the last review with that advisory council?
3. Summarize their recommendations?

Course-Level Outcomes and Assessments

1. Have you verified the Course Outcomes listed on the BYU-Idaho Outcomes & Assessments website? Please send changes or updates to Jacob Adams ().
2. If the course outcome changes are for a course with online course sections, remember that you must also communicate with the Online Course council about the changes so that course outcomes remain the same across all sections and all modes of delivery. The online course outcomes are set from the Outcomes & Assessment website.
3. How do you ensure that course-level outcomes are clearly articulated for each course in your department?
4. What are you doing to make sure each course has at least one key assessment and that all outcomes and key assessments are common across all sections and modes of delivery?
5. Describe your process for regular course-level reviews. Give an example of a course-level review of outcomes and key assessments that has been done on a multi-section or multi-teacher course within the last year.
6. Does this review compare student learning in campus courses to student learning in online courses? Please share some examples of such comparisons.
7. What are you doing to ensure that course outcomes are included in all syllabi? Has the Department Chair reviewed the course outcomes?
8. What are you doing to ensure that your courses are meeting the credit-hour policy outlined by BYU-Idaho? This is the hours of contact for the credit given policy.

Online Course Review Worksheet

Online Courses Offered / # Sections offered on Campus / # Sections offered Online / Common Outcomes Y/N / Common Key Assess. Y/N / Date of last Course Review / Have You reviewed the Course Data Portal? Y/N

Resources

1.Are there any bottlenecks in your program that slow your students down?

2.What efforts are being made to reduce the number of credits and semesters on campus to graduation?

3.What efforts are you making as a department to ensure the wise use of faculty and other resources? Be specific about scheduling, minimizing low enrolled sections, and other issues.

4. Give a detailed review of course fees in your department. What is the total average cost, in course fees, for a student to complete this program? What is the cost range for this program?

5. Have the Department Chair and Program Lead reviewed Program Sheets, Catalog Information, and Program Review Dashboard in preparation for this program review? What strengths or weaknesses have been identified in the review?

Changes anticipated or needed?

What curriculum changes are anticipated or needed, based on your data, for the coming year? Provide as complete a list as possible.

Recommended page length is 11-12 pages.

SampleProgram Outcomes and Assessment Plan

Program Outcome 1 – Students will demonstrate entry-level competency in financial and managerial accounting, accounting systems, tax, and auditing.(Skilled Professionals)

Direct Measure: Final Project Grades from ACCTG 460

Indirect Measures: Alumni and Capstone Student Surveys

Program Outcome 2 - Students will prepare and analyze accounting information in accordance with current professional standards.(Skilled Professionals)

Direct Measures: Grades on Tax Returns from ACCTG 321 and ACCTG 322

Indirect Measures: Alumni and Capstone Student Surveys

Program Outcome 3 - Students will employ decision-making abilities by identifying issues, analyzing alternatives, and choosing reasoned solutions to problems.(Creative & Critical Thinkers)

Direct Measures: Simulation Exam from ACCTG 312, Group Consulting Project from ACCTG 322, and Final Case from ACCTG 499

Indirect Measures: Alumni and Capstone Student Surveys

Program Outcome 4 - Students will use research skills to access, understand, and apply relevant professional accounting guidance. (Effective Communicators)

Direct Measures: Research Paper #2 from ACCTG 301, SEC Website Project from ACCTG 302 and ACCTG 403, and Research Paper from ACCTG 344

Indirect Measures: Alumni and Capstone Student Surveys

Program Outcome 5 - Students will demonstrate effective use of technology to support accounting information.(Lifelong Learners)

Direct Measures: Major Projects from ACCTG 333 and ACCTG 456

Indirect Measures: Alumni and Capstone Student Surveys

Sample Assessment Calendar

Program Level Student Learning Outcomes / Assessments / Year 1 / Year 2 / Year 3
Fall / Win / Spr / Fall / Win / Spr / Fall / Win / Spr
1. Entry Level Competency / Acct 460 Final Proj. / A / RD / A / RD / A / RD
Exit Interview / Q1-3 / A / RD / A / RD / A
5 Yr Alumni Survey / Q1-10 / A / RD / A / RD
2. Prep and Analyzing Tax Information / Acct 321 Exam Score / RD / A / RD / A / A
Acct 322 Research Project / RD / A / RD / A
5 Yr. Alumni Survey / Q11-12 / A / RD / A / RD
3. Decision-making abilities / CPAP Test / A / RD / A / RD / A / RD
5 Yr. Alumni Survey / Q13-17 / A / RD / A / RD
4. Research Skills / Paper #2 / A / RD / A / RD
Alumni Survey / Q. 18-25 / A / RD / A / RD
5. Effective Use of Technology / 333 Project / A / RD / A / RD
456 Project / A / A / RD / A / RD
Alumni and Capstone Survey / A / A / A
RD / A / A / A
RD / A / A / A
RD

A= Assess the Outcome

RD = Review the Data

Sample Assessment Data Table

Program Level Student Learning Outcomes / % / Dist / Prof / Dev / Unsat / Target Dist. or Prof. / Actual
Dist. or Prof.
1. Entry Level Competency / % / 15 / 60 / 15 / 10 / 70 / 75
2. Prep and Analyzing Tax Information / % / 28 / 21 / 51 / 0 / 80 / 49
3. Decision-making abilities / % / 70 / 10 / 10 / 10 / 80 / 80
4. Research Skills / % / 55 / 20 / 0 / 25 / 80 / 75
5. Effective Use of Technology / % / 20 / 30 / 20 / 30 / 80 / 50

Distinguished: Excellent level of achievement; exceeds expectations

Proficient: Good level of achievement; meets expectations

Developing: Fair level of achievement; minimally meets expectations

Unsatisfactory: Low level of achievement; fails to meet expectations

Program Outcomes Assessment Rubric

Criterion / Distinguished (3) / Proficient (2) / Developing (1) / Unsatisfactory (0) / Score
Program Outcomes / Clear & Complete / Outcomes clearly define expected student learning and cover all areas that are significant to the program / Most outcomes define expected student learning, but may lack clarity or neglect some areas that are significant to the program / Outcomes do not define expected student learning or do not cover most of the areas that are significant to the program / No outcomes are defined
Aligned / All program outcomes are clearly mapped to key supporting courses and to the Student Outcomes / Most program outcomes are mapped to key supporting courses or to the Student Outcomes / Some program outcomes are mapped to key supporting courses or to the Student Outcomes / Outcomes are not mapped to courses or to Student Outcomes
Assessment Plan / Specific Plan / All outcomes have a clear assessment plan, with specific key assessments and a clear data collection schedule / Most outcomes have specific key assessments, but no data collection schedule is established / Some outcomes have specific key assessments, but no data collection schedule is established / No key assessments or data collection schedule have been established
Quality Measures / Each outcome has at least one direct measure and one indirect measure that is targeted specifically to that outcome / Most outcomes have multiple measures, but not all measures are targeted specifically to one particular outcome / Some outcomes have multiple measures, but the measures are not targeted specifically to one particular outcome / No outcomes have key assessments or measures of achievement
Assessment Data / Collected & Reported / All assessment data has been collected per the assessment plan, and applicable data is reported in the 4-level format* / Most assessment data has been collected per the assessment plan, but all applicable data is not reported in the 4-level format* / Little assessment data has been collected per the assessment plan, and applicable data is not reported in the 4-level format* / No assessment data has been collected
Reviewed / All collected assessment data is published or uploaded annually and reviewed by a department committee / Most collected assessment data is published or uploaded annually, but it is not reviewed by a department committee / Little of the collected assessment data is published or uploaded annually, and it is not reviewed by a department committee / No assessment data is published annually or reviewed
Analysis and Improvement Plan / Program Changes / Program changes since the last review are described, along with a clear analysis of the corresponding impact on student performance / Program changes since the last review are described, but with little or no analysis of the corresponding impact on student performance / Some program changes since the last review are described, but with no analysis of the corresponding impact on student performance / No program changes are described
Analysis & Plans / Program strengths and improvement plans are described clearly and are based on evidence from collected data / Program strengths and improvement plans are described, but are not based on evidence from collected data / Program strengths and improvement plans lack clarity and supporting evidence from collected data / No program strengths or improvement plans are described
Total: / /24

* The 4-level format refers to reporting assessment data in terms of the percentage of students who achieved Distinguished, Proficient, Developing, and Unsatisfactory levels of performance on the program outcome.

BYU-Idaho Institutional Alignment Rubric

Core Themes / Exceeds expectations (3) / Meets expectations (2) / Needs Improvement (1) / Score
Develop disciples of Jesus Christ who are leaders in their homes, the Church and their communities / Building Testimonies
High-Quality Education
Reach
Cost / Overtly seeks to build testimonies and true conversion. / Some evidence that the gospel is incorporated into the curriculum. / Little evidence that the gospel is incorporated into the curriculum.
Gives students leadership opportunities. / Leadership opportunities available to some students in the program. / Few leadership opportunities available to students.
Principles of the learning model are easily seen in all areas. / Learning model principles seem to be present in courses. / No evidence of learning model principles in curriculum.
Individualized nurturing attention to students. / Advising of students is sporadic or relegated to advising centers. / Little to no advising and what is done is for motivated students.
Easy to navigate for students. / Curriculum is fairly easy to navigate. / Curriculum is challenging to navigate.
Set up to educate the “whole” person. / Some breadth to curriculum. / Very little breadth to curriculum.
Primary focus is on employment rather than graduate school preparation. / Post-graduation employment is at least a part of the program. / Most emphasis is on theory and graduate school preparation.
StudentOutcomes are apparent in program outcomes. / Program outcomes are mapped to Student Outcomes. / Little linkage between StudentOutcomes and program outcomes.
Career planning and success are integral to the curriculum. / Career planning is present but not integrated into the curriculum. / No introductory career planning built into the curriculum.
Has a capstone project to prepare students for the workplace. / Has capstone project that is an elective for students. / No capstone project or project is very theoretical.
Scalable to an ever increasing number of students. / Scalability is possible but no apparent plan to use it. / Scalability is questionable.
Fees are minimized. / Fees are not minimized and appear to increase regularly. / Fees imposed on students are excessive or are for non-essential items.
Emphasizes student ability to be life-long learners. / Shows desire to have students as life-long learner. / Life-long learning not part of outcomes.
Prepares students to be teachers in their homes, church, and work. / Some focus on helping students be successful in the home, church, and work. / Primary focus on research or graduate school.
Quality as measured by student success after graduation is assessed. / Quality of program is inferred by student jobs but not directly. / No measure of students’ success after graduation.
Students build people around them as a result of the program. / Student focus is mixed between service and monetary success. / Student’s main focus is on monetary rewards of the program.

1