BYU-Idaho Program Review Template and Instructions As of May 8, 2017

BYU-Idaho Program Review Template and Instructions as of May 8, 2017

Program Review Purpose:

1.  to raise quality of education by instilling a culture of outcomes and assessments at all levels

2.  to ensure that what is happening in the program is meeting program and university outcomes

3.  to align programs to university priorities and strategies

A dashboard that summarizes information from each program in your department will be provided to you at least a month before your scheduled review. Your time in Curriculum Council will be spent discussing metrics from this dashboard as well as how well your outcomes are being achieved and areas that need attention and/or improvement. In addition to the dashboard, the material asked for below is critical to understanding your programs. Much of the data provided below will be pulled from this report and used to populate the Outcomes and Assessment site for accreditation. We expect a separate packet of information for each program within your department.

Introduction - What overarching goal do you have for your students upon completion of this program of study?

You may include:

1.  A short paragraph description of the degree offered.

2.  Maybe include a short paragraph discussing numbers specific to your majors such as detailing the number of majors, % face to face students, % online students, number of FT faculty and number of PT faculty and other numbers from your dashboard.

3.  A short paragraph about what % of students are employed at graduation or shortly thereafter, where your graduates are getting work, and how the information is being gathered.

4.  How are you incorporating the Purdue/Gallup survey findings into this program?

Program Outcomes -

1. Have you verified the Program Outcomes listed on the BYU-Idaho Outcomes & Assessments website? Please send changes or updates to Jacob Adams ()

The program outcomes page of the Outcomes & Assessments website is linked to the Catalog as the official statement of program-level outcomes. As program outcomes are modified on this page, remember that these such changes should be approved by the Curriculum Council, and if the changes are deemed significant, they may require submission to the Northwest Commission on Colleges and Universities.

2. Complete the Assessment Plan, Mapping, Assessment Calendar and Data Table using the following as sample formats:

1.  Assessment Plan, Mapping, Calendar and Data

Sample Outcome Statements, Mapping to Core Themes, and Assessment Plan

1.  List the program outcome statements and map each to one of the four core themes.

2.  Describe the assessment plan for each program outcome including, at minimum, one direct measure from key supporting courses and one indirect measure for each outcome (more would be better).

Program Outcome 1 – Students will demonstrate entry-level competency in financial and managerial accounting, accounting systems, tax, and auditing. (CT #3 - Preparing Students for Future Roles)

Direct Measure: Final Project Grades from ACCTG 460

Indirect Measures: Alumni and Capstone Student Surveys

Program Outcome 2 - Students will prepare and analyze accounting information in accordance with current professional standards. (CT #2 – Providing a Quality Education)

Direct Measures: Grades on Tax Returns from ACCTG 321 and ACCTG 322

Indirect Measures: Alumni and Capstone Student Surveys

Program Outcome 3 - Students will employ decision-making abilities by identifying issues, analyzing alternatives, and choosing reasoned solutions to problems. (CT #2 – Providing a Quality Education)

Direct Measures: Simulation Exam from ACCTG 312, Group Consulting Project from ACCTG 322, and Final Case from ACCTG 499

Indirect Measures: Alumni and Capstone Student Surveys

Program Outcome 4 - Students will use research skills to access, understand, and apply relevant professional accounting guidance. (CT #2 – Providing a Quality Education)

Direct Measures: Research Paper #2 from ACCTG 301, SEC Website Project from ACCTG 302 and ACCTG 403, and Research Paper from ACCTG 344

Indirect Measures: Alumni and Capstone Student Surveys

Program Outcome 5 - Students will demonstrate effective use of technology to support accounting information. (CT #4 – Maintaining a Wholesome Environment)

Direct Measures: Major Projects from ACCTG 333 and ACCTG 456

Indirect Measures: Alumni and Capstone Student Surveys

The Assessment Calendar should indicate the timeline for measuring how well the program outcomes are being achieved. Show on the calendar where analysis and review of the assessment data is taking place.

Sample Assessment Calendar

Outcomes / Assessments / Year 1 / Year 2 / Year 3
Fall / Win / Spr / Fall / Win / Spr / Fall / Win / Spr
1. Entry Level Competency / Acct 460 Final Proj. / A / RD / A / RD / A / RD
Exit Interview / Q1-3 / A / RD / A / RD / A
5 Yr Alumni Survey / Q1-10 / A / RD / A / RD
2. Prep and Analyzing Tax Information / Acct 321 Exam Score / RD / A / RD / A / A
Acct 322 Research Project / RD / A / RD / A
5 Yr. Alumni Survey / Q11-12 / A / RD / A / RD
3. Decision-making abilities / CPAP Test / A / RD / A / RD / A / RD
5 Yr. Alumni Survey / Q13-17 / A / RD / A / RD
4. Research Skills / Paper #2 / A / RD / A / RD
Alumni Survey / Q. 18-25 / A / RD / A / RD
5. Effective Use of Technology / 333 Project / A / RD / A / RD
456 Project / A / A / RD / A / RD
Alumni and Capstone Survey / A / A / A
RD / A / A / A
RD / A / A / A
RD

A = Assess the Outcome

R = Review the Data

Assessment Data

2.  Summarize the assessment data that your plan has generated over the past two years using the Distinguished, Proficient, Developing, and Unsatisfactory levels where appropriate in either Percentages of students who score in each category or raw numbers of students who score in each category.

3.  Have you sent this data to Jacob Adams () to upload onto the BYU-Idaho OA website?

Sample Assessment Data Table

Outcomes / Assessments / % or n / Dist / Prof / Dev / Unsat / Target Dist. or Prof. / Actual
1. Entry Level Competency / Acct 460 Final Proj. / % / 15 / 60 / 15 / 10 / 70 / 75
Exit Interview / Q1-3 / % / 30 / 65 / 5 / 0 / 80 / 95
5 Yr Alumni Survey / Q1-10 / % / 20 / 31 / 10 / 39 / 70 / 51
2. Prep and Analyzing Tax Information / Acct 321 Exam Score / % / 28 / 21 / 51 / 0
Acct 322 Research Project / % / 35 / 35 / 25 / 5
3. Decision-making abilities / CPAP Test / n / 123 / 1143 / 77 / 3
5 Yr. Alumni Survey / Q13-17 / n / 117 / 1200 / 55 / 15
4. Research Skills / Paper #2 / % / 10 / 70 / 15 / 5
Alumni Survey / Q. 17-25 / n / 130 / 1120 / 86 / 11
5. Effective Use of Technology / 333 Project / n / 227 / 1025 / 124 / 29
456 Project / % / 28 / 48 / 12 / 12
Alumni and Capstone Survey / n / 17 / 1233 / 10 / 0

Distinguished: Excellent level of achievement; exceeds expectations

Proficient: Good level of achievement; meets expectations

Developing: Fair level of achievement; minimally meets expectations

Unsatisfactory: Low level of achievement; fails to meet expectations

Analysis and Improvement Plan

1.  Establish Target Thresholds for each outcome and discuss whether the program outcomes are being met at an acceptable level.

2.  Summarize conclusions that have been drawn from your analysis and discussion of the assessment data.

3.  How do you analyze and make decisions based on your assessment data?

4.  Who are the people that need this information and how do you provide it to them?

5.  Describe program improvement efforts that have been made based on the assessment data?

6.  Describe any improvements in student learning based on your data?

Rubrics – Score your program on the Program Outcomes &Assessment Rubric and the Institutional Alignment Rubric. Your self-evaluation based on these rubrics should be discussed in a department faculty meeting as you review each program. Your self-evaluation score will be discussed in more detail in the Pre-Review meeting with the Academic VP about a week prior to your Program Review in the Curriculum Council.

1.  When were the department faculty meetings held to discuss these rubrics? ______

2.  In 4-5 sentences, please share your summary of the discussions and be prepared in our pre-review to discuss your program’s strengths and weaknesses based on your assessment data and these two rubrics.

1.  Program Outcomes & Assessment Rubric

Criterion / Distinguished (3) / Proficient (2) / Developing (1) / Unsatisfactory (0) / Score
Program Outcomes / Clear & Complete / Outcomes clearly define expected student learning and cover all areas that are significant to the program / Most outcomes define expected student learning, but may lack clarity or neglect some areas that are significant to the program / Outcomes do not define expected student learning or do not cover most of the areas that are significant to the program / No outcomes are defined
Aligned / All program outcomes are clearly mapped to key supporting courses and to the Core Themes / Most program outcomes are mapped to key supporting courses or to the Core Themes / Some program outcomes are mapped to key supporting courses or to the Core Themes / Outcomes are not mapped to courses or to Core Themes
Assessment Plan / Specific Plan / All outcomes have a clear assessment plan, with specific key assessments and a clear data collection schedule / Most outcomes have specific key assessments, but no data collection schedule is established / Some outcomes have specific key assessments, but no data collection schedule is established / No key assessments or data collection schedule have been established
Quality Measures / Each outcome has at least one direct measure and one indirect measure that is targeted specifically to that outcome / Most outcomes have multiple measures, but not all measures are targeted specifically to one particular outcome / Some outcomes have multiple measures, but the measures are not targeted specifically to one particular outcome / No outcomes have key assessments or measures of achievement
Assessment Data / Collected & Reported / All assessment data has been collected per the assessment plan, and applicable data is reported in the 4-level format* / Most assessment data has been collected per the assessment plan, but all applicable data is not reported in the 4-level format* / Little assessment data has been collected per the assessment plan, and applicable data is not reported in the 4-level format* / No assessment data has been collected
Reviewed / All collected assessment data is published or uploaded annually and reviewed by a department committee / Most collected assessment data is published or uploaded annually, but it is not reviewed by a department committee / Little of the collected assessment data is published or uploaded annually, and it is not reviewed by a department committee / No assessment data is published annually or reviewed
Analysis and Improvement Plan / Program Changes / Program changes since the last review are described, along with a clear analysis of the corresponding impact on student performance / Program changes since the last review are described, but with little or no analysis of the corresponding impact on student performance / Some program changes since the last review are described, but with no analysis of the corresponding impact on student performance / No program changes are described
Analysis & Plans / Program strengths and improvement plans are described clearly and are based on evidence from collected data / Program strengths and improvement plans are described, but are not based on evidence from collected data / Program strengths and improvement plans lack clarity and supporting evidence from collected data / No program strengths or improvement plans are described

* The 4-level format refers to reporting assessment data in terms of the percentage of students who achieved Distinguished, Proficient, Developing, and Unsatisfactory levels of performance on the program outcome.

BYU-Idaho Institutional Alignment Rubric

Criterion / Exceeds expectations (3) / Meets expectations (2) / Needs Improvement (1) / Score
Developing Disciples and Leaders / Building Testimonies / Overtly seeks to build testimonies and true conversion. / Some evidence that the gospel is incorporated into the curriculum. / Little evidence that the gospel is incorporated into the curriculum.
Gives students leadership opportunities. / Leadership opportunities available to some students in the program. / Few leadership opportunities available to students.
Principles of the learning model are easily seen in all areas. / Learning model principles seem to be present in courses. / No evidence of learning model principles in curriculum.
Quality Education / Individualized nurturing attention to students. / Advising of students is sporadic or relegated to advising centers. / Little to no advising and what is done is for motivated students.
Easy to navigate for students. / Curriculum is fairly easy to navigate. / Curriculum is challenging to navigate.
Set up to educate the “whole” person. / Some breadth to curriculum. / Very little breadth to curriculum.
Primary focus is on employment rather than graduate school preparation. / Post-graduation employment is at least a part of the program. / Most emphasis is on theory and graduate school preparation.
Preparing Everyday Students / Core themes are apparent in program outcomes. / Program outcomes are mapped to core themes. / Little linkage between core themes and program outcomes.
Career planning and success are integral to the curriculum. / Career planning is present but not integrated into the curriculum. / No introductory career planning built into the curriculum.
Has a capstone project to prepare students for the workplace. / Has capstone project that is an elective for students. / No capstone project or project is very theoretical.
Scalable to an ever increasing number of students. / Scalability is possible but no apparent plan to use it. / Scalability is questionable.
Fees are minimized. / Fees are not minimized and appear to increase regularly. / Fees imposed on students are excessive or are for non-essential items.
Emphasizes student ability to be life-long learners. / Shows desire to have students as life-long learner. / Life-long learning not part of outcomes.
Prepares students to be teachers in their homes, church, and work. / Some focus on helping students be successful in the home, church, and work. / Primary focus on research or graduate school.
Quality as measured by student success after graduation is assessed. / Quality of program is inferred by student jobs but not directly. / No measure of students’ success after graduation.
Students build people around them as a result of the program. / Student focus is mixed between service and monetary success. / Student’s main focus is on monetary rewards of the program.

Course-Level Outcomes and Assessments