Self-Study Program – Program Assessment System

Description Overview of the Program:

The Self-Study Program is a comprehensive team of university constituents that promotes an ongoing institutional culture of assessment and monitors the effectiveness of an improving assessment system. It also coordinates the activities of the Institutional Research Task Force (IRTF), established in lieu of an Office of Institutional Research, and of the University Scholar Series (USS), developed to establish a program of professional development activities for all members of the university community.

Scope and boundaries of the program:

The primary function of the Self-Study Program is to prepare the reports and other documentation necessary for the reaccreditation of the university by the Middle States Commission for Higher Education and prepare the university for its site visit in March 2005. Due to the lack of an Office of Assessment, an Office of Institutional Research, and a Center for Excellence in Learning and Teaching, Self-Study has also taken on the functions these offices would serve: the development and the management of a university-wide assessment system, the establishment of an Inventory of Resources and other sources of data collection (surveys), and the development of workshops and materials to educate the university community about topics related to assessment, teaching, and learning . The Self-Study Program is participatory and transparent; our activities encourage participation from all sectors of the university and are communicated to all.

The office does notdevelop program assessment systems make designs for programs or units, conduct program assessment systems reviews, interpret survey results, write reports for individual units, make institutional decisions, or set policy. There are no hidden agendas for data collection and there is no evaluation of performance done by the self-study office.

Program goals / Description of key program processes / Top five products of the program (current and future)
Current:
  1. Produce Self-Study documents for MSCHE in preparation for MSCHE site visit (Self-Study Design, Preliminary Report, Revised Report)
  2. Create a culture of assessment for the continuing improvement of teaching and learning at the university
  3. Facilitate the formation and functioning of the Self-Study Subcommittees
  4. Generate useful assessment data
  5. Provide professional development activities for the university community to strengthen assessment
  6. The integration and alignment of the assessment systems.
Future:
1. To develop a meta- assessment system for the university that aligns institutional measures of effectiveness, strategic plans, operational plans, functional unit assessment systems, and program assessment systems into annual budgeting and decision making processes
  1. To bring every unit in the university to a point where it has an ongoing effective program assessment system for the improvement of teaching and learningor and university effectivensseffectiveness
  1. To support the facilitatedthe development and implementation of program assessment systems by providing the necessary infrastructure and support for an ongoing assessment system
4.To align assessment systems with the budget process
  1. To institutionalize an ongoing professional development program for the university community to strengthen assessment practices.
/
  1. Designs: provides assistance and guidance in the development of programassessment systems for academic and administrative units of the university and documents results through annual assessment reports for PAS..
  2. Organizational management: coordinates the activities of 14 subcommittees and IRTF through weekly reports, monthly meetings, bi-semester fora, emails, and Blackboard postings
  3. Professional Development: coordinates University Scholar Series to provide faculty and organizational development through workshops, seminars, and personal interactions.
  4. Data collection: creates and maintains Inventory of Resources, Self-study database, surveys, focus groups, and interviews
  5. Data analysis: Subcommittees review data and analyze data using standard statistical procedures.
  6. Communication: Communicates everything through email, phone mail, Blackboard postings, UDC Spark, University newspapers, and website
  7. Collaboration: encourages team participation by broad segments of the university community (students, faculty, staff, administrators, alumni, and community members)
  8. Writing: Facilitates the integration from a series of inputs, the synthesis, and articulation, evidence supporting communication of teams’ efforts in a coherent and meaningful presentation.
/
  1. Meta-assessment system (future)
  2. Weekly reports, minutes of meetings, Self-study reports (current and future)
  3. Standard reporting system (standard operating procedures with timelines (future)
  4. Nine USS workshops & other sessions
  5. Inventory of resources, Self-Study database (ongoing), online inventory of resources (future on Blackboard and website)
  6. Standardized and coordinated surveys (future)
  7. Facilitators for workshops
  8. Assessment manual
  9. Numerous program assessment systems designed and assessed.

Performance Criteria for the Self-study Office

  • Data is collected in a timely manner using the appropriate measures/instruments to match the diverse needs/requirements of programs with minimal error.
  • Articulates and communicates a consistent clear vision of the assessment initiative and the institutional commitment through constant formal and informal communications that is supported with a growing expertise, expanding resources, and aligned strategic planning.
  • Facilitates and engages the entire community in an extensive ongoing dialog of how to improve the campus quality through a diverse set of assessment practices, improved processes, better analyses, and a strong commitment to help each other strengthen our respective programs.
  • All members of the self-study team are service-oriented, committing extensive time, energy, and talents to carry out the required mission and tasks in a timely manner.
  • Constantly promoting an advanced understanding of the value of assessment through identifying appropriate measures, systematic processes, and annual documentation of accomplishments and action plans.
  • Self-study office information is unbiased, easily accessible, and collected in a honest effort.

Measurement / Evaluation / Evaluation
Qualities / Criterion
Factor / Weight / Vehicle / Instrument / Current Standards / Future Standards / Accountability
Accuracy:
Matches needs
Appropriate measurements done
Minimal error
Timing / Data is collected in a timely manner with appropriate measures to match the diverse needs/requirements of programs with minimal error.
Completeness: All concerns are identified / 30% / All reports reviewed upon submission
(Review process: external and internal reviews) / Report rubric (that will measure the validity and the usefulness of the data) (future)
Statistical Sampling (future)
Oral and written feedback (current) / 30% of the reports meet our standard / 50% of the reports should meet our standard / Marie
Helene
Transparency: / Availability of information on multiple means of communication / 25% / Reports and communications are posted on multiple venues: Blackboard, UDC website, UDC Spark, fliers, posters, voice mail, and in Self-Study Office. / % of material posted on more than one venue / 80% of material is posted on more than one venue / 95% will be posted on more than one venue / Marie
Helene
Conceptual framework of assessment: / Number of units that have improved assessment systems through PAS / 20 / Assessment systems;
Assessment Survey
Assessment reports / Rubric for assessment of program assessment systems (future) / 20% of constituents currently have improved their program assessment system in the last 2 years / 80 % of constituents will document improvement in their PAS in last 3 years / Marie
Conceptual framework of assessment: / % of constituents demonstrating the rudimentary understanding of assessment / 10 / Assessment systems;
Assessment Survey
Assessment reports / Rubric for level of assessor performance (future) / 40% of constituents currently demonstrate a rudimentary understanding of assessment / 80 % of constituents will demonstrate a rudimentary understanding of assessment / Marie
Helene
Leadership:
Vision
Expertise
Strategic thinking
Support / The program provides a clear vision of where the assessment initiative is heading through constant formal and informal communications that is supported with expertise, resources, and strategic planning.
Performance of subcommittees / 5% / Timely submission of reports;
Survey / Checklist / 10% of subcommittees currently submit reports on time / 50% of subcommittees will submit reports on time / Marie
Helene
Collaboration:
Extension
Inclusion
Interaction
Bottom-up participation / The program includes the entire community in an extensive dialog of how to improve the campus quality through a diverse set of processes, analyses, and commitment to help each other.
1) Number and 2)level of participation / 5% / Attendance at meetings;
Surveys / Sign-in sheets / 10% of targeted population / 20% of targeted population / Marie
Helene
Dedication:
Commitment
Follow-through
Service orientation
Engagement / All members of the self-study are service-oriented, committing extensive time, energy, and talents to carry out the required mission and tasks in a timely manner.
Percentage of time devoted to assessment tasks / 5% / Surveys / Responses on surveys
Response rate of surveys / Index of satisfaction
10% response rate / Index of satisfaction
50 % response rate / Marie
Helene
Conceptual framework of assessment:
Community buy-in
Value of assessment
Assessment processes
Annual documentation
Action plan / The program is constantly promoting advanced understanding of the value of assessment, its appropriate measures, systematic processes, and annual documentation of accomplishments and action plans.
Number of units that have improved assessment systems through PAS
% of constituents demonstrating the rudimentary understanding of assessment / 30% / Assessment systems;
Assessment Survey
Assessment reports / Rubric for assessment of assessments (future) / 40% of constituents currently demonstrate a rudimentary understanding of assessment / 80 % of constituents will demonstrate a rudimentary understanding of assessment / Marie
Helene
Transparency:
Unbiased, open access
Easy access
Honesty / The program provides information that is unbiased, with open access, easily accessible, and honest in effort.
Availability of information on multiple means of communication / 25% / Reports and communications are posted on multiple venues: Blackboard, UDC website, UDC Spark, fliers, posters, voice mail, and in Self-Study Office. / % of material posted on more than one venue / 80% of material is posted on more than one venue / 95% will be posted on more than one venue / Marie
Helene