Minutes of the Meeting of the Provost’s Assessment Advisory Committee
November 12th, 2004
Present: Wilkinson, Wagner, Reynolds, Moore, Herman, DiStefano, Haldar, Bakhru, Beditz, Myers, Bouchard, Filippone, Horney, Faerman, Persily, Pryse, Singh, Chaiken, Sherman, Walker, Phillips, Mahan, Szelest
Szelest distributed a revised draft, dated November 12, 2004, which incorporated the comments received to-date from the committee members. Primary purpose of today’s meeting is to detail the document changes from the first draft based on this input.
Results of mapping working group - Bouchard explained that rather than explicitly map out which administrative units would be asked to develop assessment plans, along with the Senate Councils they would be required to share their assessment plans with, the working group of Bouchard, Lorang, and Moore believed it more appropriate that each Vice Presidential division would develop it s own assessment plan, and timetable, which would detail the units involved, and require them to share assessment plans and results with appropriate Senate Councils. Szelest indicated that the Advisory Committee will annually review the division plans– of the VPs, and how they are shared with the Senate Councils. The working group focusing on the document section dealing with linking assessment, planning, and resource allocation will be discussed in agenda item #3.
First page and paragraph have been reworked to reflect the primary focus of the institution, as laid out in (1992) mission statement, strategic plan, statement of strategic values, as well as subsequent memo of understanding with SUNY. This is one way of tackling the issue raised at the first meeting regarding how we integrate the research and service missions of the University into the assessment plan. We may recommend to the Provost and President that the strategic plan be revisited to clarify and prioritize goals. What to focus on? Research? Undergraduate education? Faerman pointed out that now (while we have an interim President) is probably not a good time for revisiting the current strategic plan. Szelest agreed, but pointed out that it is within the purview of this committee to ask the President and/or Provost to revisit the strategic plan if there are implications for carrying out assessments, down the road. Perhaps something the Committee could revisit in a year or two.
Singh asked for clarification on the degree to which Szelest (Advisory Committee?) are involved in determining the mission and goals of the administrative units. Szelest responded that on the academic side, Director of Assessment is a facilitator to help the academic units develop their mission and goals within the broader framework of the University’s mission and goals. Szelest will help the administrative units to do that, but they will develop their own mission and goal statements, as well as assessment plans. They must be self-defined in order to be meaningful to them.
Bouchardraised the issue of whether or not there can be workshops to help the administrative units develop plans? How will they know what to do? Szelest indicated that there should be, probably led by himself, working with VP divisions. Would point to Appendix E – template for administrative units – Szelest would help these units to go through a process for administrative units that is parallel to that which the academic units now go through for program review, as facilitated by the Director of Assessment (e.g., define mission, state goals, develop evaluative metrics, analyze data, incorporate results into ongoing operations)
Page 2 – Szelest indicated that changes on this page reflect the expectation that little steps over a six or seven year period will take place with regard to assessment plan implementation. Slow and steady wins the race. Don’t need to do everything at once.
Faerman stressed the need to make clear that assessment activity is ongoing, we do not want to give the impression that activity is only periodic. Assessment cycle means everything is part of the overall plan. The time when you are not collecting the data, you are doing something, planning or analyzing. Assessment plans should reflect a schedule of activity, and some of that activity should be discussing results and thinking about, and devising ways to incorporate the results into the curriculum or operations for administrative units. Szelest agreed to rework language in the document to make this more clear.
Herman asked for clarification regarding whether or not each unit would develop an assessment plan, and share it with whom? On what cycle? Szelest responded that unit assessment plans would be developed under the direction of each VP. VPs may choose to phase in units. But once they begin, annual reports would be required to ensure ongoing activity.
Chaiken asked that the charge of this committee be clarified. Why are we asking units to undergo assessment? Szelest responded that Middle States guidelines have changed. Since our re-accreditation in 2000, the direct assessment of student learning outcomes, and assessments of institutional effectiveness across the institution have become the primary focus of Middle States. New SUNY mandates for assessment in the major and General Education also require our attention to assessment. These external mandates are driving our actions to a large degree. But in so doing, it is in our best interests to make these experiences as positive as possible, to do what makes sense for us.
Persily asked how graduate school accreditation fits in with this? Szelest noted that specialized accreditation meets Middle States standards. Program or school/college documentation toward accreditation is applicable to meeting the Middle States standards.
Principle of Assessment. Szelest added #12 – resource constraints, based on member comments to the previous draft. Comments from Committee reviewers and our experience with assessment in the major to-date indicate that embedded assessment is one way to deal with resource constraints – across both academic and administrative units.
Principle # 13 – faculty and staff need to be involved. Director of Assessment or Asst VP for Strategic Planning and Assessment can’t do it (external) for them. Students are involved in the development of assessment. Share results with students. Szelest noted that our last Middle States review was good, but many faculty were (are) unaware of the SUNY Student Opinion Survey and other assessment activities. Insights from students and faculty/staff should be welcome. And we need to get information out.
Discussion ensued regarding what program review or assessment materials, particularly learning objectives, might be made public. Faerman indicated that we have been very public with sharing General Education learning objectives. Desfosses asked what sharing results with students look like? Assessment results are not generally discussed in departments. Szelest indicated that some accreditation documents(e.g. Middle States site team reports and external reviewer reports) are not public. On principle, results of surveys and assessments should be shared with students. Intentions to do so should be part of the unit assessment plan. Singh asked how student perceptions are worked into assessments? Szelest replied that students have not really been an integral part of assessment design previously. Faerman noted that results of survey or plans that result from survey should be. It is important to include students in assessment cycle and the development of plans to use the results. The emphasis should be on making students part of the process, from design on, not just sharing results. Desfosses noted that some assessment findings may be subject to student grievances. Bakhru noted that in the SED doctoral program review process – the report was confidential. Szelest noted that the assessment portion of program reviews, as now conducted, are appropriate to share publicly. See page 33-34, section V, this is the assessment component of the program reviews. Chaiken asked What else can we (should we) share besides assessment? Goals? Where do we categorize goals/courses? Szelest responded that the assessment plan is a small part of the program review document. This committee could take up what specific parts of the program review self-study to publicly share down the road.
Chaiken noted that he can see how this process might bring benefits to the institution – improvements where needed, where the potential lies. But what are the benefits to students? Sherman added that ideas for improvement need resources and we have no authority to reallocate funds. Szelest responded that part of the assessment process is to find efficiencies from within. Find out what works and what doesn’t. Faerman added that on the academic side, the question of whether or not undergraduates are getting what they need (learning outcomes) is a primary purpose, and that is not zero sum. An additional benefit to alums is the reputation of the university.
Pryse raised the issue that some courses meet multiple program learning goals (e.g., for major, for cross-listed major, for Gen Ed) and this can be overwhelming to assess. One of the results of the assessment process was the development of program mission statements, goals, and learning objectives included on syllabus. SIRF could be tied to learning objectives as an efficient way of determining from a student perspective whether or not courses address stated learning outcomes. Now, assessment (SIRF) is not keyed. Could rewrite SIRF. Then students get to see them and respond. Don’t know if resources are available to do this though. Faerman noted that something similar for General Education has already been done. Very labor intensive. A direction we could go. Important because faculty sometimes think they address learning goals and this might not be clear to students. Szelest noted that this should be something we should put on the short list for 2005 agenda items.
The following changes to the draft document were described by Szelest:
Page 6, middle of page– “The Provost’s Assessment Advisory Committee is charged with annually reviewing and promoting this flow of information.”– this is one of our responsibilities.
Page 7 – reiterates the need for annual review of assessment activities.
Page 9, bullet “e” – committee reps from VP areas are expected to help promote assessment in their VP areas. Previous language only noted the “committee members” in general.
Page 10, Mid-page –Institutional level outcomes. Re-written to make it clearer that institutional level assessments are occurring. Institutional Research has traditionally conducted and reported on this activity. IR will be asked to develop a formal assessment plan for these types of institutional level outcomes. The Assessment Advisory Committee will review that. General Education, which are also institutional level outcomes, are pointed out, and dealt with in a later section.
Page 14 – assessment and resource allocation – this is an area for further discussion as we go forward – concentrate attention at next meeting. Academic unit – goal is to promote student learning. Support units, goal is to promote efficiencies and effectiveness. VP level to allocate resources in the pursuit of divisional goals. We need to be clear of our purposes. Come to consensus on how to link assessment and resource allocation. Assessment should inform decision making, not necessarily drive that resource allocation. Assessment results are not part of any resource allocation formulae. Committee members are asked to comment on this section in particular.
Next steps – Short term goal is to draft an Institutional Assessment Plan that meets Middle States expectations and also sets standards for UAlbany. We will make a recommendation to the Provost. Have a further comment period. Committee members are asked to share this document with colleagues and student groups.
An electronic version of the latest draft will be shared with the Committee members.
It was agreed that the Senate Council chair members of the Committee will share the draft with their Council membership for additional comment, preferably by November 30, and comments received after that date can be discussed at the December 10th meeting. Committee members are urged to comment by November 30 if possible.
Extending the comment period (with a new draft based on new comments) into January was also discussed. Next steps to be discussed further at next meeting.
Mahan questioned the context of moving forward in academic units. How to initiate conversation? Szelest replied that we still need to discuss this, but it may be left up to each VP. [Szelest retrospective comment: perhaps we should have a side meeting of the VP reps to discuss how we foresee implementation in the non-academic units? And what might be good approaches that would facilitate successful implementation?]
Pryse raised the issue of whether or not there may be a need for more resources than Wilkinson and Szelest to help units. Previous director of assessment trained students to run focus groups. What other resources are available? Szelest replied that this is something for the 2005 agenda – we are currently reconfiguring the Director of Assessment position into CETL to take advantage of the support resources there, and to help integrate assessment into the teaching process. Finance and Business has dedicated part of an FTE to promoting assessment-related activities, and Student Affairs has already meaningfully engaged assessment over the past three years with leadership from the VP’s office. The challenges ahead may not be as daunting as they might appear. That said, identifying challenges is precisely our job. Our future recommendations may include training and recognition awards, as well as recommendations for increased staff or funds for professional development opportunities at conferences. Another item for the 2005 agenda.
Persily raised the importance and benefits of sharing best practices across the campus. Presentation from this and other universities would be helpful, and there may be a good deal to learn from the programs that are currently going through specialized accreditation. Szelest noted that was an important item to consider for the 2005 agenda.
Minutes respectfully submitted by Barbara Wilkinson and Bruce Szelest.