The IMPACT seminar evaluation
The IMPACT seminar was evaluated on the basis of annual reports that the seminar participants were obliged to produce by the end of each year. Thus qualitative information collected through the annual reports provided the data used by the evaluator to determine the multiple aspects of change initiated by the seminar with respect to the participants, university and community. The seminar participants were asked to structure their reports to include the short description of the following elements:
strategic professional and personal goals,
short-term professional plans and revision of such if needed,
summary of their achievements,
personal change as an effect of IMPACT seminar
benefits of participation in IMPACT seminar(at personal and university/community levels)
suggestions For Future IMPACT classes
The material has beenprocessedthrough the QSR Nudist software. The evaluator has created the following categories (nodes) to detect different concepts (constructs) mentioned inIMPACTfellows’ reports:
1.Time/Stress Management. In reflection over the seminar topic about the strategies that may be helpful in improving the quality and quantity of productive activity, a number of IMPACT fellows stressed the importance of new techniques that they learned while approaching the issues of overwhelming workload. New ways of structuring and planning their workday are highlighted by several fellows as useful strategies they learned.
2. Perception of Self (Self-Awareness). This node refers to the changes in personal and social identity as a result of attending the seminar.It is an axiom of cultural anthropology that we are not endowed at birth with conscious self-awareness, but we have the propensity to attain it (Foster, 2002; Herder, 1800; Rousseau and Herder, 1986). Therefore, this node is critical in assessment of how and why IMPACT seminar triggered change at the levels of social identity and, perhaps, even worldview. The change in self-awareness that a number of IMPACT fellows attribute to the seminar is described as positive and empowering, a result of continuous professional maturation and rise in self-esteem.
3. Learning (New Skills). An openness to learn new ways of doing things is a central motif that passes through all IMPACT fellows’ reports and therefore is important as a criterion of IMPACT success. Not only does this node reflect real progress in secondary socializations to the norms of the institution, as is also evidenced by the two aforementioned nodes, but, more importantly, it captures the openness of the seminar participants to new modes of thinking and behavior, as well as their adaptability and flexibility to new social environments.[1] Therefore, this node has a strong nexus with “Attitude Change” node (see Table 1).
4. Attitude Change.The evaluator used the definition of attitude as a complex of beliefs and feelings that people have about specific ideas, events, or other people. This definition is based primarily on Cohen’s (1964) and Muzafer and Hovland’s (1961) work. Some change of attitude is expected to take place during the course because the seminar stimulates cognitive and psychological adjustment to new information flows by allowing participants observe their own behavior in a new social setting (Bem, 1967). The intensive seminar is designed to foster an environment in which participants begin to trust their new colleagues enough to analyze publicly their own beliefs about the institution and their role in it. Such reflections are encouraged throughout their year of participation in monthly team meetings. The IMPACT seminar provides fellows with the resources that enable them to perform at a higher level and achieve academic success. The “Attitude Change” is important because a number of fellows note change in their attitude towards themselves and others around them (see Table 1).
5. Empowerment. This node is a reflection of the fact that many IMPACT fellows attest to being empowered in consequence of the positive experiences and encounters developed in the course of the seminar. At the level of personality development empowerment means a culmination of self-realization (Ozer and Bandura, 1990).It means that an individual has come to an understanding of his or her own worth and through the resources available through the program, self-realization becomes possible. Note that empowerment does not necessarily imply involvement of only an individual agent. In practice, agency may be implemented at the individual level or in groups. At the institutional level empowerment is critical for involvement of faculty in decision-making. Because institutions of higher education generally are hierarchical organizations,faculty and administration often have different professional agendas. Hence, faculty members are often skeptical about the university administration decisions. Empowerment is the positive personal change that contrasts with the distrust that some faculty have towards upper administration. Empowerment in the sense of institutional transformation leads to the increased interest in the decision-making process.
6. Increased Productivity.As already stipulated earlier, time and stress management was not only set as a priority by many IMPACT participants, but also a number of the program seminars and workshops are routinely allocated to address this particular problem. Keeping time/stress management in focus, IMPACT participants were encouraged to develop an action plan for improvement with accountability and benchmarks. The action plans allowed IMPACT participants compare the results of their research and teaching activity before and after attending the seminar. The better results in a very limited framework of the seminar (1 year) are reported almost unanimously by the IMPACT participants. There are signs of continuous improvement in productivity not only in terms of research and publication record, but also in terms of more effective teaching techniques, etc.
7. Integration of Personality.One of the main benefits of the seminar that has been constantly stressed by its participants was learning the practical skills of how to become an integrated personality. Here lies the importance of IMPACT in creating a conducive social environment which serves as a source of the external stimuli which, in turn, bring about the integration.Social environment and significant others (including role models) provide us with the stimuli to imitate which results in creating common patterns of behavior(Dewey, 1916). Although the integration of personality usually manifests itself through a variety of behaviors, an integrated personalities self-identify themselves as “being together”, “being whole”. Several IMPACT fellows reported becoming “whole” as a result of attending the seminar (see Table 1).
8. Integration of Teaching, Research and Service. Among the personal changes that IMPACT fellows ascribe to the positive influence of the seminar is the necessity to effectively incorporate their teaching into their research and service. Apart from “keeping the focus” and “staying in focus” which are repeatedmotifsin the fellows’ reports, the integration of research and teaching is perceived as an important strategy to be employed by the junior faculty in the process of their professional development. It is by utilizing this strategy that the continuous enhancement of the scholarly work of teaching and research becomes a virtually limitless opportunity (Cross, and Steadman, 1996).
9. Leadership. One of the positive changes that the seminar has brought about in IMPACT fellows is genuine interest in leadership. As one of the recurrent themes in fellows’ annual reports, interest in leadership is seen as not just a personal enterprise but as a collaborative effort. Indeed, one of the main goals of the seminar was to discuss and explore the challenges and opportunities of academic leadership. This effort should lead to the formation of a cadre of leaders who may take up the sustainability of institutional change. Availability of role models is one of the most important factors that affect leadership as a process. The formation of integrated personality, in theory, relies on the exposure to role models that show a successful career is possible (Henry et al., 1994; Tierney andBensimon, 1996; Wunsch, 1994). Several of IMPACT fellows expressed their desire to serve as role models to other junior faculty, a change which is attributed to attending IMPACT.
10. Sense of Community. This node captures the ways IMPACT fellows experience the feeling of belongingness to a University as a community (McMillan and Chavis, 1986; Sarason, 1974).[2] A sense of community manifests itself in the interest in being a member and sharing responsibility for community action (Sarason, 1974). Thus, by showing a strong interest in community action, IMPACT fellows perceive the benefits that might acrue from their participation in community.
11. Collegiality. Collegiality in the context of an educational institution means an ability on the part of individual faculty to effectively participate in decision making given the existing diversity of opinions (Bess, 1988). A mutual definition of a problem and a shared process of decision making are important components of collegiality and are essential for professional development. Collegiality dwells upon the ethical code of bahavior which is based on mutual respect and allows collective understanding of common problems (Cahn, 1993; Baldridge, 1971;Johnson, 1997). Therefore, this node includes sub-node “Diversity” to capture the value and significance that IMPACT fellows place on differences among and between them. Beyond its intrinsic value, collegiality has been referred to by IMPACT fellowsas a variety of strategies UTEP as an institution of higher education has to employ in order to address the consequences of historically pervasive white-man bias in academia. Not only is the persistence of the aforementioned bias evident from the fact that the majority of academiauntil the 1970s has been disproportionately white men (Cahn, 1993), but it is also manifested in the homogeneity of views and opinions clustered around the viewpoint of non-Hispanic white men. Collegiality was mentioned by IMPACT as an important skill gained through participation in the seminar by IMPACT fellows 15 times (the highest frequency among the university benefits), a result reflecting increased awareness of and newer thinking about the building of a university-wide community.
12. Networking.This node refers to junior faculty’s ability to build ties with their colleagues. Networking fosters professional development through access to information and resources (social capital) that support scholarly growth.[3]As shown in Table 1, several IMAPCT fellows reported that creation of a network of associates as one of the most visible benefits of the seminar.
13. Certification (Institutionalization). A number of IMPACT fellows supported the view that IMPACT seminar should become institutionalized. Moreover, some even proposed that it should become mandatory for all junior faculty, given the perceived benefits of the program.
Nodes 1-8 were further merged into a tree node “Personal Benefits of IMPACT Participation” and nodes 9-13were combined into a tree node “Institutional Benefits of IMPACT Participation”. The node tree is shown in Figure 1. This aggregation of individual nodes is provisional because of a certain degree of overlap between them. For example, the “Leadership” node can be interpreted as being beneficial for junior faculty as individuals as well as for UTEP as an educational institution. The institutional benefit, in the evaluator’s opinion, is more important because leadership is a collective asset through which the dissemination of best practices and transmission of knowledge and skills is sustainable (Tierney and Bensimon, 1996). The summary of nodes and correspondent quotes from IMPACT fellows’ reports is presented in Table 1.
The comparison of the annual IMPACT fellow reports across years revealed that earlier cohorts of IMAPCT fellows tended to underscore the less concrete outcomes, such as attitude change. There is a notable shift in the reports of more recent cohorts towards tangible outcomes, such as publications, presentations, etc. Also noteworthy is the tendency of more recent cohorts to adopt a more realistic attitude towards their own abilities and endeavors.
Bibliography
Baldridge, J. V. 1971. Power and Conflict in the University.New York: John Wiley.
Bem, D. J. 1967. Self-Perception: An Alternative Interpretation of Cognitive Dissonance Phenomena. Psychological Review 74: 183–200.
Bess, J. 1988. Collegiality and Bureaucracy in the ModernUniversity: the Influence of Information and Power on Decision-Making Structures.New York, NY: College Press.
Cahn, S. M. 1993. Affirmative Action and the University: A Philosophical Inquiry.Philadelphia, PA: TempleUniversity Press.
Chavis, D. M., and G. Pretty. 1999. Sense of Community: Advances in Measurement and Application. Journal of Community Psychology 27(6): 635-642.
Cohen, A. R. 1964. Attitude Change and Social Influence. New York: Basic Books.
Cross, K. P. and M. H. Steadman. 1996. Classroom Research: Implementing the Scholarship of Teaching.San Francisco, CA: Jossey-Bass.
Dewey, J. 1916 Democracy and Education. An Introduction to the Philosophy of Education.New York: Free Press.
Forster,M. N. 2002. Herder: Philosophical Writings. Cambridge: CambridgeUniversity Press.
Henry, J.S, Stockdale, M.S., Hall, M., and W. Deniston. 1994. A Formal Mentoring Program for Junior Female Faculty: Description and Evaluation. Initiatives 56(2): 37-45.
Herder, J. G. 1800. Outlines of a Philosophy of the History of Man, translated by T. Churchill, London: J. Johnson.
Johnson, A. G. 1997. The Gender Knot: Unraveling Our Patriarchal Legacy.Philadelphia, PA: TempleUniversity Press.
McMillan, D. W., and D. M. Chavis. 1986. Sense of Community: A Definition and Theory. American Journal of Community Psychology 14(1): 6-23.
Muzafer, S., and C. I. Hovland. 1961. Social Judgment.New Haven, CT: YaleUniversity Press.
Ozer, E., and A. Bandura. 1990. Mechanisms Governing Empowerment Effects: a Self-Efficacy Analysis. Journal of Personality and Social Psychology 58:3: 472-486.
Portes, A. 1998. Social Capital: its Origins and Applications in Modern Sociology. Annual Reviews of Sociology 24:1-24.
Putnam, R. D. 2000. Bowling Alone. New York, NY: Simon and Schuster.
Ross, E. A. 1901. Social Control. New York: Macmillan.
Rousseau, J. J., and J. G. Herder. 1986. On the Origin of Language, translated by J. H. Moran and A. Gode. Chicago: The University of Chicago Press.
Sarason, S. B. 1974. The Psychological Sense of Ccommunity: Prospects for a Ccommunity Psychology. San Francisco: Jossey-Bass.
Tierney, W. G., and E. M. Bensimon. 1996. Promotion and Tenure: Community and Socialization in Academe. Albany, NY: StateUniversity of New York Press.
Wunsch, M. A. 1994. Giving Structure to Experience: Mentoring Strategies for Women Faculty. Initiatives, 56 (1): 1-10. Washington, D.C.: National Association for Women in Education.
TABLE 1. Quotes of IMPACT Participants and Nodes Created in QSR Nudist.
Node / Selected QuotesPersonal Benefits of IMPACT Participation
1 / Time/Stress Management
Total quotes: 19 / “IMPACT … helped to reduce my stress level greatly, solved some real problems and tooled me to avoid these problems in the future.” “Closing my door for a couple of hours a day has helped tremendously in that it provides me with uninterrupted time. I will most definitely maintain this initiative.” “Following the seminar last May, I purchased a time-management system which I have been using continuously. This, perhaps, has been one of the most visible and welcome changes in my life.”
[need a line between these two like you did for the other categories]
2 / Perception of Self (Self-Awareness)
Total quotes: 11 / “Self Development – am I utilizing my network of people and am I
listening? Gotta self reflect on this on a regular basis…” “Reflection came about more subtly; assessment became a normal part of my routine, even though I didn’t really realize that I was assessing my progress.”
“The importance of self-reflection to truly understand my own mission to students, colleagues, UTEP, and El Paso/Cd. Júarez community. This understanding helps sharpen diplomacy when it comes to saying “no” to committee work that has little or nothing to do with my mission. Most of all, since service work must be done, knowing my mission allows me to be proactive as to which committees and other serves activities to participate in order to increase my circle of influence for changes that I would like to see take place.”
3 / Learning (New Skills)
Total quotes: 16 / “… able to benefit from the new ideas I have learned…” “Learned more about the tenure and promotion process at UTEP and in other institutions.” “…I left with new ideas, suggestions and reassurance from colleagues who had dealt successfully with similar challenges.”
4 / Attitude Change
Total quotes: 7 / “Attitude is perhaps the major change I have experienced this year as an IMPACT fellow…Thus this last year, more then others, I have challenged myself to keep focus on the issues where my chances of creating change are greater—yet without compromising my ethics” “Another challenge with change is busyness, loss of focus, overworked. This is combated by encouraging people to plan well, manage time.”
5 / Empowerment
Total quotes: 6 / “One of the main changes in perspective that I am trying to maintain is to see myself as being in control of my situation instead of being buffeted about by forces out of my control.” “I now see myself as a “facilitator”, whereas before I saw myself as expendable, part of the herd, a follower.”
“I now see myself as a manager and not as a teacher or researcher.”
6 / Integration of Personality
Total quotes: 7 / “Stronger and integrated faculty (integrated within themselves
and between colleges) leads to a stronger institution.” “…I could see myself more as a “whole person” and member of a whole campus community, not just my own department or college.”
7 / Increased Productivity
Total quotes: 4 / “I have seen a large improvement in my productivity over the past year. I expect that productivity increases will continue into the near future. That increased productivity will help to make me a more valuable asset to UTEP through improved scholarly work, teaching, and service.”
“More writing was accomplished as a function of time. For example, three major articles were published in this period.”
“The program has allowed me to focus more clearly on obtaining funding
by targeting relevant funding opportunities…”
8 / Integration of Teaching, Research and Service
Total quotes: 8 / “The integration (R/T/S) and the creation of synergy were very important lessons. This does not have to be an artificial add-on, nor does it have to be done as an afterthought.” “… the integration of research and teaching became a natural progression.” “…improved my skills in integrating teaching, research, and service (kill
as many birds as possible in one stone)” “Since IMPACT is designed for faculty to develop ways of integrating teaching, research, and service once we begin a proactive approach
towards such integration, I feel the major benefit to the University is
that it will have more efficient faculty members who won’t burn out within a few years.”
Institutional Benefits of IMPACT Participation
9 / Collegiality
Total quotes: 17 / “The greatest impact that I see the IMPACT Fellowships having is in increasing collegiality among the faculty.” “… the fact that the participants get to meet colleagues from various colleges and departments helps in creating bonds of friendship and possibilities to collaborate on multi-disciplinary and inter-disciplinary endeavors.” “The different points of view about the university operation are extremely important, especially for young faculty.”
10 / Sense of Community
Total quotes: 5 / “… by hearing experiences of people outside my department…, I gained very helpful perspective about what facets of my experience at UTEP are fairly “universal”…” “…all of us, assistant, associate, and full professors are experiencing many of the same issues.” “Almost all untenured faculty shared most of my feelings and concerns. This was tremendously encouraging …”
“The IMPACT program … exposes the faculty member to the University process as a whole”
11 / Leadership
Total quotes: 11 / “Impact can help cultivate future leadership at UTEP by providing faculty members who have thought holistically about teaching, research, service and leadership and have some training in integration.” “Participation in IMPACT has stimulated an interest in leadership issues in me.” “I plan to continue my studies in leadership, and to incorporate lessons learned in my professional and private life.” “I do see myself as a role model, and as a person who will help others achieve tenure.” “…I have selected certain people, whom I view as role models and emulate some of their practices.” “There is something to learn from everyone’s leadership style and I made it a point to try to pick and choose certain characteristics from people that I met, adopt some of them, tailor some of them to meet my needs and certainly to avoid others.”
12 / Networking
Total quotes: 9 / “We have developed a new support network for women in engineering at UTEP. The network has been established to promote success of women…”“…rely on this network of people for help, advice, mentoring, and encouragement.”
13 / Certification (Institutionali-zation)
Total quotes: 4 / “I strongly believe that having a certificate (yes, a paper certificate) for each fellow who completes the IMPACT program is extremely
important, particularly if such a certificate has the signature of
someone from the upper administration – such as the provost – who cares about the success of this program. I am willing to personally pay for the forms, and for the administrative time to prepare such certificates (for our cohort).” “… I would encourage the university to make Impact mandatory, as part of new faculty orientation.” “I would recommend that UTEP make IMPACT mandatory for all new faculty, and offer it as part of new faculty orientation.”
1