BIHTEK: Benchmarking as a tool for improvement of Higher Education Institution Performance, 530696-TEMPUS-1-2012-1-BE-TEMPUS-SMGR
Report on the project progress for the period of 15.10.2012 – 10.04.2014 (18thproject months)
By Mila Zarkh and Nora Skaburskiene
Structure:
Detailed follow-up of workpackages as indicated in the final submission:
Workpackage P 1: The Analysis of Benchmarking Approaches in EU and Bosnia and Herzegovina
Workpackage P 2: Principles of good benchmarking in the development of B&H Higher Education
Workpackage P 3: Strengthening capacity for implementation of BM methodology in B&H Universities
Workpackage P 4: Capacity building of Universities in Bosnia and Herzegovina
Workpackage P 5: Pilot project for benchmarking in B&H Universities
Workpackage P 6: Dissemination
Workpackage P 7: Sustainability
Workpackage P 8: Quality Control and monitoring
Workpackage P 9: Management of the project
Conclusions of the CEENQA team:
Workpackage P 1: The Analysis of Benchmarking Approaches in EU and Bosnia and Herzegovina
Deliverable 1.1 Meeting in Gent:preparation of a document with analysis of present situation of benchmarking approach in EU (15/11/2012)Deliverable 1.2 Meeting in Sarajevo: development of a document (including a comparison)on Benchmarking procedures in B&H in relationship with the EU HEI's; Final document drawn up during the meeting (15/01/2013)
The meetings in Gent and Sarajevo were held in order to carry out an analysis of the present situation on the benchmarking approaches in EU and B&H. Within the framework of these meeting, EU partners, all BH universities and the representatives of quality assurance agencies as well as the ministry representatives during a first seminar in Gent and a second in Sarajevo were supposed to draft an analysis paper which has been documented in a report. From the interview with the project coordinators we have learned that this report has been made available to all project partners via e-mail, but not yet published on the project website. Since already considerable time has passed, we recommend to put the website up-to-date and to provide all necessary documentation on trainings and meetings which have been held so far in order to facilitate access to all relevant documentation as well as easy knowledge sharing. Especially considering the fact that the dissemination phase at the B&H Universities is approaching and this phase will demand an advanced ownership and extensive preparation, an up-to-date website and a common base of project relevant documents are very important. For further comments on the project website, please cf. analysis of the workpackage 6.
to be quite a helpful tool in spite of lacing documents.
The project management team also stressed the fact that the implementation of the BIHTEK project is a consequent and sustainable continuation of the work done during the two tempus Projects ESABIH and SHEQA. The quality criteria developed during ESABIH project will be used as a base for benchmarking on the programme level, whereas those from SHEQA project are to be used for the institutional benchmarking. It is also planned to use the benchmarking software purchased during the SHEQA and ESABIH projects and to further train the skills of the involved staff.
Workpackage P 2: Principles of good benchmarking in the development of B&H Higher Education
Deliverable 2.1 Seminar in Zenica: examples of good practice in EU; better understand the meaning of benchmarking in HEIs. Joint work of EU experts, BH partners from HEI, HEAARS and HEA on benchmarking handbook (15/03/2013)Deliverable 2.2 Workshop Banja Luka: document with framework of benchmarking methodology in B&H starting from the results of previous activities; draft proposal of framework for benchmarking at B&H HEI's. The benchmarking handbook team is determined, the results will be published (15/04/2013).
Deliverable 2.3 Development of the benchmarking handbook, focus on benchmarking models and methodologies with detailed procedures for application in B&H HEI's. This handbook will be guidelines for good practices for effective benchmarking and an ongoing platform to promote exchange and good practices for benchmarking in higher education (15/10/2012).
Deliverable 2.4 Conference in East Sarajevo:current situation in Higher education area concerning Benchmarking as a modern tool for improved performance at a HEI. Presentation of the Handbook and press conference (15/11/2013).
The meeting in Zenica was planned for making the B&H academic staff with examples of good benchmarking practice carried out bythe EU universities, whereas the meeting in Banja Luka was focused on determination of the most adequate and relevant benchmarking models and methodology for B&H universities. Those were put to a document which established the theoretical framework of the BM handbook which was developed by the project core group (University of Mostar and KAHOSL) and which was developed and discussed with all the partners. Their feedback on the framework has been documented, but the document itself has not yet been published. Based on these meetings and discussions, a mixed team composed from B&H and EU experts has developed the benchmarking handbook, the didactical and methodological core of the project.The results were documented in minutes which we recommend to make available to all partners in order to facilitate good dissemination work at B&H universities and to be able to easily follow up the steps of the project progress.
The project partners were asked for their feedback on the first handbook draft during the meeting in Sarajevo in December 2013 and were also able to deliver their remarks during a phase of one month. The feedbacks have been followed up and some changes made. The project management team informed us that an update of benchmarking handbook is planned. The team considers the handbook to be almost ready, it still will be enriched with conclusions from the training sessions in Tuzla and East Sarajevo,example templates and case studies as well as examples of good practice after the implementation of the pilot projects. The next steps towards improving the handbook will be made during the conference in Mostar. From informal feedback after the meetings and workshops the coordinators know that the B&H partners use the benchmarking handbook and find it all in all clear, easy understandable and useful. The analysis of the results gained from the participants surveying showed that the participants evaluate the handbook as (rather) useful; some narrative comments were made, e.g. recommending to include more examples of existing full benchmarking procedures or to improve the handbook in general. It would definitely be very helpful if the next update of the handbook could be made available before the implementation of pilot benchmarking projects.
Additionally, the project coordinators also signaled that the communication flow is all in all going well and reliable. They are frequently getting positive feedback from partner institutions and individual participants. We have noticed that the feedback on our tasks for “homework” – individual development of the results from the first training in Tuzla for better preparation of the subsequent trainings - was not always sufficient, which could be due to the fact that homework always means additional workload and the participants already said that they were overloaded with regular duties. This situation might change once the real ownership phase – the dissemination at home institutions and implementation of pilot projects will start.
Workpackage P 3: Strengthening capacity for implementation of BM methodology in B&H Universities
Deliverable 3.1 Training step by step towards BM: Theory of benchmarking, benchmarking as a learning process for universities, defining priorities, targets, criteria, indicators and benchmarks, scoring the institution, analysis and action plans, implementing and concluding the benchmarking cycle etc. Seminars will be organized in Tuzla, Bihać and East Sarajevo (15/04/2014).Deliverable 3.2 Study visit to the university of Vigo: exact application of BM methodologies; implement the best BM with special emphasis on improvement of performance of the institution, cooperation with public authority (HEA,HEAARS, ministries) (15/05/2014).
Deliverable 3.3 B&H staff disseminates good practice and knowledge to staff and students in B&H Universities through organizing dissemination seminars (15/06/2014).
For aims of capacity building and strengthening for implementation of benchmarking methodology in B&H Universities, three meetings (January 2014 in Tuzla, February 2014 in Bihać and March 2014 in East Sarajevo) were planned as subsequent training units. Due to partly violent political protests in Bosnia and Herzegovina’s bigger cities in February, the meeting in Bihać has to be canceled. Since during the meeting in East Sarajevo the knowledge level of the participants turned out to be already quite advanced, the project management decided to catch up with organizing the Bihać meeting sometime after the pilot projects have started. It is most probable that during these phase, the B&H partners will need punctual assistance and a “forum” for their questions which emerge “on-the-job”.
The planned study visit to Vigo had to be moved to June, due to the change of coordinator and emerging financial/accounting problems, which is against the previous plan to include the gained knowledge from direct learning from an EU university implementing benchmarking to the dissemination phase.
At the current stage, the B&H partners are supposed to overtake the ownership for the dissemination activities at their home institutions and finalize those by the end of May 2014. Besides some instructions, the duration and training design is completely up to them. They will document their activities with agenda plans, reports and photos. We consider the starting point for dissemination activities as rather complicated due to the fact that one of the planned trainings as well as the study trip to Vigo were skipped, so that time for training and strengthening the implementation skills was missing and the BiH partners usually do not discuss openly the proposals of each other so it is hard to estimate how well every single partner is able to conduct dissemination trainings at his/hers institution. Therefore it is of a major importance to maintain good contact to the partners and also facilitate the access to all relevant documents as soon as possible. During the interview, the idea emerged to start a blog in order to facilitate the dialogue among partners during the dissemination phase at the B/H universities. CEENQA will check whether there is a good and easy manageable option and will report on that in order to help catch up the missing time. The project core group will report on the progress of these activities based on the single reports from the BiH universities.
As further activities from this workpackage, the conference in Mostar (September 2014, fine-tuningof BM procedures and action plans), a meeting in Düsseldorf (October 2014, Final training for implementation plans of the pilot projects and project management meeting) as well as the post-poned meeting in Bihać after the start of the Pilot Projects (2015) are planned.
Workpackage P 4: Capacity building of Universities in Bosnia and Herzegovina
Deliverable 4.1 Purchase of equipment for QA offices necessary for implementing of benchmarking at partner universities (4th month of project) (15/12/2013).Deliverable 4.2Four representatives of B&H Universities will develop action plan for implementing BM at home institution. The action plan should include procedures for the implementation of BM in the university activities (15/08/2014).
Due to troubles with financing caused by the change of coordinator and to the fact that there is still (as of March 14th 2014) no a valid grant agreement, the planned equipment was not bought yet. In the second middle of March 2014, a final approval from the KAHOSL is expected; but still some additional time will be needed due to the obligatory tender procedure based on TEMPUS regulations. Also the connected administrative procedures in B&H could take some time so that at this moment, it is not clear yet when the purchase will be finalized. In order to follow up the recommendation No. 1 by the EACEA, the purchase of the equipment should be started as soon as possible. Preferably it would happen before the pilot projects start.
The teams for the implementation of action plans and pilot projects have not been yet set up. Ivana Zovko will contact the partners in order to ask them to start thinking of that in order to facilitate the planning and the implementation. According to the project plan, at least internal benchmarking is planned; whether the partners decide to do external benchmarking, is up to them. However, some institutions have already signaled their interest to cooperate: The university of East Sarajevo and the University of Zenica will do external benchmarking and also some faculties of Zenica and Mostar. As well as the faculties of Healthcare of the Universities of Mostar and Zenica will cooperate. Faculties of Law in Zenica and East Sarajevo.
Workpackage P 5: Pilot project for benchmarking in B&H Universities
Deliverable 5.1. Three areas for benchmarking for pilot projects (governance, quality assurance, curriculum reform, external impact reputation, competitiveness, etc.) to be chosen. Each B&H University will select a team for realization of pilot project (four members). The selected team will develop a detailed methodology for implementation of benchmarking (15/11/2014).Deliverable 5.2.Meeting in Paderborn: Presentation of the methodologies developed by B&H HEIs for realization and expected results. EU experts and QA Agencies will be consulted(15/01/2015).
Deliverable 5.3Implementation of the developed action plan and methodology at home institution. Reports about implementation of pilot project before Zenica;EU experts will prepare suggestions and comments on reports with the aim to improve the results. Results of this activity will be published. (15/06/2015))
Deliverable 5.3meeting in Zenica: presentation of the reports on pilot projects;EU experts will prepare suggestions and comments on reports with final aim to improve the results (15/07/2015).
The pilot projects are scheduled to be implemented from November 2014 on. They are currently being in planning; during the interview on the project progress, the project coordinators have signaled that they plan to ask the B&H staff to already think of responsible working groups for the pilot projects and set up them ideally before the conference in Mostar, at latest before the meeting in Düsseldorf. We consider it recommendable and helpful since during these two meetings, final fine-tuning of action plans is supposed to be done.
Workpackage P 6: Dissemination
Deliverable 6.1.: Development of the website (15/10/2015)Deliverable 6.2.:Articles, news, newsletter, reports (15/10/2015)
Deliverable 6.3:Dissemination conference (15/09/2015)
So far, the dissemination activities to wider public were rather modest since the work done within the first project year and the first months of the second year was focused on developing the relevant documents and capacity building, so that the focus of project activities was laid rather on internal processes and need. By the time being, only an overview about benchmarking relevant online sources was published on the public site of the project website and an interview for local media with the vice-rector of the University of Sarajevo have taken place. The link to the interview was published on the university’s website. It is planned to publish the benchmarking handbook as a booklet comparable to those of ESABIH and SHEQA projects. After the conference in Mostar, phone calls media to media for enhancing visibility (scheduling new interviews with B&H and European partners would be a good way to go, too) and also first publications of the work done so far will be published. The project management also plans to inform external stakeholders such as relevant companies and further ministries about the project. We recommend for this purpose to issue short newsletters and articles for the website of partner universities but also for independent sending via e-mail.
As for facilitating the internal dissemination, we have already mentioned in the analysis of the results of the workpackage 1 that the project website should be updated and missing documents made available as soon as possible. We have learnt that the delay concerning the website update has been caused by the change of coordinator and the respective administrative difficulties. Nevertheless, the project management team agreed to follow-up this recommendation in the next time. The project partners could enhance the visibility of the project by putting a link to the institution’s websites. By the time being, only the University of East Sarajevo and HEA have done so. For some institutions, the website administration is connected to a complicated bureaucratic process. Still, where possible, this activity should be followed up as part of dissemination activities.
We were also wondering whether there was a way to check the frequency of the website use, but apparently, there is no technical option of following up how many visitors the project website had so far. The project coordinators have however confirmed that the B&H partners do use the website whenever they need any information. A clear majority of participant has indicated in the questionnaires that they have used the project website already for information research, so that all in all the website seems to be a useful and helpful tool for the participants. It was also the recommendation No. 2 to further develop the project website made by the EACEA representative Ms Piia Heinämäki in her e-mail of November 2013 which should be followed up as soon as possible.
Workpackage P 7: Sustainability
Deliverable 7.1 Increased level of cooperation and benchmarking in the B&H. Sustainability of the project outcomes and outputs will be achieved as a result of support (financial,...) provided by university and government structures (15/11/2013).Deliverable 7.2 Action plan for implementation of benchmarking (15/10/2014).
Deliverable 7.3 Quality assurance strategy by every B&H HEI(15/09/2015).
This workpackage is to be understood as an overarching achievement of determining benchmarking models for B&H Higher education, purchasing equipment and training of staff from B&H universities. Those elements being part of the sustainability concept of the project, some measures have already been taken and are commendable, such as trainings of B&H staff which have shown all in all good results and also the development of the first working version of the benchmarking handbook which is already in use and also quite good evaluated by the participants (17 out of 21 marked 5 to 6 points). The benchmarking handbook indeed shows some clear and easily understandable objectives, methods and activities for implementation of benchmarking and further improvement of the handbook is planned. The next step towards successful implementation of sustainability goals is setting up an action plan for implementation of benchmarking at all B&H HE institutions which is planned for November 2013, but the project coordinators have already signaled that they envisage to ask for first appointments of the implementation plan working groups before the conference in Mostar, so before September 2014.
Other activities from this workpackage (establishing benchmarking at all B&H Universities during the 33th month of project duration, appointing a team for making a quality assurance strategy based on benchmarking) will start in the third year of the project implementation and will be followed up in the next report.
Workpackage P 8: Quality Control and monitoring