”Improving cost effectiveness in the assessment of laboratories”
Copenhagen, September 10, 1997
Joint EAL/EUROLAB Workshop in cooperation with EURACHEM
No 2/97
Foreword
EAL, European co-operation for Accreditation of Laboratories, (now merged with EAC to EA) and EUROLAB, European Federation of National Associations of Measurement, Testing and Analytical Laboratories, have jointly arranged biannually a workshop since 1992. Issues of interest to both parties have been treated on these workshops. In many cases there have also been different opinions and positions and therefore the discussions have been very interesting. Often are the discussions the most interesting part of these workshop. It has therefore been the intention to document the discussions in the proceedings from the workshops. It is however difficult to in a practical way reflect the exact wordings of the discussions. In these proceedings the discussions are extracted from the notes written by the EUROLAB Secretary during the workshop and the written discussions are therefore not reproduced in the exact wordings used during the workshop and there is even the possibility that there may be misunderstandings. If so, the EUROLAB Secretary is responsible for such misunderstandings.
Introduction by the EUROLAB President, the Secretary General of EAL and the Vice-Chairman of EURACHEM
Claes Bankvall, President EUROLAB, SP
The President of EUROLAB, Mr Bankvall, opened the workshop and thanked the Force Institute represented by Mr Larsen and Mrs Jürgenssen, for the hosting and the arrangement of the workshop. He also welcomed the participants and especially the participants from EURACHEM since it was the first time the members of the EURACHEM Executive Committee had the opportunity to participate in this type of workshops. In the future it might be possible to activate also other sectorial groups.
This is the third of the joint workshops. They are considered by EUROLAB to be very important and they are valuable tools in the co-operation between the laboratories and the accreditors. Another very important tool in this co-operation is the Permanent Liaison Group between EAL and EUROLAB.
The President pointed out that the activity of the participants in workshop is the key to the success of it.
Peter van de Leemput, Secretary General of EAL, RvA
The Secretary General of EAL, Mr van de Leemput was the representative of EAL. Mr van de Leemput presented the apologies of Mr Kaarls who did not had the possibility to participate.
This is the last workshop to be arranged in co-operation between EAL EUROLAB and EURACHEM. This is not due to any dissatisfaction but is due to the merger between EAL and EAC to EA.
The workshop is a good opportunity for fruitful discussions and the workshop has a historical background since it has been arranged several other times and the understanding between the accreditors and laboratories has improved greatly during the years.
The costs for accreditation have been high during the last ten years, mostly due to the development of quality systems and of the conformity assessment system. The costs have now been stabilised and the new version of the ISO Guide 25 may introduce possibilities to lower the costs for accreditation, but it is important to remember that the main purpose of accreditation is to satisfactorily assure the confidence in the competence of laboratories to their customers and this leads to high demands on the assessors which in turn leads to rather high costs.
Veikko Komppa, Vice-Chairman of EURACHEM, VTT
The Vice-Chairman of EURACHEM Mr Komppa welcomed the participants and gave the regards from Marie Walsh who was unable to attend the workshop. This is the first time EURACHEM is participating in this sort of workshops. Earlier, EURACHEM has been represented by single laboratories who have been members of EUROLAB.
The organisation of EURACHEM is very similar to the organisation of EUROLAB, but it has a different role than EUROLAB. EURACHEM is organising analytical laboratories and has members in 25 European countries. The work in the joint EEE-groups is by EURACHEM considered as valuable tools for quality assurance in Europe.
Cost effectiveness optimisation of the
accreditation process
Internal Audit and review: a way to
minimize surveillance
Harry Gundlach
Raad voor Accreditatie
The Netherlands
Discussion on ”Internal Audit and review: a way to minimize surveillance”
Mr Bryden asked about the qualification criteria and especially how to demonstrate costumer satisfaction. The laboratories are already performing investigations of the satisfaction of their customers, e.g. by sending out questionnaire to their customers. Are these sorts of investigations enough to show customer satisfaction or are there additional requirements?
Mr Gundlach could not exactly answer the question at the moment since it is still under discussion. A customer satisfaction questionnaire will most surely be a part of the way to show the customer satisfaction.
A question concerning the meaning of independent internal audits was raised by the audience. Mr Gundlach answered that the audits shall be performed in accordance with ISO 10011, have a qualified lead auditor, cover at least all requirements of EN 45001. The audit team shall be independent from the activity to be audited and finally audit reports shall form a basis for the management report.
Mr Gundlach was asked how this leads to a reduction in costs. He answered that since the accreditors are using the internal audits which are already performed by the laboratories instead of performing their own audits with assessors the costs will be lowered. There will of course be traditional audits but they will be fewer.
Mr Steck asked if Mr Gundlach could assure that the national accreditation bodies will accept all internal audits? Mr Gundlach could not assure that all national accreditation bodies will accept all internal audits but he assured that the main goal for the accreditors is to have a good accreditation and not to earn money.
Mr Andersson raised several questions and comments in connection with Mr Gundlach presentation. He was wondering what the requirement that internal audits should form the basis for accreditation reviews may lead to if they are formalised to the equivalent in extent to third party assessment. He also stressed that additional requirements to EN45000 should not be introduced and that the approach of internal audits and review as a way to minimise surveillance might be just a way to move costs within the laboratories. The most important matter is to have a level playing field in Europe.
Mr Gundlach stated that it is not the accreditors’ intention to introduce new requirements to EN45001 but clarify the already existing requirements
Mr Tsimillis pointed out that the picture shown by Mr Gundlach was not correct since there is requirements for internal audits even if the accreditation is performed in the more traditional manner. Mr Gundlach agreed.
Mr Larsen stressed that the laboratories already perform internal audits and that EAL wants to increase the frequency of the internal audits. He asked Mr Gundlach if he was prepared to work against such an increase. Mr Gundlach promised that RvA should take up the matter for discussion in EAL. But he also stressed that a high frequency of internal audits leads to a lower frequency of the surveillance audits which in turn leads to lower costs for the laboratories.
Mr Eberhardt pointed out that demands for an increase of the frequency of internal audits in turn leading to fewer surveillance audits just is a shift of costs and in fact may lead to an increase of the total costs for accreditation. He also mentioned that the majority of customer complaints concerns cost for the test.
Mr Gundlach stated that customer complaints concerning the cost for the test are not the issue for the accreditors and that the added value is gained since the laboratories already are performing internal audits. With this approach these audits will be used in the accreditation process.
Mr Eggimann agreed with Mr Andersson. He especially stressed that the customer satisfaction is measured by the laboratories acceptance on the market. He also repeated that no additional standards or requirements should be added. He also saw a great danger in the audit reports forming the basis for the management review, with such a system the laboratories will form an internal police, which Mr Eggimann is strongly against. All levels in a laboratory should be responsible for the corrective actions etc. mentioned in the internal audits reports.
Mr Gundlach answered that the management should not deal with all the internal audit reports but only with summaries of them and therefore there should be no internal police. Good auditors are meeting the requirements in ISO10011. Mr Gundlach stressed that good industry are usually not satisfied with only complying with the lowest requirements but is aiming at a higher level of quality and that the words ”at least” not is equivalent with the word ”more”.
The cost effectiveness of accreditation -
A Laboratory perspective
Richard Worswick
Laboratory of the Government Chemist, LGC
United Kingdom
Discussion on ”The cost effectiveness of accreditation - A Laboratory perspective”
Mr Steck described the situation at his own laboratory. They have four different quality management system in use and one common assessment for several different systems would be very valuable for his laboratory. He was however not in favour of too much harmonised standards since different standards are used for different activities in the laboratory. Mr Steck also mentioned that in his view the new revised ISO Guide 25 will be negative for smaller laboratories while it will be beneficial for larger laboratories.
Mr Ots was surprised of the lack of correlation between good results in interlaboratory tests and accreditation of the laboratories. Earlier investigations have shown different results and in fact some of the proficiency testing schemes were started to find out if accreditation lead to better performance.
Mr Worswick was also surprised by the results. He is in favour of accreditation but would have preferred a better outcome for the accredited laboratories. It seems however as the accredited laboratories are better in making corrective actions and also to improve from the proficiency tests. Maybe the situation is becoming similar to the one in the ISO 9000 community where the performance of an organisation is not at all affected by an ISO 9000 certification.
Mr Solumsmoen supported Mr Worswick’s view that customer complaints very seldom concern the paperwork in the quality system.
Mr van de Leemput pointed out that it seems like the personnel in the laboratories are becoming the most important factor for the quality in laboratories. He was wondering if accreditation of people will be an important issue for the laboratory community in the near future.
Mr Worswick agreed on the great importance of the laboratory personnel and mentioned that he had noticed several problems in the recent transformation of his organisation LGC. Changes of an organisation is always problematic. The culture of the organisation is extremely important and such things as rewards are becoming important tools to build up such a culture. Mr Worswick is however hesitating to accredit people.
Assessment of laboratories competence
The effective and efficient assessment of the competence of testing laboratories applying for accreditation
Hans-Peter Ischi
Swiss Accreditation Service, SAS
Switzerland
Discussion on ”The effective and efficient assessment of the competence of testing laboratories applying for accreditation”
Mr Bryden asked if the concept of the 3 categories of laboratories is formally expressed in the accreditation certificate and if it is in agreement with ISO Guide 25. Mr Ischi answered that the Swiss accreditation board has applied the concept of the three categories since the start of laboratory accreditation and the category of the laboratory is mentioned in the extended annexes to the scope of accreditation. Both the ISO Guide 25 and the document EAL P10 is of great help when using the three category approach.
Mr Albutt liked the idea of having different levels. It seems however as if there are different procedures around Europe and there is not a level playing field in Europe. According to Mr Ischi there is not a big difference between the way to apply the scope of accreditation in different countries. All accreditation bodies have the same goal even if there are different procedures. All members in EAL have agreed upon EAL P10 and therefore there should not be big differences. Mr Albutt responded that in his opinion EAL P10 was accepted around Europe but not implemented.
Mr Forstén discussed how far it is possible to go with accreditation and if it is possible to accredit personal competence and how far. It is not possible to go much further in the accreditation of quality systems documentation. In the revised ISO Guide 25 there are more issues concerning the customer and how to deal with customers. This is the borderline of the laboratories and accreditation must understand that in many cases they are not asking for the same things as the customers, e.g. laboratories are very seldom discussing the measurement uncertainty with their customers, but it is a never ending story in the discussions with the accreditors. To the accreditors there should be more important things than the paperwork to focus on.
Mr Ischi argued that a good assessor can feel if there is a good culture and a good management in the laboratory just after a short visit. When a type C laboratory is accredited the focus is put on the validation process and the fitness for purpose approach is used and the most important question is ”what are the needs of the customers?”.
Mr Solumsmoen stressed that the contract review is important both to type B &C laboratories. The part of EN 45000 dealing with these matters are not used today.
Mr Ischi agreed that Mr Solumsmoen’s comment is correct. In EN 45001 it is written that the laboratories shall understand the demands of the clients.
Mr Worswick expressed doubts in the accreditors competence to assess personal competence among laboratory personnel.
Relevant elements of laboratory competence
Hans Andersson
SP - The Swedish National Testing
and Research Institute
Sweden
Discussion on ”Relevant elements of laboratory competence”
Mr Andersson stated that complaints from the customers of the laboratories very often are related to ”bad” contract reviews.
Mr Steffen would like to add costumer co-operation to the agenda of meetings between the accreditors and their stakeholders and also to the agenda for the future Advisory Committee. Mr van de Leemput also agreed especially in two points, the competence and the feedback. But one should remember that there are different definitions of the word competence. A commonly agreed definition should be used. He also noticed that legislation is a one-way system with very little feedback. The laboratories has an automatic feedback due to their contacts with their customers.
Intercomparisons/proficiency testing
Comparison to independent reference values as a tool for creating confidence
Paul de Bièvre
European Commission
Institute for Reference
Materials and Measurement
Belgium
Discussion on ”Comparison to independent reference values as a tool for creating confidence”
Mr Golze pointed out that uncertainty should be used in connection to the procedures used and what is technical possible to achieve. Mr van de Leemput agreed and pointed out that feedback from the customers about what is needed concerning uncertainty is very valuable. He also believe that what is needed is too much affected by what is possible.
EAL Interlaboratory comparisons: a prerequisite for establishing mutual recognition agreements between accreditation bodies
Klaus Brinkmann
Physikalisch-Technische Bundesanstalt
Germany
Discussion on ”EAL Interlaboratory Comparisons: A prerequisite for establishing and maintaining mutual recognition agreements between accreditation bodies”
Mr Steck was a little bit shocked by the fact that there are demands on the laboratories for mandatory participation in proficiency test schemes. Mr Steck supports the idea of proficiency tests, but he questions if all laboratories have to participate in all proficiency test schemes.
Mr Brinkmann stressed that for the MRAs it is very important to assure confidence across the borders and the proficiency test schemes are important tools to assure this confidence. It is not necessary for all laboratories in one country to participate in all international proficiency test schemes, just a few laboratories in each country have to participate in each PT scheme.
Mr Forstén pointed out that the ISO Guide on proficiency tests was influenced early by calibration and there are many areas in testing where it is impossible to apply the thinking from calibration. Mr Brinkmann agreed.
Mr Steffen pointed to the work in the joint EAL-EURACHEM-EUROLAB group on the use of proficiency test in the accreditation procedures which is important. It is also important to avoid starting new schemes but to use old ones already in function. Mr Brinkmann agreed.
Mr Larsen mentioned that to be able to make corrective actions in time the laboratories need to read the report from the proficiency tests soon after the test has been performed. Therefore he asked how long time it takes from the test results are reported to the report is distributed in the EAL proficiency test schemes. Mr Brinkmann answered that it takes four weeks.
Improving cost effectiveness in the assessment
of laboratories
Alan Bryden, LNE
Discussion on ”Improving cost effectiveness in the assessment of laboratories ”
Mr Tsimillis was surprised by the lack of consistency in the results of the interlaboratory test reported by Mr Bryden especially since the 44 participating laboratories all were notified bodies. Mr Bryden agreed but could not explain the lack of consistency.
Mr Steffen asked for a list of the participating laboratories to be sent to the joint EAL-EURACHEM-EUROLAB group on the use of proficiency test in the accreditation procedures. Mr Bryden will do so.
Mr Worswick mentioned that a possible solution is to exclude the laboratories which are performing badly. But he is hesitating to use PT schemes to exclude the badly performing laboratories, instead PT schemes should be used to improve the performance of the laboratories.