Getting the Seal
Description of the procedures
1.Content
1.Content
2.Introduction
3.Phases in getting the EuroRec Seal
4.Information phase
1.Access to the Seal
2.Availability of the Seal
3.Comments and interpretations
4.Requesting additional information or commenting a criterion
5.Export
5.Documentation phase
1.Selection & “Registration”
2.Documenting compliance to the criteria
3.Documentation template
4.EuroRec Partners Assistance
6.Testing phase
1.Submitting “documentation”
2.Application tests
3.Costs of testing the application
4.Conducting the tests
5.Final Test Log Report
6.Appeal
7.Retest
7.Granting phase
1.Request for granting a Seal
2.Granting the Seal
8.Limitations of the Seal
9.Renewing the Seal
10.Alternative “External” Procedure
1.Basis of a “external” EuroRec Seal
2.Request by a supplier
3.Request by an external certification authority
4.Granting an “external” EuroRec Seal
2.Introduction
This document provides information on the process of obtaining a EuroRec Seal for your application. This document defines the main stream process as well as an alternative process. Testing and quality assessment within the main stream process is done within the EuroRec Network. The EuroRec Seal is granted on top of a third party testing and quality assessment in the alternative process.
The EuroRec Seal granted is to be considered as a “Quality Label” and not as a certificate. Only officially accredited organisations can deliver “Certificates”.[1]
The main stream process is based an initial auto-evaluation and documentation approach, completed with an on-site testing and quality assessment of the application submitted to quality labelling.
The alternative process is based on a validation by EuroRec of the quality assessment procedures and the content of a testing session performed by a recognised third party. The compliance to the criteria of the Seal needs to be guaranteed by that organisation.
3.Phases in getting the EuroRec Seal
This section distinguishes the following phases of the process of obtaining the EuroRec Seal for your application:
- Information phase, acknowledging the content of the Seal, including composition of the Seal as well as the accepted “interpretation” of each of the criteria.
- Documentation phase, when the supplier documents as convincingly as possible how the application complies with each of the criteria. This documentation needs to be “accepted” by EuroRec before starting the next phase. This phase can also be considered as the “auto-evaluation” by a supplier of his or herown product.
- Testing phase, where an on-site validation is done of the application by EuroRec or one of its partners and on basis of the documentation produced.
- Granting phase when the EuroRec Seal is finally granted for a given country and intended user group.
4.Information phase
1.Access to the Seal
The Seal is available for free on the web site of EuroRec. Click on the icon “Free Access to the EuroRec Seal”.
Registration is required. You will be requested to register if you are not yet registered.
Registration is for free.
It is NOT required to apply for a Seal in order to be granted access.
This registration is not required if you already have a username and password to access the “private” zone of the EuroRec website and to access the EuroRec Repository.
In that case, simply login with your username and your password.
2.Availability of the Seal
The EuroRec Seal is accessible for free.
Click on the icon “Free Access to EuroRec Seal”. You will obtain a full window dedicated to the EuroRec Seal Level 1 and Level 2.
Select the Level you want to apply for and click on e.g. “View list of Criteria Seal Level 2”
The Seal is displayed in English. You can select from many other European languages if you wish.
The English version of the criteria is the referenced version for granting the EuroRec Seal.
Do not pay attention to column “weight”. This is not used within the EuroRec Seal Quality Labelling. All criteria are mandatory and they are all equally mandatory.
The criteria are listed by an internally maintained number. The criteria are exportable to Excel, enabling you to order and annotate them in any way that you wish.
3.Comments and interpretations
Clicking on opens a window with background information about each criterion, including how to interpret that criterion, as illustrated on next page.
The system displays some factual details related to creation and version of the statement as well as the original reference statement for that criterion.
The lower part of the window displays the comments / interpretations related to that criterion.
4.Requesting additional information or commenting a criterion
The Web interface offers the user the possibility to add a comment by clicking the checkbox at the end of the row and confirming it by clicking on the “add comment” button.
This generates an interface enabling the user to add a comment. This comment is send automatically by mail to the EuroRec repository Maintenance collaborators.
5.Export
“Export Statements & Interpretations to Excel” results in an Excel file as illustrated:
5.Documentation phase
1.Selection “Registration”
The process leading to the EuroRec Seal starts with making a choice between Level 1 (containing 20 low level, very “generic” criteria) and Level 2 (containing 50 standard criteria).
Registration requires completion of a “Request for the EuroRec Seal – Seal Level 1 or 2 – Supplier Form”, available on the EuroRec Web Site.
Click “Apply as EHR vendor for this seal”
This form should be completed and sent electronically to EuroRec . The last section of the form “About the effective Quality Assessment Testing of the Application or Component” will be completed later on in the procedure.
Submitting a request formmakes no formal commitment, and your details will remains confidential.
2.Documenting compliance to the criteria
Compliance to each of the criteria needs to be documented in anevidenced way, criterion by criterion.
This documentation encompasses:
- The criterion (with number)
- Description of the scenario used to illustrate compliance
- Evidence of compliance, usually through print screens. In principle this should be an illustration of the application screen before execution the scenario and an illustration of the application screen after executing the scenario. The number of illustrations provided is not limited. The illustrations should clearly evidencing that the criterion can be met by the system.
The documentation needs to be in English. The screens are in principle in the language of origin of the application. Data / Fields on the screen and points of attention should be annotated by supplementary labels in English as illustrated below in Section 3.
3.Documentation template
A way to document your application is to produce preliminary versions of the final “Test Log Report”, as illustrated.[2]
4.EuroRec Partners Assistance
EuroRec has across European partners willing to assist vendors in preparing the final test. This includes explaining the criteria, the procedure and – if agreed on with the supplier – a pre-validation of the documentation or even a pre-testing the application.
The nearest available partner organisation will be informed of your request once you have submitted this.
The list of the ProRec centres with contact data as well as the list of the Partner Organisations are available on the web site: click on “Who are we”.
The “services” are provided on basis of an agreement between the vendor and the service provider. That agreement can only be a service agreement without success guarantee. The fee agreed on between parties needs to be paid before an appointment can be given for the testing phase.
Remark: 1. not all the ProRec centres are providing assistance services. 2. Not all partner organisations providing assistance services are ProRec centres.
6.Testing phase
1.Submitting “documentation”
The documentation, illustrating compliance to the criteria, will first be evaluated by the partner organisation that will do the final “testing” of the application[3] and that will request – if the testing goes well – the Quality Label from EuroRec.
2.Application tests
An appointment for final testing is made after approval of the documentation as being sufficient and convincing.
The final testing is usually undertaken at the premises of the system vendor, but can be undertaken at any place agreed between parties.
Tests are chaired by the testing organisation, this being either EuroRec (the central organisation) or a EuroRec Partner Organisation (ProRec centre or any other ‘accredited’ organisation entitled by EuroRec to do final testing for the Seal).
The delegation of the testing organisation will be one or two persons with experience in validating EHR systems, to be designated by EuroRec or the EuroRec Partner Organisation.
The testing organisation cannot be involved in providing “assistance services” for the same vendor, not can it have done so in the past (e.g. for other products of the same vendor). The testing also cannot be undertaken by another person belonging or linked to that organisation.
3.Costs of testing the application
The fees for testing the application are available on the web site. Click on “Rates for granting a EuroRec Seal Level 2”. The fees listed in the document do not include travel and hotel costs for the testing staff.
The fee can be limited to granting the Seal or may inlcude the testing of the application and granting the Seal.
Testing will only be done after payment of the appropriate fee and the costs.
4.Conducting the tests
The testing organisation will use the documentation provided by the supplier as a basis for guiding the tests. The testing organisation has the right to retest each of the criteria, even if “convincingly documented” by the supplier.
5.Final Test Log Report
The testing organisation documents – if any – why an application does not meet a criterion, while producing a final test report.
The supplier will receive a copy of the test log report.
6.Appeal
The supplier may contest any of the conclusions reported in the test log. This can be done by generating complementary documentation and sending by registered surface mail to the testing organisation within two weeks of the testing date.
The testing organisation will accept or reject the arguments, modifying or not its advice to EuroRec.
The testing organisation adds the appeal documentation to the dossier submitted to EuroRec when requesting finally the Seal to be granted for that product.
7.Retest
The number of retests is unlimited. The required fee and costs are due each time.
Retests can usually be avoided by a good and intense pretesting assisted by a partner organisation.
7.Granting phase
1.Request for granting a Seal
The final test log report is send to EuroRec together with a “Request for granting a EuroRec Seal – Level 1 or 2 – Certifier Form” available on the web site under “certifier forms”.
The testing organisation should clearly confirm the compliance of the application with all the criteria or document the criteria that were not met, including anyarguments to justify why a criterion might not be applicable to a given application, considering legal, regulatory or context / domain of use.
The testing organisation should include, if any, “appeal documentation”, even if it is accepted by that organisation.
2.Granting the Seal
The EuroRec Seal is always granted the by the central EuroRec organisation. EuroRec has the freedom not to grant the Seal. Such a decision and its justification will be communicated to the supplier and treated as confidential information.
The supplier receives a confirmation.
The awarding of a Seal can be announced in a press release by supplier as well as by EuroRec or in a joint communiqué.
The awarding of a Seal is also announced on the web site of EuroRec ,
8.Limitations of the Seal
The EuroRec Seal is granted for an identified version of an EHR system or module and for use in a given a specified territory / country by a specified class of users (general practitioners, nurses, private physiotherapists, hospital environment, specified specialty…).
The version as well as the territory and the domain of application need to be specified in the Supplier’s Request.
Extensions can be requested during the process of getting a EuroRec Seal. Such request should be done before agreeing on a date for the final testing.
There is no limitation in using the EuroRec Seal as long as linked to the same version of the application and addressing the listed territory and/or a domain. Major regulatory changes in a territory will require retesting of the application, at the discretion of EuroRec. The supplier commits himself to inform EuroRec when such changes occur.
9.Renewing the Seal
Renewing the EuroRec Seal equals a new EuroRec Seal for e.g. a new version. It is permitted to document that a given functionality / attribute of the application has not changed for a number of criteria.
The testing organisation may choose to retest a selection of “unchanged” criteria. The testing organisation is thereby not bound by the “result” of a previous test.
10.Alternative “External” Procedure
A EuroRec Seal Level 1 or Level 2 may be granted to one or several EHR systems or modules, based on a quality labelling or certification procedure performed by another organisation, e.g. a nationally organised certification.
1.Basis of a “external” EuroRec Seal
EHR systems or modules may be granted a EuroRec Seal after undergoing third party quality labelling or certification procedure.
This is only possible on the basis of a verifiable confirmation by the “certification authority” that all the functions and requirements listed in the appropriate Seal were included in the testing process successfully undergone by the supplier’s product.
2.Request by a supplier
A supplier may apply for a EuroRec Seal on the basis of an external quality labelling and certification. The supplier should address this request to EuroRec and to the relevant Certification Authority in parallel.
EuroRec will then contact that authority and request information that proves compliance to the EuroRec criteria.
The Seal will be granted as soon as this compliance is proven. There is no need for retesting in this case.
The external certification authority will be specified on the EuroRec Seal as the testing organisation.
3.Request by an external certification authority
An external certification authority may be interested in getting a European recognition of the EHR systems or modules certified by them.
EuroRec will then investigate if all the (applicable) criteria will be tested / validated during the upcoming certification process, possibly suggesting some changes in order to obtain full compliance to the EuroRec Seal criteria.
The certification authority is, from then on, authorised to publish the possibility of being granted a EuroRec Seal through that certification process.
4.Granting an “external” EuroRec Seal
The supplier needs to confirm an interest in the EuroRec Seal. The EuroRec Seal will be granted after completing a request and paying the requested fee. See “Rates for granting a EuroRec Seal Level 1” or “Rates for granting a EuroRec Seal Level 2” on the EuroRec web site.
EuroRec SealDescription of the Procedures – 2011/12Page 1 of 1
[1]See the EHR-QTN deliverable “Roadmap towards sustainable quality labelling and certification across Europe”.
[2]Illustration offered by Forth, Greece for the application ICS-M.
[3]The organisation doing the final testing of the application may be EuroRec, the central organisation.