Summary Report of Thenational Conversation on the Discussion Paper on the Assessment Of

1

Summary Report of theNational Conversation on the Discussion Paper on the Assessment of Non-Accredited Degree Applicants

The Qualifications Board (QB) supports regulatory excellence as well as the sustainability and confidence in the profession of engineering by developing and maintaining national guidelines and model guides on the practice of engineering in Canada. While it does not have legal authority to regulate the profession, QB seeks to establish national harmonized practices that foster international and inter-provincial labour mobility.As part of its responsibility, QB develops and maintains the national examinations syllabi, one of the tools used by regulators to assess non-Canadian Engineering Accreditation Board(CEAB) accredited degree applicants.

Over the last few years,QB has identified considerable disparity in the use of the syllabi across jurisdictions and is thus in the process of developing a national model guide on the assessment of non-CEAB accredited degree applicants. While regulators currently follow a similar assessment process but use different tools, several are in the process of changing important elements of their processesin response to legal and political drivers. To avoid further disparity, QB has decided to undertake a national stakeholder engagement process to learn about regulator processes and collect views on the current use of examinations syllabi. The intent was to identify criteria and elements of a Pan-Canadian process that would provide enough flexibility for regulators to work within their acts, while building trust and fairness into academic credential assessments.

It is important to note that QB does not have a particular end-result in mind and sees its role as a convener in addressing education assessment disparity. QB is committed to supporting regulators moving forward, by continuing to develop and maintain associated guidelines and syllabi if required.

METHODOLOGY

In September 2016, QB held a workshop on the use of syllabi in assessing non-CEAB accredited degree applicants to identify issues related to their current use and identify criteria for a new potential assessment process. The identified criteria were compiled into a discussion paper that highlighted the main drivers for change and risks associated with the current process. The discussion paper was approved by QB for distribution and consultation at its January 2017 meeting.

In February and March 2017, QB embarked on a national process that engagedregulators (including several of their Board of Examiners), the Engineers Canada Board, the Canadian Engineering Accreditation Board (CEAB) and the Chief Executive Officers Group. The goal of this engagement exercise was to increase QB’s understanding of educational credential assessment practices usedand identify opportunities for greater harmonization across the countrywhile providing regulators with enough flexibility to work within their own engineering acts.

This summary report highlights general and specific feedback received from engaged stakeholders. It is meant as a communications tool to inform further discussions around a Pan-Canadian process, and as a result, can be distributed to interested individuals. Itis important to note that thissummaryrepresents the general views expressed by stakeholders and not necessarily QB or Engineers Canada’s positions on the issues.

STAKEHOLDER FEEDBACK

The following section presents the general feedbackreceived during the engagement process:

Awareness of disparity:Regulators are aware that disparity exists in the assessment of education credentials across the country – both between CEAB and non-CEAB degree applicants, and between jurisdictions for the same non-CEAB applicant. Some express great concern, in light of mobility, that a lower standard used in one jurisdiction becomes the default threshold for others.A national conversation around this topic was welcomed and mentioned as needed.

The background information on the accreditation process was a key element of the engagement process: Individuals generally appreciated receiving the historical information that led to the establishment of the CEAB and the national examinations syllabi.Admissions officials involved in the engagement process were particularly concerned over their lack of direct involvement in national discussions on accreditation. The perceived influence of higher education institutions on the accreditation processwas a source of angst in all jurisdictions. Regulators were generally pleased with QB’s position and approach to discussing education standards for licensure.

The CEAB “minimum path”and the Engineers Canada syllabi are different standards:There is a greater understanding among members of committees dedicated to assessing academic credentials across all jurisdictions that there is a difference in assessment practices between CEAB and non-CEAB accredited degree applicants. The debate that exists for some jurisdictions and individuals is whether an applicant fits into the CEAB or the QB syllabidefinition of an engineering education. For some these thresholds should be the same, while for others the CEAB threshold is the “gold standard” – far above that which would be required for licensure (hence, confirmatory examination exemptions). Individuals were surprised to learn that a Canadian accredited program does not have to meet discipline-specific course requirements (e.g. there is no explicit or implicit CEAB requirement that a Canadian mechanical engineering degree have a course in heat transfer).While there was some debate on which standard is higher, it is understood that these differences could pose legal defensibility challenges.

The use and interpretation of syllabi vary across and within jurisdictions: The engagement process confirmed that syllabi are not uniformly interpreted by regulators and that volunteers exert a certain level of judgement in their use. Some reviewers have the philosophy that all examination topics listed in the Engineers Canada examinations syllabi must be present in every applicant’s background; others, due to the binning processes used in their jurisdiction, assign a fixed number of confirmatory examinations chosen by the applicant from core or elective topics.

Formal education requirement: All regulators currently have formal education requirements included in their legislation. Some jurisdictions require four-year bachelor degrees, while others require a minimum two-year postsecondary education in an area cognate to engineering. It was surprising for some to hear that these thresholds were not the same across the country.

Syllabi do not easily accommodate multi-disciplinary degrees: All jurisdictions acknowledge that they often pull from multiple syllabi in an attempt to match with the education held by the applicant.This “syllabi matching” is anad hoc practice that seems to be done for the benefit of the applicant. In effect, this is a substitution of discipline depth for breadth. No jurisdiction, however, currently admits and registers professionals by discipline;the discipline-specific assessment is something that is done only during the education credential assessment stage of admissions. There is recognition of the fact that accredited degrees are increasingly becoming specialized and that there is a limit to Engineers Canada’s capacity to develop and maintain syllabi for each emerging discipline.

Breadth and depth assessment must be kept part of any assessment process: There was strong national consensus on the need to assess breadth and depth of every applicant (CEAB degree and non-CEAB degree), although there is currently not an agreement on tools to achieve that objective.It is understood that the CEAB accreditation process provides sufficient assessment of breadth and depth.

Confirmation of the academic credential is the goal: Regardless of the methodology, there is a national consensus that ultimately, the intent is to assess if the applicant is an engineer.Regulators feel that examinations would likely always be a part of any assessment process that requires confirmation of an individual’s knowledge.

Use of technical examinations to address deficiencies: There is considerable difference of opinion between jurisdictions on the assignment of examinations in lieu of formal education in specific missing discipline subject areas. Some assessors argue that the Engineers Canada syllabi form a definitive knowledgebase and that the passing of examinations in required subject areas normalizes all applicants in a legally defensible standard;othersbelieve that formal education is a much richer experience – one that cannot be matched by self-study from a textbook in order to pass an examination. Some jurisdictions assign courses from accredited schools as a means of providing a qualifying path for a deficient applicant.

Academic credential confirmation is only one step of the professional licensure process: Itwas stressed that the ultimate goal is to license competent individuals. Experience in lieu of formal education is used by almost all regulators in their assessment processes. Recognizing differences in legislation across the country, some express concern over how much formal education is required and can be exempted (two v. four year degrees). It is felt by some jurisdictions that competency assessment can normalize both academic and experience qualifications.

The advantages of national mobility are widely acknowledged by regulators. There is a general openness to address credential assessment disparity in a pan-Canadian solution,but regulators are reticent to adopt changes without understanding workload impact and without widespread adoption across the country.

SPECIFIC FEEDBACK ON PROPOSED CRITERIA

In addition to general feedback on the current assessment process, QB was also engaging individuals on the following proposed criteriafor a possible Pan-Canadian assessment processes:legallydefensible;comparable to CEAB; harmonized while allowing for flexibility; beneficial to regulators and applicants; and individualised and transparent.No additional criteria were suggested by regulators during consultations. The following section will provide an overview of the feedback received on each criterion.

Legally defensible

There was discussion on how regulators can seek to minimize legal risks associated with the assessment process by contracting out part of the responsibilities to third parties, including but not restricted to World Educational Services (WES) to assess degrees and assign formal education courses for deficiencies.

There was also discussion around the legal defensibility of the current practice of using “syllabi-matching” in the academic assessment process. The ruling in the Mihaly case has prompted several regulators to consider the defensibility of their protocols and how individualized they are. Each regulatorhasits own perspective on what elements constitute a legally defensible assessment process. It was mentioned that the Mihaly case highlighted the importance of psychometric testing.

It was generally acknowledged by all that legal challenges would result no matter what process was used, but that it should be the objective of each regulator to have a process that protects the public and respects human rights and fairness. In light of mobility, it is understood by all that the practice of one jurisdiction is affecting others and that not fulfilling Agreement on Internal Trade obligations could result in costly legal challenges.

It was explained by the QB representatives that a professional legal opinion was being sought on both the defensibility of a typical assessment process for non-CEAB degree holders vis a vis the assessment of CEAB-degree holders, and on the assignment of technical examinations in lieu of formal education to address deficiencies; and to determine the legal impact of regulators adopting different assessment processes throughout the country.

Comparable assessment process to CEAB

This criterion received the most interest and engagement from individuals that took part in the engagement process. As a result, this section is also clustered by key area:

On the term comparability:The terminology associated with comparability v. substantial equivalency was also raised by several individuals in different jurisdictions as a point of confusion. It was explained that the term “substantial equivalency” is a CEAB term that is used to describe an institution that has undergone review by CEAB and has met their standards. There are not many substantial equivalency schools approved by CEAB so to avoid confusion, “comparable” should be used.

On the CEAB degree being the education standard:There is a general misunderstanding in the country on the educational threshold for licensure. Some individuals feel the CEAB standard is (or should be) the standard across all jurisdictions and that the CEAB AU “minimum path” for the degree should be the minimum required for practice. Others feel it is because the threshold of the CEAB is so high that exemptions are awarded by regulators. It was generally agreed that in many cases a mapping of university courses to the QB Syllabus for a particular discipline would result in fewer AUs. It is felt by most that the standards (CEAB vs. non-CEAB) should be somehow linked and comparable.

On CEAB establishing the standard for education and regulators, for licensure:Recent events have led regulators to remind everyone that the legislated mandate to set (and assess for) the educational threshold for licensure rests with the regulator, and that the CEAB was created for the sole purpose of accrediting programs.

On regulators adopting an educator role:Several volunteers are both Board of Examiners members and higher education institutions (HEI) professors. It was highlighted that it is important that both roles remain distinct, and acknowledged that it is often difficult to separate one role from the other when conducting an assessment. Each regulator has a responsibility to provide a route to licensure for applicants who are found to fall short of the educational standard. There is debate on whether the regulator can provide a route that consists of passing an exam when it is known that the individual does not have formal education in the subject area. Conversely, some do not accept courses in lieu of examinations. A question was posed on whether the fact that professors develop examinations is legally defensible from a regulator’s perspective as professors prepare exams for their course material, and not what is necessary knowledge for practice.Some regulators felt it would be safer for the regulator to refer applicants with deficiencies to CEAB-accredited schools to apply the same standard for all applicants.

On the quality of programs:Since the quality of a university program is assessed as part of the current process using mutual recognition agreements, the Foreign Degrees List (FDL), and the International Institutions Degree Database (IIDD), concerns were expressed on whether the quality of a program could and should be measured as part of a new assessment process. The defensibility of the IIDD and FDL was raised as a possible concern because these databases are not maintainable. It was felt, however, that these are indeed useful tools for admissions officials as rough measures of program quality. The use of a third party assessment (e.g. World Education Service or WES) was generally seen as more defensible, but concerns were raised about cost and burden to the applicant and if it can be used as a substitute to IIDD/FDL to assess the quality of a degree.

Promote harmonization while providing flexibility

There was a national consensus on Engineers Canada providing national guidance on a process that would improve harmonization. It was also recognized that establishing a process that would be completely uniform across the country was highly improbable due to different regulations and policies and that flexibility was considered to be very important.

Beneficial to regulators and applicants

There was a national consensus on the need to ensure that a new process should benefit both regulators and applicants. Application fees and timeliness was an important area of concern for some individuals. It was also highlighted that the process should ensure that qualified persons become licensed and that unnecessary barriers are removed. It was suggested by a few regulators that a risk-based approach might be desirable to expedite the process and ensure that Board of Examiners’ work is not too burdensome.

Individualised and transparent

While there is a national consensus on an individualized approach using a combination of academic and experience/competencies, there is no common definition of what constitutes an individualized assessment. Some regulators feel that unique characteristics of their current assessment protocol address some of the concerns raised in the discussion paper. This is a contentious difference in opinions between regulators that has the potential to affect mobility (in particular, jurisdictions raised flags over alternative accreditation processes being used across the country). It was agreed by all that this national engagement process on academic credential assessment should result in improved trust between stakeholders. It was also stated that careful balance must be sought between providing more options and flexibility while being transparent and predictable.

NEXT STEPS

There was general agreement that the discussion paper identified the important criteria upon which to build a pan-Canadian assessment process. Regulators felt that, where possible, tweaks would be preferred over radical changes to current admission practices. Ultimately, each regulator values mobility and wishes to improve its legal defensibility position in such an environment, especially given the cost associated with legal challenges. Moving forward, discussions remain around specific tools that can be used to:

•Check the quality of institution/program/degree

•Assess breadth of engineering knowledge

•Confirm depth of engineering knowledge

It is recognized that different tools can be used by regulators to assess applicants.QB wishes to reiterate that there are no preferred options at this stage and that input is welcomed and sought on mechanisms that would help to harmonize thresholds throughout the country, promote fairness and streamline processes.

A workshop with regulators, Engineers Canada Board, QB and CEAB representatives will be held in Gatineau in April 2017. Following the workshop, further national engagement sessions will be held on this summary report. It is expected that supporting feasibility studies will be conducted to evaluate potential alternatives and inform the development of a draft direction for the National model guide on the assessment of non-accredited degree applicants that will be presented at the QB meeting in September 2017, followed by additional national engagement sessions in the fall 2017. It is anticipated that a National guideline on the assessment of non-accredited degree applicants will be completed in 2018.

ENQUIRIES

Please direct your enquiries or comments about this summary report to:

•Frank Collins, P. Eng., Chair of the Qualifications Board Syllabus Committee at or at 506-452-6063.

•Mélanie Ouellette, MA, MBA, Practice Lead, Qualifications, Engineers Canada at or at 613-232-2474 ex. 225.