Quality Assurance and Accreditation of MOOCs: Current Issues and Future Trends
Serpil Kocdar and CengizHakan Aydin, Anadolu University
,
Abstract
Quality assurance and accreditation is a key issue that needs to be addressed in MOOCs.However, research focusing particularly on quality assurance and accreditation has remained limited so far, leading to a lack of comprehensive literature reviews on the issue. In this regard, the purpose of this presentation and associated paper is to examine divergent opinions in MOOCs debate regarding quality assurance and accreditation, analyze current initiatives, discuss future trends and make recommendations for future practices. The paper is expected to contribute to the efforts of establishing policies for quality assurance and accreditation for MOOCs, and guide policy makers, administrators and practitioners in universities, quality assurance and accreditation agencies and governments as well as MOOC providers.
Keywords
MOOCs, quality assurance, accreditation, open and distance learning
Introduction
Originating from the Open Educational Resources movement, Massive Open Online Courses (MOOCs) have attracted significant attention from commercial organizations, universities and governments in recent years (Grainger, 2013; Nath & Agarwal, 2014; Yuan & Powell, 2013). The first MOOC Connectivism and Connective Knowledgewas facilitated by George Siemens and Stephen Downesin 2008 at the University of Manitoba. After that, a group of Stanford computer science professors opened up their courses to anyone in the world with access to a computer and an Internet connection in the fall of 2011 (Downes, 2008; Grainger, 2013; Kelly, 2014). Following these initial samples, MOOCs have quickly gained popularity and expanded as they promise to offer the opportunity for massification of courses for free or at a very low cost.
Even though their popularity among learners, there are conflicting opinions regarding the value and importance of MOOCs.Some researchers argue that they open up access to education decreasing the cost of university-level education. However, others claim that they are just a way of making money and more about learning income than learning outcomes (Conole, 2013; Kelly, 2014; Yuan & Powell, 2013). Haggard et al. (2013, p.4) mention on the criticisms of some of the researchers who argue that the MOOC format suffers from “weaknesses around access, content, quality of learning, accreditation, pedagogy, poor engagement of weaker learners, exclusion of learners without specific networking skills.” Daniel (2013) emphasizes the importance of quality of the learning experience in MOOCs and points to the high drop-out rates. Although some of the research have shown that learners rate the MOOC experience positively, many learners find MOOCs a confusing experience (Haggard et al., 2013; Knox, 2014), which might be one of the reasons of high drop-out rates. In parallel to this, Poyiadgi (2014) states that if the quality, delivery and learners’ perceptions on the value of the MOOC are not at an expected level, course completion rates can be low.These criticisms indicate that assuring quality is one of the crucial aspects of MOOCs that should be addressed.
While mechanisms to ensure quality or accreditation are well established in formal education institutions, such mechanisms are not in place for MOOCs formally (Conole, 2013). Despite having some sort of internal quality review practices, MOOC providers are not reviewed for quality by external parties (Eaton, 2014). Questions have begun to be raised about how to judge their quality externally and whether there is a role for accreditation (Eaton, 2012; YuanPowell, 2013; Haggard et al., 2013), butcurrently there is no consensus on how these practices should be carried out. Eaton and Uvalic-Trumbic (2014) state that if new providers are to play a significant role in higher education, we need criteria for quality and agreement on who should make judgments about quality in this emerging sector. Therefore, quality is a key issue that needs to be discussed and compromised if MOOCs are going to be sustainable in the longer term(Conole, 2013).
In this context, the purpose of this paper is to examine issues and divergent opinions in MOOCs debate regarding quality assurance and accreditation, analyze current initiatives that aim to assure quality of MOOCs or accredit them, discuss future trends and make recommendations for future practices. The paper is expected to contribute to the efforts of establishing policies for quality assurance and accreditation of MOOCs. Research focusing particularly on quality assurance and accreditation has remained limited so far, leading to a lack of comprehensive literature reviews on the issue. This paper intends to fulfill this gap in the literature and guide policy makers, administrators and practitioners in universities, quality assurance and accreditation agencies and governments as well as MOOC providers.
Issues and Divergent Opinions
One of the issues regarding quality assurance and accreditation of MOOCs in the literature is that there is no consensus on what constitutes quality in MOOCs. According to the results of the MOOC Quality Project conducted by the European Foundation for Quality in E-Learning, it is difficult to define quality since the nature of MOOCs are constantly changing and new forms are appearing (Ehlers, Ossiannilsson, & Creelman, 2013). Eaton and Uvalic-Trumbic(2014) argue that the flexible and changeable learning process possessing a greater openness creates a challenge for recognition. In addition to these, Read and Rodrigo (2014) point out that as it is not easy to differentiate between a MOOC and other types of online courses, it is difficult to specify a quality model for MOOCs.
Another issue is the lack of expertise of some traditional universities in online learning. According to some researchers, quality of teaching or pedagogy is at risk since most of the current MOOCs are from research-intensive universities (Grainger, 2013). Similar to this, Daniel (2013) criticizes MOOCs in terms of their quality and asserts that MOOCs are evolving in ways that may actually obstruct the development of open education.
MOOCs are frequently criticized in terms of possessing a lack of reliable assessment methods which threatens the quality of learning.Eaton and Uvalic-Trumbic (2014) state that MOOCs are not really higher education because the vital component of rigorous student assessment is missing.Due to the concerns about the weak assessment methods and quality of MOOCs, most institutions have decided not to award credits for MOOCs (Yuan & Powell, 2013).Gaebel (2014) implies that it is difficult to understand the role of MOOCs for disrupting higher education unless they award credits. Kelly (2013, p.2) states that at San Jose State University, which has partnerships with Udacity and edX, the Philosophy Department refused to offer edX’s version of a popular Harvard course called Justice as they regard such courses as a serious compromise of quality of education.There is considerable concern that MOOCs would not promote real learning innovation, and would be used just to save costs rather than improve quality (Gaebel, 2014). According to Haggard (2013), accreditation has two aspects in terms of MOOCs: Securing the revenues from course fees andthe issue of how the learning is assessed, authenticated and valued by employers. He points out that the latter is a less discussed issue for now.
Although concerns about the quality of MOOCs have prevented many participating institutions to offer credits for MOOCs as part of their traditional awards (YuanPowell, 2013), some universities have started to offer credits to their MOOCs in order to reduce students’ enrolled time in campus, and therefore intend to reduce cost of degrees (Haggard et al., 2013).Fisher and Fox (2014) demonstrate that it is possible to receive an undergraduate computer science degree via MOOCs and they state that such way of earning a degree provides learners alternative learning pathways through multiple institutions which enables to personalize the curriculum.For instance, the Georgia Institute of Technology, Udacity and AT&T have collaborated to offer the first accredited Master of Science in Computer Science which is delivered through a MOOC platform (see In addition, Udacity offers a limited number of basic college credit-bearing courses for a low fee in collaboration with San Jose State University (Morris, 2013). Two British MOOCs have been offered for credit:First Steps in Teaching and Learning(Oxford Brookes University) andVampire Fictions(Edge Hill University) (see /2014/06/06/moocs-and-academic-accreditation-in-the-uk/). Some institutions such as University of Maryland University College, Georgia State University, and Penn State World Campus are exploring ways to award credits to MOOC completers who are able to demonstrate competencies equivalent to on-campus offerings (Hollands &Tirthali, 2014).
Supporting these practices, in the Statement on Massive Open Online Courses by the Quality Assurance Agency (QAA) of the United Kingdom (UK), it is declared that “MOOCs offer a form of learning that may be capable of formal recognition towards the entry requirements for a higher education qualification, or for credit through providers’ existing recognition and accreditation systems” (QAA, 2014). Poyiadgi (2014) and Read and Rodrigo (2014) claim that for MOOCs to really succeed, it is necessary for the higher education institutions to apply same measurements of impact and quality control mechanisms in their standard formal education programs. On the other hand, Hollands and Tirthali (2014) state that Dr. Douglas Hersh, formerly Dean of Educational Programs at Santa Barbara City College, CA, emphasized that his college had to achieve the requirements of Western Association of Schools and Colleges and the Accrediting Commission for Community and Junior Colleges which demand that all learners should be offered equivalent student services and regular effective contact regardless of the delivery mode. According to him, it would be not only difficult but also expensive to offer such services involving library services, academic counseling or tutoring for massive amount of MOOC participants. Moreover, costs of accreditation would be a problem especially for the for-profit providers as it is costly to offer fully accredited education. Similarly,Alston and Brabon (2014) state that the massive nature of the MOOCs presents some operational issues and the validation process of a MOOC offering university credit requires to be supplemented by traditional forms of pedagogy, delivery and support. Another point they mention is that the process of accreditation transform the learning dynamic of MOOCs and have a negative impact on the conception of openness in terms of its costs and requirements.
Current Initiatives
New offerings of higher education such as MOOCs have not attracted adequate attention from traditional quality assurance and accreditation bodies both in the United States of America(USA) and Europe yet (Eaton & Uvalic-Trumbic, 2014). For instance, MOOCs are not directly subject to accreditation by the NVAO, the accreditation agency of the Netherlands and Flanders, and it is unlikely that MOOCs will be subject to accreditation in the Netherlands or Flanders in the foreseeable future(NVAO, 2014). Although there is little interest on the issue, there are a few initiatives started by the MOOC providers and agencies or councils responsible for or interested in quality assurance and accreditation. For instance, in Coursera, MOOCs are peer-reviewed prior to delivery. In order to ensure the quality of the courses, Coursera has introduced aprotocol for quality assurance and courses are reviewed by the home institution (see This protocol outlines some processes such as course description pages, course development agreements, early uploading of course materials, on-going class monitoring and post-course feedback. American Council for Education (ACE) has accepted 11 courses, five from Coursera, five from Udacity and one from edX, for credit, and will continue to review and externally quality-assure them (Gaebel, 2014). Hollands and Tirthali (2014) mention on the statement of Karen Vignare, Associate Provost at the Center for Innovation in Learning of University of Maryland University College, informing that if the college or the MOOC platform such as Coursera or Udacity has gone through the extra steps of having ACE accreditation reviews, the MOOC would be accepted.
Another initiative is the MOOC Quality Project conducted by the European Foundation for Quality in E-Learning (EFQUEL), in which many well-known researchers were involved to discuss quality of MOOCs (Ehlers, Ossiannilsson, & Creelman, 2013). Researchers mention on some factors regarding the MOOC quality as: the pre-course information available to learners, pedagogical approaches, whether some kind of declaration of contents can be agreed upon showing clearly the type of course, level of student commitment, course schedule and deadlines, technical requirements, role of teacher or tutor, availability and level of interaction, availability of credentials.
Rosewell and Jansen (2014) mention on the development of the OpenupEd Quality Label, a self-assessment and review quality assurance process for the new OpenupEd MOOC portal which was launched by the European Association of Distance Teaching Universities (EADTU). Read and Rodrigo (2014) claim that the 32 benchmarks of OpenupEd Quality Label represent a good first step toward MOOC quality control but will inevitably need to be refined as more experience of applying them has been obtained. Similarly, the Quality Matters Program offers a quality review process and recognition through a rubric for continuing and professional education. The rubric can be used to evaluate and improve college-based, non-credit courses including MOOCs (see
Perhaps the most remarkable initiative is the establishment of a global quality platform by the Council for Higher Education Accreditation (CHEA) as a national accreditation organization that formally works for recognition of accrediting bodies in theUSA.CHEA and its International Quality Group(CIQG) aim to review non-institutional education providers for protecting learners as a response to the explosion of non-traditional provision including MOOCs and increasingly international higher education (Eaton, 2014; MacGregor, 2014).This quality platform is intended to provide a voluntary, non-governmental external review of non-institutional providers undertaken by an expert team and based on self-evaluation, using standards with a primary emphasis on learning outcomes.Similarly, QAA, the independent body responsible for quality of higher education in the UK,isdeveloping resources to facilitate good practice to support MOOC providers (QAA, 2014). The UK Quality Code for Higher Education provides a useful reference point for providers in their internal quality assurance processes.
Future Trends
Although some institutions have remained hesitant in terms of credentialing MOOCs, there are clear signs that MOOCs will become a significant and possibly a standard element of credentialed and accredited university education (Grainger, 2013; Haggard, 2013) as long as they are regarded as a means of lowering costs, increasing revenues and building capacity in higher education. Currently, methods for assessing learning, which leads to credentialed outcomes, are not highly developed (Haggard, 2013). However, some new methods are starting to emerge such as Signature Track or Pearson VUE’s examinations centers. Some MOOC providers announce that they are beginning to partner with proctoring firms, and various tools such as webcams, microphones, and screen sharing are used to keep an eye on exam takers remotely in order to create verified assessments (Kelly, 2014). Such external recognition or validation of MOOC assessment verification services seem to continue to increase (Grainger, 2013).Another trend is the emergence of new types of awards. One example is the use of open badges which allow learners to get recognition for their studies and aggregate a series of badges that could be converted into a degree or a diploma (Eaton Uvalic-Trumbic, 2014).
Currently, there is no voluntary, independent and formal third-party review process to evaluate the quality of MOOCs (CHEA, 2014).However, Grainger (2013) implies that accreditation of MOOCs by external bodies and university providers is highly likely. On the other hand, there is uncertainty regarding the identity of the accrediting body. Eaton and Uvalic-Trumbic(2014) mention on three alternatives, one of which is the review of MOOCs by the current quality assurance and accreditation agencies by expanding their scope. Another alternative is examining quality of MOOCs by colleges and universities. Third one is to develop a capacity for a separate quality review process, focused explicitly on the non-institutional sector. They state that they are exploring the feasibility of this third alternative by the CHEA International Quality Group (CIQG).
Hollands and Tirthali (2014) mention on the suggestion of Minghua Li, Professor, School of Public Administration at East China Normal University. According to Li, major online platform providers should collaborate to create a course accreditation system and accordingly they could make the federal government allow the application of financial aid to pay for online courses offered outside of college degrees, including MOOCs.
Bergeron and Klinsky (2013) propose establishing a new private sector body, “Modern States Accrediting Agency,” which could be recognized by the U.S. Department of Education as an approved accreditor to qualify learners for federal aid. Modern States Accrediting Agency would ensure the quality of the specific courses and make the credits transferable into the traditional system as well. In addition, they suggest that the Agency could recognize a MOOC and develop proctored tests for learners who complete that MOOC. Similarly, Hollands and Tirthali (2014) imply that establishment of an accrediting organization for MOOCs and other non-traditional education will provide learners to accumulate a portfolio of credentials that serve as a viable supplement or alternative to a college degree.
Discussion
Obviously, different forms of learning in higher education pose challenges for quality assurance (Eaton & Uvalic-Trumbic, 2014).New frameworks for quality assurance and accreditation are required for MOOCs to support different approaches and delivery models in higher education (YuanPowell, 2013).MOOCs cannot be considered apart from the open and distance learning (ODL) literature which has a well-established body of scholarshipon ODL in higher education (Adams, Yin, Vargas Madriz, & Mullen, 2014; Daniel, 2012).Although quality assurance and accreditation practices in ODL are still at a development stage when compared with face-to-face education (Jung, Wong, Li, Baigaltugs, & Belawati, 2011), some ODL associations, qualityagencies and ODL universities have considerable experience on the issue that could not be ignored. For instance, ODL quality practices introduced the concept of quality in course level, in addition to the two types of quality assurance and accreditation practices in face-to-face education which are definedas institutional and program accreditation. Accordingly, course level review processes and quality rubrics appeared in the literature such as Chico Rubric, Quality Matters Rubric or Online Learning Consortium Quality Scorecard. Course developers can either use these rubrics as a guide or in some cases they can get a quality label after a course review process to certify the quality of their online course design and other online components. However, these kinds of initiatives are voluntary and have not been considered as a formal accreditation review yet. In this regard, proliferation of these course level quality assurance and accreditation practices might be a solution for assuring quality of MOOCs or accrediting them.