Literature Review Narrative but Try to Organize by Research Questions

Self-Study and Curriculum Review

of the

Information and Learning Technologies (ILT) Program,

ID & Adult Learning,

at University of Colorado at Denver

Michelle Boyer, Lisa Bradshaw, Cheryl Ide,

Annie Persson, Michele Sutherland

July 27, 2005

Introduction 2

Background 3

Research Questions 4

Method 5

Findings 7

Implications for Practice 31

Possible Future Actions 34

References 35

Appendix A (Alumni Survey) 37

Appendix B (Faculty Survey) 41

Appendix C (Student Survey Questions) 43

Appendix D (Questions About ILT History) 45

Appendix E (Sample University Faculty Inquiry) 46

Appendix F (Program Comparison Table) 47

Appendix G (Course and Program Comparison) 62

Introduction

Simulations, project management, Captivate, Flash, return on investment (ROI), learning management systems, EPSS, SCORM, outsource training, design documents… All of these terms relate to jobs in the emerging field of instructional design in the corporate world. As more and more companies turn to e-learning and Web-based training, more instructional designers need training in this rapidly changing field. Two years ago, all but one of our group of researchers began Master’s degrees in Information and Learning Technology, Instructional Design & Adult Learning, at University of Colorado at Denver. Now, as we approach the completion of our program, we have had time to reflect on what we need to be successful in our field and how those needs have been met.

As a team, we chose to delve into a self-study of the ILT program. We also pursued a curriculum review of other programs alongside our own. We explored these topics out of respect for our program and from the desire to find opportunities to make positive changes for future Adult Learning/Instructional Design students in the program. Our group of researchers included four members for whom this course was the final requirement in the Master’s degree program. Michelle Boyer recently transitioned from an eleven-year career as an educator into an Instructional Designer position. Lisa Bradshaw is an Online Communication Developer for an educational grant at Metropolitan State College of Denver. Cheryl Ide is an e-learning consultant, on-line trainer, and documentation specialist. Annie Persson has a background in technical writing, and recently accepted a design position with a local company. Michele Sutherland is a Special Events Coordinator for a prestige cosmetics brand. All are students of the Adult Learning/Instructional Design track in the ILT program.

From our diverse backgrounds, we found one major similarity amongst ourselves; all of us have had concerns about gaps in the curriculum of our Master’s program in Adult Learning/Instructional Design. We have invested a great deal of time and money to advance our learning and improve our careers. While we are nearly finished with our program, and this research will not benefit us directly, we hope to contribute to improvements for the benefits of future students.

Some of the learning experiences we had have been incredible, and they made us more aware of ourselves as learners and educators. However, there are still areas in which we wish we had received more guidance and experience. There are parts of our chosen field that we only learned about on the job, instead of in the classroom. With the goal of performing a self-study of the program’s history and curriculum, and a curriculum review of other top ILT programs, we will recommend ways for CU-Denver’s ILT program to improve and adapt to changing conditions in the professional field. We want to make recommendations that will enable the program to provide all of its Adult Learning/Instructional Design graduates with a more comprehensive preparation for the business world where we intend to pursue our careers.

Background

While we gained a great deal of information and learning experiences vital to developing our skills as instructional designers, we believe there were gaps in our learning. Some of our classes were cancelled due to low enrollment, and each member of our group has taken classes or independent studies from departments outside of the ILT program to complete not only electives, but also core classes. However, these substitutions expanded our horizons and added depth to our studies, especially in several business aspects of instructional design; yet, questions remain about content specific to Instructional Design.

We also made note of the changing nature of the ILT program. This field of study, while not new, has changed dramatically with the advent and adoption of Web-based training. At the start of our program, two years ago, we followed a specific plan of study in which classes were offered sequentially, and K12 and Adult Learning cohorts had been eliminated. Shortly after the start of our program, the ILT competencies were changed along with the ILT portfolio requirements. Currently, students enrolling in the program have a different plan of study, including an internship requirement, and competencies for the ILT portfolio have expanded.

In the rapidly evolving field of e-Learning instruction in both academia and on the job, change is a requirement. In order for us to be well prepared students in a business driven world, we needed guidance in the business aspect of Adult Learning/Instructional Design. We needed to understand the importance of ROI, requiring management buy-in for training, proper feedback from trainings that are executed, and the latest acquisitions in trends and applications in the field. Our perspective on e-learning was different because just as educators must prove their worth as teachers with increased student test scores, we must prove our worth as trainers and instructional designers with proof of return-on-investment. However, since one of our group members had not heard of the term “return-on-investment” until she took a Business School course upon the cancellation of an ILT elective, we feel there is a disconnect between the academic and business realities of our field in the program.

As corporate educators, we bring a different slant to what we feel is currently a K12-based program. In casual conversation amongst our peers, we brainstormed about ways the ILT program could be structured to better meet the needs of ID & Adult Learning students. This is the main thrust of our research.

Research Questions

1. How has the CU-Denver ILT program evolved?

· What are some of the significant changes that have been made in the past to improve the program? How did students and faculty receive the changes? Were there any decisions to revoke changes and revert back to previous policy, and if so, why were the decisions made?

· Are there proposed future changes to the program? If so, what changes are proposed, and what are the reasons for choosing them?

We sought to learn about the background and history of the CU-Denver ILT program, when and how it evolved, the ways that it has changed since its creation, and how students and faculty received those changes. We wanted to find out if any of the changes could be revisited for clues that might help us recommend future changes to the program that could improve it further.

2. How does the ILT Instructional Design and Adult Learning curriculum map to professional standards?What courses fulfill each of the standards?

· AECT: what official accreditation standards currently in use map to the ILT curriculum?

· ASTD: what standards map to the ILT curriculum?

· ISPI: what standards map to the ILT curriculum?

We wanted to examine the professional standards of these three organizations, and how the CU-Denver ILT program’s curriculum has been designed to fulfill them. We investigated possible areas in our program’s curriculum that may not fully address the standards, to recommend ways in which the curriculum could better prepare students to fulfill the standards.

3. How well does the ILT program prepare graduates for careers in the adult instructional design/instructional technology/educational technology field?

· Do current students feel that the ILT program is preparing them for their careers in ID? If not, how do they feel the program could be improved?

· Do alumni of the program feel that the ILT program has prepared them for their careers in ID? If not, how do they feel the program could be improved?

We surveyed current students and alumni of the CU-Denver ILT program as well as faculty members, to learn if these groups feel that the program effectively prepares its graduates for their careers. We gathered suggestions and feedback that we hoped would enable us to make recommendations for improvement in the program.

4. How does the ILT program compare to other top programs nationwide?

· What are the top programs nationwide? Who feels that these programs are exceptional? What criteria are used to evaluate a program that is considered one of the top programs?

· What are some distinctive features of these programs? What are their similarities and differences?

· Based on this analysis, what could be incorporated into the CU-Denver ILT program that would improve it?

We examined other top instructional design and related programs across the United States, to determine if the top programs have features in common that make them exceptional. We investigated the ways these programs are given their high ranking. We sought to use our findings to compare characteristics of these other top programs with the CU-Denver ILT program, and to recommend strategies and methods that the other programs are using which might be incorporated into the CU-Denver ILT program to help improve it.

Method

Each member of our team participated in a portion of the data collection process for this study by developing data collection tools, administering them to participants or evaluating and interpreting the results. This study required contact with current ID & Adult Learning track students of the ILT program, ID & Adult Learning alumni of the ILT program, program faculty, program advisors and/or chairs, and selected program correspondents from other schools throughout the country. As current students ourselves, we considered ourselves subject matter experts in the area of study and hoped to provide additional insight where applicable, while remaining objective.

Data Collection Procedures

This study required several methods to properly answer our research questions. Our matrix depicts how data was collected for each question posed, followed by a more specific explanation for each question.

Method / Literature Review / Correspondence with SMEs / University Faculty / Rubrics and Comparison Tables / Surveys with Current ILT Students / Faculty / Alumni
Question 1 / X / X
Question 2 / X
Question 3 / X / X
Question 4 / X / X / X

Figure 1: Data Collection Methods

For the first question, How has the CU-Denver ILT program evolved? we combined literature reviews with personal interviews of the CU-Denver ILT program chair and former faculty members. Since the inception of this program, many changes have been implemented for the improvement of the program. Understanding the program’s history and the reasons for some of the changes enabled us to make better recommendations for the future without “reinventing the wheel” or making suggestions that may have already proven unsuccessful. Reviewing an old course catalog and an accreditation report provided a clear understanding of the history of the ILT program and allowed for more concise interviews with faculty members.

The second question, How does the ILT Instructional Design and Adult Learning curriculum map to professional standards?What courses fulfill each of the standards? was answered through literature reviews. Using a copy of the ILT Rubrics and a comparison table, we compared the ILT competencies with AECT standards. As a requirement of the Current Trends and Issues in Instructional Technology class, several members of this team mapped program competencies against various industry standards, and we used our own research for those projects in finding answers to this question. Standards for several professional organizations were easily available, and we constructed additional comparisons as part of the final report.

The third question, How well does the ILT program prepare graduates for careers in the adult instructional design/instructional technology/educational technology field?, required administering online surveys to current students and alumni of the ILT program. These focused on personal experiences and opinions of students of the program and sought to gain insight and suggestions for future program suggestions. In this question we tried to identify areas which were consistently lauded or lacking within the program so that these areas can be brought to the attention of the curriculum developers. These surveys can be found in Appendices A and B respectively.

Our final question, How does the ILT program compare to other top programs nationwide? required multiple data collection methods. This question combined literature reviews of other programs nationwide, anonymous surveys of CU-Denver ILT faculty, and follow-up correspondence with similar program administrators nationwide. By giving CU-Denver ILT faculty the option to remain anonymous in our survey we hoped to increase our potential for “honest” answers, just as faculty evaluations administered to students are anonymous for each class at the end of a term. The literature review of other programs was conducted both online, by visiting the Web sites of others schools and programs, and through requesting program information from the institutions themselves. We have contacted program leaders at targeted schools to ask more in-depth questions that arose in our study.

Data analysis

Both our data collection and analysis methods were varied. Our surveys were evaluated for trends and percentages by simple comparative analysis of each question’s responses. Charts and tables were generated in Excel for selected findings. For any personal interviews and written correspondence conducted, each interviewer followed a standard procedure of writing all questions out in advance so that answers could be recorded directly during the interview. If multiple sources were interviewed for the same purpose, the interview questions were included in the appendix section. Results of these interviews, as well as any open-ended questions given in our surveys, were reviewed for significant consistencies, inconsistencies and trends.

Findings