Benefits of Joint Research by

International Education (IE) and Institutional Research (IR) Offices: UCLA

Caroline West

Director

UCLA Office of Analysis and Information Management

In higher education institutions, offices that report up different lines often do not realize their potential formutual assistance in improving services and student outcomes. International education offices usually operate outside of the enrollment management and accountability arenas that bring institutional research (IR) and traditional on-campus academic units into contact. But if international education offices seek out IR staff they maygain access to information that can place their operations within the wider frame of reference of institution-wide student academic progress and success. The two offices at UCLA recently initiated a research relationship and the accompanying PowerPoint presentation outlines early results.

The UCLA Office of Analysis and Information Management (AIM) conducts research to fulfill a dual role—providing statistics, data, and information about UCLA to both the public and the campus. The office gathers and analyzes data on a broad range of policy and practice issues, including extensive data on prospective and current students, and on alumni. In addition, the office staff brings together personnel with highly refined skills in data collection and analysis and has organized an extensive store of knowledge used in UCLA’s planning and evaluation processes.

The UCLA International Education Office (IEO) collects data based on its study abroad population. Data is collected from students’ online registration, online program evaluations, and through retroactive analysis of degree progress reports.

Although the offices identified areas of overlapping research when they first met, mutual efforts using their respective access to unique data sets yielded more nuanced and robust results. For both offices, reporting on outcomes and understanding student preferences and trends is becoming vital to effective programming. Thus, collective efforts on data gathering, analysis, and interpretation can enhance credibility and help uncover trends; and such efforts have enormous value for both units. One major gap in the IEO’s research is inaccessibility to data about the general UCLA population. Together with AIM, IEO is able to compare the study abroad population with the general student population.

The first project that AIM undertook was to determine whether the demographic profile forparticipants in IEO programs mirrored the campus population as a whole. The goal was to identify underserved groups for which IEO might need to develop different kinds of overseas offerings or special communications. Institutional research used its database to identify students who had participated in either the UC Education Abroad Program, where students enroll in classes offered by foreign universities, or in UCLA’s Summer Travel Study Program, where a UCLA professor teaches at a foreign location. We looked at participation by gender and ethnicity.

The charts illustrate that while many groups participate roughly in proportion to their overall share in campus population, Asian men under-participate and Hispanic women participate heavily in summer programs. Further research indicated that at least some of the Asian male underrepresentation could be related to the high percentage of Asian students majoring in science and engineering. These data suggest that development of a Travel Study Program in engineering might appeal to this population in ways that current programs cannot.

Another future research topic that surfaced in conversations between the two offices was time to degree. While an IEO might be able to track graduation outcomes for its participants, IR is set up to do an analysis that compares them to similar students who have not participated in IEO programs, to provide a fair comparison.

A more difficult challenge is research about ability of returning participants to apply their units towards degree requirements. The IEO has conducted a long-term data collection and analysis project in this area. The results are shown in the final PowerPoint slide--information from over 22,000 student course records of UCLA students who participated on the University of California Education Abroad Program.

First, the data’s shortcomings: they were gathered one-by-one by multiple researchers over a period of several years. During that time, it is highly probable that human error could be embedded in some of the records. Second, the records reflect only a snapshot in time of the students’ Degree of Progress Report (DPR). If the student had petitioned for courses to apply towards their major/minor/college requirements after the snapshot was taken, this research would not capture the result. Finally, some college requirements such as foreign language, would not reflect in these results.

The data is compelling however, in demonstrating that, even with certain gaps of results, over half of the courses students took abroad were successfully applied towards college/major/minor requirements. Thus, it is perhaps safe to say this number is actually higher since these results do not reflect courses that students may have decided to apply towards requirements, after the snapshot was taken. The ratio is also certainly higher since foreign language requirements satisfied from courses taken abroad are not reflected in the data. Finally, for unidentifiable reasons, grades for a significant portion of the data (about one-third) were not captured by the researcher.

Students are often concerned about how courses taken abroad can be used towards their degree. This data is important because it helps advisors more confidently assert that completing major/minor/college requirements abroad is not only possible but probable. Students are also often concerned with their home GPA in taking courses abroad. Our results show that, of the data gathered, over 60% of grades were A- or better. It is worth noting that South Korea was ranked 4th among the top destinations that students have applied major and GE credit.

UCLA Summer Travel Study also gathers its own data based on student online applications and evaluations. One significant finding is that 40% of participants are leaving the U.S. for the first time. This figure confirms anecdotal evidence that short-term, summer programs are often a gateway for students to begin their abroad experience.

Finally, the IE and IR offices can cooperate in analyzing survey data. While IE offices can survey participants about their experiences, they do not have comparison data for non-participants. At UCLA, the institutional research office was able to compare IE participant responses to the UC undergraduate engagement survey (UCUES) to responses from students who hadn’t participated in programs administered by IEO. UCUES is a census survey of all enrolled students, administered every two years, that asks them about academic engagement, skill development, and student development issues.

We found that there were negligible differences between the two groups, including their responses to questions about interaction with different ethnic groups and understanding of and comfort with international differences. Since demographic questions on the same survey indicate almost two-thirds of UCLA respondents have at least one grandparent born abroad, and that the student body itself is very diverse, this is not surprising. However, the results suggest that in discussing the benefits of international education for our students, a more nuanced approach may be useful.

Finally, one of the ways in which the offices have benefitted from collaboration is deeper understanding of the student population, both by pooling data and by gaining understanding of the limits of the data and of how to interpret data withincontext. Both offices agree that we are learning a great deal about data resources, data limitations, and caution in interpreting research results for this group of students through our new relationship. Student populations will differ widely among campuses and data is often best interpreted at the local level. In addition, in many cases data can be incomplete or imperfect. It is important to discuss the extent to which conclusions and decisions can be made based on such shortcomings. Nevertheless, data can be a helpful tool to understand or uncover campus specific trends.

9/8/09Page 1