LibQUAL+TM Supplementary Results
Part Two: Trend Analyses from 2001 - 2004
Prepared by ITS Research Consulting
Introduction
Part Two of the supplementary report analyzes trends over time in LibQUAL+™ Survey responses for the University of Texas at Austin. These analyses focus on longitudinal analyses from 2001 – 2004 for the LibQUAL+™ dimensions Affect of Service and Information Control, and for analyses from 2003 – 2004 for the dimension Library as a Place.
Part Two of the report is comprised of three sections:
· Section 1 describes the three dimensions of service and explains the preliminary analyses of the data.
· Section 2 focuses on differences between faculty, graduate students, and undergraduates in terms of their responses to the three primary dimensions across time.
· Section 3 examines trends across time within each discipline in terms of the three primary dimensions. Trends within specific libraries could not be examined, as this data was collected for 2004 only.
Section 1: Describing the dimensions across time
Overview
This section focuses on describing the LibQUAL+™ Survey responses used for examining trends across 2001-2004 for the dimensions of Affect of Service and Information Control, and across 2003-2004 for the dimension Library as a Place. The analyses are based on desired, perceived, and minimum ratings, as well as adequacy (the difference between minimum and perceived dimensions) and superiority (the difference between perceived and desired dimensions) gaps. Reliability analyses for the three dimensions are discussed below.
Preliminary Analyses
Recoding Discipline
Respondents at UT-Austin self-classified themselves into one of fifteen academic disciplines, and some disciplines had very low numbers of respondents (e.g., General Studies, with N = 2 for 2003). General Libraries requested that only two disciplines (“Other” and “Undecided”) be recoded into a new category. In addition, the number of disciplines changed from 2001 to 2002. The ‘Comm/Journalism’ discipline was not included in the 2001 survey and will not have data for all four years.
Scale Scores
The items that are administered as part of the LibQUAL+™ Survey have changed over time. There were 56 items administered in 2001, 25 items in 2002, 25 items in 2003, and 22 items in 2004. In 2004, two former dimensions (Access to Information and Personal Control) were reduced and combined into a new dimension called Information Control. In order to track the Information Control dimension across all four years, the relevant Access to Information and Personal Control scores from previous years were combined to create Information Control scores for each year 2001-2003. We computed scale scores only with items that were asked consistently across time (for example, the “electronic information resources” item, added in 2003, was omitted from these analyses).
Another major change in item content related to Library as Place occurred between 2002 and 2003, which resulted in a lack of consistency for this dimension from 2001 to 2004. Table 1.1 shows the items that were used for the Library as Place dimension across all four years. Due to the lack of continuity for items belonging to Library as Place, this dimension could be compared only for 2003 vs. 2004.
Table 1.1: Items for Library as a Place dimension from 2004 to 2001.
Quiet space for individual activities / Quiet space for individual activities
A haven for quiet and solitude / A haven for quiet and solitude
A comfortable and inviting location / A comfortable and inviting location / A comfortable and inviting location
Library space that inspires study and learning / Library space that inspires study and learning
Space that facilitates quiet study / Space that facilitates quiet study
Community space for group learning and group study / Community space for group learning and group study
A place for reflection and creativity / A place for reflection and creativity
A getaway for study, learning, or research / A getaway for study, learning, or research
A contemplative environment / A contemplative environment
Reliability Analyses
Reliability analyses were used to assess whether questions within each dimension were answered in a similar fashion. Table 1.2 shows the results of these analyses, which used Cronbach’s Alpha to assess the internal consistency of each dimension for the 2001-2004 data.
Table 1.2: Cronbach’s alphas for Affect of Service and Information Control, 2001-2004
Dimension / Rating / 2001 / 2002 / 2003 / 2004Affect of Service / Minimum / 0.92 / 0.93 / 0.94 / 0.94
Perceived / 0.94 / 0.93 / 0.93 / 0.94
Desired / 0.88 / 0.90 / 0.92 / 0.91
Information Control / Minimum / 0.86 / 0.89 / 0.90 / 0.91
Perceived / 0.84 / 0.85 / 0.86 / 0.87
Desired / 0.82 / 0.82 / 0.81 / 0.88
Note: Alpha < .60 = poor; between .6 & .7 = moderate; between .7 & .8 = good; and > .80 = high reliability.
The Affect of Service and Information Control scales showed high reliability across time (minimum alpha > 0.8). There was a small increasing trend in the reliability of the items measuring Minimum standards for Information Control through time (0.86 to 0.91). The overall reliability shows it is reasonable to present results for each dimension as a whole, rather than presenting results for individual items within the dimensions.
The reliability of the Information Control scale (Table 1.2) was compared to that of the two scales that were combined to create it (Access to Information and Personal Control, Table 1.3). The Information Control and Personal Control scales showed high reliability across time (alpha ranging from 0.81 to 0.91), while the Access to Information scale showed only moderate to good reliability across time (alpha ranging from 0.65 to 0.82). Thus the combination of the two scales did not reduce internal consistency.
Table 1.3: Cronbach’s alphas for Access to Information and Personal Control, 2001-2003
Dimension / Rating / 2001 / 2002 / 2003Access to Information /
Minimum /
.71 /
.78 /
.82
Perceived / .69 / .65 / .68
Desired / .67 / .73 / .71
Personal Control / Minimum / .88 / .91 / .90
Perceived / .85 / .86 / .84
Desired / .84 / .85 / .82
Note: Alpha < .60 = poor; between .6 & .7 = moderate; between .7 & .8 = good; and > .80 = high reliability.
The next table shows the results of the reliability analysis using Cronbach’s Alpha to assess the internal consistency of the Library as Place dimension for 2003 and 2004. The Library as Place scales showed high reliability across time (minimum alpha = 0.84). Reliability is also consistent across these two years.
Table 1.4 Cronbach’s alphas for Library as a Place, 2003-2004
Dimension / Rating / 2003 / 2004Library as Place / Minimum / 0.87 / 0.87
Perceived / 0.84 / 0.86
Desired / 0.86 / 0.87
Note: Alpha < .60 = poor; between .6 & .7 = moderate; between .7 & .8 = good; and > .80 = high reliability.
Section 2: Responses by Faculty, Graduate,
Undergraduate Status and total
Overview
Within the University of Texas at Austin, each status (faculty, graduate, and undergraduate) was compared to the others in terms of minimum, perceived, and desired scores for each of the three 3 primary dimensions that remained consistent across time. General staff and library staff were excluded from these comparisons due to generally small numbers of respondents (7 excluded in 2001, 13 excluded in 2002, 11 excluded in 2003, 68 excluded in 2004).
Method
To determine whether significant differences existed between faculty, graduate student, and undergraduate student responses for each of the three dimensions across time, a series of 3 (faculty, grad, undergrad) X 4 (2001, 2002, 2003, 2004) ANOVAs were conducted for each dimension. Each analysis included Status (undergraduate, graduate student, faculty) and Time (2001, 2002, 2003, and 2004) as between-subjects variables. Although it is likely that some respondents were sampled in multiple years, there was no identification variable that could be used to match cases across time. The analyses treated minimum, perceived, and desired quality ratings, along with the derived adequacy and superiority gap variables, as the dependent variables. For each analysis, if Status or Time was significant at the p < 0.05 level, pairwise comparisons were conducted (using Sidak’s adjustment for multiple tests) between the three status groups and the three timepoints. The term “significant” is used to denote a pairwise difference significant at the p < 0.05 level. If the Status by Time interaction was significant, then simple-effect tests of Time within Status were conducted.
Results
Affect of Service
Minimum Rating: The results for the minimum rating showed significant differences by Status (F2, 2432 =18.82, p<0.001) with all three groups significantly different from each other (FMean=6.33 > GMean=6.13 > UMean=5.85).
Perceived Rating: The results for the perceived rating showed significant differences by Status (F2, 2767 =23.32, p<0.001) with undergraduates significantly different from both faculty and graduate students (FMean=7.15 and GMean=7.02 > UMean=6.74). Significant differences were also observed over Time (F3, 2767 =2.99, p=0.030) where 2003 differed from 2001 (2003Mean=7.07 > 2001Mean=6.88).
Desired Rating: The results for the desired rating showed significant differences by Status (F2, 2432 =5.37, p=0.005) with undergraduates significantly different from both faculty and graduate students (FMean=7.78 and GMean=7.77 > UMean=7.60).
Adequacy Gap: There were significant differences by Time (F3,2432 =7.52, p<0.001) with 2001 and 2002 having smaller gaps than 2003 (2001Mean=0.70 and 2002Mean=0.77 < 2003Mean=1.08).
Superiority Gap: The results showed significant differences by Status (F2, 2432 =5.97, p=0.003), with undergraduates having larger gaps than faculty (FMean= -.63 < UMean= -.86), and by Time (F3, 2432 =5.47, p=0.001) with 2001 and 2002 having larger negative gaps than 2003 (2001Mean= -0.90 and 2002Mean= -0.80 < 2003Mean= -0.62) and 2001 having a larger negative gap than 2004 (2001Mean= -0.90 < 2004Mean= -0.65).
There were no significant interactions between Status and Time for any dimension, suggesting that the three status groups do not differ from one another in terms of the extent of change across time.
Figure 2.1 displays the desired, perceived, and minimum mean ratings for Affect of Service by Status (Faculty, Graduate, and Undergraduate) and Total. The Total figure shows the trends for desired, perceived, and minimum mean ratings when faculty, graduate, and undergraduates’ responses are combined; this figure is useful for the interpretation of any main effects for Time. For each mean, the 95% confidence intervals are plotted by the lines extending above and below the means. The adequacy and superiority gaps represent the difference between the minimum and perceived and the perceived and desired respectively.
Figure 2.1: Trends across time for Affect of Service by faculty, graduate, undergraduate, and combined groups.
Information Control
Minimum Rating: The results for the minimum rating showed significant differences by Status (F2, 2434 =36.36, p<0.001). That is, when collapsing across the years of the study, all three groups differed significantly from each other (FMean=6.85 > GMean=6.55 > UMean=6.24). Significant differences were also observed though time (F3, 2434 =6.52, p<0.001). That is, when collapsing across the three groups, overall 2002 minimum ratings differed from 2004 (2002Mean=6.39 < 2004Mean=6.70). Results also showed a significant interaction effect (F6, 2434 =2.55, p=0.019) which was primarily due to differences between status across years; i.e., 2002 graduate students responded significantly lower than 2004 graduate students (2002Mean=6.38 < 2004Mean=6.82). No differences were seen within status by year.
Perceived Rating: The results for the perceived rating showed significant differences by Status (F2, 2769 =4.01, p=0.018) in that faculty had higher scores than undergraduates (FMean=7.04 > UMean=6.89). Significant differences were also observed though time (F3, 2769 =6.65, p<0.001) in that 2001 and 2002 were lower than 2004 (2001Mean=6.91 and 2002Mean=6.88 < 2004Mean=7.12).
Desired Rating: The results for the desired rating showed significant differences by Status (F2, 2434 =21.92, p<0.001) with faculty and graduate students having higher scores than undergraduates (FMean=8.28 and GMean=8.20 > UMean=7.97). Significant differences were also observed though time (F3, 2434 =12.27, p<0.001) in that 2002 was lower than both 2003 and 2004 (2002Mean=8.00 < 2003Mean=8.24 and 2004Mean=8.26), and 2001 was lower than 2004 (2001Mean=8.09 < 2004Mean=8.26).
Adequacy Gap: There were significant differences by Status (F2, 2434 =18.08, p<0.001) in that faculty had smaller gaps than both graduates and undergraduates (FMean=0.19 < GMean=0.49 and UMean=0.66). In addition, there was a significant interaction effect (F6, 2434 =2.39, p=0.027), which was primarily due to differences between graduate and undergraduate students in 2004. Although graduate and undergraduate responses did not differ overall when collapsed across all four years, in 2004 undergraduates had a larger gap than graduate students (2004UMean=0.77 > 2004GMean=0.32).
Superiority Gap: There were significant differences by Status (F2, 2434 =3.94, p=0.020), where faculty had a larger negative gap than undergraduates (FMean= -1.24 < UMean= -1.07).
Figure 2.2 displays the desired, perceived, and minimum mean ratings for Information Control by Status (Faculty, Graduate, and Undergraduate) and Total. The Total figure shows the trends for desired, perceived, and minimum mean ratings when faculty, graduate, and undergraduates’ responses are combined; this figure is useful for the interpretation of any main effects for Time. For each mean, the 95% confidence intervals are plotted by the lines extending above and below the means. The adequacy and superiority gaps represent the difference between the minimum and perceived and the perceived and desired respectively.
Figure 2.2: Trends across time for Information Control by faculty, graduate, undergraduate, and combined groups.
Library as Place
Library as a Place was composed of a consistent set of items for 2003-2004 only (see Table 1.1 above). Accordingly, this dimension was compared for 2003 vs. 2004 only.
Minimum Rating: The results for the minimum rating showed significant differences by Status (F2, 1416 =29.98, p<0.001) with faculty significantly lower than both graduate students and undergraduates (FMean=5.16 < GMean=5.81 and UMean=5.98).
Perceived Rating: The results for the perceived rating showed significant differences by Status (F2, 1416 =7.43, p<0.001) with both faculty and graduate students significantly lower than undergraduates (FMean=6.43 and GMean=6.50 < UMean=6.78). Significant differences were also observed over Time (F3, 1416 =6.84, p=0.009) in that 2003 was significantly higher than 2004 (2003Mean=6.67 > 2004Mean=6.47).