Information Technology Survey
Spring 2010
BA 6303
Business Research Methods
Prepared for:
Dr. Sudhir Chawla
Prepared by:
Karen Mallott
Turner Phipps
Josh Neiswander
Executive Summary
Each year, an information technology survey is distributed to randomly selected classes of students at Angelo State University. The purpose of this survey is to gage students’ satisfaction with current technology services and measure the performance of I.T. customer service. In addition, this survey attempts to identify trends in the student body population in order to update product and service offerings accordingly. Students are asked for their opinions and preferences on potential new offerings. The results of this survey are used by the I.T. department to improve and modify services, add new services, and address any concerns that arise from unfavorable results.
During the spring 2010 semester, surveys were distributed to randomly selected classes of students between March 22nd and April 9th. Both paper-based and electronic versions of the survey were distributed. A total of 401 valid surveys were returned and utilized for this report.
The survey consists of three parts. Part I of the survey is based on the SERVQUAL instrument by Zeithaml, Parasuraman, and Berry and measures students’ satisfaction with three dimensions of service provided by the I.T. department at ASU. Part II has been developed by the I.T. staff and is modified each year to obtain students’ opinions on services currently provided by I.T., familiarity with the variety of services offered, and is designed to gain feedback on adding potential new services. Part III of the survey gathers information on student demographics and provides space for students to write individual comments or make recommendations to the I.T. department.
The three SERVQUAL related dimensions utilized in Part I of the survey are as follows:
STAFF – this factor consists of questions concerning the following dimensions:
· Assurance - Knowledge and courtesy of employees and their ability to inspire trust and confidence
· Empathy – Providing care and individualized attention
· Responsiveness - Willingness to help customers and provide prompt service
SERVICES – this factor consists of questions concerning the following:
· Tangibles – Appearance of physical facilities, equipment, and personnel
PROFESSIONALISM – this factor consists of questions concerning the following:
· Reliability - Ability to perform the promised service dependably and accurately
Once the questions in Part I were grouped into the appropriate factor, a SERVQUAL score was calculated for each factor using the model below:
Q = PERCEIVED – DESIRED
The SERVQUAL score measures the difference between a student’s perceived satisfaction level and a student’s desired satisfaction level. A mean score was then calculated. The graph below shows the SERVQUAL mean score for each factor:
The largest discrepancy between a student’s desired level of service and the service they actually perceive is under the Staff factor (-.77), followed closely by the Services factor (-.75), with the most favorable results appearing under the Professionalism factor. Under the Professionalism factor, there is only an average difference of -.044 between a student’s desired and perceived service level.
Below are a few recommendations we feel will improve these scores:
§ Provide additional workstations and encourage staff do be proactive in directing students to open workstations and/or other computer labs during busy times.
§ Improve computer response time during log-in or application loading.
§ Increase the variety of software and special equipment available in the computer labs and increase the reliability of the software and hardware in operating smoothly.
§ Increase the response time to problems and/or suggestions submitted by students.
This report goes into additional detail about the SERVQUAL instrument and the factors utilized in this survey; as well as providing additional detail and analyzed results of all three parts of the survey.
Introduction
There are currently 5 computer labs on campus available for student use, with one of those labs (MSC 111) being available 24 hours a day Monday through Thursday. In addition, there are over 100 applications available in the labs. Each lab is equipped with scanner stations, black and white laser printing, headphones and microphones. Color laser printing is also available for an additional fee. Lab assistants are always on duty to answer questions and are trained on a variety of software applications.
With the constant innovations and advances in technology, students’ technology preferences and needs are frequently changing. The I.T. survey was originally developed in 1997 as a way to gage those student needs and to provide I.T. with a platform upon which to make changes and improvements each year. The survey measures the performance of the various services being provided by I.T.; it provides information on student demographics and growing trends; and it seeks feedback from students on what new products and services they would like to have available at ASU. This year, additional questions were introduced to the survey to obtain some initial feedback pertaining to a new program ASU is hopeful to implement in the near future. This program would require that all students attending ASU have a laptop. Professors would actively promote the use of these laptops in classroom activities, and the hope is that students would consistently use the laptops for homework, classroom work, etc.
Survey Distribution, Design and Methodology
During the spring 2010 semester, surveys were distributed to randomly selected classes between March 22nd and April 9th. The majority of surveys completed were paper-based surveys passed out in classes. Professor approval was obtained prior to distribution of the surveys. A number of surveys were also taken by students directly online. A link was provided on the desktop of every computer on campus inviting students to participate in the survey online and give their opinion. A total of 513 surveys were completed with 441 of those being paper-based and 72 taken online. Of those 513 surveys, 401 were valid and utilized for this report. Surveys were determined invalid if they were less than 70% complete, or if students’ answers were biased (i.e. questions were all answered in the same direction such as all 9’s, all 1’s, etc.).
The survey itself consists of three parts:
Part I uses the SERVQUAL instrument developed by Zeithaml, Parasuraman, and Berry. It consists of twenty-four questions scaled from 1 (low) – 9 (high) scale. Each question has two scales. One measures the students’ desired service level and the other one measures their perceived service level. The SERVQUAL methodology seeks to determine a mathematical difference between a desired level of service and the level of service actually perceived. The initial SERVQUAL research yielded seven dimensions, but this was later scaled down to five dimensions to maintain distinction. The five distinct SERVQUAL dimensions are:
§ TANGIBLES
Physical facilities, equipment, and appearance of personnel.
§ RELIABILITY
Ability to perform the promised service dependably and accurately.
§ RESPONSIVENESS
Willingness to help customers and provide prompt service.
§ ASSURANCE
Knowledge and courtesy of employees and their ability to inspire trust and confidence.
§ EMPATHY
§
§
Caring, individualized attention the firm provides its customers.
Due to the limited number of surveys typically returned for the I.T. survey, it is not possible to maintain all five SERVQUAL dimensions; therefore, the five dimensions have been combined into three. The three ASU survey service dimensions utilized, and the questions from the survey included in each dimension are as follows:
Factor 1 focuses on STAFF and consists of questions concerning the following dimensions:
§ Assurance - Knowledge and courtesy of employees and their ability to inspire trust and confidence
§ Empathy – Providing care and individualized attention
§ Responsiveness - Willingness to help customers and provide prompt service
The following questions from the I.T. survey were included under this factor:
Q2 - The staff’s ability to instill confidence in me to use the computer lab software and hardware
Q4 - Computer lab staff who deal with me in a caring fashion
Q5 – Receiving a prompt response to my computer lab problems and/or suggestions
Q6 - Computer lab staff who have my best interest at heart
Q7 – Computer lab staff who understand my computing needs
Q8 – Computer lab staff who give me personal attention
Q15 - Being directed by staff to an open workstation at another computer lab if the first computer lab is full
Factor 2 focuses on SERVICES and consists of questions concerning the following:
§ Tangibles – Appearance of physical facilities, equipment, and personnel
The following questions from the I.T. survey were included under this factor:
Q16- Convenience of normal operating hours to personal schedule
Q18- Computer response time during log-in or application loading
Q20- Relying on the computer lab software and hardware to operate smoothly
Q21- The computer labs containing state-of-the-art computers and peripherals
Q22-The variety of software available in the computer lab
Q23-The variety of special equipment (i.e. scanners and color printers) available in the computer labs
Q24- Disk space provided to me on the server (my P: drive) is adequate for my data storage
Factor 3 focuses on PROFESSIONALISM and consists of questions concerning the following:
§ Reliability - Ability to perform the promised service dependably and accurately
The following questions from the I.T. survey were included under this factor:
Q3- Staff who has the knowledge to answer my questions about computer lab policies, operating hours, software, and hardware
Q9- Staff who have the knowledge to answer my questions about software and hardware.
Q11- Services provided at times listed
Q12- A visually appealing computer lab (i.e. neat, clean, and organized)
Q17- Computer lab staff who are dressed appropriately for their position
Q19- Counting on the computer labs to have the software and hardware I need to complete my assignments
The following questions were not utilized due to having a double loading factor:
Q10 – Providing computing services as promised in the ASU catalog, computer lab brochure, or web page
Q1 – The computer lab staff’s willingness to help me
Q14 – Ability to find an available workstation in one of the six labs on campus
Q13 – Computer lab staff can be distinguished from other students and are easy to identify
The variables were grouped in SPSS in the following manner to make up each factor:
Factor 1: A4+A8+A10+A12+A14+A16+A28
Factor 2: A32+A36+A40+A42+A44+A46+A48
Factor 3: A6+A18+A22+A24+A34+A38
After determining which questions should be assigned to a particular factor, the following
model was used to develop a score for each question:
Q = PERCEIVED – DESIRED
The scores for the questions within each factor were then added together. Finally, that
score was divided by the total number of questions that made up a particular
factor to come up with a mean score. The mean score was each factor was calculated in
Microsoft Excel as follows:
Factor 1 score = (Q2+Q4+Q5+Q6+Q7+Q8+Q15)/7 = -.7691
Factor 2 score = (Q16+Q18+Q20+Q21+Q22+Q23+Q24)/7 = -.7945
Factor 3 score = (Q19,Q11,Q12,Q3,Q17,Q9)/ 6 -.4372
We have also compared the mean scores calculated in 2010 against the scores calculated in the five previous years. A summary of those figures is shown below:
It should be noted that all three factors have been improved as compared to their scores last year.
To ensure internal reliability of the scale, a Cronbach Alpha was run on each of the three factors individually as well as running all three factors against Q25 (What is your OVERALL satisfaction level with ASU computer labs?). Cronbach’s Alpha will generally increase as the intercorrelations among variables increase, therefore, a larger number typically indicates greater internal consistency. A reliability of .70 or greater is required to guarantee internal consistency. Following are the Conbach Alpha scores we have calculated. Score T is the score for running all three factors against Q25.
Since all calculations are greater that .70 we can conclude sufficient internal consistency and reliability of this scale.
Statistical Analysis
T-TEST
A t-test is typically run in order to make inferences about the mean of a population. It will help determine if the means between two variables differ. Our null hypothesis is that there is no difference in the means of the two variables we are comparing. Our alternative hypothesis is that there is a difference in the means. We have selected a significance level of .05 to run our analysis and are assuming normal distribution. It is important to note the types of data that can be included in a t-test. The dependent variables must be interval or ratio scale (in business, categorical data is treated as an interval scale, therefore for purposes of this report may be included in this test). The independent variables must be nominal or two level categorical data. There are three questions in the I.T. survey that can be utilized as independent variables in the t-test and for which we can compare means:
Q26 – Do you own a personal computer or laptop?
Q58 – Gender
Q62 – Student lives on or off campus
These three questions were first run against the three SERVQUAL factors. We calculated the difference between the perceived and desired values for each question for each respondent. We then added that number together for all the questions within a given factor. We also ran our independent variables against Q41, Q42-54 (“satisfied” liker scale questions only) and Q55. We have listed all significant results below. Results were significant if the T Significance value was less than .05. It’s important to note that if the F Significance was > .05, we looked at equal variances; but if it was < .05 we looked at unequal variances. For purposes of this report, a t-test only needs to be run against the three SERVQUAL factors and not the individual SERVQUAL questions, but for information purposes, we did run it against each of the SERVQUAL scores for individual questions and have put ** by those results. The larger the SERVQUAL score, the larger the gap between a student’s desired level or service and their actual perceived level.
OWN/DON’T OWN A COMPUTER OR LAPTOP (Q26)
# / Question / F Sig. / t / Sig. (2-tailed)**Q5 / Receiving a prompt response to my computer lab problems/suggestions / 0.825 / 2.077 / 0.038
Q5 – students who did own a person computer felt they received a more prompt response to their problems/suggestions (-.8333 mean SVQL score) than students who did not own a personal computer (.7710 mean SVQL score). This is likely due to those students not owning computers relying more heavily on computer lab computers to get their work done, and therefore needing problems/suggestions handled more quickly than others.