2007PRIORITIES SURVEY FOR ONLINE LEARNERS

SUMMARY OF RESULTS

CONCORDIAUNIVERSITY, ST. PAUL

Introductory Comments

The Noel-Levitz Priorities Survey for Online Learners (PSOL) was completed by students in Concordia’s online cohort programs in August and September of 2007. Student invitations to participate were sentvia email, and the survey was available for online completion only. Attempts were made to contact 461 students. 225 students completed all or some of the survey for a response rate of 49%. A comprehensive institutional report of results was received from Noel-Levitz in September 2007.

The same survey was taken by a sample of Concordia’s online cohort students in November and December of 2005. The report that follows provides information on strengths and challenges, comparisons of 2005 and 2007 results, and comparisons of 2007 results with a national group of students taking the PSOL.

PSOL Executive Summary of Findings

Institutional strengths and challenges related to specific survey items are identified in the Noel-Levitz report. Strengths are items of both high importance and high satisfaction. Challenges are items of both high importance and low satisfaction. 2005 and 2007 findings are quite similar.

The following items are identified as institutional strengths in both 2005 and 2007 reports:Participation in this program enhances my job effectiveness, Faculty care about my learning, and Faculty effectively challenge me to engage course content. New strengths identified in 2007 are Faculty are responsive to my needs and Working with the same cohort throughout the program enhances my learning. These items were identified as strengths in 2005 and are no longer listed as such in 2007:Adequate online library resources are provided (although still listed as having significantly higher CSP satisfaction compared to national group), Program residencies effectively start students’ learning experiences, Billing and payment procedures are convenient to me, and Faculty create a learning environment that effectively engages me to learn from other students.

The following items are identified as institutional challenges in both 2005 and 2007 Noel-Levitz reports:The quality of feedback from instructors contributes effectively to my learning, Student assignments are clearly defined in the syllabus, Program assignments are clear and reasonable, Tuition paid is a worthwhile investment, and Faculty provide timely feedback about student progress. The institution responds quickly when I request information is listed as a new challenge in 2007, however, CSP satisfaction on this item is significantly higher than for the national comparison group. The quality of online instruction is excellent was listed as a challenge in 2005 but is no longer identified as such in the 2007 report.

2007 CSP comparative results remain quite positive. As in 2005, CSP results are generally better in comparison to a national group of PSOL respondents. The national group is comprised of online learners from a wide range of colleges and universities, including community colleges. 2007 CSP results are very similar to or slightly weaker than results in 2005.

As they were in 2005, 2007 CSP results related to overall satisfaction with the institution, on scales (groupings of similar items), and on individual national items are generally significantly higher in terms of satisfaction than for the national comparison group. 2007 CSP satisfaction levels are significantly higher than national averages on 4 of the 5 national item scales (groupings of similar items) and significantly higher on 17 of the 26 individual national items (including 8 benchmarks – both high CSP importance and high satisfaction).

On the 3 items related to overall satisfaction with the institution, the 2005 and 2007 results on So far, how has your college experience met your expectations and Rate your overall satisfaction with your experience here thus farare similar and a little better in 2005 on All in all, if you had to do it over, would you enroll here again.

On the 5 over-arching scales, performance gaps in 2005 and 2007 are similar in Institutional Perceptions, Academic Services, and Enrollment Services. The gap in Instructional Services is a little smaller (better result) in Academic Services in 2007 and a little smaller in 2005 in Student Services.

2005 and 2007 performance gaps are similar on 5 of the 10 institutionally developed items and smaller in 2007 on 3 other items. As in 2005, the items with the smallest performance gaps in 2007 are on Working with the same student cohort throughout the program enhances my learning and Program residencies effectively start students' learning experiences and the largest on The quality of feedback about my work from instructors contributes effectively to my learning (although improved in 2007) and on The balance between program pace and quality meets my educational goals.

Results differed substantially in 2005 on many items when comparing CSP respondents in bachelor’s programs with those in master’s programs. The response averages of these two groups are much closer in 2007. The 2007 CSP bachelor’s average satisfaction level is a little higherthan the 2007 master’s average on Rate your overall satisfaction with your experience here thus far. 2007 subgroup results are similar on all five scales. On the institutional items, the largest response difference between 2007 subgroups is on The balance between program pace and quality meets my educational goals, with the bachelor’s group having a higher satisfaction level and smaller performance gap. Results were a little better in 2005 for students in bachelor’s programs than they are for the 2007 bachelor’s group on the overall experience items and results are mixed comparing 2005 and 2007 master’s respondents. On the scales, performances gaps are smaller for bachelor’s students in 2007 than in 2005. Master’s gaps in 2005 were somewhat better in 2005 than in 2007. Similarly, results are a little better in 2007 than in 2005 for bachelor’s students on most institutional items and generally somewhat better for master’s students in 2005.

A sample of students in face-to-face cohorts completed the Adult Learner Inventory during the same time frame that cohort students completed the PSOL. On the 7 institutionally developed items common to both surveys, the 2005 and 2007 online learner satisfaction levels are higher and the performance gaps smaller.

Demographic Information

The PSOL contains a number of demographic items. Concordia responses on a few items related to programs are summarized below. 2005 and 2007 breakdowns are similar on other characteristics like gender, age, and race/ethnicity. Somewhat different M.A./B.A. profiles are indicated in the first table. 56% of 2007 respondents are in programs in the College of Business and Organizational Leadership and 43% in the College of Education.

2005 / 2007
Educational goal / Number of respondents / Percentage of respondents / Number of respondents / Percentage of respondents
Bachelor’s degree / 74 / 37.4% / 129 / 57.3%
Master’s degree / 108 / 54.5% / 86 / 38.2%
Other goal / 14 / 7.1% / 8 / 3.6%
No answer / 2 / 1.0% / 2 / 0.9%
2005 / 2007
Current program / Number of respondents / Percentage of respondents / Number of respondents / Percentage of respondents
M.A. in Christian Outreach / 22 / 11.1%
M.A. in Education, Early Childhood Emphasis / 35 / 17.7% / 30 / 13.3%
M.A. in Education, Differentiated Emphasis / 6 / 2.7%
2005 / 2007
Current program / Number of respondents / Percentage of respondents / Number of respondents / Percentage of respondents
M.A. in Human Services, Family Life Education Emphasis / 20 / 10.1% / 8 / 3.6%
M.A. in Human Services, Criminal Justice Leadership Emphasis / 18 / 9.1% / 11 / 4.9%
M.A. in Organizational Management / 8 / 4.0% / 10 / 4.4%
M.A. in Organizational Management, Human Resources Emphasis / 3 / 1.3%
Master of Business Administration / 7 / 3.1%
B.A. in Child Development / 31 / 15.7% / 37 / 16.4%
B.A. in Family Life Education / 6 / 3.0% / 16 / 7.1%
B.A. in Criminal Justice / 13 / 6.6% / 29 / 12.9%
B.A. in Human Resource Management / 4 / 2.0% / 8 / 3.6%
B.A. in Innovation and Marketing Management / 11 / 5.6% / 15 / 6.7%
B.A. in Organizational Management and Leadership (Communication) / 22 / 11.1% / 33 / 14.7%
B.A. in Information Technology Management / 10 / 4.4%
Associate of Arts program / 3 / 1.5%
No answer / 3 / 1.5% / 2 / 0.9%

Survey Structure

On 26 national PSOL items and 10 institutionally supplied items,respondents are asked to identify the degree to which that factor/quality/experience/service is important to them and then also the degree to which they are satisfied with it. On those items, Noel-Levitz calculates performance gaps (average importance level minus average satisfaction level). Smaller gaps indicate better performance. Only importance scores are solicited on 18additional items mainly focusing on enrollment factors. Only satisfaction scores are requested on 3 items related to the overall university experience. Similar national importance-satisfaction items are also grouped into 5 scales: Institutional Perceptions, Academic Services, Instructional Services, Enrollment Services, and Student Services.

Strategic Planning Overview

The Noel-Levitz report includes a strategic planning section. The following definitions are used.

Strengths are items with high Concordia importance overall and also high Concordia satisfaction overall. More specifically, strengths are items both above the midpoint in importance and also in the upper quartile (25%) of satisfaction scores.

Challenges are items with high Concordia importance overall and also low Concordia satisfaction overall.

They are items above the midpoint in importance and also in the lower quartile in satisfaction or the top quartile of performance gap scores.

Benchmarks are items of relatively high Concordia importance for which Concordia scores are statistically significantly higher than or lower than the national average in either satisfaction or importance.

There are no items on which the Concordia satisfaction level issignificantly lower than the national average.

2005 at the end of an item indicates that the item was listed in the same category (as a strength, challenge, or benchmark) in the 2005 Noel-Levitz report. The items not so marked are new strengths, challenges, or benchmarks in 2007. The large number of items in the same category both years indicates that results are generally consistent. Items identified as strengths in 2005 that are no longer listed as strengths in 2007 are Adequate online library resources are provided (but still listed as a high satisfaction benchmark), Program residencies effectively start students’ learning experiences, Billing and payment procedures are convenient to me, and Faculty create a learning environment that effectively engages me to learn from other students. The only new 2007 challenge is The institution responds quickly when I request information. This item is, however, also listed as a positive satisfaction benchmark, indicating that it is a challenge for many institutions.

(Items in each section are listed in descending order of average importance to CSP respondents.)

2007 Strengths and Challenges
Strengths
Faculty are responsive to student needs.
Participation in this academic program enhances my job effectiveness. (Campus item) – 2005
Faculty care about my learning. (Campus item) – 2005
Working with the same cohort throughout the program enhances my learning. (Campus item)
Faculty effectively challenge me to engage course content. (Campus item) – 2005
Challenges
The quality of feedback about my work from instructors contributes effectively to my learning. (Campus item) – 2005
Student assignments are clearly defined in the syllabus. – 2005
Program requirements are clear and reasonable. – 2005
Tuition paid is a worthwhile investment. – 2005
Faculty provide timely feedback about student progress. – 2005
This institution responds quickly when I request information.
2007 Benchmarks
Higher Satisfaction vs. National Online Learners
The quality of online instruction is excellent. – 2005
Faculty are responsive to student needs. – 2005
Student assignments are clearly defined in the syllabus.
Program requirements are clear and reasonable.
Instructional materials are appropriate for program content. – 2005
2007 Benchmarks
Higher Satisfaction vs. National Online Learners
Adequate online library resources are provided. – 2005
This institution responds quickly when I request information. – 2005
There are sufficient offerings within my program of study.

Comparative Results on Overall Experience

Statistical significance tests were conducted only in comparisons of Concordia’s overall average satisfaction levels with those in the national comparison group. Concordia satisfaction levels are statistically significantly higher than the national comparison group on all three of the items. The national comparison group is comprised of students completing the PSOL as of May 2007, including students in community and technical colleges. The pattern seen in the first table of 2007 CSP results significantly higher than the national group and similar to or slightly lower than 2005 CSP results is pretty typical of results on most PSOL items. On these items related to overall satisfaction with the institution, the 2005 and 2007 results are similar; about the same on So far, how has your college experience met your expectations and Rate your overall satisfaction with your experience here thus far and slightly better in 2005 on All in all, if you had to do it over, would you enroll here again.

Since the master’s vs. bachelor’s percentages on Concordia’s 2005 and 2007 samples are quite different, results for those CSP subgroups are also included in the tables that follow. No comparisons with national subgroups are available. Results were a little better in 2005 for bachelor’s students than they are in 2007. 2005 and 2007 master’s results are similar.

Average scores are provided. (7 = much better than expected, very satisfied, or definitely yes to 1 = much worse than expected, not satisfied at all, or definitely not)

2005 / 2007
Item / CSP
Overall / CSP
Bachelor’s / CSP
Master’s / CSP
Overall / CSP
Bachelor’s / CSP
Master’s / National
So far, how has your college experience met your expectations? / 5.55 / 5.68 / 5.50 / 5.56 / 5.58 / 5.55 / 5.07
Rate your overall satisfaction with your experience here thus far. / 6.24 / 6.39 / 6.18 / 6.21 / 6.26 / 6.10 / 5.78
All in all, if you had to do it over, would you enroll here again? / 6.37 / 6.57 / 6.29 / 6.30 / 6.32 / 6.29 / 5.93

Comparative Results on Scales

Survey results on the 26 national items are grouped into 5 over-arching scales in the Noel-Levitz analysis. Average satisfaction levels of 2007 Concordia respondents are significantly higher than the national scores across all scales except Enrollment Service(a little higher on this one). A performance gap is calculated by subtracting average satisfaction from average importance. A smaller performance gap indicates better perceived institutional performance in that area.

Concordia average levels of satisfaction are statistically significantly higher than national averages on 4 of the 5 scales and on 17 of the 26 individual national items comprising the scales and significantly lower on no items. 2007 CSP performance gaps are better than the national gaps and very consistent with 2005 results. Performance gaps in 2005 and 2007 are similar in Institutional Perceptions, Academic Services, and Enrollment Services. The gap in Instructional Services is a little smaller (better) in Academic Services in 2007 and a little smaller in 2005 in Student Services. A complete listing of overall results on national importance/satisfaction items is in Appendix A and bachelor’s/master’s results are in Appendix B. 2007 bachelor’s and master’s results are similar on all scales. Performances gaps are smaller for bachelor’s students in 2007 than in 2005. Master’s gaps in 2005 were somewhat better in 2005 than in 2007.

Averages are provided. (7 = very important or very satisfied to 1 = not important at all or not satisfied at all)

2005 / 2007
Scale / Response Category / CSP
Overall / CSP
Bachelor’s / CSP
Master’s / CSP
Overall / CSP
Bachelor’s / CSP
Master’s / National
Institutional Perceptions / Importance / 6.50 / 6.59 / 6.48 / 6.46 / 6.45 / 6.48 / 6.45
Satisfaction / 6.05 / 6.03 / 6.10 / 6.04 / 6.05 / 6.04 / 5.72
Performance Gap / 0.45 / 0.56 / 0.38 / 0.42 / 0.40 / 0.44 / 0.73
Academic Services / Importance / 6.28 / 6.47 / 6.19 / 6.24 / 6.23 / 6.25 / 6.30
Satisfaction / 5.93 / 6.01 / 5.92 / 5.87 / 5.86 / 5.90 / 5.63
Performance Gap / 0.35 / 0.46 / 0.27 / 0.37 / 0.37 / 0.35 / 0.67
Instructional Services / Importance / 6.51 / 6.63 / 6.48 / 6.48 / 6.44 / 6.53 / 6.39
Satisfaction / 6.05 / 6.08 / 6.09 / 6.08 / 6.08 / 6.11 / 5.71
Performance Gap / 0.46 / 0.55 / 0.39 / 0.40 / 0.36 / 0.42 / 0.68
Enrollment Services / Importance / 6.43 / 6.63 / 6.31 / 6.29 / 6.28 / 6.30 / 6.42
Satisfaction / 6.09 / 6.01 / 6.15 / 5.92 / 5.94 / 5.95 / 5.83
Performance Gap / 0.34 / 0.62 / 0.16 / 0.37 / 0.34 / 0.35 / 0.59
Student Services / Importance / 6.24 / 6.49 / 6.10 / 6.23 / 6.23 / 6.24 / 6.31
Satisfaction / 5.92 / 5.98 / 5.90 / 5.83 / 5.84 / 5.89 / 5.61
Performance Gap / 0.32 / 0.51 / 0.20 / 0.40 / 0.39 / 0.35 / 0.70

Institutional Item Comparisons

2005 and 2007 performance gaps are similar on 5 of the 10 institutionally developed items and smaller (better) in 2007 on 3 other items. As in 2005, the items with the smallest performance gaps in 2007 are on Working with the same student cohort throughout the program enhances my learning and Program residencies effectively start students' learning experiences and the largest on The quality of feedback about my work from instructors contributes effectively to my learning (although improved in 2007) and on The balance between program pace and quality meets my educational goals. The largest response difference between 2007 bachelor’s and master’s groups is on The balance between program pace and quality meets my educational goals, with the bachelor’s group having a higher satisfaction level and smaller performance gap. 2007 performance gaps are smaller for the bachelor’s group in 2007 on 5 items and better in 2005 on 2 items. Master’s gap results were better in 2005 on 8 of the 10 items.

Cell entries and the range of possible scores follow the same pattern as in the previous tables.

2005 / 2007
Item / Response Category / CSP
Overall / CSP
Bachelor’s / CSP
Master’s / CSP
Overall / CSP
Bachelor’s / CSP
Master’s
Participation in this academic program enhances my job effectiveness. / Importance / 6.69 / 6.74 / 6.63 / 6.63 / 6.56 / 6.71
Satisfaction / 6.36 / 6.44 / 6.33 / 6.35 / 6.37 / 6.33
Performance Gap / 0.33 / 0.30 / 0.30 / 0.28 / 0.19 / 0.38
The quality of feedback about my work from instructors contributes effectively to my learning. / Importance / 6.76 / 6.77 / 6.75 / 6.72 / 6.68 / 6.78
Satisfaction / 5.94 / 6.04 / 5.99 / 6.08 / 6.03 / 6.15
Performance Gap / 0.82 / 0.73 / 0.76 / 0.64 / 0.65 / 0.63
Working with the same student cohort throughout the program enhances my learning. / Importance / 6.46 / 6.50 / 6.53 / 6.50 / 6.49 / 6.50
Satisfaction / 6.51 / 6.55 / 6.57 / 6.45 / 6.47 / 6.43
Performance Gap / -0.05 / -0.05 / -0.04 / 0.05 / 0.02 / 0.07
The balance between program pace and quality meets my educational goals. / Importance / 6.66 / 6.76 / 6.61 / 6.65 / 6.63 / 6.66
Satisfaction / 6.18 / 6.31 / 6.11 / 6.15 / 6.25 / 6.00
Performance Gap / 0.48 / 0.45 / 0.50 / 0.50 / 0.38 / 0.66
Faculty create a learning environment that effectively engages me to learn from other students. / Importance / 6.52 / 6.54 / 6.51 / 6.49 / 6.47 / 6.52
Satisfaction / 6.29 / 6.42 / 6.25 / 6.20 / 6.19 / 6.20
Performance Gap / 0.23 / 0.12 / 0.26 / 0.29 / 0.28 / 0.32
Faculty care about my learning. / Importance / 6.55 / 6.64 / 6.49 / 6.63 / 6.61 / 6.66
Satisfaction / 6.24 / 6.32 / 6.20 / 6.32 / 6.30 / 6.31
Performance Gap / 0.31 / 0.32 / 0.29 / 0.31 / 0.31 / 0.35
Faculty effectively challenge me to engage course content. / Importance / 6.55 / 6.53 / 6.56 / 6.50 / 6.47 / 6.54
Satisfaction / 6.32 / 6.44 / 6.26 / 6.24 / 6.23 / 6.20
Performance Gap / 0.23 / 0.09 / 0.30 / 0.26 / 0.24 / 0.34
2005 / 2007
Item / Response Category / CSP
Overall / CSP
Bachelor’s / CSP
Master’s / CSP
Overall / CSP
Bachelor’s / CSP
Master’s
Program residencies effectively start students' learning experiences. / Importance / 6.55 / 6.61 / 6.58 / 6.33 / 6.30 / 6.35
Satisfaction / 6.39 / 6.49 / 6.37 / 6.26 / 6.25 / 6.27
Performance Gap / 0.16 / 0.12 / 0.21 / 0.07 / 0.05 / 0.08
Bulletin boards effectively promote my learning. / Importance / 6.27 / 6.45 / 6.26 / 6.37 / 6.34 / 6.41
Satisfaction / 6.04 / 6.12 / 6.03 / 6.15 / 6.22 / 6.04
Performance Gap / 0.23 / 0.33 / 0.23 / 0.22 / 0.12 / 0.37
Chats effectively promote my learning. / Importance / 6.44 / 6.55 / 6.41 / 6.44 / 6.44 / 6.44
Satisfaction / 6.09 / 6.27 / 6.06 / 6.09 / 6.13 / 6.02
Performance Gap / 0.35 / 0.28 / 0.35 / 0.35 / 0.31 / 0.42

ALI Comparison

A sample of students in face-to-face cohorts completed the Adult Learner Inventory during the same time frame that cohort students completed the PSOL. On the first 7 institutionally developed items above common to both surveys, the 2005 and 2007 online learner satisfaction levels are consistently higher and the performance gaps smaller. Much smaller response rates on the ALI (26% in both years) may be a mitigating factor.

Sources of Information and Enrollment Factors

Respondents were asked to rate the importance of various sources of information about the institution and enrollment factors. 2007 responses are similar to 2005, adjusting for the different B.A./M.A. profiles. Not surprisingly for online students, the most important source of information about the institution is the university website. The enrollment factors with highest average importance for Concordia respondents are Work schedule, Convenience, Program requirements, Reputation of Institution, and Flexible pacing for completing a program.