Open Source vs. Proprietary Software for Regional Public Universities: A Study of Knowledge Transfer
Bradley Moore, Wayne Bedford, and James Todd
Abstract
This is a preliminary study of the effectiveness of knowledge transfer when using Open Office to teach office applications such as word processing, electronic spreadsheets, and presentations in stead of Microsoft Office. Two sections of a microcomputer applications course were evaluated with performance tests – one with Microsoft Office 2003 and the other with Open Office (v2.02). Simple t-tests were used to determine whether differences in test means exist. The results can be used as part of the criteria to determine if a university should adopt open-source software.
1
Open Source vs. Proprietary Software for Regional Public Universities: A Study of Knowledge Transfer
Open-source software is often seen as a low cost alternative to the use of proprietary software by businesses, schools and other organizations that frequently find resources scarce, particularly when operating in the public sector(Duderstadt, 2002). However, other factors such as training cost, file compatibility, and support availability should also be considered before adopting open-source solutions(Alfonsi, 2005; O’Hara & Kay, 2003). Schools and universities that use proprietary software for instructional purposes also should consider how well the open source software provides the necessary knowledge level for the students. Since Microsoft Office currently has 95% of the office suite market (Hamm, 2006), it is important that the level knowledge transfer between open source and proprietary software is evaluated. This paper presents a study of knowledge transfer conducted at a regional university during the fall 2006 semester.
Study Design
Thisstudyis based on a previous proposal (Bedford, Moore, & Todd, 2006)to examine the knowledge transfer that may occur when students are taught application suites (word processing, electronic spreadsheets, etc.) using open source products in lieu of proprietary products like Microsoft Office. The study was initiatedusingtwo sections of afreshman microcomputer applications course which is required for all students with a declared major in business and an elective for other students. The make-up of any classis generally varied and may contain students of all statuses (freshman to senior) and from all colleges,therefore, providing a good representation of the university population. Section one consisted of 28 students while section two had 27. Both sections met in the newest state-of-the-art computer lab with all systems configured with both Microsoft Office 2003and Open Office Suite (v2.02).
The course is presented as part lecture, part self-taught tutorial from the text, and part laboratory with comprehensive case assignments. Both sections used the same text and associated assignments, but section two received lectures and completed the assignments with Open-Office rather than Microsoft Office 2003. Each component application (Word, Excel, etc.) is presented and a skills test is given to determine the level of knowledge and skill the student has attained. The student may use the text and any notes taken during the lectures.
The first day of class, both sectionswere informed that they would be participating in a pilot test that would compare the use of Open Office Suite (v2.02)with Microsoft Office 2003 Suite. At the completion of this first class period, the students were asked to complete a survey that providesa self-reported evaluation of the student’s basic skill level and background knowledge and allows for some comparison by section.
The results of individual application tests and the final exam will be evaluated to determine whether differences exist in the degree of skill attained as represented in the following hypotheses:
H1:There will be no difference in test scores between students using Microsoft Word 2003 and students using Open Writer.
H2:There will be no difference in test scores between students using Microsoft Excel 2003 and students using Open Calc.
H3:There will be no difference in test scores between students using Microsoft PowerPoint 2003 and students using Open Impres.
H4:There will be no difference in overall test scores between student using Microsoft Office 2003 and Open Office.
Survey
The survey instrument (appendix) provides the instructor with a self-reported evaluation of the student’s experience with computers, operating systems and applications, particularly Microsoft. It also allows the students to indicate their interest in the course and what they expect to gain. Of particular interest to this study are the responses to questions 2, 3, and 4 (table 1), since they relate to prior experience. The students were allowed to select more than one response to each question which may explain the high comfort and in-home usage levels in section 2; however, the reported usage of Microsoft Office is consistent in both sections and illustrates its high usage. Previous usage of Microsoft Office may facilitate the knowledge transfer.
Table 1 – Survey ResultQuestion / Section 1 / Section 2 / Total
2 – Comfort / 71.4% / 96.3% / 83.6%
3 – PC at home / 57.1% / 88.9% / 72.7%
4 – MS Office / 60.7% / 70.3% / 65.5%
Discussion
Each section was given the same classroom lecture and tests. As for the lectures, the only difference was the actual application that was being covered, i.e. – MS Word vs. OO Writer, MS Excel vs. OO Calc, or MS PowerPoint vs. OO Impress. Probably the most frustrating aspect of this was the text. The text presented MS Office and contained extensive graphics for many of the steps the student was required to accomplish. This was not available for students in section 2 since they were required to use Open Office. For this reason, the classroom lecture for section 2 had to be modified to contain similar graphics during the lecture to point out what the student should be seeing when accomplishing the same steps that their counterparts accomplished during the similar step. This forced the student to take more notes or recall from memory what they were to do during a particular step in their assignments.
The Word/Writer skills test required the students to open a document from a network drive and reset the margins, add a title page with specific information and then make changes throughout the rest of the multiple-page document. They also were to add a header and footer, convert two questions to bulleted items, and take several “references” and place them in alphabetical order.
The Excel/Calc test proved to be a major hurdle for both sections. It consisted of one of the cases contained in the text with the addition requirement to create an x/y scatter chart with a trend line. This case was very similar to a tutorial that the students had accomplished as an in-class laboratory assignment and two other homework assignments.
Although a very short instructional portion of the course, presentation application software (PowerPoint/Impress) seems to be an enjoyable part of the class. The test required the students to create a presentation from an existing outline, modify fonts on the master slide, add appropriate transitions and animations, and add a summary slide.
The final exam is considered comprehensive due to the nature of the exam: covering Word/Writer, Excel/Calc, PowerPoint/Impress, and Access. Students were required to integrate the applications learned throughout the entire semester. For example, linking spreadsheet data to a word processing document so that the spreadsheet data can change as it is updated and incorporate data found in a word processing document into a presentation application.
Results
Results for the tests in each of the two sections were recorded and evaluated for differences. Due to the small sample size, simple t-tests were used to compare the means of the samples. Each test was administered independently, so more complex methods of evaluation were unnecessary. Table 2 presents the results of the comparisons. Excel’s data analysis tool pack was used to perform all calculations with a level of significance (α) of 0.05. The number of observations for each test is less than the number of registered students due to no shows who received a score of zero. These scores were deleted from the data since these students chose not to participate in the test. The t-tests clearly indicate that, for this data, there are no significant differences between the section averages.
Table 2. Comparison of DifferencesWord / Writer / Excel / Calc / PPT / Imp / Off2003 / OpenOff
Mean / 76.5 / 77.1 / 75.1 / 76.9 / 85.9 / 86.1 / 75.4 / 77.9
Variance / 48.9 / 56.3 / 129.2 / 164.2 / 51.9 / 85.2 / 241.1 / 187.7
Observations / 24 / 24 / 17 / 19 / 23 / 24 / 24 / 23
Pooled Variance / 52.60 / 147.75 / 68.92 / 215.00
Hypothesized Mean Difference / 0 / 0 / 0 / 0
df / 46 / 34 / 45 / 45
t Stat / -0.2985 / -0.4379 / 0.1054 / -0.5784
P(T<=t) one-tail / 0.3833 / 0.3321 / 0.4582 / 0.2829
t Critical one-tail / 1.6787 / 1.6909 / 1.6794 / 1.6794
P(T<=t) two-tail / 0.7667 / 0.6642 / 0.9165 / 0.5659
t Critical two-tail / 2.0129 / 2.0322 / 2.0141 / 2.0141
alpha=.05
Conclusion
Although limited in scope, this simple study indicates that there is a knowledge transfer present when using open source products to substitute for proprietary products such as Microsoft Office. This indication could make the adoption of open-sourced products more feasible to the cost conscious university considering a switch to more economical software. However, the potential adopter should keep in mind the limitations of this study: small sample size, limited to Open Office vs. Microsoft Office, and a high self-reported experience level with Microsoft Office that could influence the test grades. Further evaluation of knowledge transfer should be considered before a final decision is made.
Preparing students with the necessary skills for today’s work environment should be the primary consideration of all educational institutions. Open-source products are seen as opportunities to reduce some overhead costs, but the transition should be undertaken with care. While cost is important, the contribution to the student’s skill and understanding can not be undervalued. This study should provide another useful evaluation tool.
References
Alfonsi, B. (2005). Open Source in the Classroom. IEEE Distributed Systems Online, 6(6), 4.
Bedford, W., Moore, B., & Todd, J. (2006, May 2007). Proposed Pilot Study to Evaluate Open Source Academic Computing Environment for Regional Public Universities. Paper presented at the Academic Business World, Nashvill, TN.
Duderstadt, J. (2002). The Future of Higher Education in the Knowledge-Driven, Global Economy of the 21st Century. University of Toronto, Canada: University of Toronto.
Hamm, S. (2006, July 3, 2006). More To Life Than The Office BusinessWeek.
O’Hara, K. J., & Kay, J. S. (2003). Open Source Software and Computer Science Education. Journal of Computing Sciences in Colleges, 18(3), 7.
Questionnaire
Expected graduation year:
2006
2007
2008
2009
2010
2011
[1] I have some some experience with:
(check more than one as needed)
a a PC running Windows 3.xx
b a PC running Windows 95 or 98
c a PC running Windows NT or 2000 or XP
d a PC running Windows ME
e Macintosh computers using OSX
f UNIX/Linux
g DOS
[2] When it comes to computers, I would characterize myself as:
(check more than one as needed)
a comfortable and confident
b good enough with things I know - nervous with new things
c largely clueless but willing to try
d hopeless - a total basket case
[3] Before this semester I was using:
(check more than one as needed)
a a personal PC computer at home
b a personal Macintosh/G3/iMac computer at home
c UWA computer labs
d Internet access through UWA
e Internet access through a commercial service provider
f Internet access not described above
[4] Prior to this semester, I was using and felt comfortable with the following software:
(check more than one as needed)
a Netscape Web browser (any version)
b news reader (any version)
c Internet Explorer's Web browser (any version)
d FireFox Web browser
e a free mail service (e.g. Hotmail or NetCenter)
f Eudora mail
g Microsoft Office Suite (Word, Excel, Access, PowerPoint, Outlook)
h Open Office (Writer, Calc, Draw, Impress)
i Corel WordPerfect Office X3
j FTP on a PC
k Fetch on a Mac
l telnet
m a web page construction tool (e.g. FrontPage, Dreamweaver, NVu, Notepad)
[5] Have you ever done any computer programming? If so, please describe which languages and the nature of your background.
[6] Do you have a working computer at home right now? If so, please describe it and any peripherals it has (modem, printer, scanner, zip/jazz drive, etc)
[7] Are you currently trying to get a personal computer connected to the UWA network? If yes, how is that going?
[8] I am interested in this course because I want to be able to:
(check more than one as needed)
a learn to use a computer
b download free software from the net
c use the Internet to find a job someday
d run a small business that exploits the net
e join a virtual community
f understand what the Internet is good for
g I feel like I know my way around the Internet
h have some fun in my spare time
[9] I expect the Internet to become a part of my life because:
(check more than one as needed)
a I like to be where the action is, plus it's fun
b I know online resources will be valuable for my work
c I want access to online resources for my personal benefit
d It just seems to be an unavoidable fixture of the future
e Whoa - I'm not so sure it's going to be a part of my life
[10] I would characterize my attitude toward this course as:
(check more than one as needed)
a excited and enthusiastic
b excited and enthusiastic but a little nervous
c very nervous about learning new computer skills
d concerned about how much time I'll be spending on all this
e confident that this is a valuable time investment
f curious to see if this is a worthwhile time investment
g frankly skeptical about this time investment
1