Annual Report for NSF CRCD Sponsored Grant 0087977
Participants
1. What people have worked on your project?
Faculty PI:
Glenn D. Blank, male, white, USA PI Blank is directing the multimedia content development component of the project and spent approximately 25% of his time on the project.
Faculty co-PI’s:
William M. Pottenger, Ph.D., male, white, USA. Co-PI Pottenger is directing the research component involving the detection of trends in textual data mining, and spent approximately 25% of his time working in this and the associated areas of textual data mining.
G. Drew Kessler, Ph.D., male, white, USA. Co-PI Kessler is directing the research component involving the collaborative interface and infrastructure, and spent approximately 25% of his time working on the project.
Faculty consultants:
Morgan Jennings, Metropolitan State College, Denver, CO, female, white, USA
Debra Dirksen, Metropolitan State College, Denver, CO, female, white, USA
Graduate students – Research Assistants
David R. Gevry, male, white, USA. David’s role in the project is to conduct research in the development of a methodology to detect emerging trends. He was responsible for design and implementation of automated techniques for the emerging trend detection system as well as database design and development.
Jeffrey Heigl, male, white, USA. Jeff’s role in the project is artistic design of user interfaces and multimedia content as well as interfacing with other research assistants in the development of use case scenarios.
Soma Roy, female, Asian-Indian, India. Soma’s role in the project is to conduct research in the development of a methodology to detect emerging trends. She was responsible for the design and implementation of the multimedia based tutorial to detect emerging trends. She has also designed multimedia content including but not limited to constructive exercises in the domain of Object Oriented Software Engineering.
Shreeram Sahasrabudhe, Asian-Indian, India. Shreeram’s role in this project is to redesign the user interface including development of new multimedia content.
Qiang Wang, male, Asian, China.Qiang has developed a framework for collaborative tools using peer-to-peer communication scheme with a “Show Me” interface (Video FAQ) for recording communication seesions. He has also developed an instant messenger and chat collaborative tool.
Undergraduates and graduates – hourly
Dan Darr, male, white, USA. Aided development of Trend Detection Text Mining Infrastructure.
Martin Herr, male, white, USA. Multimedia programming. Martin spent approximately five hours per week on the project and was full-time last summer.
Sumit Jain, male, Asian-Indian, India. Multimedia programming.
Aaron Sherrick, male, white, USA. Designed and implemented the tracking database.
Chris Janneck, male, white, USA. Worked as a narrator.
2. What other organizations have been involved as partners?
The Pennsylvania Infrastructure Technology Alliance (PITA), a collaboration of the Commonwealth of Pennsylvania, LehighUniversity, and CarnegieMellonUniversity, has partnered with us by agreeing to fund three S.T.A.R. (Students That Are Ready) Academy students and one R.A (Research Assistant). Through this partnership, we are able to promote S.T.A.R. (Students That Are Ready) Academy, a year round successful intervention academy to promote academic achievement in STEM for at-risk (economically disadvantaged) middle and high school students in Allentown, Bethlehem, and surrounding communities. We hope to utilize the existing partnerships (since 1989) developed by the S.T.A.R. Academies and will extend the S.T.A.R. Academies into K-12 classrooms to reach a greater number of students and teachers.
3. Have you had other collaborators or contacts?
PI, Glenn Blank wrote the prospectus of a new book “The Universal Computer: Introducing Computer Science with Multimedia” and sent it to two major publishers for review, together with two new chapters for the book and the multimedia material.
Co-PI, Pottenger contacted Dean Catherine Chew at NorthamptonCollege and initiated a collaboration to scale the CIMEL framework to junior college students. He is also working through Professor Mike Berry at UTK (The University of Tennessee Knoxville) to publish a survey of trend detection techniques in the well-known Springler-Verlag series on Computer Science.
Activities and Findings
1. Describe the major research and education activities of the project.
We are developing the beta version ofCIMEL, a multimedia framework for constructive and collaborative, inquiry-based learning for introductory, upper-level and graduate computer science courses. Constructive learning goes beyond learning by receiving knowledge, to learning by building systems, with immediate, visual feedback. Collaborative learning encourages students to interact with instructors and librarians, via both live links and remote-controlled “show me” sessions or by reviewing a multimedia FAQ of recorded “show me” sessions. Inquiry-based learning guides the student into pursuing exploratory research in a community of students and scholars. A reference librarian persona will suggest research topics, then help extract content from both traditional library resources as well as dynamically mined material, answer typical questions and help construct annotated bibliographies, reviews and research proposals. The framework will be an integrated, multi-tracking model, allowing students to select content according to their background, interests and course requirements. Within this general framework, we have alreadydeveloped content for two courses in the Computer Science curriculum, ranging from Introduction to Computing (CS1) at the freshman level to Object-Oriented Software Engineering (OOSE) at the graduate level. In the fall and spring of 2002, we gathered evaluation data for the CS1 and OOSE courses, which our evaluators analyzed.
Since the project began in October 2000, we have developed a prototype and have nearly completed the beta version of the CIMEL framework with OOSE including Abstract Data Types and Programming Languages as our sample content.
Multimedia
For the OOSE course, we developed a multimedia unit on Abstract Data Types (ADTs), as a way to formalize the meaning of classes in connection with object -oriented design. In the past, graduate students have found it difficult to master this material from lecture and textbooks alone. Figure 1 provides a snapshot of the new user interface, showing the content menu on the left, presentation area on the right and button bar on the lower right. The CIMEL framework can be accessed from with user name “guest” and password “guest”. (A new login procedure, which gathers information about new users, is under development and will be in place soon.)
Figure 1: Screen Capture from The Universal Computer (as of March 2002)
Figure 1 illustrates several features of the CIMEL multimedia framework:
- Multimedia personae model a community of learners and instructors. The personae include a professor (shown here), teaching assistant, a reference librarian, and two students. In addition to graphical images, they speak in audio and/or text boxes. These personae model students and teachers studying material together, working through interactive and constructive exercises, and suggest exploratory research on relevant topics using online information.
- A tracks list at left displays the content of a lesson as a sequence of screens. The menu uses check marks to show progress and highlights the current screen in red. An external XML file maintains a menu of screen titles, facilitating maintenance of course content.
- The icons at the bottom give learners access to various tools, including the collaborate and explore (emerging trends) tools under development for this project, as well as the BlueJ and JavaEdit programming environments developed elsewhere.
- The preferences icon presents a panel of options letting the user adapt the environment according to his or her personal learning style, including turning text boxes or audio on/off, toggling auto-advance or wait for next page, setting the timing rate where there is no audio narration, etc. A user may change these settings at any time during these sessions and they will be recorded locally and on a network drive for the next session.
- A just the facts mode lets users switch to viewing non-interactive content (text and graphics) presented in HTML pages. From there, one can switch back to rich media mode via hyperlinks anchored to the corresponding Flash page. There are also links to interactive screens, which remain in Flash.
Interactivity and constructive exercises
Interactivity is a key aspect of CIMEL content. Interactive quizzes and constructive exercises help students learn by doing. Personae provide feedback guiding a student through each exercise. In Figure 2, the TA persona changes expression and provides feedback hinting at what is wrong with a user choice in a multiple-choice question.
Constructive exercises are much more complex, challenging a learner to build solutions to problems by dragging and dropping pieces of structures into place, incrementally. Figure 1 is a snapshot from a sequence of screens, in which a user gradually builds an abstract data type for Apple. In this exercise, the learner goes through a series of screens, incrementally building up an ADT specification for class Apple. The learner has already constructed the signatures section of Apple ADT by first choosing a member function, then in Figure 1 selecting that function’s arguments. Later on in the exercise, the learner builds the preconditions and postconditions for Apple ADT. Finally, the learner runs simulations running the functions, preconditions and postconditions, testing them for completeness. At each step, feedback helps the learner learn from mistakes as well as correct actions.
Inquiry-based exercises facilitate learning by doing research. For example, after studying ADT for collections, the screen shown in Figure 3 asks to the student to investigate how the design of similar ADTs in the most recent JDK. The following screen then asks a follow up question designed to find out what the student has learned from the inquiry-based exercise.
Figure 2: TA persona responds to a wrong choice in a multiple-choice question
Figure 3: The student is given resource links help with a solution to the question.
Inquiry Based Learning through the Detection of Incipient Emerging Trends
One of the goals of the CIMEL multimedia framework is to offer students a mechanism to expand upon the knowledge that is presented in the course-work by exploring with current research trends. This inquiry-based approach provides the students with opportunities to explore the research literature related to the course and gain a better perspective of where technology is headed. To facilitate this goal we have developed an inquiry-based learning module for the CIMEL multimedia framework that guides the student through the process of detecting incipient emerging trends in key topic areas related to the course material. Much like the constructive exercises in the multimedia, this module provides the students an opportunity to pursue course related topics in a more hands-on manner. Through the detection of incipient emerging trends the students see the role that current topics play in course related research areas.
We have completed the first steps in this effort by completing the development and evaluation of a methodology for detecting incipient emerging trends. We discuss the experimental findings for this evaluation later in the results section of this report. Summarized below is the methodology for incipient emerging trend detection that is integrated into the inquiry-based learning module.
Detection of emerging trends starts with the selection of a main topic area such as “inheritance” in OOP. Knowledge in this area is required as the use of domain knowledge at various stages of identification of emerging trends is necessary. The objective is to discover emerging trends in course-related areas. Recent conferences and workshops are searched for discussion on the main topic area giving special attention to workshop websites and technical papers for possible emerging trends (i.e. topics within the domain of the main topic area). A web search engine (e.g. Google, Yahoo, etc.) is used to find additional trends and find further evidence of references to the candidate trends. There are three possible scenarios. One, if any candidate emerging trends were identified from the conference search we perform a web search with the combined query of the candidate trend and a helper term. Two, if the identified candidate trend is a general topic in other areas (areas other than main topic area), a web search is performed with the combined query of the main topic and candidate trend and a helper term (e.g. recent research, novel, emerging trend, etc.). Otherwise, this web search is made using a combined query of the main topic area and any possible helper terms. After choosing one of these queries, the algorithm [4] is followed to identify candidate emerging trends. If the first scenario applies then “main topic area” should be read as “candidate emerging trend” for the algorithm. This algorithm identifies trends as emerging trends. For further verification a database search of research abstracts focused around the main topic area is performed using a newly found candidate emerging trend from the year of origin of the main topic area to the current year. If the frequency of documents referencing the search terms increases over the years, and there is a viable author and venue spread, the candidate emerging trend is confirmed as a bona fide trend with respect to the main topic. The methodology is explained in details in [4]. This methodology is demonstrated with a case study in the next section.
Tutorial and Semi-Automation of the Methodology
We have developed a multimedia tutorial as a CIMEL module to guide students through the process of emerging trend detection. Through the detection of incipient emerging trends the students will see the role that current topics play in course related research areas. Initial results of evaluation of an early form of the methodology [CRCD Annual Report 2001] showed that students had difficulty in understanding the methodology for detecting emerging trends. This motivated the development of a tutorial for emerging trend detection as a part of CIMEL courseware. The students are shown the significance of emerging trend detection and taken through an example of the methodology for detecting emerging trends.
The tutorial for emerging trend detection starts (Figure 4) by introducing to the student what we mean by emerging trends, specifically an incipient emerging trend and its importance in Computer Science Education.
Figure 4 What is an Emerging Trend
Next the tutorial explains why Inheritance in Object-Oriented Programming could be a good area for research on emerging trends. At the end of the tutorial, students are asked to find some more emerging trends in the given area of Inheritance. The tutorial then explains where one should look for discussions of possible (candidate) emerging trends like recent conferences, workshop links, poster sessions, etc. The students get to explore conference sites like ECOOP and OOPSLA, two well-known conferences on Object-Oriented Programming in a more hands-on-manner while the librarian persona in the background prompts the students where to look for possible emerging trends. Also, a demonstration of how web searches are done using popular web search engines (e.g., Yahoo and Google) is shown. In the next step, the tutorial walks through the algorithm step by step using a specific example and showing variable values at each step of the algorithm. Next comes the algorithmic result verification step. This involves three sub steps – verification using document spread, author spread and venue spread. The students get to see simple examples explaining technical jargon like Author Set, Co-Author Set, etc.
After explaining the process of emerging trend detection, students are asked in an assignment to find two more emerging trends in the main topic area of Inheritance. However, while the tutorial explains all the steps involved in detection of emerging trends in detail, the student gets to do the assignment in a somewhat controlled and guided process. Certain steps of the trend detection process have been automated. The students get to see a list of relevant conferences discussing Inheritance with links taking them directly to the conference web sites / journal pages etc. In step 2, the students get to see the algorithm together with the list of helper terms. The “Download Algorithm” button allows the student to download the algorithm to their computer so that they can print it and take a closer look at it.
In our initial evaluation of the methodology [CRCD Annual Report 2001], many students complained about this step. One of the reasons was that it is very time consuming to calculate the occurrences of terms in a document. We addressed this point by automating parts of the web-mining algorithm. The number of occurrences of the candidate emerging trend and the helper terms in the document is handled using an automatic tool. The only inputs to this term extraction tool [5] are a URL and the candidate emerging trend.
Figure 5 Step 3 of the Assignment
The next step is to verify candidate emerging trends using document count, author and venue spread. Again, to make the trend detection process easier, this step has been automated [5]. Students are only required to enter the Candidate Emerging Trend (Figure 5) which they have already identified in steps 1 and 2 while the database search tool automatically generates document count, unique author sets, unique co-author sets, a list of unique venues, etc. pertaining to the chosen candidate emerging trend. The tool also provides a link to the corresponding abstracts, which can be accessed by clicking on individual document titles. This feature of the tool is important, as the student still has to make his or her own decision, considering the information provided by the tool and using the heuristics provided in the tutorial, to validate a candidate emerging trend. Step 5 involves recording the results found in steps 1 through 4 (step 4 repeats steps 2 and 3 for each candidate emerging trend), using the template provided through the “Download Template” button. The students get to see a sample assignment submission filled out using the case study presented in the tutorial as an example by clicking on “View Table for the Case Study” button. Once completed, the assignment can be submitted to the instructor (or teaching assistant) using the “Submit” button. [1] details the script used in designing the tutorial. [2] contains a detailed explanation of these tools as well as the design and implementation of a area specific abstract repository.