Final Reflection PaperJune 8th, 2008.

Constantine Lazarakis

I. ReflectionsOn What I Learnt.

What I learnt isquite a challenging question. This course introduced me to a plethora of new topics, techniques, ways of thinking, and learning by collaborating in a group.

In an attempt to answer, I will break down this complex question, into smaller pieces starting with the question: What did the course set out to teach?

The course description indicates that the course focuses on “…the design and evaluation of interactive systems from a user-centered perspective”. Presumably, by completing the course I should be able:

  • To find and interpretresearch literature in the general areas of study within the field of human computer interaction.

The course offered –beyond the textbook- several articles on human computer interaction. The textbook and the articles acted as stepping stones into the communities of HCI and in a broader scope, interactive design. They provided the grounds to familiarize myself with the terms used in these disciplines. I did not have many opportunities to do a lot more research due to lack of time, but I now posses the basic knowledge to navigate my way through an article or book in HCI or Interactive Design and make meaning out of it. The most interesting articles involving HCI, for me, where:

1.“As We May Think” by Vannevar Bush; an excellent visionary. Even though the article was written more than sixty years ago it still fascinates. The author’s ability to build such fantastic scenarios (e.g. the Memex) of the future-to-be is beyond implicit creative genius.

2.“Five Reasons for Scenario-Based Design” by John M. Carroll; this article presents the challenge of two competing mental states; that of the action of design and that of reflecting on that design. “There is a fundamental tension between thinking and doing: doing obstructs thinking…But frequently it is more a matter of trading off priorities”. Scenario building seems to offer a solution to this mental conflict.

3.“Share Globally, Adapt Locally: Software Assistance to Locate and Tailor Curriculum Posted to the Internet” by Gerry Stahl, Tamara Summer, Robert Owen. This article shows how important it is for the designer to thoroughly understand the user’s world in order to provide a design that will actually solve the user’s problem “…just storing unrelated educational material on the Internet does not by itself solve the problem. ..Teachers need productivity software for locating,...searching,…adapting,…organizing,…sharing their experiences…”.

4.Also the articles on heuristic evaluations and cognitive walkthroughs offered practical guidance as to how to go about these tasks.

  • To analyze the interaction between people, the work they do, the information systems they use, and the environments in which they work and learn.

I have to say that the course struck me as very much oriented towards the topics of the way people interact and learn in small groups and the topic of interaction design.The topics –especially in the textbook - of what work they do and the information systems they use, were not my focus in this course; the former topic is a very general one; the latter is a matter of technology which is ubiquitous and hence, familiar. The bright exception was the information conveyed through the article: “The Research Cyber Studio: Supporting Researchers as LifeLong Learners” by Gerry Stahl. The topics of “Domain-oriented design environments (DODE),… dynamic websites… [and] latent semantic analysis (LSA)” will surely change and enhance the way researchers work and interact.

All articles on CSCL provided an excellent window to the concepts of how people interact, how they learn alone and in groups, how appropriate software, which enables collaborative work over the internet, can be utilized to make collaborative learning a more common way of enhancing knowledge in a global community.

If I had a choice, I would have opted for two separate classes: One in Interactive design and another in CSCL. CSCL is a paradigm for learning critically and deeply when confronted with any topic. It should be a separate, required course for all disciplines.

To analyze the design of the interactive computer systems from a user-oriented approach.

This, for me, was one of the most interesting parts of the course, since I have always been interested in the discipline of ergonomics and human factors, which have “…closely overlapping goals with HCI” (Human Factors Society, 2005, text p.11). I believe that more time should be spent analyzing designs of interactive computer systems in this course. I feel I needed the time to become more adept at this discipline, which can only be enhanced by doing more of it. A designer, trying to emphasize the thoughtful attention to detail required, when designing for humans, once said: “Perceive the human being as an egg; can an egg withstand a poorly designed egg-holder? No; it will break”.

‘No interactive computer system should be released for production, without analyzing and taking into account all the possible small details concerning the target end-user’ was also the theme of this course.

Chapters 3 (What is cognition? Cognitive frameworks), 10 (Identifying needs and Establishing requirements)and 11 (Design, Prototyping and Constructionin conjunction withJ.Carroll’s article mentioned above, where the more interesting readings in learning to analyze the design of an interactive computer system from a user-oriented approach. Chapters such as 7 (Data Gathering) and 8 (Data Analysis, Interpretation and Presentation) were familiar from previous academic work.

To analyze the results of the evaluation of interactive computer systems from a user-oriented approach.

To analyze the results of an evaluation of an interactive computer system from a user-oriented approach implies that one is ‘comfortable’ with the following issues:

  • Analytical Evaluation, especially:
  • heuristic evaluations where knowledge of typical users is applied, guided by heuristics such as guidelines and standards to identify usability problems and
  • Walkthroughs which involve experts in walking through scenarios with prototypes of the application.
  • The DECIDE framework which guides evaluation.

Both were adequately explained in the text Chapters 12 and 13. Articles and papers such as “How to Conduct a Heuristic Evaluation” by Jakob Nielsen; Gregory Abowd’s notes on “Heuristic Evaluation” and “Performing a Cognitive Walkthrough” were helpful, practical guides in the limited time we had to perform these analytical evaluations. Articles such as “Task Analysis for Groupware Usability Evaluation…” (D. Pinelle et al.) provided useful insight on the limitations of traditional discount evaluation methods --such as the above-- in evaluating groupware, since they are oriented around individual work. Their new modelling technique “Collaborative Usability Analysis” sets out to provide a better evaluation method for groupware usability.

  • A hands-on experience in actually designing and evaluating human computer interaction and computer mediated human-human interaction, through the course design project.

The pinnacle of the course was the actual design of a prototype by each group and the evaluation of other groups’ designs. The process started in week 4 with the presentation of our use cases. Gradually, to move forward and turn our conceptual design into a prototype, we had to spend long hours chatting and trying to explain –to everyone’s satisfactory understanding- what our individual ideas or findings meant. We had to take into consideration the evaluations of others, of our work, and work together to improve our design. No one member of the group could claim expertise in all aspects involved in completing the project. Synchronous collaboration was the only way forward. I, for example, have limited knowledge in actually using web-site design tools, whereas I have more experience in conducting research on a subject. I had to explain my ideas in detailed writing to Zdravko and Dorian in order for them to implement them in the prototype. Our team took full advantage of our diverse backgrounds to produce what we consider a fairly complete low-fidelity prototype. So useful was this experience that I have written a separate, small section on it below.

II. Format of the Course

If there are some issues I feel should be included in the format of the course, these are:

  • More emphasis on case studies with analysis of why the interfaces were designed as they were. An analysis of the way of thinking behind sites such as Amazon’s, Google’s and others could have provided insight on how experienced designers think; what empirical evidence has taught them. Also what to avoid in design.
  • Maybe basic knowledge and use of some specific tools and techniques for designing should either be a prerequisite or be briefly taught in this class (if time allowed). Having knowledge of the possibilities of what can be designed helps in heuristic evaluation eventually. The best heuristic evaluations of the class –it seems- were made by people with experience in using design software tools and in designing interfaces. I felt lacking since I do not posses such knowledge and I believe that affected my ability to judge a design as thoroughly as some of my class mates did.
  • As I mentioned above CSCL should eventually become a course in itself. My experience in this format was that CSCL overshadows HCI, simply because it is a far more complex and ‘deep’ topic, with a much larger scope and importance to society than the comparatively limited scope even of interaction design.

III. If I had another 10 weeks

  • I would have liked to have more time to spend on ‘case-studies’ of interaction design. What I mean by this is to study the thought and the work that went into the design of interactive experiences in various fields, including commercial sites (Amazon,EBay) critical missions (e.g. medical, military, utility, manufacturing process), fast pace/ fast decision environments (e.g. stock market screens, fighter aircraft screens), and others.
  • I would also have liked to get acquainted with the various software design tools as mentioned above (Format of the Course).

IV. What should be implemented and adopted by the CSCL community?

This idea may appear crude and heavy handed but I think it is worth pursuing:

First of all, for all practical purposes, CSCL has proven a successful theory which does explain how expertise is achieved through collaborative learning within small groups.

The CSCL community has to lobby and persuade those responsible for the direction of education in their respective countries, that schools need to be restructured“…so that they become the kinds of environments that support the process of expertise, in particular progressive problem solving as it applies to competence and understanding”.2

To persuade someone about one’s cause, one has to find something that is interesting to the other party in this cause. In the case of CSCL the people that have to be persuaded must:

  • Have CSCL ‘translated’ to them in a way that is understandable in the context of how they are used think (usually politically).
  • The concept of CSCL must also have to be communicated to the decision makers as something that makes economic sense to society as whole; i.e. that the benefits accruing to society are economically higher than the costs that may be involved to implement CSCL in schools and other public learning institutions or even the public sector in general.

I believe that CSCL has the critical mass needed –in terms of empirical research supporting the theory and in terms of an organized CSCL society-- to launch itself out of the academic community, into the halls of power that influence the direction of education. It has to gradually be promoted as a critical topic, which can positively affect the competitive advantage of a country, in a world where value added is increasingly dependent on superior expertise.

V. What I learnt beyond techniques, articles and text:

Group Learning Dynamics and Human Experiences from our Group Collaboration.

  • This course provided for my first experience of learning to work in a group of people I had never even seen or heard of before. We had no information ofeach other’s personalities, educational backgrounds, age, career paths, work ethic, etc. What I gradually came to understand, is that the dynamics of such a group are influenced more by meritocracy than by any other form of influence. The deciding factor on the direction and the level of ‘sophistication’ and quality at which the collaborative project is ultimately placed, is the qualityof postings and knowledgeable ideas, brought to the chatting sessions, by each individual group member; which, in turn, can only happen,after extensive and thorough work on and immersion in each weekly subject matter.
  • The level of meaning making rose as others strived to keep up or surpass the level of thoroughness and originality of other members’ postings. There was a sense of pride because of good work. In this positive upward spiral, however, there were costs for the group. Our group experienced the loss of one member, somewhere mid-stream in the course. This member’s comments before resigning from group-work, reflected the fact that this member felt she could not keep up with the ‘demands’ or the level of quality of work the group had established already mid-course. So, the empirical evidence, from at least this group learning experience, is that collaborative learning enhances the level of quality of work done; in the process, it increases the workload for the contributions and consequently, the need for discipline and tight time management. The overall experience, for those who ‘survive’, brings about positive feelings of self-actualization1.
  • Another factor that influences the group’s performance –especially in the beginning stages of this course-- is each individual member’s tolerance for ambiguity. The course has many factors that produce feelings of stress and frustration; I list the ones I felt were most prevalent:
  • A course which essentially (eventually, thankfully) teaches two subjects: HCI and CSCL.
  • The necessity to collaborate with ‘strangers’ on-line, on new concepts (above) in a new environment (wiki), to produce something that is partially or totally new to each one of the members.
  • If one has a tolerance for a, seemingly, chaotic or less structured environment and perseveres, one gradually realizes the advantage of being free to think creatively and contribute knowledgeable ideas, without the fear of going beyond a structured ‘norm’.
  • A welcome factor that I experienced for the first time in a course was the fact that grading was not the focus of the course; learning was. It projected a sense of maturity –of greater perspective- to the objective of the course, which really was enhancing meaning-making through small group collaboration.
  • An additional factor that helped to enhance meaning-making was the encouragement given to groups to go back and continuously iterate/improve on their work. It felt like the creation of a painting or of an architectural design. You had a chance to go back and put final touches to the group work and make it better, after reflection on the issues involved.Transition from creative thinking to work needs this freedom. Overall, a very fulfilling experience.

1 Maslow’s Hierarchy of Needs (Self-actualization: morality, creativity, spontaneity, problem solving, lack of prejudice, acceptance of facts).

2“Computer Support for Knowledge-Building Communities”, Marlene Scardamalia, Carl Bereiter (1994), The Journal of the Learning Sciences, 3(3), 265-283.

1