Torturous Transcript of Ann’s Info. Seeking
Key
[> - input to computer
[< - computer output
[- - current computer state
… - continuation across action
Clip no. & time on compilation / Transcript / Interpretation / Discussion1)
0:00:09 / Topics of interest
1I’m looking for digital library stuff.
2In particular I have four topics I’m looking for information about.
3One is usability of digital libraries, European cognitive modelling and its relationship to HCI, usability of calendars and diaries and material on the relationship between AI and HCI.
2)
0:00:32
4[- ACM DL search page] / Chooses ACM DL first – familiarity?
5Terms such as digital libraries…
6[>types ‘digital libraries’ into search box’]
7[< search box contains ‘digital libraries’]
8… seems kind of bizarre to look just for titles so. / Default search criteria seems bizarre
9If I put libraries I’m going to miss those with library so I’ll put…
10[> changes ‘libraries’ to ‘librar’] / Stems word manually / Believes stemming necessary – isn’t there an option to stem words for you?
11[< search box contains ‘digital librar’]
12… exact phrase and try ‘full text’.
13[> clicks full text option]
14[< full text selected] / Full text search
15Errm, do I want to limit my search?
16[> scrolls down]
17[< extra search options scroll up from bottom of page]
18Errm, what’s the choice?…
19[> click on criteria pop up menu]
20[< long list of criteria]
21… all journals and proceedings, errm, it looks like all other things are errm significantly more restricted.
22[> let go of pop up menu]
23[< pop up menu disappears]
24Errm, actually probably looking through full text is probably a waste of time… / What do search options mean?
25[> clicks on full text option]
26[< full text option deselected] / Not full text search / Why change of mind?
27… but looking through the abstracts might help.
28[> clicks on abstract option]
29[< abstract option selected] / Abstract search / Why abstracts better?
30Errm, I don’t know what those numbers after the things are.
31Anyway, let’s try searching and…
32[> clicks on search]
33… see what we come up with.
34[< user log in dialog displayed]
35Ok so the first thing I have to do is remember my ACM account number…
36… need that so early on, especially as I thought…
37[< search results page appears]
38… Middlesex subscribed to it.
39Oh, no matches, whoopee. / No matches / Why not? – gives up at this point with dl
3)
0:02:02
40[- ACM DL search page]
41oops, might help if I could spell…
42[> typing ‘calendar diaries’]
43… diaries, so any words errm well lets stick with that…
44[> scroll down page]
45[< bottom of search page in view]
46… and search again.
47[> click search]
48[< search results page shown] / Successful query
49Ah ok.
50Developing calendar visualisers for the information visualiser.
51Errm.
52The diary study, well, I happen to know what that work is. / Recognises work from details
4)
0:02:44
53[- ACM DL search results page (results from 3)]
54I don’t know what happens when I tick them. / What is ticking for?
55[> scrolls down]
56[< more search results appear]
57Ooh.
58[> scrolls up]
59[< top of search results shown again]
60Calendars in the office, an assessment of user needs…
61[> clicks tick box for ‘calendars in the office, an assessment of user needs] / Selecting articles / What does selecting do?
62[< tick box ticked]
63… does definitely look relevant.
5)
0:03:08
64[- ACM DL search results page (results from 3))
652000 AD I wonder what that might be… / View abstract
66[> clicks on 2000 AD abstract link]
67[< abstract of article displayed replacing search results page]
68… its given me an abstract errm…
69[> scrolls down]
70[< bottom of abstract displayed]
71… oh its to do with dates, no that’s not relevant / Needs to view abstract to decide if relevant – not much info in list
Lack of discriminability?
72[> clicks browser back button]
73[< search results page displayed replacing abstract page] / Back to search results
74Programming languages. / Long list of articles to wade through
6)
0:03:30
75[- ACM Dl search results page (results from 3) with several articles ticked]
76I’ve clicked these things now and I haven’t got a clue what I’m meant to do with them. Hmm. I don’t know what a binder is… / What to do with articles once they’ve been looked through?
77[> clicks and holds on binder link]
78[< browser pops up menu of possible actions – open page etc.]
79… help… / Lucidity – what is Ann supposed to do now?
80[> releases mouse]
81[< page is replaced with page containing list of articles selected]
82… please choose a binder from your bookshelf.
83I don’t have a binder, and I don’t have a bookshelf as far as I’m aware. / Lucidity – interface is referring to items Ann has no idea about
84Errm OK I’d better just have a quick shuft through…
85[> clicks on first article in list] / Look through abstracts of articles / Needs to look at abstracts of articles selected to check if relevant
86[< page is replaced with abstract of first article]
87… I guess.
7)
0:04:05
88[- ACM DL abstract of an article previously selected]
89Hmm, so this is about…
90[> scrolls down]
91[< bottom of abstract is shown]
92… new technologies for diaries, it kind of looks, looks like something I need to know… / Checking relevance of article by reading abstract and keywords / Assessment of relevance
93[> scrolls up]
94[< top of abstract shown]
95… about, so where the pdf?
96[> clicks on pdf link]
97[< page clears, adobe acrobat starts up]
98[- blank browser page]
99[20 seconds]
100Oh joy. I can’t remember how big it was now so don’t know how I’ll know when we’re getting there. / Lack of feedback on size of file and time to download / What is this? Lack of info on system state?
101[10 seconds]
102[- acrobat viewer within netscape showing article with thumbnails to the left hand side]
103Ohh. Oh, so you get tiny little thumbnails of each page.
104Isn’t that sweet.
105[> scrolls down thumbnails]
106[< thumbnail frame scrolls down]
8)
0:05:10
107[- acrobat viewer of article from 7)]
108Can’t now remember when it was, which conference, or which year. / Looking for year of article / Important info (year) is embedded in article, not explicit.
Hard to assess relevance if criteria are embedded
109[> scrolls article down]
110[< article scrolls down to bottom of first page]
1111994. Ah, so its actually very old.
9)
0:05:25
112[- acrobat viewer of article from 7)]
113It’s, I think its at least worth saving.
114[> Clicks browser ‘File’ -> ‘Save’ menu]
115[< Menu pops up]
116[< Save dialog appears]
117OK. Save it as source…
118[> selects source from file type pop up menu]
119[< type changed to source]
120… presumably I’ll be able to…
121[> clicks save]
122[< dialog disappears ]
123[< file save status window appears]
124… open it again. Gosh its big. Oh….
125[> clicks cancel on file save status window]
126[< file save status window disappears] / Cancel save article because of time to download
127… This is just silly.
128Just silly.
129If I’m going to print it out then…. / Print instead / Print chosen because quicker than save – what would be done with saved versions anyway… if Ann’s going to read it she’s going to print it
130[> scrolls article up]
131[< top of article displayed]
132… if I’m going to read it then I’m going to need it printed so….
133[> clicks on print icon]
134[< print dialog displayed]
135… Let’s just print it and be done.
136[> clicks on print]
137[< print dialog disappears]
138[< print status appears]
10)
0:06:00
139[- ACM DL abstract of paper (different to 7))]
140Hmm, so that’s another 1000k.
141[> clicks on pdf link]
142[< acrobat page appears with article only (no thumbnails)]
143Oh.
144But at least this one came up quickly, that’s much nicer.
145Presumably because didn’thave to hang around for the errm cute errm little thingerprints things, thumbnails, or whatever they’re called. / Lack of thumbnails speeds up download.
Preferred.
146Percentage has come up, that’s much nicer. / Feedback on system state – percentage done
147I much prefer that.
148[> scrolls down]
149[< scrolls to end of article]
150OK.
151That’s a vast improvement.
152Who cares which print button you use. / Multiple print buttons – maybe there is a different (acrobat vs. Netscape).
153[> clicks print button]
154[< print dialog appears]
11)
0:06:50
155[- ACM DL list of selected articles (from 6))]
156Its really hard to tell, I must say, just from these little bits…
157[> clicks on link to abstract of an article]
158[< abstract of article displayed (page replaces list of selected articles)]
159… which one’s relevant. / Difficult to assess relevance of doc from list description – need to look at abstract.
Discriminability?
160[> scrolls down]
161[< bottom of abstract shown]
162Oh. Looks like…
163[> clicks on abstract link]
164[< page scrolls to bottom]
165… design, human factors, and theory…
166[> clicks browser back button]
167[< abstract page returns to top]
168… it doesn’t tell use very much about it at all and I can only assume it hasn’t been errm digitized for the library which seems strange really. / No abstract for article / Assumption about lack of article in library – based on familiarity?
Is it a wrong assumption?
169[> clicks browser back button]
170[< list of selected articles replaces abstract page]
171OK.
172Well, I’m prepared to assume that’s not relevant.
173These are all very old. / Age as a criteria / How explicit is age in the interface?
12)
0:08:00
174[-ACM DL first page of article in acrobat viewer in netscape]
175What we’re about to get I think is that is not giving me errm…
176[> scrolls down]
177[< bottom of first page of article comes into view]
178… ah CSCW 90 Proceedings, it is giving me. / Found citation info. / Again, important info (citation this time, year before) is implicit in article rather than being explicitly represented.
Hidden state? Related to OSM?
13)
0:08:10
179[-ACM DL article displayed in acrobat viewer (from 12)), print status overlayed on top]
180Its hard to know how to manage these things without actually printing them out and then using that to decide whether or not that was actually worth doing.
181I mean you have to commit to printing before you can really work out whether it was worth the effort. / Management of articles and assessment of relevance ultimately done by printed copy.
Though there are different levels of management going on.
14)
0:08:40
182[-ACM DL abstract of article (bottom of abstract)]
183Digital document readers, electronic books…
184[> scrolls up]
185[< top of abstract page shown]
186… OK so this is more to do with…
187[> clicks on pdf link]
188[< article is displayed in embedded acrobat window (no thumbnails)]
189… digital libraries I think than it is to do with diaries. / Finds articles relevant to first search (dl) – raises issues of interleaved searching
190[> scrolls down]
191[< first page scrolls to bottom]
192Againg, I think its probably something we need to know about…
193[> scrolls down]
194[< top of second page shown]
195… and be aware of and…
196[> clicks on print icon]
197[< print dialog displayed]
198… these are probably good authors to be followed up / Input into future searches – authors from found article
199[> clicks on print button]
200[< print dialog disappears]
201[< print status window appears]
15)
0:09:20
202[- ACM DL list of selected articles (from 6))]
203Where would I find, I’m I’m intrigued about this bookshelf, where would I find my bookshelf? / Resolving lucidity issue
204[> scrolls down]
205[< list of selected articles scrolls down – my bookshelf details appear]
206Oh, my bookshelf.
207[> scrolls down]
208[< page scrolls to very bottom]
209[> scrolls up]
210[< page scrolls up to show some selected articles and some of my bookshelf]
211Please select a binder from your bookshelf.
212Well I can’t be bothered right now so lets go back. / Organisation concept supported not used – why not/ why bother with it anyway?
213[> clicks browser back button]
214[< list of search results shown (results from 3)]
215Don’t think I need that. OK.
16)
0:09:50
216[- ACM DL list of search results]
217Now then, am I going to find much on…
218[> clicks browser back button]
219[< search formulation page displayed replacing search results]
220… European cognitive modelling? / Search interest: European cognitive modelling
221Let’s, let’s go the other way and…
222[> enters an author in the author search box]
223… just look for one or two authors.
224Let’s search for something by Rasmussen or Hollnagel I think. / Search by known author: Rasmussen
225[> clicks search button]
226[< list of search results shown in page replacing search page]
227Right, err, um, who the Rasmussen.
228Lars, oh there’s lots of Rasmussens.
229Errm, I don’t think I want VLSI or randomized load balancing.
230[> scrolls down a bit]
231[< next search result comes in to view]
232Ap ba ha ha ha
233Alright.
234[> scrolls down]
235[< page scrolls down to end]
236There are lots of Rasmussens in the world and none of them is the one I was looking for. Errm… / Lots of search results, but none relevant / Good discriminability, but none relevant – needle in haystack?
237[> clicks browser back button]
238[< search page replaces list of results page]
239… do I want, I wanted Jens Rasmussen. / Why not trying search for Jens Rasmussen?
240[> starts editing author search box]
241[< text in search box cannot be read]
242I wonder if Hollnagels pub published anything anything in ACM. / New search: Hollnagel
243[> clicks search button]
244[< search results page replaces search page (no results)] / No results / Is Hollnagel spelt correctly?
245Nope. This is partly why I find it hard to…
246[> clicks back button on browser]
247[< search page replaces search results page]
248… find stuff by these guys.
249OK. Then so the other thing I wanted was…
250[> types artificial intelligence into subject search box]
251[< artificial intelligence appears in subject box – note that Hollnagel is still in author box]
252… artificial intelligence, oops, intellignece, and human interaction, well it could be human computer interaction or human factors…
253[> adds human to end of search terms]
254[< subject search is now ‘artificial intelligence human’]
255… so lets.
256Which one worked last time?
257I can’t even remember which one I did last time.
258Was it full text or abstract?
259Lets go for full text.
260[> clicks on full text tick box]
261[< full text selected]
262[> clicks search] / Search for ‘artificial intelligence human’ but with author ‘hollnagel’ / Problem with system state – not aware that its using Hollnagel as well as artificial intelligence human.
263Errm, I don’t know what reviews or index terms would be.
264[< search results page replaces search formulation page (note – no matches)] / No matches
265No matches. Great.
266[> clicks back]
267[< search page replaces search results page]
268So even on full text.
269Of course, its possible, no.
270What happens if I turn off the human and try again?
271[> removes human from subject search terms] / Reformulating search terms
272[< subject search is now ‘artificial intelligence’ (note – author search is still ‘Hollnagel’)]
273I would expect to get a fair amount.
274[> clicks search] / Search for ‘artificial intelligence’ but again with author ‘hollnagel’ / Again, problem with system state – not aware that its using Hollnagel as well as artificial intelligence human.
275[< search results replace search formulation page – note no matches]
276No matches. Ha ha ha.
277[> clicks browser back button]
278[< search page replaces search results page]
279Right OK lets.
280[> clicks browser back button]
281[< error page saying document not found in cache replaces search page]
282Documents results from expired in cache.
283Lets try a bit of reloading.
284[> clicks browser reload button]
285OK, I don’t know what that question meant. What ever it was will
286[< page of search results is displayed replacing error page]
287OK, I didn’t want search results.
288It wasn’t.
289I just wanted to go home again. / Lost? / Appears now to be lost/ confused about current location/ how to get to home page.
17)
0:12:40
290[-ACM DL main home page] / Spends some time looking at page
291[> clicks on publication list link]
292[< publication list page replaces ACM DL home page]
293Lets try doing a bit of intelligence guessing on which ones might be relevant.
294[> scrolls down one page]
295[< first set of publications scrolls into view]
296CACM well I get that anyway so I don’t need to look at it. / Recognition of already received journal
297Intelligence.
298AI in practice.
299Captures the excitement of AI in practice.
300[> clicks on Intelligence journal link] / Views journal page / View journal based on description
301Wow.
302[< Intelligence journal page replaces journals list page]
303Erm.
304[> scrolls down page]
305[< bottom of page appears]
306Volume ten.
307[> clicks on volume 10 link] / Views vol. 10 of journal / Lots of scrolling involved in looking through long list of contents
308[< content for volume 10 replaces journal page]
309Table of contents.
310News.
311[> scrolls down]
312[< more contents of volume ten come into view]
313Oh dear.
314Book reviews.
315Announcements.
316Authoring tools for AI.
317[> scrolls down]
318[< more contents come into view]
319Ontologies is borrowing from euggh.
320She we have just a little shufty?
321[> Clicks on article link] / Views article details / View abstract given description in list – is it relevant?
322[< abstract of article displayed replacing list]
323[> scrolls down]
324[< bottom of abstract shown]
325Doesn’t seem to want to tell us anything apart from wanting to give us full text which is going to be a… / No abstract, so have to look at full text
326[> clicks on full text link] / View article
327[< full text replaces abstract page]
328… nightmare.
329[> starts scrolling down]
330Oh.
18)
0:14:10
331[- blank web browser window]
332We’re going to work much more efficiently if we have a new navigator window…
333[> clicks browser file -> new navigator window] / 2nd window
334[< New window (2nd window) appears with home page]
335… and have that one running in the background.
336[> selects ACM DL from bookmarks]
337OK so I wonder if I’m going to have to.
338[< ACM DL main page appears in browser window]
339Ah OK so.
340[> clicks publications link from main page]
341[< ACM DL journals and conference proceedings page replaces main page]
342Proceedings by subject as well in the foreground.
343[> scrolls down]
344[< page scrolls to show long list of publications]
345Gosh. / Large list of proceedings by subject
346Ooof.
347I don’t know what any of these things are. / Don’t know whether relevant
Discriminability
348[> clicks International Conference on Autonomous Agents publication link] / View ICAA conference
349[< International Conference on Autonomous Agents page replaces publications list]
350Well, I know what an agent is.
351Agents 99.
352Related SIGs.
353SIGCHI fine.
354[> clicks on most recent conference link] / View specific conference proceedings
355[< Table of Contents page starts to appear (some delay)]
356Now of course, all I’m doing is downloading two lots of stuff.
357Hermatic, believable emotional agents [inaudible]
358Eugh.
359Pets with evolving emotional intelligence.
360Gosh.
19)
0:15:40
361[- ACM DL content of conference proceedings (from 18))]
362It would be so much easier to just sit there with a paper copy and flick through and actually see. / Difficult of skimming through and getting an idea
20)
0:15:55
363[- ACM DL content of conference proceedings (from 18))]
364I don’t think any of its particularly useful. / Assessment of relevance – not relevant, but had to look at conference page and some particular proceedings
365[> clicks browser back button]
366[< details of conference page (from 18)) replaces content of conference page]
367Errm.