Cape Gateway Portal (v.1)

User Test Report

VERSION HISTORY

VERSION / DATE / DESCRIPTION / AUTHOR / Notes
1 / 5 Aug 2004 / Draft / S Vosloo
1.1 / 10 Aug 2004 / Final / S Vosloo

CONTENTS

1.Introduction TO REPORT

2.Introduction to Cape Gateway

The Cape Gateway portal

Design and usability goals

Phase two design and usability goals

Statistics

3.Introduction to User Test

Profile of participants

Structure of sessions

4.General feedback

Quotes

5.Homepage

What is this site about?

What strikes you about it?

What would you click on first?

What don’t you read on it?

Do you read the page footer?

Other notes / comments

6.Main SECTIONS: navigation and introduction page

Your Life

Topics

Publications

Directories

Your Government

Tenders

Jobs

7.TASK: Find a Job

Jobs Find page

Results page

Job page

8.TASK: FIND YOUR NEAREST SCHOOL

9.TASK: SEND AN EMAIL TO THE PREMIER

10.TASK: find your nearest library

11.Task: look for a tender that is relevant to your business

Tenders Find page

Results page

Tender page

12.Portal Search

13.LAYOUT

14.NAVIGATION AND LABELS

15.Multilingualism

16.TOPICS and your life

17.IF YOU CAN’t find information, what next?

18.General

What the users liked

Do you read the explanatory text at the top of a page (directly underneath the page title)?

Users’ understanding of different spheres of government

Do you know what a PDFs is?

19.GENERAL User SUGGESTIONS / COMMENTS

20.WHAT CONTENT DO YOU WANT ON THE PORTAL?

21.SURVEY RESULTS

22.HIGHLIGHTS AND KEY ISSUES

APPENDIX A

POST-TEST SURVEY

APPENDIX B

CAPE GATEWAY PORTAL USER TEST REPORT 1

1.Introduction TO REPORT

In June 2004 a series of user tests were conducted on by Steve Vosloo, Design and Usability Project Leader at the Centre for e-Innovation, Provincial Government of the Western Cape. The website was launched in March 2004 and the tests were carried out to learn lessons from version one of the project and inform improvements for version two. A limited time frame constrained the tests somewhat in terms of their rigour and has lead to a less formal approach to the usability report.

The Cape Gateway team intends to conduct more tests in the next few months as part of an ongoing iterative user-centred design approach. Despite the constraints, the findings of these tests have proved extremely valuable in guiding our thinking on the next version of the site (and confirm the popular belief that any user testing is better than no user testing).

We subscribe to the Steve Krug (a leading usability consultant - belief that “the point of testing is not to prove or disprove something - it is to inform your judgment. It provides invaluable input which, taken together with experience, professional judgment and common sense, will make it easier to choose, and with greater confidence, between ‘a’ and ‘b’.” The goal of the user testing was to gather as much feedback as possible from a few “fresh eyes”.

The key findings of the report are described in 22. HIGHLIGHTS AND KEY ISSUES.

For feedback or comments on the report email Steve Vosloo on .

2.Introduction to Cape Gateway

The Cape Gateway project provides easy access to government information in the Western Cape. Phase one of the project involved the creation of a consolidated database of government-related information. The Centre for e-Innovation, the primary e-government body within the Provincial Government of the Western Cape, runs the project and is aware of the digital divide issues facing the majority of the citizens of the province. It thus has 3 channels for information distribution: a walk-in centre (142 Long Street), a call centre (0860 142 142) and the portal ( When a citizen poses a question to the walk-in centre or call centre operators, the operators search for the answer on the portal.

The Cape Gateway portal

The Cape Gateway portal, which consists of over 25,000 pages and documents related to government, was designed primarily for the following target audience:

  • All online citizens of the Western Cape.
  • Online businesses of the Western Cape.

Secondary users include:

  • Non-profit organisations. They are seen as “infomediaries” who provide a vital channel for information exchange between government and citizens.
  • Staff members of the Cape Gateway walk-in and call centres.
  • Government staff.

Clearly this is a very varied set of end-users, with different levels of Web literacy and education, different first and second languages and different information needs. The portal is available in three languages at these addresses:

There is currently more English content than Afrikaans and Xhosa content on the portal. We decided to show as much content as possible, so on Xhosa versions of the portal there will be English or Afrikaans content in places, to make up for missing bits and pieces. (This has been a contentious issue within the design team: mix-and-match to fill the gaps or only show Xhosa content when the user chooses Xhosa, while clearly communicating to him/her that there is more content for that page or section in the English or Afrikaans versions of the portal.) The choice of which 2nd language to display is based upon the browsing habits of each user.

While the portal was designed to be easily accessible for the widest possible audience, the primary audience is intermediate or advanced Web users, especially those on slow Internet connections. The main design focus was to provide readable and relevant content, easily found through intuitive navigation. Thus the design is clean, consistent and page weight is kept at a minimum. A dedicated content team has gathered and rewritten content from the 13 government departments and also ministries.

It is accessible via a connected Web browser (preferably Internet Explorer 5.5 and above or Netscape 6 and above), with screen resolution ideally set to 800x600.

Design and usability goals

The following were design goals for the current version of the site:

  • Given the large amount of content on the site, the user must be able to easily, quickly and logically get to any piece of content.
  • Content must be presented from a user’s perspective, and not only reflect the structure of Government.
  • The site must seamlessly present three spheres of Government: national, provincial and local.
  • The site must easily present content in three languages: English, Afrikaans and Xhosa.
  • The search must produce usable and accurate results that quickly get the user to the correct content.

Phase two design and usability goals

Phase two involves a number of high-level goals:

  • Improving the usability of the portal (based on changing requirements from users and the existing results of usability tests).
  • Improving the content management system that drives the portal (in terms of usability, performance, data structure, etc.)
  • Adding more content to the site.
  • Building stronger relationships with stakeholders, e.g. local governments in the Western Cape.

Statistics

For usage stats on the site go to:

Statistics are only relevant from July onwards.

3.Introduction to User Test

Because of time constraints the sessions weren’t held in a usability lab and filmed; simple one-on-one discussions were held around a PC.

Profile of participants

The participants were selected through word-of-mouth and chosen according to first language and level of Web literacy. Thirteen participants were chosen and are representative of the selection criteria as follows:

English / Afrikaans / Xhosa
Beginner /  / 
Intermediate /  /  / 
Advanced /  /  / 
Age / All under 35
Gender / Women: 8
Men: 5
Highest level of education / High school: 4
Tertiary: 8
Business owners / 4 participants
Apply for government tenders / 2 participants
Occupation / Various, including ballet dancer, artist, students, NGO workers and volunteers, designer, project manager and travel consultant.

As expected, users with different levels of Web literacy had different experiences of the site. Two of the users have very low Web literacy and struggled with even the most basic elements of the site. We don’t recommend that the site should cater to real beginners in future versions. The site should clearly highlight the other distribution channels: email, call centre and walk-in centre. The very beginner users should use those channels.

Ideally the selection of participants should have also included pupils, teachers and older surfers.

Structure of sessions

Each user was asked to perform a series of tasks and offer their comments on certain site pages/sections. After the tasks a short survey was completed. See Appendix A for the user test questions and survey.

At the time of the tests the homepage lead item was about Drug Treatment and Counselling – see Appendix B for a screenshot. This lead a number of participants to initially think it was a health site.

4.General feedback

In general the users were impressed with the portal and thought that it had “got it right”. They found it clear and informative. The issues that were highlighted are improvements to the site; there are no “show-stoppers”.

Quotes

Below is a selection of comments from the participants.

“I like it. Neat and easy to read.”

“I’ve always wanted to find this sort of information. Cool site.” (Travel consultant)

“Very clear and logically laid out.”

“Easy site to browse.”

“It’s great. I love it. I’m going to visit it more.”

“Like the colours.”

“Friendly site. Well-structured and easy to work through.”

“I’ll be back!”

“Fabulous for school projects.”

“This is really so interesting. Government really does care and is doing something.”

5.Homepage

User were shown the homepage and prompted with questions.

What is this site about?

“Cape Gateway logo says ‘tourism’”

“Gateway = tourism”

“Tourism”

“Aimed at foreigners”

“Travel agency for government”

“Cape: not just Western Cape – could also be Eastern Cape or Northen Cape”

“Job & investment opportunities in the Western Cape”

“Information about City of Cape Town”

“Logo and tag line is explanatory”

“Site for the Government of the Cape. Logo and tag line is clear.”

“Tag line does explain the site, but should be more prominent”

“Government site”

“Main image isn’t explanatory”

“Health information”

“Company offering health services”

Afrikaans user on Afrikaans version of the homepage: “Why is the Latest Tender in English? I’m not happy about it!”

“Branding and purpose not clear”

“Needs clearer identity”

What strikes you about it?

“Top navigation is good. Everything is clear and obvious.”

“Main image”

Users all liked the images and big green link titles.

What would you click on first?

“Your Life and Topics. Would use them if I was searching for something.”

“Your Life, Topics or About”

“Left column more clickable because of colourful links”

What don’t you read on it?

“The tag line” (easy access to government information and services)

“Top right blurb ... but did read middle 2 lines of it”

 “Lots of text … some gets lost”

“When there are many lines of links, e.g. NEWS, the content gets lost”

Latest Tender description

Do you read the page footer?

Yes:44%

No:56%

Other notes / comments

Top right blurb gets lost. It was suggested that there is too much information being packed into a small space. Perhaps create a clear link to a page with all of that text and more, e.g. ABOUT page. Link might say “More about Cape Gateway” or “What is Cape Gateway?” Then at the top of the homepage there’d simply be:

Easy access to Western Cape government information

Email:

Call: 0860 142 142

Visit: 142 Long St, Cape Town

NEWS and WHAT’S NEW confused users – what is the difference? (SV: Perhaps change WHAT’S NEW to NEW CONTENT and move below POPULAR CONTENT.)

“Why are these particular articles on the homepage?” Perhaps display a heading like ‘INTERESTING CONTENT’.

“Not sure what the articles are about (what is the criteria for them being there?)”

“Is there a theme for the articles? How often do they change?”

“Does main article cover everything below it (left & right cols)?” SV: the size of the lead image is overpowering.

“Line down the centre means I don’t get prompted either way. The balance means I don’t look at either side.”

“Make one side more prominent than the other”

SV (from Steve Krug): Provide more “editorial guidance”, e.g. make one column wider than the other because certain content is more popular or important than other content.

6.Main SECTIONS: navigation and introduction page

Users were asked to describe what they expect to find within each main section of the site. Then they’d click through to the first page in each section and give comments.

Your Life

Expectation

“How my life relates to government”

“Expect to find services I can use as a citizen”

“What life is like in the Western Cape … to attract visitors”

“Demographic group I fall into”

Some users expect stats, e.g. how does unemployment affect me?

Once-seen

Users “get it” immediately.

They were generally interested in Housing, Education and Employment.

Topics

Expectation

“Current affairs, events, AIDS, poverty, etc.”

“Random interest topics”

“Current topics”

“The general topics of the site … subject categories”

Once-seen

“Summary of all services”

Most users do understand why Topics is necessary in addition to Your Life.

SV note: Users like Your Life and Topics; the titles grab them. Explore other alternatives for “Topics” – something that’ll still engage users but move away from the current affairs perception. Having said that, all users immediately “got it” after opening the page, so perhaps it should be left as is.

Publications

Expectation

“Government policies and public, formal docs”

“Statistics”

Once-seen
  • “Like the single column format … easy to assimilate information”
  • Most users got it; very clear section.

Directories

Expectation

“I understand Directories to be listings e.g. of NGOs and services, etc.”

“Expect lots of links to other sites because this is a portal”

“Contact names and numbers, e.g. for WESGRO, politicians, etc.”

“Yellow pages for government”

“Site map”

Once-seen

Users scroll down the whole page (very long) to see everything.

“Hard to differentiate clickable from non-clickable headings”. SV: insert space after every top-level heading?

Most users got it. It helped them to see an expanded view of all categories and sub-categories even though the page is long. This mitigates the problem of subjects being categorised differently by different people.

Your Government

Expectation

“Information about local and national government relating to me”

“General stuff about government”

“People in government, e.g. ministers, addresses, tel no’s etc.”

“What government is up to”

“How I can get involved in government”

Once-seen

Too much introductory text - (“text and me are unlikely bedfellows”). Some read it and others don’t – but all agree there is too much text and not enough to grab the eye.

“Show more faces and organograms”

“Put anchor links at top”

“In introductory text describe that this section has services, publications, contacts, etc.”

SV: Reduce text. Move page links to the top.

Tenders

Expectation

Obvious: information about government tenders.

BUT “not sure if only Western Cape tenders or national tenders too?”

Once-seen

Users “get it”

SV: Need colourful button/hook to OPEN TENDERS

Overview is long, but necessary. Some users do read/scan through it. Anchor links at top are helpful.

Jobs

Expectation

Obvious: jobs provided by government.

Once-seen

Users “get it” and like the section. Clear and easy to use.

7.TASK: Find a Job

Users were asked to look for any job for which they could apply.

Jobs Find page

Beginner and intermediate users didn’t click the browse button. SV: didn’t they realise that it was a button? Not all users are familiar with the term “browse”. Change to “View all jobs”

Almost every time users used the search interface they got 0 results. SV: maybe only show search interface when more than X number of jobs open? If less than, say 25 jobs, only allow them to browse jobs.

“Level” means nothing to users (“government jargon”). Some users equated it with level of education. SV: remove it from interface.

“Strange that the introductory text says that not all jobs are shown, and then the button is labelled BROWSE ALL JOBS”

Results page

“Like it, clear and informative”

Remove Level as it's own column.

When 0 results: change “Or, browse all jobs” to “Or, view all jobs”. Remove line break between “or” and “browse”.

Mixing of languages problematic, e.g. “Why is job title in English and GSC in Afrikaans?”

Job page

Users like it.

Show level here but also link it to a GLOSSARY entry.

8.TASK: FIND YOUR NEAREST SCHOOL

Users needed to find their nearest school.

They used Your Life, Topics, Directories or the search.

In the search they typed “schools”.

Answer: Your Life > General Education & Training (Grades R - 9) > Find-a-school

Note on Service page:

  • Put URL to external site at the top, above Description.

Most were successful in this task.

9.TASK: SEND AN EMAIL TO THE PREMIER

Users were asked to find the Premier of the Western Cape’s email address.

They looked in Your Government

  • > Provincial Government (Overview)

> Contacts … “no”

> Parliament site … “no”

For those who made it to STRUCTURE page, the following terms were “government-speak”

  • LEADERSHIP
  • POLITICAL
  • ADMINISTRATIVE
  • ORGANISATION

They also said that the horizontal lines add too much “noise” – needs work