Chapter 4

Evaluating Interface Designs

Outline

I. Introduction

      The determinants of the evaluation plan include:

– stage of design (early, middle, late)

– novelty of project (well defined vs. exploratory)

– number of expected users

– criticality of the interface (life-critical medical system vs. museum exhibit support)

– costs of product and finances allocated for testing

– time available

– experience of the design and evaluation team

II. Expert Reviews

      Methods:

– Heuristic evaluation

– Guidelines review

– Consistency inspection

– Cognitive walkthrough

– Formal usability inspection

III. Usability Testing and Laboratories

• Participants should be chosen to represent the intended user communities

• Participation should always be voluntary, and informed consent should be obtained.

• Videotaping participants performing tasks is often valuable for later review and for showing designers or managers the problems that users encounter.

• Many variant forms of usability testing have been tried:

– Paper mockups

– Discount usability testing

– Competitive usability testing

– Universal usability testing

– Field test and portable labs

– Remote usability testing

– Can-you-break-this tests

IV. Survey Instruments

• Need clear goals and focused items that help attain goals.

• Users could be asked for their subjective impressions about specific aspects of the interface

• Other goals would be to ascertain

– users background (age, gender, origins, education, income)

– experience with computers (specific applications or software packages, length of time, depth of knowledge)

– job responsibilities (decision-making influence, managerial roles, motivation)

– personality style (introvert vs. extrovert, risk taking vs. risk aversive, early vs. late adopter, systematic vs. opportunistic)

– reasons for not using an interface (inadequate services, too complex, too slow)

– familiarity with features (printing, macros, shortcuts, tutorials)

– their feeling state after using an interface (confused vs. clear, frustrated vs. in-control, bored vs. excited).

V. Acceptance Test

• Set objective and measurable goals for hardware and software performance.

• Measurable criteria for the user interface can be established for the following:

– Time to learn specific functions

– Speed of task performance

– Rate of errors by users

– Human retention of commands over time

– Subjective user satisfaction

VI. Evaluation During Active Use

• Interviews and focus group discussions

• Continuous user-performance data logging

• Online or telephone consultants

• Online suggestion box or e-mail trouble reporting

• Discussion group and newsgroup