Reliability is the question of whether one would get the same result if the test were to be repeated, and validity is the question of whether the result actually reflects the usability issues one wants to test. Standard statistical tests can also be used to estimate the confidence intervals of test results and thus indicate the reliability of the size of the effects. Validity is a question of whether the usability test in fact measures something of relevance to usability of real products in real use outside the laboratory.
6.1 Test Goals and Test Plans。 Before any testing is conducted, one should clarify the purpose of the test since it will have significant impact on the kind of testing to be done. A major distinction is whether the test is intended as a formative or summative evaluation of the user interface. Formative evaluation is done in order to help improve the interface as part of an iterative design process ( thinking-aloud test ). And summative evaluation aims at assessing the overall quality of an interface ( a measurement test ). Test Plans, No usability testing should be performed without first having tried out the test procedure on a few pilot subjects.
6.2 Getting Test Users The main rule regarding test users is that they should be as representative as possible of the intended users of the system. Within-subject testing does have the major disadvantage that the test users cannot be considered as novice users anymore when they approach the other systems after having learned how to use the first system. In order to control for this effect, users are normally divided into groups, with one group using one system first and the other group using the other system first.
6.3 Choosing Experimenters 6.4 Ethical Aspects of Tests with Human Subjects 6.5 Test Tasks 6.6 Stages of a Test. A usability test typically has four stages: 1. Preparation 2. Introduction 3. The test itself 4. Debriefing
6.7 Performance Measurement User performance is almost always measured by having a group of test users perform a predefined set of test tasks while collecting time and error data. Once the components of the goal have been defined, it becomes necessary to quantify them precisely. Typical quantifiable usability measurements include
6.8 Thinking Aloud By verbalizing their thoughts, the test users enable us to understand how they view the computer system, Constructive Interaction. co-discovery learning. retrospective testing. Coaching Method
6.9 Usability Laboratories Having a permanent usability laboratory decreases the overhead of usability testing (once it is set up, that is!) and may thus encourage increased usability testing in an organization.