Impact of Pre-Study Exploration on System Usability Scale and Task Success Rates for Automotive Interfaces

Measurement of usability with the System Usability Scale (SUS) is successfully applied to products in many industries. The benefit of any measurement scale, however, is limited by the repeatability of the associated testing process. For SUS, these factors can include sample size, study protocol, previous experience, and pre study exposure to the system being tested. Differences in user exposure can influence the usability assessment of interfaces which could affect the validity of SUS scores. A customer clinic was conducted on a steering wheel/instrument cluster and a center display screen, to see the difference in SUS scores of participants with “Free Exploration (participant had a few minutes to interact with the system)”, “Guided Exploration (participant was given a couple of tasks prior to the study)”, “No Exploration (no interaction prior to the study)” and “Repeated task Exploration (tasks were asked again after the no exploration phase concluded)” in a between subject design study. All four scenarios were analyzed for their impact on SUS score, task success rates and adjective rating. Under the “No Exploration” test method the estimates of mean SUS score, adjective rating and task success rates are reliable and repeatable with a good correlation to external quality metrics.