This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Sample Size in the Application of System Usability Scale to Automotive Interfaces
ISSN: 0148-7191, e-ISSN: 2688-3627
Published March 28, 2017 by SAE International in United States
Annotation ability available
There is a strong business case for automotive interfaces to undergo usability testing throughout their product development lifecycle. System Usability Scale (SUS) is a simple and standard measure of usability. To meet the timing needs for product development, usability testing needs to be performed in a quick, cost effective manner. Hence the required sample size of participants for a usability study is one of the critical factors. To determine an acceptable sample size, a Monte Carlo simulation using SUS scores from eleven different in-vehicle automotive interface usability studies was used to create 500,000 subsamples of different sample sizes. The percentage of subsamples with mean scores within the confidence interval of the population mean was calculated. At a subsample size of thirty-five, 95% of the subsamples have a mean SUS score within the 95% confidence interval of the population mean.
CitationChandran, S., Forbes, J., Bittick, C., Allanson, K. et al., "Sample Size in the Application of System Usability Scale to Automotive Interfaces," SAE Technical Paper 2017-01-1383, 2017, https://doi.org/10.4271/2017-01-1383.
- Macefield, R., “How to Specify the Participant Group Size for Usability Studies: A Practitioner’s Guide,” Journal of Usability Studies: 34–45, Nov. 2009.
- Schmettow, M., “Sample Size in Usability Study,” Communications of the ACM, 2012, doi:10.1145/2133806.2133824
- Nielson, J., Landauser, T.K., “A Mathematical Model of the Findings of Usability Problems,” presented at Interchi ’93, Amsterdam, April 24–29, 1993.
- Virzi, R.A., “Refining the Test Phase of Usability Evaluation: How many subjects are enough,” The Journal of the Human Factors and Ergonomics Society, 1992, doi:10.1177/001872089203400407
- Turner, C.W., Lewis, J.R., Neilsen J.,” “Determining Usability Test Sample Size,” International Encyclopedia of Ergonomics and Human Factors: 3084-3088, 2006.
- McKean, J., “Information Masters: Secrets of the Customer Race,” (Wiley, 1999), 9780471988014
- Pyzdek, T., “The Six Sigma Handbook, Second Edition,” (New York, McGraw-Hill, 2003), 978-0071840538
- Spyridakis, J. H., Fisher, J. R., “Usability testing in technical communication: The application of true experimental designs,” Technical Communication: 469–472, 1992.
- Hwang, W., Salvendy, G., “Number of People Required for Usability Evaluation: The 10±2 Rule,” Communications of the ACM :130–133, May 2010.
- Perfetti, C., “Eight is Not Enough,” https://articles.uie.com/eight_is_not_enough/, accessed May 2006.
- Grosvenor, L., “Software Usability: Challenging the Myths and Assumptions in an Emerging Field,” Unpublished Master’s Thesis, University of Texas, Austin.
- Woolrych, A., Cockton, G., “Why and when five test users aren't enough’” presented at IHM-HCI 2001 Conference, France, 2001
- Faulkner, L., “Beyond the Five-User Assumption: Benefits of Increased Sample Sizes in Usability Testing,” Behavior Research Methods, Instruments, & Computers 35 (3), 379–383, 2003.
- Molich, R., “A Critique of "How to Specify the Participant Group Size for Usability Studies: A Practitioner’s Guide" by Macefield,” Journal of Usability Studies: 124–128, May 2010.
- John Brooke 1986 Brooke, J., “System Usability Scale (SUS): A Quick-and-Dirty Method of System Evaluation User Information,” Digital Equipment Co Ltd, Reading UK, 1986.
- McLellan, S., “The Effect of Experience on System Usability Scale Ratings,” Journal of Usability Studies: 56–67, Feb. 2012.