requirements analysis for customization using subgroup differences and large sample user testing: a case study of information retrieval on handheld devices in healthcare

requirements analysis defines the goals and evaluation criteria of system design. we introduce a methodology for requirements analysis for customization based on large sample interactive testing, with the premise that analysis of user behaviour with prototypes leads to requirements discoveries. the methodology uses a relatively large sample (1) to identify relevant user subgroups, (2) to observe significant empirically determined group differences in the context of task and tool use and (3) to estimate the groups’ different requirements and derive design implications. between 20 and 50 participants are used per test, rather than the three to five often recommended for user testing. statistical relationships are investigated between subgroups in terms of background variables, questionnaire items, performance data, and coded verbal statements. customization requirements are inferred from the significant differences observed between empirically determined groups. the methodological framework is illustrated in a case study involving the use of clinical resources on handheld devices by three groups of physicians. the groups were found to have different needs and preferences for evidence-based resources and device form factor, implying opportunities and necessities for group customization requirements. relevance to industry in safety-critical domains such as health care, it is essential to assess user needs and preferences regarding devices and systems to inform appropriate customizations. we present a methodological framework and case study that demonstrates how large sample user testing can supplement typical methods of requirements analysis to provide contextualized, quantitative accounts of group differences and customization requirements.