Developing and Validating Measurement Scales During Pandemic Conditions: A Case Study with the Scale for Habitat Usability

Abstract

At NASA, habitat evaluations often employ subjective measures. Some measures are frequently used, well-established tools, whereas others are homegrown measures tailored to specific projects. The variety of measures used makes evaluation comparisons across projects difficult. Additionally, some of these measures are burdensome, may be too specialized, or may require an expert to use and interpret, limiting their utility. Taken together, these drawbacks suggest the need for a new measurement tool. To that purpose, a team at NASA worked on developing a new scale for measuring habitat usability, the Scale for Habitat Usability (SHU). The SHU is intended to be a quick, multi-faceted measure for evaluating habitat usability across the development lifecycle. However, like many research projects, the development of the SHU faced setbacks due to the COVID-19 pandemic. Pandemic prevention protocols precluded in-person data collection, forcing the team to take some non-traditional approaches to scale development. This paper reports the steps the team took to complete the project.

Get full access to this article

View all access options for this article.

References

Acemyan

C. Z.

Kortum

(2016). Does the polling station environment matter? The relation between voting machine layouts within polling stations and anticipated system usability. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59(1),1066-1070. https://doi.org/10.1177/1541931215591299

Acemyan

C. Z.

Kortum

(2018). Does the type of presentation medium impact assessments of thebuilt environment? An examination of environmental usability ratings across three modesof presentation. Journal of Environmental Psychology, 56, 30-35.

Alexander

V. D.

(2013). Views of the neighbourhood: A photo-elicitation study of the built environment. Sociological Research Online, 18(1), 1-26. https://doi.org/10.5153/sro.2832

Bangor

Kortum

Miller

(2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies, 4(3), 114-123. https://doi.org/10.5555/2835587.2835589

Boateng

G. O.

Neilands

T. B.

Frongillo

E. A.

Meglar-Quiñonez

H. R.

Young

S. L.

(2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontier in Public Health, 6, 149. https://doi.org/10.3389/fpubh.2018.00149

Brooke

(1996). SUS: A quick and dirty usability scale. In Jordan

P.W.

Thomas

Weerdmeester

B. A.

McClelland

I. L.

(Eds.), Usability evaluation in industry (pp. 189-194). Taylor & Francis. https://doi.org/10.1201/9781498710411-35

Capra

M. G.

(2005). Factor analysis of card sort data: An alternative to hierarchical cluster analysis. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 49(1), 691-695https://doi.org/10.1177/154193120504900512

Carpenter

(2018). Ten steps in scale development and reporting: A guide for researchers, Communication Methods and Measures, 12(1), 25-44. https://doi.org/10.1080/19312458.2017.1396583

Gabrielli

Mirabella

Kimani

Catarci

(2005). Supporting cognitive walkthrough with video data: A mobile learning evaluation study. In Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices & Services (pp. 7782). https://doi.org/10.1145/1085777.1085791

10.

Hornbæk

Law

E. L. C.

(2007). Meta-analysis of correlations among usability measures. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 617626). https://doi.org/10.1145/1240624.1240722

11.

Hinkin

T. R.

(2005). Scale development principles and practices. In Swanson

R. A.

Holton

E. F.

(Eds)., Research in organizations: Foundations and methods of inquiry (pp. 161-179). Berrett-Koehler. https://www.drrichardgreen.com/uploads/3/4/5/2/34520924/swanson_and_holton-2005research_in_organizations.pdf#page=180

12.

Nunnally

J. C.

(1978). Psychometric theory. McGraw-Hill.

13.

Robertson

Kortum

(2019). An investigation of different methodologies for rating product satisfaction. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 63(1), 1259-1263. https://doi.org/10.1177%2F1071181319631071

14.

Robertson

Kortum

(2020). Validity of three discount methods for measuring perceived usability. Journal of Usability Studies, 16(1), 13-28. https://uxpajournal.org/wpcontent/uploads/sites/7/pdf/JUS_Robertson_Nov2020.pdf

15.

Tullis

Wood

(2004). How many users are enough for a card-sorting study? In Proceedings UPA (Vol. 2004). Minneapolis (EUA): Usability Professionals Association (UPA). https://www.researchgate.net/publication/254164354_How_Many_Users_Are_Enough_f or_a_Card-Sorting_Study

16.

Stamps

A. E.

(1990). Use of photographs to simulate environments: A meta-analysis. Perceptual and Motor Skills, 71, 907-913. https://doi.org/10.2466/PMS.71.7.907-913

17.

Xiang

Liang

Fang

Chen

Hedblom

Qiu

Gao

(2021). The comparisons of on-site and off-site applications in surveys on perception of and preference for urban green spaces: Which approach is more reliable? Urban Forestry and Urban Green, 58, 126961. https://doi.org/10.1016/j.ufug.2020.126961