Text analysis of motivations for (not) donating smartphone sensor data

Abstract

Open-ended answers in surveys capture rich motivations but are costly to code by hand. We study respondents’ stated reasons for (probable) refusal to share smartphone-sensor data, using two closely related Dutch questionnaires fielded in 2017–2018: a LISS panel study and a Statistics Netherlands (SN) consent survey. The responses in the LISS panel were coded manually, while there are no manual codes available for the SN consent survey. We transfer an 11-category motivation taxonomy from the LISS panel to the SN consent survey via a transparent NLP pipeline, using rule-based keyword extraction and elastic-net logistic regression. The manually coded responses in the LISS panel can serve as training set and offer the possibility to evaluate the responses classified by the NLP pipeline. The cross-validated AUCs applied to the LISS data are high for core categories like Privacy, Safety and Due to emotions. The results from both surveys show the following reasons for refusal: privacy dominates; control and brief refusals (without reason) follow. On the reasons per task, it follows from the consent survey, that location elicits comparatively more control/surveillance, while the house photo is most often flagged as effortful. Demographic contrasts are modest but suggestive (e.g., more effort among ages 50–67; more control/surveillance mentions among younger groups). For this application, it is assumed that for practical use, the LISS-trained classification model is transferable to the SN consent survey.

Keywords

text-mining techniques survey open-ended questions natural language processing smartphone sensor data

Get full access to this article

View all access options for this article.

References

Struminskaya

Toepoel

Lugtig

, et al. Understanding willingness to share smartphone sensor data. Public Opin Q 2020; 84: 725–759.

Struminskaya

Lugtig

Toepoel

, et al. Sharing data collected with smartphone sensors: willingness participation and nonparticipation bias. Public Opin Q 2021; 85: 423–462.

Keusch

Struminskaya

Antoun

, et al. Willingness to participate in passive mobile data collection. Public Opin Q 2019; 83: 210–235.

Chai

. Text mining in survey data. Surv Pract 2019; 12: 1–14.

Seni

Elder

. Ensemble methods in data mining: improving accuracy through combining predictions. Synthesis Lectures on Data Mining and Knowledge Discovery 2010; 2: 1–126.

Grimmer

Stewart

. Text as data: the promise and pitfalls of automatic content analysis methods for political texts. Polit Anal 2013; 21: 267–297.

Roberts

Stewart

Tingley

, et al. Structural topic models for open-ended survey responses. Am J Pol Sci 2014; 58: 1064–1082.

Straka

Hajič J

. UDPipe: trainable pipeline for processing CoNLL-U files performing tokenization, morphological analysis, POS tagging and parsing. In: Calzolari N, Choukri K, Declerck T, et al. (eds) LREC 2016 proceedings. Portorož, Slovenia: European Language Resources Association (ELRA), 2016.

Wijffels

. UDPipe: tokenization, parts of speech tagging, lemmatization and dependency parsing with the ‘UDPipe’ ‘NLP’ toolkit, https://CRAN.R-project.org/package=udpipe (2023).

10.

Friedman

Tibshirani

Hastie

. Regularization paths for generalized linear models via coordinate descent. J Stat Softw 2010; 33: 1–22.

11.

Tay

Narasimhan

Hastie

. Elastic net regularization paths for all generalized linear models. J Stat Softw 2023; 106: 1–33.

12.

Daumé III

. Frustratingly easy domain adaptation. In: Proceedings of the 45th annual meeting of the association for computational linguistics, 2007, pp.256–263.

13.

Pan

Yang

. A survey on transfer learning. IEEE Trans Knowl Data Eng 2009; 22: 1345–1359.

14.

de Groot

. Onderzoeksdesign win consent survey 2018, 2018.

15.

Hornik

Rauch

Buchta

, et al. textcat: N-gram based text categorization, https://CRAN.R-project.org/package=textcat (2023).

16.

Universaldependenciesorg. Universal dependencies (UD), https://universaldependencies.org/ (2014–2024).

17.

Swets

. Signal detection theory and ROC analysis in psychology and diagnostics: Collected papers. Scientific Psychology Series. Hillsdale, NJ, US: Lawrence Erlbaum Associates, Inc., 1996.

18.

Fawcett

. An introduction to ROC analysis. Pattern Recognit Lett 2006; 27: 861–874.

19.

Allen

. The relationship between variable selection and data agumentation and a method for prediction. Technometrics 1974; 16: 125–157.

20.

Stone

. An asymptotic equivalence of choice of model by cross-validation and Akaike’s criterion. J R Stat Soc Ser B (Methodol) 1977; 39: 44–47.