Mobile phone usage is typically measured via self-reporting. However, scholars have questioned the validity of self-reported data, which may lead to Type I or Type II errors. Using an online survey (n = 777), this study compared self-reported and log mobile phone usage data using a simplified version of the mobile data donation method. The results showed that people generally underreported their mobile phone usage in terms of time duration, the number of notifications, and apps used. Moreover, self-reported data may either have no additional effect on or overestimate the communication findings depending on the outcome variables. This challenges the Type II error explanation and suggests that the effect sizes of self-reported data might not be underestimated after all. Instead, past research examining mobile use and pertinent outcomes may have false-positive findings and Type I errors. Given the potential inaccuracies of self-reported data, future research on mobile media and communications should go beyond self-reported data to enhance the validity of findings.
BartelsL. M. (1993). Messages received: The political impact of media exposure. American Political Science Review, 87(2), 267–285. https://doi.org/10.2307/2939040
2.
BoaseJ.HumphreysL. (2018). Mobile methods: Explorations, innovations, and reflections. Mobile Media & Communication, 6(2), 153–162. https://doi.org/10.1177/2050157918764215
3.
BoaseJ.LingR. (2013). Measuring mobile phone use: Self-report versus log data. Journal of Computer-Mediated Communication, 18(4), 508–519. https://doi.org/10.1111/jcc4.12021
4.
BurnellK.GeorgeM. J.KurupA. R.UnderwoodM. K.AckermanR. A. (2021). Associations between self-reports and device-reports of social networking site use: An application of the truth and bias model. Communication Methods and Measures, 15(2), 156–163. https://doi.org/10.1080/19312458.2021.1918654
5.
ChanM.LiX. (2020). Smartphones and psychological well-being in China: Examining direct and indirect relationships through social support and relationship satisfaction. Telematics and Informatics, 54, 1–11. https://doi.org/10.1016/j.tele.2020.101469
6.
CicchettiD. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. https://doi.org/10.1037/1040-3590.6.4.284
7.
CohenA. A.LemishD. (2003). Real time and recall measures of mobile phone use: Some methodological concerns and empirical applications. New Media & Society, 5(2), 167–183. https://doi.org/10.1177/1461444803005002002
8.
CohenJ. (2013). Statistical power analysis for the behavioral sciences. Routledge.
9.
CummingG. (2009). Inference by eye: Reading the overlap of independent confidence intervals. Statistics in Medicine, 28(2), 205–220. https://doi.org/10.1002/sim.3471
10.
DengT.KanthawalaS.MengJ. B.PengW.KononovaA.HaoQ.ZhangQ. H.DavidP. (2019). Measuring smartphone usage and task switching with log tracking and self-reports. Mobile Media & Communication, 7(1), 3–23. https://doi.org/10.1177/2050157918761491
ElhaiJ. D.LevineJ. C.DvorakR. D.HallB. J. (2016). Fear of missing out, need for touch, anxiety and depression are related to problematic smartphone use. Computers in Human Behavior, 63, 509–516. https://doi.org/10.1016/j.chb.2016.05.079
13.
EllisD. A.DavidsonB. I.ShawH.GeyerK. (2019). Do smartphone usage scales predict behavior?International Journal of Human-Computer Studies, 130, 86–92. https://doi.org/10.1016/j.ijhcs.2019.05.004
14.
GötzF. M.StiegerS.ReipsU.-D. (2017). Users of the main smartphone operating systems (iOS, Android) differ only little in personality. PLoS ONE, 12(5), e0176921. https://doi.org/10.1371/journal.pone.0176921
15.
HodesL. N.ThomasK. G. F. (2021). Smartphone screen time: Inaccuracy of self-reports and influence of psychological and contextual factors. Computers in Human Behavior, 115, 106616. https://doi.org/10.1016/j.chb.2020.106616
16.
HuangY.-H. (2001). OPRA: A cross-cultural, multiple-item scale for measuring organization-public relationships. Journal of Public Relations Research, 13(1), 61–90. https://doi.org/10.1207/S1532754XJPRR1301_4
17.
HughesM. E.WaiteL. J.HawkleyL. C.CacioppoJ. T. (2004). A short scale for measuring loneliness in large surveys: Results from two population-based studies. Research on Aging, 26(6), 655–672. https://doi.org/10.1177/0164027504268574
18.
Jones-JangS. M.HeoY. J.McKeeverR.KimJ. H.MoscowitzL.MoscowitzD. (2020). Good news! Communication findings may be underestimated: Comparing effect sizes with self-reported and logged smartphone use data. Journal of Computer-Mediated Communication, 25(5), 346–363. https://doi.org/10.1093/jcmc/zmaa009
19.
KahnA. S.RatanR. (2014). Why we distort in self-report: Predictors of self-report errors in video game play. Journal of Computer-Mediated Communication, 19(4), 1010–1023. https://doi.org/10.1111/jcc4.12056
20.
KobayashiT.BoaseJ. (2012). No such effect? The implications of measurement error in self-report measures of mobile communication used. Communication Methods and Measures, 6(2), 126–143. https://doi.org/10.1080/19312458.2012.679243
21.
KwonM.LeeJ. Y.WonW. Y.ParkJ. W.MinJ. A.HahnC.GuX.ChoiJ. H.KimD. J. (2013). Development and validation of a smartphone addiction scale (SAS). PLoS ONE, 8(2), e56936. https://doi.org/10.1371/journal.pone.0056936
22.
LeeH.AhnH.NguyenT. G.ChoiS. W.KimD. J. (2017). Comparing the self-report and measured smartphone usage of college students: A pilot study. Psychiatry Investigation, 14(2), 198–204. https://doi.org/10.4306/pi.2017.14.2.198
23.
LichtensteinS.SlovicP.FischhoffB.LaymanM.CombsB. (1978). Judged frequency of lethal events. Journal of Experimental Psychology: Human Learning and Memory, 4(6), 551–578. https://doi.org/10.1037/0278-7393.4.6.551
24.
LinY. H.LinP. H.ChiangC. L.LeeY. H.YangC. C. H.KuoT. B. J.LinS. H. (2017). Incorporation of Mobile application (app) measures into the diagnosis of smartphone addiction. Journal of Clinical Psychiatry, 78(7), 866–872. https://doi.org/10.4088/JCP.15m10310
25.
LinY. H.LinY. C.LeeY. H.LinP. H.LinS. H.ChangL. R.TsengH. W.YenL. Y.YangC. C. H.KuoT. B. J. (2015). Time distortion associated with smartphone addiction: Identifying smartphone addiction via a mobile application (app). Journal of Psychiatric Research, 65, 139–145. https://doi.org/10.1016/j.jpsychires.2015.04.003
26.
LoidK.TahtK.RozgonjukD. (2020). Do pop-up notifications regarding smartphone use decrease screen time, phone checking behavior, and self-reported problematic smartphone use? Evidence from a two-month experimental study. Computers in Human Behavior, 102, 22–30. https://doi.org/10.1016/j.chb.2019.08.007
27.
OhmeJ.AraujoT.de VreeseC. H.PiotrowskiJ. T. (2020). Mobile data donations: Assessing self-report accuracy and sample biases with the iOS screen time function. Mobile Media & Communication, 9(2), 293–313. https://doi.org/10.1177/2050157920959106
28.
ParryD. A.DavidsonB. I.SewallC. J. R.FisherJ. T.MieczkowskiH.QuintanaD. S. (2021). A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use. Nature Human Behaviour, 5(11), 1535–1547. https://doi.org/10.1038/s41562-021-01117-5
29.
RadloffL. S. (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385–401. https://doi.org/10.1177/014662167700100306
30.
RydingF. C.KussD. J. (2020). Passive objective measures in the assessment of problematic smartphone use: A systematic review. Addictive Behaviors Reports, 11, 100257. https://doi.org/10.1016/j.abrep.2020.100257
31.
ScharkowM. (2016). The accuracy of self-reported internet us—A validation study using client log data. Communication Methods and Measures, 10(1), 13–27. https://doi.org/10.1080/19312458.2015.1118446
32.
ScharkowM. (2019). The reliability and temporal stability of self-reported media exposure: A meta-analysis. Communication Methods and Measures, 13(3), 198–211. https://doi.org/10.1080/19312458.2019.1594742
TaiY.FuK. W. (2020). Specificity, conflict, and focal point: A systematic investigation into social media censorship in China. Journal of Communication, 70(6), 842–867. https://doi.org/10.1093/joc/jqaa032
35.
TourangeauR.RipsL. J.RasinskiK (2000). The psychology of survey response. Cambridge University Press, pp.100–135.
Vanden AbeeleM.BeullensK.RoeK. (2013). Measuring mobile phone use: Gender, age and real usage level in relation to the accuracy and validity of self-reported mobile phone use. Mobile Media & Communication, 1(2), 213–236. https://doi.org/10.1177/2050157913477095
38.
Vanden AbeeleM. M. P.HendricksonA. T.PollmannM. M. H.LingR. (2019). Phubbing behavior in conversations and its relation to perceived conversation intimacy and distraction: An exploratory observation study. Computers in Human Behavior, 100, 35–47. https://doi.org/10.1016/j.chb.2019.06.004
39.
Vigil-ColetA.Morales-VivesF.Lorenzo-SevaU. (2013). How social desirability and acquiescence affect the age–personality relationship. Psicothema, 25(3), 342–348. https://doi.org/10.7334/psicothema2012.297
40.
VrijheidM.DeltourI.KrewskiD.SanchezM.CardisE. (2006). The effects of recall errors and of selection bias in epidemiologic studies of mobile phone use and cancer risk. Journal of Exposure Science & Environmental Epidemiology, 16(4), 371–384. https://doi.org/10.1038/sj.jes.7500509
41.
WooldridgeJ. M. (2015). Introductory econometrics: A modern approach. Cengage learning.
42.
Wu-OuyangB. (2022). Are smartphones addictive? Examining the cognitive-behavior model of motivations, leisure boredom, extended self, and fear of missing out on possible smartphone addiction. Telematics and Informatics, 71, 101834. https://doi.org/10.1016/j.tele.2022.101834
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
