Virtual research visits and direct-to-consumer genetic testing in Parkinson’s disease

Abstract

Objective

The purpose of this study was to conduct a proof-of-concept study to evaluate remote recruitment and assessment of individuals (“virtual research visits”) with Parkinson's disease who have pursued direct-to-consumer genetic testing.

Methods

Participants in 23andMe’s “Parkinson’s Research Community” were contacted by 23andMe. Fifty willing participants living in 23 states underwent a remote, standardized assessment including cognitive and motor tests by a neurologist via video conferencing and then completed a survey. Primary outcomes assessed were (a) proportion of participants who completed the remote assessments; (b) level of agreement (using Cohen’s kappa coefficient) of patient-reported data with that of a neurologist; and (c) interest in future virtual research visits.

Results

The self-reported diagnosis of Parkinson’s disease was confirmed in all cases (k = 1.00). The level of agreement for age of symptom onset (k = 0.97) and family history (k = 0.85) was very good but worse for falling (k = 0.59), tremor (k = 0.56), light-headedness (k = 0.31), and urine control (k = 0.15). Thirty-eight (76%) of the 50 participants completed a post-assessment survey, and 87% of respondents said they would be more or much more willing to participate in future clinical trials if they could do research visits remotely.

Conclusion

Remote clinical assessments of individuals with known genotypes were conducted nationally and rapidly from a single site, confirmed self-reported diagnosis, and were received favorably. Direct-to-consumer genetic testing and virtual research visits together may enable characterization of genotype and phenotype for geographically diverse populations.

Keywords

Direct-to-consumer genetic testing telemedicine Parkinson's disease genetics technology

Introduction

Technology is rapidly reshaping genetics and research studies, as new techniques identify the genetic underpinnings of many diseases.¹ Aided by the falling cost of DNA sequencing,² consumers can now assess their relative genetic risk for many disorders through direct-to-consumer genetic testing.³ For example, in Parkinson’s disease 16 genetic risk loci have been identified over the past 15 years.^4,5 Technology and rising drug development costs⁶ are fueling novel approaches to conducting clinical research and clinical trials,⁷ including enrichment for targeted genotypes, the use of social networks to conduct observational drug studies⁸ and virtual clinical trials.⁹

New collaborations continue to arise from these trends in genetics and clinical research. For example, the partnership between the direct-to-consumer (DTC) genetics company 23andMe and the Michael J. Fox Foundation for Parkinson’s disease research^10,11 has led to genotyping and collection of self-reported outcomes of over 10,000 individuals and currently represents the largest cohort of genotyped individuals with Parkinson’s disease.¹² Preliminary analyses of this data have identified new phenotype-genotype associations, risk factors, and differences in disease progression and symptoms associated with factors such as gender and body mass index.¹³ This research model has enabled broad and rapid research participation by individuals in 49 states and over 30 countries and allowed for the discovery of new genetic loci associated with Parkinson’s disease and replicated previously known associations.¹⁴ The ability to conduct more in-depth assessments and confirm self-reported data in this population could allow larger-scale participation in clinical trials and observational studies which could capture clinician-assessed phenotypic data at a single time point or longitudinally. Therapeutic trials targeted at specific genetic subpopulations could also use remote assessments to facilitate participation among geographically diverse cohorts of genotypically similar individuals. These genetic subpopulations may have different phenotypes,¹⁵ responses to current treatments,¹⁶ and almost certainly will be the preferred study population for future gene-targeted therapies, as has been the case for cancer,¹⁷ and is increasingly the case for other central nervous system disorders.¹⁸

One means of connecting to remote populations is virtual visits. These video visits, conducted via secure video conferencing, are increasingly used for clinical care,^19–22 including for Parkinson's disease,^23,24 but their application to research has been limited to date.

We conducted a proof of concept study aimed at (a) evaluating remote recruitment and assessment of individuals who have undergone DTC genetic testing, (b) assessing the level of agreement of self-reported phenotypic data with that of a specialist, and (c) soliciting feedback on virtual research visits.

Methods

Study design and participants

Members of 23andMe’s “Parkinson’s Research Community”²⁵ were invited to participate. Eligible members had contributed a DNA sample to 23andMe, reported a diagnosis of Parkinson’s disease, completed 23andMe’s Parkinson’s disease symptom survey developed in collaboration with expert Parkinson’s disease clinicians (Appendix 1), and had been active in the 23andMe community in the previous three months. To obtain a convenience sample of 50 participants, a random sample of 166 of these individuals received an email from 23andMe inviting them to participate, and to review and electronically sign the consent document and a data release authorization form. Participants had to have access to a non-public internet-enabled device. Researchers at Johns Hopkins (KCD, ERD) received contact information for interested participants and reviewed consent over the phone. Consenting participants received a web camera (Logitech C110 or C210 model) if needed, and an email link to download secure, Health Insurance Portability and Accountability Act-compliant videoconferencing software from Vidyo (Hackensack, New Jersey, USA), hosted by ID Solutions (Indianapolis, Indiana, USA). The study team (KCD, SD) also provided technical assistance by phone if necessary. Researchers performed a test connection between the remote research site and the participant’s home prior to the one-time assessment. The Institutional Review Board at Johns Hopkins Medicine approved the research protocol and consent form.

Assessments

A neurologist (ERD) conducted the 30–60 min remote assessment, which was structured to elicit information confirming the participants’ responses to the survey participants had completed upon enrollment in the 23andMe project (Appendix 1). Visits, which were modeled similar to a remote clinic visit,²⁴ included a review of participants’ histories, including symptoms of atypical parkinsonian disorders, onset of specific symptoms (age-at-onset of parkinsonian symptoms, falling, urine control problems, light-headedness, and tremor), and family history (first or second degree relative with Parkinson’s disease). The specialist completed part IA, non-motor aspects of experiences of daily living, part III, motor examination (excluding rigidity and postural stability tests which require in-person assessment), and part IV, motor complications, of the Movement Disorder Society Unified Parkinson’s Disease Rating Scale (MDS-UPDRS).²⁶ For the motor portion of the examination, individuals were asked to perform the tasks (e.g. tapping thumb and index finger, walking) in front of the camera, in line with a previous remote study.²³ Visits also included the Montreal Cognitive Assessment, a cognitive assessment tool commonly used in Parkinson’s disease.^27,28 Participants were emailed a copy of the visuospatial/executive and naming portions, and completed and demonstrated these tasks in the video call before the examiner completed the remainder of the test. The neurologist determined the most likely diagnosis based on the history and examination focused on the cardinal features of parkinsonism that can be assessed remotely (e.g. rest tremor, bradykinesia, and gait difficulties). Parkinson's disease was identified as the diagnosis if it was the most likely explanation for the individual’s condition as opposed to other causes of parkinsonism (e.g. multiple system atrophy).

Following completion of the clinical assessments, participants were emailed a survey developed for this study asking about their satisfaction with the virtual research visit, comfort in discussing their condition with the specialist, and willingness to participate in future trials remotely (Appendix 2). Completed surveys were returned via US mail, email, or fax and labeled with participant names. Responses were entered manually into an Excel database.

Analysis

Most of the analyses performed were descriptive. Cohen’s kappa coefficient²⁹ was calculated to determine the level of agreement between the participants’ self-reported outcomes on the survey that they initially completed as part of the 23andMe project (Appendix 1) and those assessed by the specialists. The analyses were conducted by NE using Excel.

Results

Study participants and feasibility

A total of 166 individuals were invited to participate, and 103 opened the email invitation. Fifty-four (32%) agreed to participate and 50 (30%) completed the one-time assessment as scheduled between 25 March 2013–5 June 2013. The reasons for not completing the assessment were: unreachable by phone (n = 2); withdrawal of consent (n = 1); and a scheduling conflict (n = 1). The baseline characteristics of those invited to participate and those that did were generally similar (Table 1). The severity of Parkinson’s disease among participants varied widely, including duration (mean 9.2 years, range from 2–28 years), motor signs (mean motor score 31, range from 18–52; higher scores indicate greater impairment), and cognitive impairment (mean score on the Montreal Cognitive Assessment was 27, range from 18–30; higher scores indicate less impairment). Overall, the characteristics of the participants in this study were similar to those in clinical trials aimed at individuals with moderate Parkinson's disease.³⁰ Six (12%) participants had genotypes associated with at least 50% increased relative risk of developing Parkinson’s disease (Table 1). The participants came from 23 states and the District of Columbia (Figure 1).

Table 1.

Baseline characteristics of study participants.

	Invited to participate (n = 166)	Completed remote assessment (n = 50)	Completed survey (n = 38)
Demographics
Age in years (SE)	63.6 (0.8)	62.4 (1.5)	63.3 (1.8)
Number (%) women	64 (38.6)	16 (32.0)	13 (34.2)
Number (%) Caucasian	153 (92.2)	48 (95.7)	34 (89.5)
Education (number (%) completing college)	101 (60.9)^a	37 (73.5)	28 (73.7)
Parkinson’s disease characteristics
Number (%) with Parkinson’s disease	166 (100.0)	50 (100.0)	38 (100.0)
Disease duration in years (SE)	9.6 (0.5)	9.2 (0.9)	9.2 (0.8)
Movement Disorder Society-Unified Parkinson’s Disease Rating Scale, part Ia: Non-Motor Aspects of Experiences of Daily Living score (SE)^b	NA	3.0 (0.4)	3.4 (0.6)
Movement Disorder Society - Unified Parkinson’s Disease Rating Scale, Part III: Modified Motor Score (SE)^c	NA	31.3 (1.1)	30.6 (1.5)
Movement Disorder Society - Unified Parkinson’s Disease Rating Scale, Part IV: Motor Complications Score (SE)^b	NA	3.5 (0.5)	3.8 (0.7)
Montreal Cognitive Assessment^d (SE)	NA	26.9 (0.4)	26.5 (0.5)
Number (%) with a family history of Parkinson’s disease	56 (33.7)	19 (38.0)	14 (36.8)
Number (%) with GBA or LRRK2 mutations	6 (3.6)	3 (6.0)	1 (2.6)
Number (%) with substantially increased risk due to other common genetic variants^e	12 (7.2)	3 (6.0)	3 (7.9)

GBA: glucocerebrosidase gene; LRRK2: leucine-rich repeat kinase 2; SE: standard error.

Responses on education are only available for the individuals who were invited to participate.

Scale ranges from with 0–24 with higher scores indicating greater impairment.

Excludes assessments of rigidity and balance. Scale ranges from 0–116 with higher scores indicating greater impairment.

Scale ranges from 0–30 with lower scores indicating greater cognitive impairment.

Over 1.5 times average risk due to common genetic variants associated with Parkinson's disease in populations of European descent (single nucleotide polymorphisms near SNCA, MAPT, PARK16, BST1, DGKQ, and STK39).

Figure 1.

Map of locations of research participants by state.

Level of agreement

Based on the history and examination, all individuals with self-reported Parkinson’s disease were judged by the clinician to have Parkinson’s disease (k = 1.00). The level of agreement between the self-reported data and that assessed by the remote clinician was also very high for age of onset (k = 0.97) and presence of family history (k = 0.85) but much lower for the presence of falling (k = 0.59), tremor (k = 0.56), and non-motor symptoms of light-headedness (k = 0.31), and problems with urine control (k = 0.15).

Post-assessment survey

Thirty-eight of the 50 participants (76% response rate) completed the post-assessment survey. Respondents were satisfied or highly satisfied with the virtual research visit and with the specialist’s ability to assess their symptoms (95%), and comfortable or very comfortable discussing their condition with the specialist (100%). Nearly all respondents were comfortable or very comfortable (97%) with the software. Satisfaction with connection quality was lower (84% satisfied or very satisfied). Nearly all (95%) reported interest in future virtual research visits, and 87% reported that they would be willing or much more willing to participate in clinical trials if they could do visits remotely from their home.

Discussion

Remote recruitment and conduct of research visits in individuals who have undergone DTC genetic testing is feasible, shows relatively good agreement with self-reported data, and is well received by participants. While the benefits of DTC genetic testing continue to be debated,^{1,2,11,31–39} this proof of concept study demonstrates its potential to identify, follow, and engage geographically dispersed populations with a given disorder and an underlying genotype in research.⁴⁰ Our 32% response rate to a single recruitment email suggests that targeted electronic recruitment in populations unrestricted by geography may improve participation in research studies. Traditional approaches requiring genetic testing of large populations at multiple medical centers require substantial investment in time and resources to satisfy regulatory needs, execute contracts, and conduct in-person assessments at each institution. By contrast, this study took less than three months and utilized a single investigator to assess individuals throughout the country. This approach lays the foundation for larger scale, broader scope studies that can target genetic sub-populations of different disorders and foster partnerships between consumer genetics companies, pharmaceutical firms, academic institutions, and patient communities. We envision the possibility of conducting remote phenotypic assessments in very large cohorts, marrying data from clinical rating scales such as the MDS-UPDRS with genetic information to better characterize and elucidate variability in Parkinson’s disease symptoms in relation to genetic markers.

The Institute of Medicine has recently called for such collaborations. In its 2012 workshop on genome-based therapies, the Institute highlighted the need to mobilize patient communities to spur research.⁴¹ In its Rare diseases and orphan products report, one of the key elements of its national strategy to promote rare diseases research and development is “[timely] application of advances in science and technology that can make rare diseases research and product development faster, easier, and less expensive.[3]”⁴²

Remote assessments are likely to play an increasingly large role in clinical studies of Parkinson's disease⁴³ and other chronic conditions.⁴⁴ Several clinical trials have been conducted over the internet^45–49 with many forgoing in-person visits altogether.^45,47–49 Previous clinical studies have found self-reported data to be valid,⁵⁰ including for self-reported diagnosis.⁵¹ This study extends these disruptive models for clinical research by laying the foundation for the inclusion of increasingly available genotypic data into remotely conducted studies.

This study has three key limitations. First, the study population was highly selected – only the most recent and active participants in the 10,000-member “Parkinson’s Community” were asked to participate. These individuals who are highly educated are also more likely to have the internet and technology savvy required to be able to install and operate videoconferencing software, as well as access to high-speed broadband connections, without which remote assessment would be much more difficult. However, many individuals show interest in Parkinson’s disease clinical trial participation, including more than 30,000 individuals registered (all remotely) in the Michael J. Fox Foundation’s “Fox Trial Finder” as of September 2014.⁵² Because less than a third (n = 50) of the invited population (n = 166) participated in the study, the results should be interpreted with caution as the low response rate may have resulted in selection bias. Similarly, while the survey response was good (76%), the respondents may have been more satisfied than non-respondents.

Second, some questions used in the background survey on falls, tremor, lightheadedness, and urine control were subjective and non-specific and may have contributed to the low level of agreement with the remote assessment. In addition, while a previous study demonstrated good agreement between remote and in-person assessments of the original UPDRS,⁵³ remote administrations of the newer MDS-UPDRS and the Montreal Cognitive Assessment scales remain to be validated.

Third, this study only included a one-time assessment. Future studies could be used to evaluate remote research visits in assessing the natural history of different genetic subpopulations, or as a screening or interim assessment in interventional studies.

Notwithstanding these limitations, this study demonstrates the feasibility and potential value of combining phenotypic and genotypic data for research participants across half the country using remote videoconferencing assessments. These novel technology applications and partnerships can accelerate research at low cost and enhance our understanding of the natural history of genetically linked disorders.

Footnotes

Acknowledgements

The authors wish to thank Tim Harris and Ajay Verma from Biogen for their thoughtful review and critique.

Funding

This study was supported by a research grant from Biogen Idec (Cambridge, Massachusetts, USA) and by the efforts of 23andMe (Mountain View, California, USA).

Conflict of interest

ERD is an advisor to, and has stock options in, Grand Rounds; is a compensated consultant to Clintrex, mc10, Shire, and National Institute of Neurological Disorders and Stroke; is an unpaid advisor to SBR Health and Vidyo; receives grant support from Auspex Pharmaceuticals, Prana Biotechnology, the Patient-Centered Outcomes Research Institute, Davis Phinney Foundation, Michael J. Fox Foundation, Huntington Study Group, and Great Lakes Neurotechnologies; and has filed a patent application related to neurology and telemedicine. KD, SD, AT and MA have no conflicting interests. EC is an employee of Biogen Idec. SM and EC are employees of 23andMe. NE was an employee of 23andMe. BR and SW were previously employees of Biogen Idec.

Ethical approval

The Institutional Review Board at Johns Hopkins Medicine approved the research protocol and consent form (IRB Application Number NA_00081180).

Guarantor

ERD

Contributorship

Contributors to the study were as follows: ERD contributed to the design, organization, and execution of the research, participated in designing and reviewing the statistical analysis, and wrote the first draft of the manuscript with KCD. KCD, SM, SD, AT contributed to the execution of the research. EDC contributed to the design, organization, and execution of the research. NE led the design and execution of the statistical analysis. BR contributed to the design of the research and participated in designing and reviewing the statistical analysis. All authors contributed to the analysis and interpretation of the data and review and edited the manuscript and approved the final version of the manuscript.

Peer review

This manuscript was reviewed by Paul Wicks (PatientsLikeMe), and two other reviewers who wish to remain anonymous.

Appendix 1. 23andMe Parkinson’s disease baseline survey

References

Topol

. The creative destruction of medicine: How the digital revolution will create better health care, New York: Basic Books, 2012.

Altman

. Direct-to-consumer genetic testing: Failure is not an option. Clin Pharmacol Ther 2009; 86: 15–18.

. Direct-to-consumer genetic testing: A comprehensive view. Yale J Biol Med 2013; 86: 359–365.

Singleton

Farrer

Bonifati

. The genetics of Parkinson's disease: Progress and therapeutic implications. Mov Disord 2013; 28: 14–23.

Nalls

Plagnol

Hernandez

. International Parkinson Disease Genomics Consortium. Imputation of sequence variants for identification of genetic risks for Parkinson's disease: A meta-analysis of genome-wide association studies. Lancet 2011; 377: 641–649.

DiMasi

Hansen

Grabowski

. The price of innovation: New estimates of drug development costs. J Health Econ 2003; 22: 151–185.

Lipset

. Telethinking with Craig Lipset. Interview by Jamie Devereaux. Telemed J E Health 2010; 16: 857–859.

Wicks

Vaughan

Massagli

. Accelerated clinical discovery using self-reported patient data collected online and a patient-matching algorithm. Nat Biotech 2011; 29: 411–414.

US National Institutes of Health. Web-based methodology trial to evaluate the efficacy and safety of tolterodine ER in subjects with overactive bladder (REMOTE). Sponsor: Pfizer, http://clinicaltrials.gov/ct2/show/NCT01302938?term=NCT01302938&rank=1 (2013, accessed 7 October 2013).

10.

Bonetta

. New citizens for the life sciences. Cell 2009; 138: 1043–1045.

11.

Fox

. What price personal genome exploration? Nat Biotech 2008; 26: 1105–1108.

12.

Teichholtz H. Foxfeed blog: 23andMe Parkinson's community hits 10,000 members, https://www.michaeljfox.org/foundation/news-detail.php?23andme-parkinson-community-hits-10-000-members-enrollment-still-free-of-charge-to-parkinson-patients (2013, accessed 3 October 2013).

13.

Hadly S. Foxfeed blog: 23andMe reports differences in Parkinson’s disease symptoms between men and women, https://www.michaeljfox.org/foundation/news-detail.php?23andme-reports-differences-in-parkinson-symptoms-between-men-and-women (2014, accessed 9 September 2014).

14.

Tung

Dorfman

. Web-based genome-wide association study identifies two novel loci and a substantial genetic component for Parkinson's disease. PLoS Genet 2011; 7: e1002141.

15.

Healy

Falchi

O'Sullivan

. Phenotype, genotype, and worldwide genetic penetrance of LRRK2-associated Parkinson's disease: A case-control study. Lancet Neurol 2008; 7: 583–590.

16.

Khan

Graham

Critchley

. Parkin disease: A phenotypic study of a large case series. Brain 2003; 126: 1279–1292.

17.

Dachs

Dougherty

Stratford

. Targeting gene therapy to cancer: A review. Oncol Res 1997; 9: 313–325.

18.

Miller

Smith

Kordasiewicz

. Gene-targeted therapies for the central nervous system. Arch Neurol 2008; 65: 447–451.

19.

Verhoeven

van Gemert-Pijnen

Dijkstra

. The contribution of teleconsultation and videoconferencing to diabetes care: A systematic literature review. J Med Internet Res 2007; 9: e37.

20.

Fortney

Pyne

Mouden

. Practice-based versus telemedicine-based collaborative care for depression in rural federally qualified health centers: A pragmatic randomized comparative effectiveness trial. Am J Psychiatry 2013; 170: 414–425.

21.

Ferrer-Roca

Garcia-Nogales

Pelaez

. The impact of telemedicine on quality of life in rural areas: The Extremadura model of specialized care delivery. Telemed J E Health 2010; 16: 233–243.

22.

Stahl

Dixon

. Acceptability and willingness to pay for primary care videoconferencing: A randomized controlled trial. J Telemed Telecare 2010; 16: 147–151.

23.

Dorsey

Deuel

Voss

. Increasing access to specialty care: A pilot, randomized controlled trial of telemedicine for Parkinson's disease. Mov Disord 2010; 25: 1652–1659.

24.

Dorsey

Venkataraman

Grana

. Randomized controlled clinical trial of “virtual house calls” for Parkinson disease. JAMA Neurol 2013; 70: 565–570.

25.

23andMe. 23andMe Parkinson's research community online, https://www.23andme.com/pd/ (accessed 19 September 2014).

26.

Goetz

Tilley

Shaftman

. Movement Disorder Society–sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS): Scale presentation and clinimetric testing results. Mov Disord 2008; 23: 2129–2170.

27.

Gill

Freshman

Blender

. The Montreal Cognitive Assessment as a screening tool for cognitive impairment in Parkinson's disease. Mov Disord 2008; 23: 1043–1046.

28.

Nasreddine

Phillips

Bédirian

. The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005; 53: 695–699.

29.

Carletta

. Assessing agreement on classification tasks: The kappa statistic. Comput Linguist 1996; 22: 249–254.

30.

Rascol O, Brooks DJ, Melamed E, et al. Rasagiline as an adjunct to levodopa in patients with Parkinson's disease and motor fluctuations (LARGO, Lasting effect in Adjunct therapy with Rasagiline Given Once daily, study): A randomised, double-blind, parallel-group trial. Lancet 365: 947–954.

31.

Bloss

Schork

Topol

. Effect of direct-to-consumer genomewide profiling to assess disease risk. N Engl J Med 2011; 364: 524–534.

32.

Gollust

Hull

Wilfond

. Limitations of direct-to-consumer advertising for clinical genetic testing. JAMA 2002; 288: 1762–1767.

33.

Hogarth

Javitt

Melzer

. The current landscape for direct-to-consumer genetic testing: Legal, ethical, and policy issues. Annu Rev Genomics Hum Genet 2008; 9: 161–182.

34.

Hudson

Javitt

Burke

. ASHG statement on direct-to-consumer genetic testing in the United States. Am J Hum Genet 2007; 81: 635–637.

35.

Kuehn

. Risks and benefits of direct-to-consumer genetic testing remain unclear. JAMA 2008; 300: 1503–1505.

36.

McGuire

Burke

. An unwelcome side effect of direct-to-consumer personal genome testing: Raiding the medical commons. JAMA 2008; 300: 2669–2671.

37.

Roberts

Ostergren

. Direct-to-consumer genetic testing and personal genomics services: A review of recent empirical studies. Curr Genet Med Rep 2013; 1: 182–200.

38.

Hauskeller C. Direct to consumer genetic testing. BMJ 2011; 342: d2317.

39.

US Food and Drug Administration. Warning Letter [to 23andMe, Inc.], http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2013/ucm376296.htm (2013, accessed 3 December 2013).

40.

Couzin

. Parkinson's disease. Streamlined clinical trials, from a home computer. Science (New York, NY). 2008; 320: 1143.

41.

Rapporteurs: Berger AC, Olson S. Genome-based therapeutics: Targeted drug discovery and development: Workshop summary. Washington, DC: The National Academies Press, 2012.

42.

Field MJ and Boat TF (eds). Rare diseases and orphan products: Accelerating research and development. Washington, DC: The National Academies Press, 2011.

43.

Achey

Aldred

Aljehani

. The past, present, and future of telemedicine for Parkinson's disease. Mov Disord 2014; 29: 871–883.

44.

Dorsey

Venuto

Venkataraman

. Novel methods and technologies for 21st-century clinical trials: A review. JAMA Neurol 2015; 72(5): 582–558.

45.

McAlindon

. Why are clinical trials of glucosamine no longer uniformly positive? Rheum Dis Clin North Am 2003; 29: 789–801.

46.

Eilenberg

Hoover

Rutherford

. From informed consent through database lock: An interactive clinical trial conducted using the internet. Therapeutic Innovation & Regulatory Science 2004; 38(3): 239–251.

47.

Jacobs

Bent

Tice

. An internet-based randomized, placebo-controlled trial of kava and valerian for anxiety and insomnia. Medicine (Baltimore) 2005; 84: 197–207.

48.

Wicks

Vaughan

Massagli

. Accelerated clinical discovery using self-reported patient data collected online and a patient-matching algorithm. Nat Biotechnol 2011; 29: 411–414.

49.

Orri

Lipset

Jacobs

. Web-based trial to evaluate the efficacy and safety of tolterodine ER 4 mg in participants with overactive bladder: REMOTE trial. Contemp Clin Trials 2014; 38: 190–197.

50.

Cascade

Marr

Winslow

. Conducting research on the Internet: Medical record data integration with patient-reported outcomes. J Med Internet Res 2012; 14: e137.

51.

Sharkey

Zoellner

Abadin

. Validity of participant-reported diagnoses in an online patient registry: A report from the NF1 Patient Registry Initiative. Contemp Clin Trials 2015; 40: 212–217.

52.

The Michael J. Fox Foundation for Parkinson's Research. Fox trial finder 2013, https://foxtrialfinder.michaeljfox.org/ (2013, accessed 8 October 2013).

53.

Cubo

Gabriel-Galan

Martinez

. Comparison of office-based versus home web-based clinical assessments for Parkinson's disease. Mov Disord 2012; 27: 308–311.