Abstract
Background
After the passage of the 21st Century Cures Act in the U.S., the Inclusion Across the Lifespan policy eliminates upper-age limits for research participation unless risk justified. Broader inclusion will necessitate the use of reliable instruments in research that characterize the health status and function of older adults with multiple chronic conditions. As there is a plethora of such instruments, the Geriatrics Research Instrument Library (GRIL) was developed as freely available online resource of data collection instruments commonly used in gerontological research. GRIL has been revised and updated by the Advancing Geriatrics Infrastructure and Network Growth (AGING) Initiative, a joint endeavor of the Health Care Systems Research Network (HCSRN) and the Older Americans Independence Centers (OAICs).
Methods
Extensive PubMed literature searches and domain expert feedback were utilized to inventory and update GRIL through the addition of instruments and compiling of instrument metadata. GRIL is hosted on the National Institute on Aging OAIC Coordinating Center website with a platform utilizing Microsoft Structured Query Language (SQL) and an Adobe ColdFusion application server. Tracking statistics are collected using Google Analytics.
Results
Presently, GRIL includes 175 instruments across 18 domains, including instrument metadata such as instrument description, copyright information, completion time estimates, keywords, available translations, and a link and reference to the original manuscript describing the instrument. The GRIL website includes user-friendly features such as mobile platforming and resource links.
Conclusions
GRIL provides a user-friendly public resource that facilitates clinical researchers in efficiently selecting appropriate instruments to measure clinical outcomes relevant to older adults across a full range of domains.
Background
Over the past several decades, a large number of instruments have been developed to evaluate a wide spectrum of domains to measure and assess the health status and care of older adults in research settings. Many newly developed instruments and variations of existing measures (e.g., short forms, adaptations for different age groups, translations) often lack the appropriate validation prior to their use.1,2 As the number of instruments continues to grow, each offering unique advantages and limitations, it is often challenging for researchers to efficiently find appropriate instruments for addressing their complex study questions in measuring different domains in older adults. As the population ages and the proportion of individuals with multiple chronic conditions increases dramatically, there is a crucial need for a curated library of instruments that facilitates investigators’ ability to select the instrument that meets the needs and purpose of studies on older adults with multiple chronic conditions.
The geriatrics research instrument library (GRIL)
(https://www.peppercenter.org/public/gril.cfm) was developed as an online resource of data collection instruments in the English language commonly used in geriatrics and gerontological research. The aim of this manuscript is to describe the development of GRIL, its features, and the procedures for maintaining its relevance and value as a resource for clinical investigators. GRIL was initially established by the Yale Older Americans Independence Center; more recent efforts to develop this resource have been undertaken through the HCSRN (Health Care Systems Research Network)-OAICs (Claude D. Pepper Older Americans Independence Centers) AGING (“Advancing Geriatrics Infrastructure and Network Growth”) Initiative, in collaboration with the OAIC National Coordinating Center at Wake Forest University School of Medicine. The HCSRN-OAICs AGING Initiative is a National Institute on Aging (NIA)-funded joint endeavor of the HCSRN 3 and OAICs 4 that seeks to build infrastructure to foster high-quality research relevant to older adults with multiple chronic conditions (MCCs). 5
Methods
In order to facilitate investigators’ ability to find instruments for their study, the GRIL resource needed to provide the following: 1) a comprehensive list of instruments across a variety of gerontological domains from which researcher users can view and select; 2) sufficient instrument metadata to provide instrument characteristics and facilitate basic comparison among instruments; and 3) an easily-navigable public website on which to present this information. Efforts by the AGING Initiative to make these developments are described below and outlined in Figure 1 (Figure 1: Process of revitalizing instruments included in the GRIL database and website). Process of revitalizing instruments included in the geriatrics research instrument library database and website.
Domain selection
The initial GRIL database development occurred between 2004 and 2013 and culminated in a total of 84 instruments across nine domains, which were determined based on the categorization of instruments used in geriatric research projects conducted at Yale, Wake Forest, and other Pepper Centers. The present initiative began the expansion of GRIL by completing an inventory of existing instruments and by requesting feedback from the AGING Initiative Steering and Advisory Committees, representing a diverse and experienced group of geriatric researchers, on instruments and domains that should be added to GRIL. Based on their expertise, new domains on frailty, caregiver burden, and medication adherence were added to existing domains including: anxiety, delirium, dementia, depression, general health status/quality of life, health behaviors, hearing, medical comorbidity, pain, physical activity/performance, physical disability, resilience, sleep, social support, and vision. Links to other non-geriatric specific instrument libraries that may be useful to investigators such as the PROMIS®2 measures, the NIH Toolbox®, 6 and COSMIN 1 were also suggested for inclusion in the GRIL resource.
The AGING Initiative recently developed the AGING Patient/Caregiver Advisory Council (APCAC), which comprises a diverse group of stakeholders living with and/or caring for individuals with multiple chronic conditions. The study team plans to engage this stakeholder council in future revisions and expansions of GRIL in order to provide feedback about the comprehensiveness of its domains.
Instrument selection
The review and expansion of instruments included in the GRIL database by the project team are illustrated in Figure 2 (Figure 2: Process of contacting GRIL domain experts for instrument review). It began with the selection of at least one expert per domain based on their relevant publications and research experience. Next, a literature search of Medline was performed by the study team to identify at least one published systematic review of geriatric instruments for each GRIL domain. An example literature search for the “anxiety” domain is as follows: (anxiety[title]) AND (systematic review[title]) AND (instrument OR measure OR screen[title]) AND (older adults OR geriatric OR complex patients OR multimorbidity[title]) Process of containing geriatrics research instrument library domain experts for instrument review.
Instruments identified in these domain-specific systematic reviews of instruments were used to populate a survey developed using the software REDCap.7–8 This survey was administered to the selected domain experts, and asked them to provide feedback on whether each listed instrument should be included into GRIL based on the findings of the provided systematic review and their professional expertise. They were also asked to offer recommendations on additional instruments that should be included in GRIL, as well as to suggest other experts in the field that should be contacted for further feedback.
To determine the comprehensiveness of selected instruments included in the GRIL database, several external resources were utilized for comparison. One resource was the United States National Library of Medicine website ClinicalTrials.gov. 9 We examined the most frequently used primary outcome measures in clinical trials by domain. Search criteria to review this standard source of instruments included trials that were completed, took place in the United States, first posted to the website no earlier than 2010, and had eligibility criteria of adults age 65 and older. Five GRIL domains (anxiety, delirium, depression, frailty, and physical disability) were evaluated using five clinical trials per domain, for a total of 25 clinical trials that met these criteria. Out of the 25 clinical trials reviewed, a total of seven instruments that were used to assess the clinical trial primary outcomes in any of the five domains were newly identified. These instruments were then reviewed by the corresponding domain experts to determine which instruments should be included in the GRIL resource. This process will be repeated on an ongoing basis across all domains to ensure the continued relevance and comprehensiveness of GRIL.
A public submission form is available on the GRIL website where users can submit new instruments to be added to GRIL. This will allow investigators to contribute to the curation of this resource in a dynamic fashion. The submission will require the submitter’s name, instrument name, instrument domain category, and the original manuscript describing the instrument development. All submitted instruments will be reviewed by the corresponding domain expert prior to its addition to GRIL.
Compiling instrument metadata
Instrument metadata incorporated into GRIL included the following: instrument description, copyright information, estimated time for completion, relevant keywords, available translations, and a link and reference to the original publication describing the instrument. Instrument metadata collection began by performing Medline searches of the original publication describing the instrument. Relevant information not found in the original publication required additional Medline or Google searches using the following criteria: the instrument name, the developers’ names in conjunction with the instrument name, instrument-specific resources found on the developers’ institution websites, and publications describing the instrument published by the original developers. Copyright information for instruments developed in the United States was additionally searched and confirmed using the U.S. Copyright Office Online Public Catalogue. 10 The process of acquiring and interpreting copyright information was assisted by staff at the University of Massachusetts Medical School Library. Each instrument’s copyright status was characterized as either “known,” “unspecified,” or “public domain,” in addition to whether the following are required for instrument use: permission, fees, training, and citation of original instrument article. If there was no clear indication of the current copyright ownership and regulations of an instrument, the copyright status was considered “unspecified”.
Website development and features
Instruments included in the GRIL database and metadata have been migrated to a new publicly available web-based platform on the OAIC National Coordinating Center website. 11 This allowed for seamless leveraging of existing programs to create an interactive, platform-responsive interface. The platform utilizes the Microsoft Structured Query Language (SQL) database, Adobe ColdFusion application server, and an array of front-end tools including jQuery, Bootstrap, HTML5, CSS3, and an extensive JavaScript library. This suite of systems was developed by the Division of Public Health Sciences at Wake Forest Medical School and vetted for security and integrity though an established protocol mandated by the institution’s Department of Information Technology. All existing and new instrument metadata can be directly added, edited, or deleted, with changes taking immediate effect, by the project team on the publicly accessible GRIL system.
Results
Instruments, domains, and resources included in geriatrics research instrument library.
P = public Domain; K = copyright Known; U = copyright Unknown; N = not applicable.
Usage details for known copyright instruments.
aInstrument authors indicate preference but not requirement.
bVaries by setting of instrument use.
The GRIL resource hosted on the OAIC National Coordinating Center website was made public in February 2020. 11 GRIL is integrated into a mobile-friendly platform that automatically adjusts the layout and menus based on the user platform, from large screen monitors and laptops to mobile tablets and smartphones. These features will allow researchers to easily navigate the website and compare the functionality of different instruments to select the one that best fits the needs of their study. A feedback survey allows users to easily notify the GRIL coordinator if they are experiencing issues or have comments related to the website. Website traffic is tracked with the use of Google Analytics, which has been developed to monitor users’ activity by domain and indicates which domains are most frequently used.
Discussion
Value of geriatrics research instrument library resource
After the passage of the 21st Century Cures Act in the United States, the Inclusion Across the Lifespan policy eliminates upper-age limits for research participation unless risk justified. 12 Thus, instruments which are reliable for use among older adults are needed for a broad range of research projects. Inclusion of older persons with high burden of multimorbidity, polypharmacy, frailty, and functional and social challenges will result in a more heterogeneous study population and more generalizable findings in research. 13 A recent publication providing a comprehensive review of indices measuring multimorbidity highlighted the importance of moving beyond counting disease. 14 The GRIL resource complements these efforts and addresses important gaps in the research of older adults by providing a free-to-use and openly accessible library of instruments that not only measures multimorbidity but also numerous quality of life, mental health, and other relevant domains that may be frequently missing for use in research studies focused on older adults.
Dissemination
The GRIL resource will be publicized via multiple venues. The AGING Initiative has cultivated an email listserv with over 1950 active accounts, which will be used to inform members of GRIL’s availability and use. A public webinar is currently being planned during which the GRIL resource and its functionality will be described. The AGING Initiative will promote the resource in its quarterly newsletter and on its Twitter account, which disseminate announcements and opportunities related to research in older adults. GRIL will also be linked on the AGING Initiative website, which had over 3,000 unique visitors in 2019. Finally, the resource will be shared as part of the AGING MCCs Scholars Program curriculum, which comprises an annual cohort of junior investigators who were competitively selected as emerging leaders in MCCs research.
GRIL will also be shared across all networks and organizations that are currently or were previously associated with its development. This will include both the HCSRN and OAICs, whose resources were brought together through the AGING Initiative. It will also be disseminated by the Research Centers Collaborative Network (RCCN), an NIA-funded network whose goal is to spur multidisciplinary efforts in aging research across six NIA centers. The overall outreach of these dissemination efforts and use of the GRIL resource will be measured using both website tracking statistics as well as references found in scientific publications that cite the use of GRIL in their work. This will allow the project team to measure the sites utility to the public, and to demonstrate the need for continued development and expansion in the future.
Future additions and limitations
Due to the large volume of instruments available in gerontological research and the resources required to establish and maintain a resource of this capacity, a geriatric research instrument library has not existed until this point. The accomplishments of this project team were made possible only through the strong collaborative efforts and strategic leveraging of existing collective resources across multiple entities and initiatives receiving funding from the NIA. The project team aims to perform an annual review of the GRIL resource with the AGING Initiative Steering Committee, Advisory Committee, relevant domain experts, and patient and caregiver stakeholders to ensure that new instruments are added and existing information is updated.
The addition of future components and features into GRIL may include the incorporation of live filters into the search functionality in order to sort instruments by various characteristics, including: 1) time required to complete the assessment; 2) date of instrument development; 3) setting where the screening has been validated or used (in-person, phone, electronically); and 4) selected keywords. A registration process on the website would allow users to sign up and to customize their GRIL accessibility, giving them the ability to make personal virtual folders to organize instruments that they have selected. The addition of a public forum for users to discuss instrument strengths and limitations based on their experience and expertise would be beneficial in providing supplemental information related to finding the appropriate instrument for their study. Both features would require the development and maintenance of a registry of users, which is currently not possible due to limited resources. As telehealth and app-based research continues to evolve, future additions to GRIL may also include instruments for these platforms as well. 15 Further indications of available translations and culturally appropriate adaptations of instruments can be further expanded upon in GRIL. Supplemental resources may also be added that could be of broader value to researchers, such as clinical guidelines in geriatric medicine or public datasets for conducting analyses of studies including older adults. Finally, the addition of quality aspects of instruments may be added, which would provide more information to researchers to compare and guide instrument selection.
Limitations were faced while developing GRIL due to copyright restrictions on instruments. The inclusion of English-only instruments limits the comprehensiveness of GRIL for non-English-speaking populations. Initial efforts were made by the team to link a complete, downloadable REDCap instrument data dictionary and entry form for each instrument to facilitate its use in studies. Acquiring copyright permission required individually contacting the author or publisher for each instrument, and describing the GRIL resource to them and how the instrument would be used. The process of receiving permission from authors, paying fees occasionally associated with obtaining permission, and developing terms of use statements for each instrument was not feasible with the limited resources and staffing available.
The instrument copyright status information included in GRIL was collected to inform the user’s selection of instruments; however, this information was also limited due to the lack of clarity surrounding the acquisition of updated information. The U.S. Copyright Office provides guidance for finding copyright information, which includes searching original instrument documentation for details, using their online catalogue, and having their office conduct searches, yet they also make the clarification that following these best practices may not provide conclusive results. 16 The GRIL resource is successful in providing initial information on copyright information to researchers as they are considering the use of an instrument. Further details about an instrument’s copyright information can be obtained by contacting authors of instruments to confirm the current copyright status and requirements.
Conclusion
The newly developed GRIL resource of 175 instruments and 18 domains provides a uniform, comprehensive means for investigators to discover and perform basic comparison of geriatric instruments for use in their study. We developed a systematic, low-burden process to add instruments to GRIL and compile metadata by allowing for public submission of geriatric instruments and routinely gathering the feedback of domain experts on those instruments for determination of their inclusion. GRIL will be publicized via a number of venues including the AGING email listserv, newsletter, Twitter, website, public webinar, and partnering organizations to bring awareness to and facilitate the use of instruments in geriatrics and gerontology.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: National Institute on Aging (P30AG021332) (P30AG21342) (R33AG057806) (U24AG0596240).
