Abstract
Objectives:
Evaluating evidence from peer-review literature for use in evidence-based design is often challenging for the design disciplines, requiring access to the peer-reviewed literature, expertise in evaluating methods and findings, and translating the results into actionable design and operational recommendations.
Purpose:
The purpose of this methods paper is to elucidate the process for systematic evaluation of research to translate evidence into practical application to improve design for occupant health and wellness.
Background:
Researchers have found strong connections in environmental design influence on health and wellness that have proven to be substantiative in the effort to improve health and well-being. Design has the capacity to encourage healthy choices and decisions within the built environment. Translation of evidence into applied design solutions may improve public health.
Methods:
A protocol is presented that culminates in the translation of evidence into design recommendations focused on improving occupant health. The protocol includes preparation for the literature search and review, search strategy, study selection, data analysis, and development of the literature review.
Results:
After evaluation of the evidence is completed, there were several positive findings in the example that stakeholders could utilize to improve the health of building occupants with programs and design to support nutrition, physical activity, and circadian entrainment.
Conclusions:
There are a variety of software tools and processes to utilize in the curation of evidence to improve the built environment with relevant design recommendations and operational considerations affecting the personal, social, and economic health of our society.
Keywords
Evidence-based design (EBD) informs design decisions about the built environment based on credible and rigorous research and evidence linking facility design to measurable outcomes (Halawa et al., 2020; Hamilton & Watkins, 2009). To further develop research, it is incumbent upon the disciplines to measure outcomes related to human behavior and health, organizational efficiencies, and building performance to validate design decisions and inform future designs, contributing to the evidence base. Evaluating evidence from peer-review literature for use in EBD is often challenging for design disciplines requiring access to the peer-reviewed literature, expertise in evaluating methods and findings to determine the veracity of results and translating the results into actionable design and operational considerations. This article presents a method for curation, evaluation, and translation to actionable design criteria to improve usability of research in the design process.
Background
Environmental design influences on health and wellness are not fully investigated, but researchers have found strong connections that have proven to be substantiative in the effort to improve health and well-being. The Centers for Disease Control and Prevention (CDC) recognize that design has the capacity to encourage healthy choices and decisions within the built environment (CDC, 2021). For instance, circadian entrainment can contribute to increased health by improving sleep (Bernhofer et al., 2014; Figueiro et al., 2017; Munch & Bromundt, 2012), reducing stress (Figueiro et al., 2017), and reducing risk of cardiovascular disease (Rockett et al., 2013) and obesity (Funderburk et al., 2020; Novak & Brownell, 2011). Using this evidence to inform design decisions about appropriate lighting systems contributes to a built environment that supports outcomes targeted for circadian entrainment, improving sleep which impacts stress, cardiovascular disease, and obesity. From a public health perspective, the environmental driver of the built environment is a crucial factor to influence overall health and disease, which in turn can impact the social and economic health of our society (Figueiro et al., 2017; Funderburk et al., 2020; Joseph et al., 2015; Schweitzer et al., 2004; Seguin et al., 2014; Zagreus et al., 2004).
Most designers and other stakeholders do not have access to, or formal training in, interpreting scientific research that supports the design of the built environment to sustain and improve health of the occupants. The demand for EBD practice is expanding, with expectations of using this process as a decision-making tool during design, construction, and development of operational protocols for occupancy. If the design process, construction, and operations all work in tandem to incorporate both active and passive design interventions to support health and wellness, there is an opportunity to improve public health.
Curation
Before starting a search, the scope of the review and the questions it will address should be developed with a rationale for each question. Developing a good review takes time, expertise, and engaging with stakeholders to ensure that the review is responsive to the topic of concern (Higgins et al., 2019). Once the search engine(s) are selected, a list of search terms translated into search queries based on the search engine criteria are developed to deploy using the search engine. Other factors to consider when conducting a search are the inclusion criteria, such as range of years for publication, type of manuscript, type of study, or literature review. For example, using the search query, information can be included such as limiting the years of publication from 2012 to 2023; and primary studies and systematic literature reviews for type of study, for instance, staff slips and falls or other topics for consideration.
One of the biggest challenges for professionals in the design disciplines is access to scholarly literature. PubMed, Google Scholar, and the Directory of Open Access Journals (DOAJ) have increased availability of peer-review research to those who do not pay for database access and search engine subscriptions. However, not all peer-reviewed manuscripts will present using these search engines and still may not be available unless purchased from the publisher. PubMed is a biomedical and life sciences database maintained by the National Center for Biotechnology Information (NCBI), a division of the National Library of Medicine (NLM) with more than 5,600 journals and greater than 35 million total citations (NCBI, 2022). Google Scholar is a search engine that indexes the full text or metadata of research literature across a wide variety of disciplines and sources (Adriaanse & Rensleigh, 2013). The DOAJ is a search engine that indexes on open-access journals and articles that are completely free online. Each of these sources have limitations: PubMed does not index some literature about building sciences that would be relevant to EBD; Google Scholar’s discipline coverage is vast but not comprehensive, results vary in quality, and the algorithm may create a hierarchy in search returns resulting in questionable validity; and DOAJ is community curated, with noted concerns about the reliability of journals and articles listed (Gul et al., 2019; Lindsey & Olin, 2013; Teixeira da Silva et al., 2018). Access to academic search engines allows for searching across multiple sources within one search providing a comprehensive list of potential relevant scholarly articles. This is the most inclusive option for a comprehensive search but is primarily available to universities and other institutions.
Once the search is complete, a screening of titles and abstracts conducted by reviewers with content expertise is the next step for study selection. The best practice for title and abstract screening is to have two independent reviewers where an analysis of interrater reliability can be conducted. Remaining articles where there is disagreement should be reviewed for a final determination (Robson et al., 2019). If two reviewers are not available, then one would screen articles and another person would verify the list of excluded studies. Another concern is the potential for selection of the same data published in multiple scholarly manuscripts (Altay & Koçak, 2021) and then used to justify an inflated prominence in the literature. Limiting searches to peer-reviewed manuscripts and excluding reports can help remedy this concern, though careful assessment is needed to discern if more than one article is published based on the same data (Carneiro-Barrera et al., 2020; Foster & Jewell, 2017).
Evaluation
As articles are reviewed, a quality assessment must be performed to determine inclusion by considering the study design to reduce bias and error. A critical appraisal is the next process that systematically evaluates the validity of the research and its relevance to a specific research question (Foster & Jewell, 2017). In brief, the key steps involved in critical appraisal for included studies is to identify the study design, methods of measurement, reported findings, and assess validity and reliability of the study.
For data collection and extraction, a full text review of the remaining articles following inclusion and exclusion criteria is performed. One strategy to use for assessment is based on population, intervention, comparisons, and outcome (Higgins et al., 2019). This tool provides an assessment strategy that is fundamental to quality assessment of peer-reviewed literature and provides a framework for extracting information and data needed to meet inclusion and provide a comparative assessment against other articles that meet inclusion. Another tool that is used specifically for systematic reviews is the PRISMA checklist (Moher et al., 2015), which provides guided structure and reporting for study design, objective, setting, patient population, methods and interventions, outcomes, appraisal of methodology, main findings, and limitations (Strudwick et al., 2018). It is a comprehensive tool that is useful in evaluation of interventional studies for inclusion in EBD strategies.
Appraisal of the methodology is based on the research design. Studies are assigned according to a hierarchy of evidence determined by the quality of the research design. Level 1 is usually a higher level of evidence, and each subsequent level is a lower quality of evidence. There are many examples of levels of evidence; some specific to medical or nursing and others specific to social sciences or other disciplines (Ackley, 2008; Daly et al., 2007; Grant et al., 2014; Younas et al., 2022). The hierarchy is not absolute and different research questions may require distinct types of evidence, which is why it is necessary to perform a critical appraisal to evaluate the strength of the findings. Systematic reviews and meta-analysis are considered the highest level of evidence, followed by randomized controlled trials, experimental studies, cohort studies, case studies, and expert opinion at the lowest level of evidence.
For evaluation, the final consideration is assessing reliability and validity. Simply put, in research, reliability refers to whether research methods can reproduce the same results when carried out in additional studies (Graziano & Raulin, 1993). The issue of reliability indicates how well the method consistently measures the outcome. Validity is a partner with reliability. Just because a method is consistent does not mean that the results are necessarily correct. Validity determines whether the result measures what it is supposed to measure. Internal and external validity are two concepts that assist in evaluating the quality and relevance of a study’s results. Internal validity (sample selection, sample size, use of validated data collection tools, form of data analysis) relates to how well a study is designed, conducted, and accuracy of analysis. External validity speaks to how applicable and generalizable the findings are to other settings, situations, or people (Harris et al., 2008). There are always differences, but the key is to determine whether the differences are significant enough to affect the applicability of the results.
Translation to Practice
Translation of research into practice involves development of assessments based on the research, collection, and validations of the data; reporting of the data; evidence-based interpretation of the results guided by application of the research findings; use of data after implementation; and a commitment to quality improvement (Toche-Manley et al., 2011). Providing EBD interventions can influence occupant outcomes by providing an environment that enhances occupant experience. The translation to practice process is outcome driven, requiring implementing EBD focused on specific occupant or building effects.
There is no widely accepted process for translating evidence into practice (TRIP), and there are limited studies that describe the process of translating evidence into practical application to improve patient care or experience (D. J. Cohen et al., 2008). Several disciplines have a documented process for translating research into practical applications for clinical use. One medical-based framework for synthesizing evidence-based care practices into actionable procedures is TRIP. The TRIP process first summarizes evidence in a checklist, then identifies local barriers to implementation followed by measuring baseline performance. Upon implementation, all patients receive evidence-based care through engagement, education, execution, and evaluation. Like design, construction, or other factors with varying conditions, medical interventions may require changes or modifications as they are integrated into practice (D. J. Cohen et al., 2008).
Difficulties exist in extraction of evidence, development of protocols, implementing the intervention, and documenting the process to evaluate the outcome. Current available studies conclude that there are a number of factors that may impact implementation of interventions (D. J. Cohen & Crabtree, 2008; D. J. Cohen et al., 2008; Herr et al., 2012; Titler et al., 2016); however, there is also great potential to better understand which types of interventions are most effective for which type of occupant outcome (Toche-Manley et al., 2011). By collecting data from EBD interventions, these applied environmental treatments can be analyzed, completing the EBD process, and contributing to the knowledge base for building science and human outcomes.
Digital Curation
A citation manager is a resource to curate, manage citations, and most have “cite while you write” capabilities. EndNote and RefWorks are proprietary citation managers that require purchase or subscriptions. Zotero and Mendeley are free downloadable citation managers. Zotero is of interest because it is developed on the Google platform and has good search and article acquisition capabilities. Each of these citation managers can assist with managing your citations, full-text articles, and notes; however, additional costs associated with subscriptions to search engines and databases are not included. Each has its own benefits and limitations, for instance, Zotero is free and built on a strong search platform but has limitations in the finer points of style accuracies, which may create issues with the references related to publication requirements.
Examples of existing tools for data acquisition include Abstrackr, SRDR Plus, and EPPI Reviewer Web. Abstrackr is an online tool that helps screen, organize, and manage the results of a topic review search, focusing on title and abstract. It allows one or more people to screen for relevance to the topic and then provides a view of the results from the screening process. Once the abstracts are chosen that are relevant to the topic, the results can be uploaded and organized for the next step in the review process. The SRDR Plus (the Systematic Review Data Repository: Plus) is an online tool for extracting, managing, and archiving data of already completed systematic reviews and the data that supports those reviews. Data can be used for new analysis or the information from the reviews can be used to produce tailored reports or reviews. Data sources include MEDLINE, PsycINFO, Embase, the Cochrane Central Register of Controlled Trials, and the Cochrane Database of Systematic Reviews. Both products were developed by the Center for Evidence Synthesis in Health, School of Public Health, Brown University, and are free to use (https://www.brown.edu/public-health/cesh/resources/software).
EPPI Reviewer Web (ER Web) is a fee-based application for a variety of literature, narrative or systematic reviews, and meta-analyses. It can be utilized for small or large-scale reviews, with the platform allowing work to be completed by one or more people. ER Web enables the user to screen articles using title and abstract and full text. Documents can be imported from PubMed, Web of Science, and PsycINFO (EPPI Centre, https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2947).
Databases on architecture and related disciplines are focused on design, history, culture, and technology. For instance, Health Environments Research & Design (HERD) journal is indexed in MEDLINE, SCOPUS, and Thomson Reuters. Articles from this journal are found in PubMed because it pulls data from MEDLINE. With specific content focused on relationships among health and environmental design, and outcomes related to health system organizations, medical professionals, and patients, the indexing for HERD is responsive to the readership. However, other databases, such as Avery Index to Architectural Periodicals or Art & Architecture Complete are not specifically focused on design for health. Another option is the EBD Translator (https://crosschek.io/ebdtranslator), a database that curates peer-reviewed published manuscripts that focus on environmental design that supports health, wellness, and sustainability. Currently, this searchable database is in beta testing. A subscription is required, but the curated content is more narrowly focused on research related to design for health.
Limitations of these programs and others include the fact that user training is required, which may be time consuming, and dependent on the users’ ability to interpret relevant scientific publications. Also, research articles for EBD may be housed in repositories other than those used by the programs described here. The FGI Guidelines (https://fgiguidelines.org/) or Whole Building Design Guide (WBDG) (https://wbdg.org/) may not be accessible by the software but are available online and provide relevant information related to building design. The FGI Guidelines offers a variety of documents that can be used by private and public entities and include guidance on standardization of equipment and space and promotion of safe practices concerning design and construction of healthcare facilities (https://fgiguidelines.org). The WBDG provides a variety of information to promote what they term as a “high-performance building.” This information includes emerging technology, federal criteria, building-related guidance such as space, building types and design objectives (https://wbdg.org/).
Protocol for Literature Search and Review
The following section describes the process for literature search, selection, data analysis, and extraction in preparation for writing a literature review. While this may be a manual process, a literature review is presented as an example of output based on curating evidence using the beta version of the EBD Translator, a software application that utilizes the proposed protocol for curation of scientific articles to support EBD. The software searches, selects, and evaluates peer-reviewed evidence to provide the researcher with automated processes to improve efficiency. The basis for the program is an article rating system (Harris & Detke, 2012; Taylor & Harris, 2019) developed to rate articles for inclusion in literature reviews.
The article rating system was validated by having the research team conduct a quality assessment of curated articles which were validated using Cohen’s Kappa Weighted to measure the agreement between two raters for inter-rater reliability (J. Cohen, 1968; Detke et al., 2013). The article rating system provides a structural method for critically evaluating published research (Concato et al., 2000; Cooper, 2001; Evans, 2003; Petticrew & Roberts, 2003) and is based on two prerequisites and a 100-point score assigned across six research design levels used in the protocol for hierarchy of evidence. The maximum possible points allocated to each study design level are weighted based on the scoring system, which provides an orderly approach for qualifying evidence (Detke et al., 2013; Evans, 2003). The example used is on the topic of improving the design and indoor environmental quality (IEQ) to support improved nutrition, increased physical activity, and environmental conditions to support improved health in the workplace.
Preparation for Literature Search and Review
In preparation for conducting a search and review of the literature, it is recommended that a citation manager be selected and set up to use throughout the process. Next, select search engine(s) and/or database(s) for the search such as PubMed, Google Scholar, or a database that may require a subscription.
Search Strategy
For the search process, develop a list of search terms and phrases that relate to the topic of the search. See Table 1 for example list related to the nutrition, physical activity, and IEQ literature review. If manually searching, develop search queries using the search engines criteria or, if using a search software that does not require search queries, prepare to search individual terms and phrases. Next, determine inclusion and exclusion criteria and utilize the search engines’ selection criteria to define for the search. Finally, conduct the search using the selected search engine(s) and or database(s) (Figure 1).
Search Terms for Nutrition, Physical Activity, and Natural and Artificial Lighting.
Search Results Using an Online Search Engine.
Study Selection
Review search results and select relevant articles to include in the review process by screening titles and abstracts (Figure 2). The title should indicate an accurate description of the research intent; and the abstract should clearly articulate the purpose, methods, results, and conclusions for rapid comprehension. Save the selected studies to the citation manager and attach full text article, if available.
Screening Articles for Inclusion by Reviewing Titles and Abstract Using an Online Search Engine.
Data Analysis
Conduct a full text review, with an evaluation that includes the study design and a ranking of articles based on a hierarchy of evidence. It is recommended that you use reviewer’s notes in the citation manager to capture the analysis of the article. The study design is heavily weighted in the quality assessment for article rating system used in this protocol (Table 2).
Quality Assessment Weighted Criteria Based on Study Design.
After identifying the study design the following criteria should be reviewed to complete the quality assessment:
Literature Review
Utilizing the remaining articles that met inclusion after quality assessment, draft the literature review. The following example is the resulting literature review (abbreviated) on evidence supporting design for nutrition, physical activity, and indoor environmental quality following the protocol, articulating the existing research on the topic, and developing a justification for further research:
Improvement of the built environment via EBD has the potential to support wellness and encourage healthy choices, thus affecting the social and economic health of our society.
The complexities of environmental and psychological influences on health and wellness are not fully investigated, but researchers have found strong connections that have proven to be substantiative in the effort to improve health. For instance, circadian entrainment can contribute to increased health by improving sleep (Bernhofer et al., 2014; Figueiro et al., 2017; Munch & Bromundt, 2012), reducing stress (Figueiro et al., 2017), and reducing risk of cardiovascular disease (Rockett et al., 2013) and obesity (Funderburk et al., 2020; Novak & Brownell, 2011).
Since 2008, obesity rates in America have increased by over 26%, reaching an all-time record of 42.4% (Warren et al., 2020). Obese adults are at a greater risk of developing a number of comorbidities such as type 2 diabetes and cardiovascular disease (Warren et al., 2020), and encumbering direct healthcare costs of approximately $480.7 billion per year (Waters & Graf, 2018). From a public health perspective, the environmental driver of the built environment is a crucial factor to influence overall health and disease such as obesity, which in turn can impact the social and economic health of our society (Figueiro et al., 2017; Funderburk et al., 2020; Joseph et al., 2015; Schweitzer et al., 2004; Seguin et al., 2014; Zagreus et al., 2004). Design elements that influence the physical space to make it easier for individuals, via passive or active choices, that positively affect health are key to promoting overall healthy behavior that in turn can affect the trajectory of obesity.
As shown in a study conducted by Seguin et al. (2014), common reported barriers to healthier lifestyles included: lack of time and competing priorities, competition with activities that promoted sedentary behavior, built environment barriers, and difficulty avoiding unhealthy food. These barriers can be commonly found in the work environment, as it has been frequently reported that there exists several obesity-promoting behaviors, such as sedentary work environments, lack of healthy food options in worksite cafeterias, and a surplus of vending machines, which often exclusively provide energy-dense choices (Lee-Kwan et al., 2017). This highlights the importance of building designers, owners, and operators working together to provide health-promoting environments.
Translation and Application of Evidence
A systematic approach to translate outcomes into actional design and operational criteria utilizes a translational research strategy, informing EBD. Increasing the use of scientific evidence to inform design and operations, including efficiencies in meeting project goals such as health and wellness for building occupants and qualification for sustainability, and health and wellness certifications will further support improvement in public health over the long-term and help to mitigate chronic disease.
Based on the literature review conducted that was focused on nutrition, physical activity (PA), and indoor environmental quality, there were positive findings that designers, building owners, and other stakeholders can utilize to improve the wellness of employees and other building occupants. Common findings in the curated nutrition studies found that utilizing choice architecture in cafeterias, deliberately increasing the availability of healthy food items, and using point-of-purchase food labeling could positively impact nutrition intake. Concerning PA, provision of workplace exercise facilities and strategically placed and attractive staircases promote an increase in activity. Other interventions found to be helpful included providing employees options for sit/stand furniture and advocating walking trails, walking to work and walking groups. Regarding indoor environmental quality, exposure to natural light and views of natural elements, when possible, were found to both increase PA and improve sleep quality. Lastly, highlighting the importance of lighting choices, overexposure to artificial light may cause circadian disruption, suppressing melatonin, and increasing risks of cancers. Providing access to natural light and specifying a circadian lighting system that mimics the natural progression of sunlight throughout the day may encourage occupant circadian entrainment. Overall, design that encourages healthy behaviors may lower risks associated with chronic disease.
Conclusion
No matter the method or what software applications/tools building stakeholders choose to use, curating queries of health and wellness scientific literature related to the built environment, provides information for relevant design considerations and operational recommendations. Improvement of the built environment via EBD has the potential to support wellness and encourage healthy choices, thus affecting the social and economic health of our society over the long term.
Implications for Practice
Curating existing scientific literature related to the built environment defines the evidence that informs design interventions for practical application.
Connecting curated articles to user experience, related diseases, indoor air quality (e.g., chemicals, pathogens), operational policies and procedures, and building assemblies provide opportunity to translate research into actionable design recommendations focused on health and wellness.
Provision of a library of documents that support or provides guidance for healthy building planning and operation supports EBD.
Programming enhanced by the provision of questions and relevant evidence and resources that support the intersection of operations and design of the built environment provides a foundation for measuring environmental and health outcomes, closing the cycle of EBD.
Footnotes
Acknowledgments
The authors would like to acknowledge the contributions of the evidence review team, including Rachel Olivarez, Abby Bean, Hannah White, Aalie Hebert, Blair Brunson, Matt Soundy, Keyanna Taylor, and Pranish Bhagat for their diligent effort to adhere to a standard of quality that greatly benefits this research.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the U. S. Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) [contract number 75D30120C08537].