Abstract
The study aim was to identify and evaluate the reliability and quality of online resources for parents of preterm infants seeking health information about their infant using the DISCERN tool and Health on Net code. An Internet search (www.google.com) was used to identify websites for parents of preterm infants on their infants’ health and health issues. For each search, the top 100 “hits” were downloaded, yielding 1200 websites. After reviewing websites for exclusion criteria and duplicates, 197 websites remained and were analyzed. According to the DISCERN tool, the websites had a moderate reliability score (mean = 29.88, standard deviation = 4.88, range: 18–40), moderate treatment score (mean = 24.15, standard deviation = 5.79, range: 10–35), and moderate overall quality score (mean = 3.41, standard deviation = 0.89, range: 1–5). Only 24 (12.2%) websites had current Health on Net code approval and no other websites met full eligibility for certification. Overall, the reliability and quality of information available online to parents of preterm infants is lacking.
Parents of infants delivered preterm are often overwhelmed with many emotions and concerns following birth. This is especially true if their infant requires care in a neonatal intensive care unit (NICU). Parents may experience disappointment, anger, guilt, or feelings of loss of control, which can be heightened due to separation from their infant as well as uncertainty and lack of knowledge regarding their infant’s medical condition.1–3 Mothers may particularly be vulnerable and unable to process all the information that is provided to them upon NICU admission as they are physically recovering from the birth process themselves.4,5 While the staff in neonatal units strive to provide consistent information to parents, verbally and in written form (e.g. brochures, books, and posters), many parents report feeling stressed and often actively seek alternative ways to find information.6,7
As technology continues to become widely used, many parents are turning to the Internet to find out more about their children’s medical health needs.8,9 In a qualitative survey conducted by Sundstrom, 9 one participant articulated: “I do that for everything—go online. I Google it. That is what everyone is doing” (p. 96). Using the Internet to seek health information has been found to be true for parents of infants in the NICU. 6 In a recent study on Internet preferences of Canadian NICU parents (n = 72), our team found that 81 percent used their smartphone to access online information related to their time in NICU. 6 This is not isolated to Canada, as German parents (n = 141) of very low birth weight infants reported to have searched for information online regarding pregnancy or prematurity similarly during pregnancy (75.2%), hospitalization (68.6%), and post-discharge (73.0%). 10 Parents identified searching for information on prematurity, explanations of abbreviations, common medical problems, and available treatments, as well as information on infant care post-discharge.6,10
While the Internet could be a beneficial resource for parents to access a breadth of information related to prematurity at their own pace, there are significant differences in health and care needs found within this highly sensitive population characterized by gestational age, birth weight, diagnoses, feeding methods, and severity of illness. 11 The care required to support an infant born extremely preterm (i.e. <28 weeks gestational age) is generally less complex compared to an infant born moderate-to-late preterm (i.e. 32 to <37weeks gestational age). 11 For example, the feeding needs and ability differs in preterm infants with significant controversy existing in the literature on best methods and outcomes due to a lack of high-grade evidence. 12 Preterm infants struggle to establish oral feeding due to immature oral and gastrointestinal systems, thereby requiring assistance with oral feeding until they can develop the necessary abilities to feed by mouth. 13 Yet, the progression between tube feeding and oral feedings remains variable both in the literature and among hospital polices. 13 Recognizing that each infant’s experience is unique, it is essential to acknowledge that online health information may be incomplete, variable, or conflicting depending on the health issue it is covering. Provision of online health information that is incomplete or inadequate could result in increased parental confusion, stress, and poorer care for infants if parents are unaware or unable to confirm the reliability and credibility of the information.
Moreover, general health information available online is not always current or evidence-based. In a systematic review conducted across all health topics, 70 percent of websites were identified to have quality issues. 14 Additionally, despite high usage, parents have acknowledged that they are not always confident in the information provided to them via the Internet. 9 Over the last two decades, there have been advancements in knowledge and implementation of evidence-based practice that has led to significant reductions in preterm infant mortality;15,16 however, there is still work being done seeking to improve morbidity outcomes in this population. As research continues to emerge rapidly, it is imperative for health information on the Internet to reflect current evidence in this field. Specific to parents of infants delivered preterm, no review of readily available online health evidence has been conducted. Therefore, the aim of this study was to identify and evaluate the quality of online resources for parents of preterm infants seeking health information about their infant.
Methods
Search strategy
This review of the available online health evidence was conducted by systematically searching Google using a search strategy combining “premature baby” with key health topics. Google was chosen as the sole search engine as it is the most frequently used search engine around the world. 17 A comprehensive Internet search using google.com was conducted on 28 May 2016. Preterm, or premature, infants are defined as an infant born before 37 weeks of pregnancy. 18 To identify the information directed to parents of preterm infants and the online information available on health issues related to preterm infants, the search was conducted combining “premature baby” and the following terms: “parenting,” “care,” and “neonatal OR neonatal intensive care unit OR NICU,” “care OR home,” “sick,” “outcome,” “health issue,” “development OR growth,” “complications OR risk,” “pain,” “skin to skin contact OR kangaroo care,” and “breastfeeding OR feeding.” The search term of “premature baby” was used as this yielded more hits in a solo search compared to “preterm baby,” suggesting this term is more widely used on Google, offering a broader reach of websites.
Websites were included if they were published in English within the top 100 “hits” per search strategy, yielding 1200 possible websites. Websites were excluded if they were newspaper articles, scientific sources (i.e. peer-reviewed published articles), products only, broken links, hospital websites that only provided information about their neonatal care unit, subscription access only, resource links only, forums, or were deemed not relevant to parenting infants in the NICU (e.g. adoption websites and Foundations). Newspaper articles were excluded due to their time sensitivity of publication and fluidity of doing a web search. Scientific sources were excluded as the goal of this review was to evaluate non-academic, online sources of information that may be used by parents as well as the likelihood of some scientific sources to be behind paywalls, limiting the ability of parents to freely access them. Forums were excluded based on the consideration that the goal of this review was to evaluate online health evidence, rather than parent-generated content. Due to the nature of this review, no ethical approval was needed.
After the removal of duplicates (n = 223) and those meeting the exclusion criteria based on title and uniform resource locator (URL) screening (n = 393), 584 websites remained. Duplicates were identified through comparing URLs, website title, and description for the same content. Upon reviewing the websites, it was identified that several of the links were different pages within the same website, identified through the stem URL. For example, the search pulled seven pages within the www.marchofdimes.org website, covering topics such as kangaroo care, feeding, and becoming a parent in the NICU. Therefore, all pages identified through the search that were from the same stem URL were considered as one when evaluating that specific website. This led to a total of 337 websites being screened, with 197 websites included after 140 were excluded upon full review for reasons identified above. This search strategy is displayed in Figure 1 in a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram. 19
PRISMA 19 flowchart of search strategy.
Two independent reviewers (B.R. and T.B.) evaluated the websites with another reviewer (J.D.) checking reliability on 30 percent of websites until agreement was reached in the use of the evaluation tools. Any disagreements that arose between the reviewers were resolved through discussion and if needed by another reviewer (M.C.Y.). Websites were identified on 28 May 2016 and evaluated over 5 months starting from 6 November 2016 to 12 April 2017. A data retrieval form was created based on the DISCERN tool 20 and Health On Net (HON) code 21 to evaluate included websites, and IBM SPSS Statistics for Macintosh, Version 22.0 was used for data analysis. Descriptive means and frequencies were used for reporting overall scores on the DISCERN tool and HON code. An independent-samples t-test was conducted to compare DISCERN scores on reliability, treatment, and overall quality based on whether a website had HON code certification or not.
Measurement tools
We objectively evaluated the websites identified using two methods: the DISCERN tool20,22 and the HON 21 code. DISCERN is a validated, 16-item instrument that has been designed to help individuals evaluate the quality of written health information online. This tool does not assess the scientific quality of evidence, but judges the reliability of a website as a source of information on treatment options. The DISCERN tool 20 is organized into three sections, rated on a 5-point scale, where 1 represents not meeting the criteria at all, 2–4 representing a partial score, and 5 represents the criteria was fully met. To ensure consistency of scoring of a partial score, the authors (J.D. and M.C.Y.) further defined the partial scores and are available upon request. Section 1, including questions 1 through 8, evaluates the reliability of the website, with a range of possible scores from 8 to 40. Questions in Section 1 evaluates the clarity of the website’s aim and whether it was achieved, relevancy, sources of information, whether it stated a date of creation, if it was balanced and unbiased, provision of additional sources of support, and if it referred to areas of uncertainty. 20 Section 2, including questions 9 through 15, assesses the information provided on the treatment options, with a range of possible scores from 7 to 35. Questions in Section 2 referred to treatment, including description of treatment, the benefits and risks of treatment, impact of no treatment, impact on quality of life, treatment options, and shared decision-making. Section 3 only contained question 16 and measures the overall quality of the website, with a possible score between 1 and 5. There are also three questions that address key aspects of the evaluation: question 6 evaluates biases, question 7 assesses links to more information, and question 4 deals with possibility of verification of the information, with a possible score between 1 and 5. We have evaluated Section 1 and Section 2 as a total mean score (range 8–40 for Section 1 and 7–35 for Section 2) and examined items 16, 4, 6, and 7 individually as well, with a possible score between 1 and 5 for each question. Also, consistent with other studies reporting on the DISCERN tool,23,24 we reported on a cut-off score of less than 40 percent to be considered “poor” and 40 percent–79 percent as “moderate” and at least 80 percent as “high.”
The HON code 21 is a set of guidelines designed to inform users on the assessment of the quality of health information on the Internet. There are eight essential principles that a website must adhere to in order to receive certification: authority (by giving the qualification of authors); complementarity (information is meant to support, not replace); confidentiality (respect the privacy of site users); attributions (cites the sources and dates of medical information); justifiability (justification of claims/balanced and is objective); transparency (refers to accessibility through providing valid contact details); financial disclosure (providing details of funding); and advertising (clearly distinguishes advertising from editorial content). 21 For this study, we entered each website into the HON code search tool to verify whether a website had already applied for and currently held certification at the time of data extraction. If a website did not already have HON code certification, we used the evaluation form to determine whether the website met each individual criterion principle.
Results
A total of 197 websites were identified and included in the analysis. Each website could cover multiple topics, and the primary ones that were identified included breastfeeding or feeding issues (51.78%), skin-to-skin contact or kangaroo care (38.07%), and illness or health issues (36.05%). Table 1 identifies the full list. Only 98 (49.75%) of websites had a date stated on all or most pages, with only 23 (11.68%) published or updated in 2015 and 25 (12.69%) in 2016.
Full list of topics covered.
Using the DISCERN tool to evaluate the quality of websites on information available to parents of preterm infants, less than half of the websites were considered to have high reliability (41.1%) and less than a third were considered to have high quality of treatment information (30.5%). The overall quality of the websites was primarily rated moderate (75.2%) to low (15.2%). Taking a closer look at the provision of sources and providing additional information, over half of the websites were considered low (62.4% and 58.4%, respectively). For those websites that did provide additional sources of support and information, most provided links to other organizations or sources of information (50.0%) or had an online forum for parents to ask questions (25.6%). Half of the websites were considered to have high-quality reporting (54.3%), thus were considered balanced and unbiased. Table 2 provides additional details on the DISCERN tool including mean values (M) and standard deviations (SDs) of scores.
DISCERN tool.
M: mean; SD: standard deviation.
Of the 197 websites that were included, 24 (12.2%) had an active HON code certification. Of the remaining websites, no websites met all principles. Table 3 outlines the percentage of websites that achieved each principle, with transparency of authorship receiving the highest score (70.05%).
HON code principles.
HON: Health On Net.
When comparing websites that had HON code certification and those that did not, there was a significant difference in the reliability scores on the DISCERN scale for websites with HON code certification (M = 33.42, SD = 4.23) compared to those without (M = 29.38, SD = 4.77, p < .001). There was also a significant difference in the treatment scores on the DISCERN scale for websites with HON code certification (M = 26.92, SD = 6.08) compared to those without (M = 23.86, SD = 5.58, p = .014). Finally, there was a significant difference in the overall quality scores on the DISCERN scale for websites with HON code certification (M = 4.00, SD = 0.83) compared to those without (M = 3.32, SD = 0.86, p < .001). Table 4 identifies the websites that had the highest DISCERN score, whether it held a HON code certification, and the topics covered. Topics covered were considered only within the sites identified by the search, not the website generally.
Websites with highest DISCERN and HON code scores.
HON: Health On Net; NICU: neonatal intensive care unit.
Discussion
With Internet use a mainstay, it is important that parents are aware that the information available online may not be of high reliability or quality. The results of our study suggest that the information directed to parents of preterm infants in the NICU is only of moderate-to-low quality in terms of reliability and information related to treatment, as assessed through the DISCERN tool, and low quality as rated through the HON code. Consistent with other studies using standardized tools to evaluate the health content of websites across various health domains, we have shown that information available online is generally not of sufficient reliability and quality as a sole source of information. In a systematic review assessing quality of health information available online, Eysenbach et al. 14 found that 70 percent of included studies reported poor quality in the websites evaluated across various health issues.
When examining the topics that were most prevalent, appearing in a least a third of websites, the most prevalent issues covered included: a general overview or expectations of being a parent to a preterm baby; illness or health concerns; skin-to-skin contact or kangaroo care; and breastfeeding or feeding issues. Other topics were less prevalent (i.e. bringing baby home, complications in the NICU, and pain) and ranged in coverage from a quarter to under 5 percent. In the top websites evaluated through the DISCERN, it is clear that websites typically covered a range of topics with the exception of some targeted websites on breastfeeding. The most prevailing topics that we identified in our review are common concerns for parents of preterm infants, as parents are often overwhelmed by the hospital admission of their infant to the NICU and are unsure of what to expect for their infants’ stay and future health outcomes. 7 Skin-to-skin contact between parents and their infant is heavily encouraged in the NICU population due the significant benefits associated with greater temperature regulation, physiological stability, attachment, pain reduction, and improved neurodevelopment.25,26 Additionally, feeding is of prevalent concern for parents as preterm infants often have difficulty establishing breastfeeding and may require expressed breastmilk or oral supplemental feeding.27,28
It is essential that parents are provided with accurate information when they need it: too much information may cause anxiety, while too little may create fear. 7 Information seeking by parents whose infant is in the NICU, whether online, through health professionals, or through peers, is thought to stem from a need to control the situation and is a method of engaging in care of their infant.7,29 Studies have found that while parents are interested in using online information to supplement the information they receive in the NICU, parents still consider healthcare providers as more knowledgeable, reliable, and sources of quality information than online sources.6,30 However, studies have also found that some NICU parents do not feel that they received sufficient information from hospital staff related to caring for their newborn, thus are turning to other sources, including online health information. 7 Parents of infants in the NICU are highly motivated consumers of information related to improving the care and outcomes of their hospitalized infant and have access to an abundance of information available online, which may or may not be as reliable as information provided by healthcare providers. The opportunity exists to encourage the facilitation between parents and healthcare providers to discuss concerns and topics in person as infants are receiving ongoing care in the NICU to ensure that the information gaps that parents are filling through online health information helps parents to identify which sources and content are reliable and credible.
When evaluating the websites based on the DISCERN tool, we found that only 40 percent of websites were considered highly reliable. This is consistent with other studies that have used the DISCERN tool to evaluate the level of online information related to pediatric populations, such as clubfoot 23 and childhood epilepsy, 24 with moderate-to-poor quality of content reported. This lack of reliability is not isolated to a particular health issue as a German study found that Internet users were able to find health information online to answer questions in an average of 5 min 42 s, yet participants did not critically evaluate the website nor could recall the specific site the information came from. 31 This is a concern as many parents are turning to the Internet to supplement the information they receive in the NICU, 6 but they may not take the time to critically evaluate the information they are obtaining. As the first point of care in the NICU, healthcare providers have the opportunity to educate parents about how to evaluate online materials, as a way to meet their information needs. Healthcare providers should actively direct parents to specific valid and reliable websites to ensure they access evidence-based information.
Using the DISCERN tool to evaluate treatment options, we found that less than a third were of high quality. While the topics covered and corresponding treatment options for preterm infant care is broad, evidence suggests improved infant and family outcomes when parents are more actively involved in the care of their infant during NICU admission.32,33 With the corresponding paradigm shift toward family-centered and family-integrated care,33,34 it should be a priority to increase parent involvement as active participants in the care of their infant, a key aspect of which is meeting parent information needs with accurate, high-quality, and up-to-date information. While some websites did include suggestions or explanations of how parents can be involved, this was not a significant focus in most websites.
Lack of reliability and up-to-date evidence-based content was a significant concern that arose from our findings, as measured through the HON code. Less than one-quarter of the websites included in our study had been updated in the past 2 years. Our findings were consistent with other studies that have found low quality of information provided by community pharmacy websites 35 and on elective orthopedic surgery. 36 As parents have been found to routinely use the Internet when seeking health information about their children, and may attempt to diagnose and treat symptoms without the need for medical intervention using this content, the lack of reliability is a significant concern. 37 It is important to note that our findings did support that websites with HON code certification were evaluated higher on all three categories on the DISCERN scale. This suggests that websites that are certified are more reliable, provide better treatment information, and are of higher quality than those without.
Given the ever changing and constantly evolving new research findings aimed to improve the outcomes of preterm infants and their parents, it is important that websites update regularly or parents are made aware that the online resources may not be current. However, parents may be unaware that information available online may not be up-to-date or follow evidence-based guidelines for care. Websites that have the HON code certification offers this reassurance to parents if they are looking for it. Yet, given that approximately only 1 in 10 of identified websites were certified, parents are limited in ways to evaluate the quality of websites. Furthermore, an important consideration with the HON code is whether online health information consumers are aware of such evaluation tools when considering the quality of such evidence. This research suggests that other measures may be used by health consumers to assess quality, which may or may not actually indicate high-quality information. For instance, a study by Diviani et al. 38 found that general online health information consumers considered the position in a search was an indication of its quality, regardless if it was a sponsored post (i.e. paid to show first) or organic result. Other studies have found that individuals used varying indicators to evaluate the quality of online health information including the design appearance of the website,31,39 medical ownership or accreditation, 40 understandable and professional writing, 31 and use of scientific resources. 31 However, the presence of a quality seal, such as HON code, was only minimally considered by online health consumers when evaluating websites.31,38 Thus, a greater emphasis is needed on educating parents on the presence and value of quality seals, such as HON code, as a way to evaluate online health information. 41
Given that parents are turning to the Internet for health information and are expressing a need for more information related to their infant’s NICU stay and future health needs, it is important that researchers and hospitals focus on translating evidence-based information into online, parent-friendly formats. 42 For instance, our team has developed a parent-friendly, evidence-based YouTube video on the use of skin-to-skin contact and breastfeeding in the NICU (tinyurl.com/PowerofParentsTouch), which parents found to be helpful and increase their confidence in their use in the future. 43 Additionally, a study by Kadivar et al. 44 in Iran found that mothers (n = 80) who received Internet-based education while their infant was hospitalized in the NICU were more satisfied in their parenting role than those who received standard care. There has been significant growth in parent’s use of social media when seeking health information about their NICU infants health needs with the goal of increasing knowledge, support, and confidence.45,46 Across Australia, parents indicated their most frequently used social media sources were Facebook, parenting websites, and blogs with the goal of seeking information and advice. 47 A recent scoping review on the role of online social support for parents of children with special health needs in the United States suggests the importance of social media for these parents on reducing isolation, increasing emotional support, and increasing informational support to improve the outcomes for their infants. 46 Knowing the growing importance of social media for parents offers an incentive for researchers and hospitals to explore ways to utilize social media to engage with parents to offer evidence-based, reliable information that parents can trust through avenues which they are already accessing through the Internet.
While this study adhered to a rigorous systematic review process, there were some limitations. First, the Internet is dynamic and constantly changing, which may have influenced the results that were obtained and analyzed. Despite the decision to exclude news articles, which are more time sensitive, the cross-sectional approach to evaluation may have been influenced by lag between initial search and completion of evaluation, limiting the ability for complete replicability of results. Additionally, the DISCERN tool is meant to evaluate the information related to treatment options, yet some of the information related to preterm infant care online is not necessarily applicable (e.g. growth and development and NICU experience), thus the tool may not have been able to adequately evaluate the quality of evidence available online. Nevertheless, the tools that were used offer a standardized way to best capture the reliability and quality of online information.
Conclusion
As the use of the Internet by parents continues to grow, there remains significant concern around the reliability and quality of information available to parents of preterm infants. As technology advances and its role and use daily life increases, this is the opportune time for researchers and healthcare providers to leverage innovative eHealth technology to provide evidence-based information in an accessible parent-friendly manner via secure websites or smartphone applications, rather than parents obtaining moderate- or low-quality preterm infant parenting information from Google. Furthermore, a focus is needed on educating parents on how to evaluate online resources to ensure they can make quality-informed decisions while using the Internet to supplement the written material provided by healthcare providers directly.
Footnotes
Authors’ Note
Talia Boates is currently affiliated to IWK Health Centre, Canada.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: JD and MCY received no financial support for the research, authorship, and/or publication of this article. BR and TB were supported from funding provided by ACOA Atlantic Innovation Fund, Project# 207518: “Chez NICU Home: Transforming health care delivery in neonatal care through greater parental presence and involvement”. Open access fees also provided by ACOA Atlantic Innovation Fund, Project Number: 207518.
