Abstract
Introduction
While occupational therapy home assessments are effective to identify environmental falls risk factors, patients may not receive these services due to workforce distribution and geographical distances. Technology may offer a new way for occupational therapists to conduct home assessments to identify environmental fall risks.
Objectives
To (i) explore the feasability of identifying environmental risk factors using smartphone technology, (ii) develop and pilot a suite of procedures for taking smartphone images and (iii) examine the inter-rater reliability and content validity between occupational therapists when assessing smartphone images using a standardised assessment tool.
Method
Following ethical approval a procedure was developed and participants recruited to submit smartphone images of their bedroom, bathroom and toilet. Two independent occupational therapists then assessed these images using a home safety checklist. Findings were analysed using inferential and descriptive statistics.
Results
Of 100 volunteers screened, 20 individuals participated. A guideline for instructing patients to take home images was developed and tested. Participants averaged 9.00 minutes (SD 4.401) to complete the task, whilst occupational therapists took approximately 8 minutes to review the images. The inter-rater reliability between the two therapists was 0.740 (95% CI: 0.452–0.888).
Conclusion
The study found that use of smartphones was to a large extent feasible and conclude that the use of smartphone technologies is a potential complimentary service to traditional home visits. The effective prescription of equipment in this trial was identified as a challenge. The impact on costs and potential falls incidents remains uncertain and more research is warranted in representative populations.
Introduction
Falls are the leading cause of injury in adults aged 65 years or older (Peel, 2011). Serious falls can result in decreased functional independence and are associated with increased morbidity and mortality. Falls in the home environment, especially in the bedroom, bathroom and toilet are one of the main reasons for hospital admission in the elderly population (Sim et al., 2015). Occupational therapists have a core role within the health professional team to assess home safety risk for patients at risk of falls by physically visiting the client’s home (Griffin & McConnell, 2001). The reducing costs and increasing availability of smartphone technology has led us to question the feasibility of using such devices within an occupational therapy context (Daniel et al., 2013). Occupational therapists are exploring the use of digital imagery for home assessments; however, we were unable to find any recent publications that addressed the feasibility of this approach or any resources to support a consistent procedure and method for the capture of digital images. Before undertaking a large-scale research project with vulnerable people, the first step is to explore if the use of smartphones is feasible. To do this, a guideline of methods and procedures for taking images of falls risk areas in the home is required to instruct people on how to take images and forward these for evaluation. This then needs to be piloted. The second step is exploring if such images can be assessed adequately by occupational therapists to identify falls risk to make appropriate recommendations.
The smartphone has been in the market since 1993 and was predominately meant for corporate users, used as enterprise devices as they were too expensive for public use at that time. Within the 25 years of evolution, the price has dropped substantially, and smartphone manufacturers have introduced features that consumers desire, including email, social website integration, audio/video, internet access and photography. In 2019 there were over 2.71 billion smartphone users globally with almost every third person worldwide owning a smartphone (Statista, 2019).
The use of smartphone images and virtual reality for home assessment have added a new perspective to the occupational therapy literature landscape with research considering the Internet (Hoffmann & Russell, 2008), telehealth (Renda & Lape, 2018), and photos (Sim et al., 2015). However, the use of virtual reality and digital images to assist in the screening of patients who may require a physical home assessment by an occupational therapist is under explored. Virtual reality is designed to simulate a person’s physical presence in a specific environment to experience and manipulate the environment as if it were the real world (Lee et al., 2003). An additional feature of this feasibility study was to pilot the wearing of virtual reality goggles by occupational therapists when assessing images to reflect the real-life situation of an individual’s home.
In summary, a review of the literature illustrates how promising new technologies are in improving access to health care, reducing costs, while maintaining patient satisfaction. Breeden (2016) and Daniel et al. (2013) found how photography, used as an archival record, allowed participants not only to share visual images representing the home environment but also to provide descriptions of fall events. However, existing literature reviews have raised the common obstacle of technology limitations and the variable skills of the individuals’ taking photos for analysis and questioned the ability to review or measure the spatial area of the patient’s environment using digital images (Sim et al., 2015). This current study aims to build on the current evidence base and fill this important gap.
As the ethics of recruiting people at risk of falls to trial an unproven method of home risk assessment were problematic, the first step was to test if this method was feasible. Our research explores the feasibility of taking photos using the general population as participants so as to trial a proof of concept (Kendig, 2015). It is envisaged that subsequent stages of this research will be conducted with older people to assist with the screening of participants who may require a physical home visit and ultimately reduce the need for physical home assessments. The proposed longer-term benefits are to minimise adverse outcomes due to falls incidents. Additional benefits include access for those living in rural and remote areas, as well as promoting public health by increasing awareness of their health issues, consequently improving levels of engagement, adherence, and satisfaction. As this research commenced in 2019, the impact that a global pandemic would have in restructuring service delivery models to embrace telehealth options was not envisaged (Zahoransky & Lape, 2020).
The “proof of concept” is one of the common methodological approaches in healthcare quality improvement research in proposing new technology and subsequently evaluate the implementation (Gewandter et al., 2014; Kendig, 2015). As an innovative approach, the development of a suite of tools was identified as an important stage of testing the process. This study aims at testing the feasibility of methods and procedures of assessing the home environment from digital images taken and forwarded by community dwelling adults to occupational therapists. As an emerging area of practice, we wish to trial the use of smartphone technology on healthy members of the community. This first stage has a narrow scope to develop a standardised photo taking guideline for use in subsequent research projects.
The research questions were as follows: 1) Is it feasible for community dwelling adults to follow a set of guidelines and submit photos for fall risk assessment by occupational therapists? Feasibility is assessed by demographic recruitment, time taken, quality of images and participants’ feedback. 2) Can occupational therapists reliably assess photos submitted for falls risk using a standardised home assessment tool? 3) What are the barriers and enablers to support the use of the photography guidelines developed for this study?
Methods
This feasibility study used a proof-of-concept methodology and followed several discrete steps as follows (Gewandter et al., 2014; Kendig, 2015). The study had full research ethics approval through the Bond University Human Research Ethics Committee, reference number KL02270. The CONSORT checklist guided the reporting of this research (Eldridge et al., 2016). 1. The pre – pilot preparation involved the development of a photo taking guideline that instructed participants in how to take, store and forward images of nominated areas of the home. The bathroom, bedroom and toilet areas were selected as these are common areas where falls occur. Using a pragmatic approach, feedback was gained from several peers and social contacts (8 individuals) and amendments made. Changes included grammatical corrections, structure and flow of the instructions and recommendations to include examples. 2. Community participants were then recruited to test the guideline and capture images of their home environments. Participants were provided with a protocol “Photo taking guidelines” and requested to email all documentation to a password protected university email address. The participants were also asked to complete a short questionnaire regarding the time taken to complete the task, the ease of following the photo taking guideline and their experience in forwarding the images to the research email. The researcher estimated participants would not spend more than 30 minutes to complete the full process. All stored data was deidentified using a naming convention method to ensure privacy and confidentially of participants. 3. Once participants had submitted images of their bathroom and bedroom, two occupational therapists risk assessed these images using the adapted Westmead Home Safety Assessment (WeHSA) checklist (Clemson et al., 2014). The WeHSA consists of 72 items that are rated using 2-point scale (hazard/no hazard) with an overall content validity index of 0.80 by different content analysis of literature and the expert review process. As a feasibility study, we selected 24 items of the WeHSA to examine the inter-rater reliability between two occupational therapists. These items were selected as they related to the areas of the bathroom, bedroom, and toilet. The therapists were required to wear virtual reality goggles to assess smartphone photos. If the smartphone photos were not applicable in virtual reality setting, occupational therapists assessed the photos by using the computer. At the completion of the assessments, therapists were asked to complete a short questionnaire on the process including time taken to assess the image, their thoughts on the process, and suggestions for improvement. 4. Once all data had been collected, this was analysed using descriptive and inferential statistics using SPSS. Additional information follows.
Participants
Two groups of participants are identified for this study.
Group 1: Community dwelling people were recruited from the community and through personal networks to take digital photos of their bedroom, bathroom and toilet areas of their home. Volunteers were screened against the inclusion criteria prior to completing written consent. Participant inclusion criteria were 1) aged between 18 to 75 years; 2) English speaker; and 3) access to digital devices with a camera. The exclusion criteria were participants: 1) unable or unwilling to provide consent; and 2) unable to use smartphone technology and 3) registered as an occupational therapist (to avoid bias). This study aimed to recruit 20 participants to inform a sample size calculation.
Group 2: Two Australian registered occupational therapists (OT-A and OT-B) with a minimum of 5 years clinical experience in adult rehabilitation to simultaneously assess the images and provide feedback on the process. This was a sample of convenience.
All data was inputted into the SPSS system for further analysis including descriptive and inferential statistics. This included variables of number of eligible participants, demographic distribution, time required for the study, quality and format of the photos and feedback gained from both participants and occupational therapists using Likert scale questions.
Outcome Measurent
Outcome measures included: 1) Feasibility of eligibility and recruitment
The number of eligible participants, response rates to questionnaires and refusal rates for participation, study procedures, intervention attendance and engagement, were recorded and presented in a CONSORT flow diagram. Missing data was used as a measure of the feasibility of enacting the study protocol in this setting. We compared the total number of potential data sources with the actual number submitted. Baseline demographics were analysed using descriptive statistics. Categorial data was presented as frequencies and percentages, while continuous data is presented as means and standard deviation. 2) Feasibility of the time and resources used for data collection
The procedure of taking smartphone photos and that was taken for assessment was collected. Participants were asked to record the time spent on downloading the smartphone application, taking photos, and sending these to the mailbox. Occupational therapists were asked to record the time used in assessing smartphone photos. Dropout rates and the number of failure photos in the data collection section was also collated. 3) Protocol quality measured by participant feedback
Community participants and occupational therapists completed a satisfaction survey indicating their level of satisfaction with the five-point Likert scale. The results were presented with Mean (SD) for continuous variables that have an approximately normal distribution for each group (A: participants and B: OTs). 4) Description of photo files submission
The number, quality and format of each photo file returned was recorded by the researcher. The researcher determined the quality of photos into high or low resolution and calculated how many photos would be assessed. 5) Examine the inter-rater reliability of environmental falls risk
The research team adapted the Westmead Home Safety Assessment tool and selected 24 out of 72 items to be examined. These items aligned to assessment of bedroom, bathroom and toilet. The two assessors (OT-A and OT-B) then independently assessed the images for environmental falls risk. The assessment results are presented with Mean (SD), Median (range) and the ICC 1k Intraclass Correlation Coefficient.
Results
Feasibility of Eligibility and Recruitment
Demographic characteristics of the community dwelling people.
A total of 100 potential eligible individuals were invited to participate based on convenience and snowball sampling. Thirty declined to participate with “not interested” as the main reason for non-participation. A total of 49/70 (70%) of people invited did not follow through to consent to be included in the study. One individual (1/21) consented to the study but did not fulfil the inclusion criteria (see Figure 1).
Feasibility of the Time and Resources used during the Data Collection Process
The research team aimed to recruit a minimum of 20 individuals to complete this study within 3 months. Following the recruitment process, the researcher requested invited participants to complete a written consent form and to provide feedback “promptly” to simulate the primary care setting. The estimated time for individuals to return photos and surveys to the research email was approximately 1 week, however a small proportion of missing values related to data fields on demographic characteristics. Participants also provided feedback that the time for completing the photo taking process was on average time of 9.00 minutes (SD 4.401).
Protocol Quality Feedback
The quality of photos.
Summarised Description of Images Received
The participants submitted 49 smartphone photos from 20 participants, of which only 10 photos were able to be viewed with the virtual reality goggles. Of these, there were only two photos that could be assessed using the virtual reality goggles. Among all the panorama photos received, 45% were in low quality as deemed by the size (small) and the resolution (fuzzy) of the images.
The Inter-Rater Reliability
Intra-class correlation between OT-A and OT-B.
Two-way mixed effects model where people effects are random and measures effects are fixed.
aThe estimator is the same, whether the interaction effect is present or not.
bType C intraclass correlation coefficients using a consistency definition. The between-measure variance is excluded from the denominator variance.
cThis estimate is computed assuming the interaction effect is absent, because it is not estimable otherwise.
CONSORT Recruitment Flow Diagram.
Discussion
This prospective pilot found that it is feasible for community dwelling adults to follow a set of guidelines and submit photos for fall risk assessment however more detailed instructions are required. Occupational therapists can assess photos for falls risk using a standardised tool, however further research is warranted that compares face to face visits with photography. The results of this feasibility pilot will be used to determine whether further research can be done with the same procedure for full trial in the hospital setting. By identifying the qualitative or the quantitative issues with feasibility and acceptability, it impacts on the result of recruitment, inter-rater reliability and content validity, thus the procedures developed can be modified (Lancaster et al., 2004).
The demographic characteristics of the study sample showed how gender, education background, previous falls history, years of using technology and physical supports related to the acceptance of the new technology. Most of the population in the study sample were aged between 18–39, however there were no correlations found between the age group and the years of using smartphone. The results showed that some of the older population group had more than 10 years of using a smartphone, and this can be explained by the technology acceptance model that how users come to accept and use a technology (Davis, 1989). The theory posited is that the acceptance of technology and the actual use of technology is predicted by the person’s perceptions of the usefulness and the ease of use. This model also suggested that the perception of usefulness and ease of use are mediated by external factors including individual differences, social influences and facilitating conditions. Once older adults are engaged in a technology, they tend to highly utilise it (Bothorel et al., ). For example, when the iPad first appeared in the market, their target consumers were maturely aged businessmen who could apply it to their work. These high-tech products are also mostly affordable for the mature to older populations who do not have financial stress. This is counter to the idea that young people are more proficient with technology (Kon et al., 2017). Therefore, promoting the adoption of popular, easy-to-use features may foster user satisfaction and further use of smartphone technology features. As a feasibility study, we predict that if the adapted guidelines and the smartphone applications are easily to follow and operate, all individuals would be able to take smartphone photos independently despite their age.
Participants had different opinions on their capability to take smartphone photos independently. The study results showed that 85% of the participants reported having no difficulties to take the photos within 15 minutes and did not require any assistance with photo taking, however some individuals were concerned if they had captured the important features of the home environment.
On inspection only 10 of the 49 photos were of a quality that was able to be assessed by the occupational therapists. This is a very low yield and indicates the challenges of self-reported ease and actual delivery of quality images.
The intra-class reliability of the environmental risk factors identified by the two occupational therapists in this study reflected the usefulness and the effectiveness of the photos in identifying the home hazards, with single rater measures as 0.741 and average measure as 0.888. It was believed that a good inter-rater reliability between OT-A and OT-B was indicated. The therapists believed that if participants worried the photos had not given the best information regarding their home, more photos could be taken. This project has demonstrated that technology can be used as an alternative to on-site home visit and has the potential to improve the throughput and efficiency of health services. The other benefits included being able to complete home visit in situations that would otherwise not permit a home visit such as poor living conditions, rural and remote locations, and complex family situations. Therefore, smartphone photos could be in the future an alternative to actual home visits from occupational therapists. The current global pandemic further reinforces this need and validates or research efforts as consumers indicate a desire for more telehealth options (Tenforde et al., 2020).
While most of the participants in our study sample found the application easy to operate, some participants recommended additional support from a technical person or input from occupational therapists to assist throughout the photo-taking process. This phenomenon can be explained by the effectiveness of verbal and written instructions (Marcus, 2014). (Paasche-Orlow, 2011) demonstrated that providing verbal and written discharge instructions together are more effective in imparting information on care to be followed at home than either verbal only or written instructions only. For the hospitalised patient, education is generally a feasible approach to provide sufficient information and training to enable safe care at home, especially for patients with diabetes who to obtain better knowledge about the name of the medication, dosage, dose frequency and the duration for the insulins were to be used (Kimmel et al., 2010). Pictographs or videos that are easily understood by both patients or their family or friends can be supplementary to written instructions and verbal communication for better comprehension. We acknowledge that the role of occupational therapists completing home visits cannot be completely replaced by technology, however, the introduction of pictographs or videos could eliminate the costs of hiring a technical person as assistant.
We observed that user skill can be a major obstacle. Firstly, some volunteers changed the format and size of the photos to send them through email, however these photos were unsuitable for viewing in the “Cardboard Camera” application selected by the project. Only 10/49 photos were successfully imported into the application and occupational therapists were required to assess the panoramic photos with computer as an alternative. Secondly, technology is rapidly changing over time, and it is extremely challenging for health professionals to explore or update latest technology devises that are suitable and appropriate for patients (Atwal et al., 2014). Previous researchers identified the use of Internet, telehealth and conventional photos in assisting occupational therapists to assess home environment, however there are still no clear methods that provides a comprehensive outcome (Hoffmann & Russell, 2008). In practice, occupational therapists have been using different technologies such as virtual reality in rehabilitation to improve patient outcomes (Rose et al., 2005), wearable devices for fall detection and prevention (Mannini et al., 2015) and the use of different assistive technology for people with visual impairment (Alves et al., 2009). As technology improves, more universal and comprehensive assistive technology solutions are expected to assist health professionals with identifying environment risk factors in the home.
Our study showed a good reliability of ICC 0.851 (95% CI 0.623–0.941) between OT-A and OT-B in using smartphone photos to identify environmental risks. The average time that OT-A and OT-B took to assess the digital photos were 10 minutes and 4 minutes respectively. We infer that while the smartphone photos provided a significant outcome in reducing time costs and travelling costs for therapists, both therapists provided feedback that they preferred completing home visits rather than reviewing digital photos. We highlight that data for this study was collected prior to the global pandemic. They questioned if the panoramic photos could picture all the components of the home, including the door jambs (whether patients can step in/out of the room), the stability of furniture, and the length or width of hobs, toilet or bed.
As a feasibility study to identify the environmental risks in the home with photos, no baseline conditions of individuals were provided. This may have impacted the participation rate (20% of all invited) as potential participants were not necessarily in the group of possible longer-term beneficiaries of home visits. This was an ethical related limitation as the protocol needed to be shown to be feasible prior to targeting a more vulnerable population. Recent studies have had similar challenges in recruitment (Latulippe et al., 2022). In reality, the baseline conditions of individuals are necessary for occupational therapists to determine their functional capability, including mobility and transfers (Griffin & McConnell, 2001). For example, if the individual was required to use mobility aids, occupational therapists would investigate into different perspective in the house, such as having adequate spatial area to manoeuvre the wheelchair or scooter in the house safely (Atwal et al., 2014). The occupational therapists in our study commented that digital photos could be helpful as a screening tool to prioritise the workload of completing a physical home visit, but independently agreed that home visits are still irreplaceable by technology at this stage.
Limitations and Strengths
There are several adaptations which should be considered for a future trial based on the findings of this study. Firstly, the generalisability of the study sample could be improved by recruiting participants from primary care including acute and rehabilitation units. In the current study, convenience sampling and snowball sampling had been used for recruitment, which had inherited bias (Heckathorn, 2011). It allowed the study to take place where otherwise it might not be possible to conduct because of a lack of participants. This method was the most cost-effective method, and the data is readily available for the research to collect, however providing limitations of a narrow demographic population or a favourable population. Most of the participants in this study were aged 18–39 and a generally healthy group, thus future research is necessary to examine the capability in using new technology in the elderly population group. This study did not collect information on participants medical conditions or disability, so we make no inferences of how the presence of physical and cognitive conditions impacts the ability to manipulate a phone and follow a set of instructions. Future research would explore recruiting a more representative sample of community dwelling people which may benefit from a falls risk home assessment. While our sample was small (20), in accordance with recent research findings in the usability testing research domain, this exceeds the suggested threshold number of five participants that are necessary to provide useful and effective feedback when using protocol for interactive prototype evaluation (Følstad, 2017) Secondly, this is a feasibility study aimed at examining if participants are able to follow guidelines and to use the technology application effectively. Therefore, this study only focused on three specific areas as a trial.
While acknowledging limitations, we believe this study is timelier than we originally predicted and adds to the rapidly growing evidence of modalities for telehealth. We differentiate our work as it was conducted in Australia, a land of great geographical distances and variable internet connections. Indeed video conferencing has recently been explored by others (Latulippe et al., 2022) and concluded that smartphones may have potential. Photos can be stored to be assessed at a time suitable for a therapist. They also require less bandwidth and can be saved, stored and forwarded more efficiently than videos. Taking a photograph is an age old practice, whereas taking a video requires a level of skill and training. As a pilot, several areas for refinement and future research are required before a full trial can commence with authentic users of occupational therapy home visits. These include a refinement of the guidelines, an improved mechanism to assess the quality of the images provided and a more streamlined method of submitting images. For example, the use of online government platforms and cloud storage systems may be a more efficient way of storing and transferring images versus the email method used here. The next stages of the research plan will aim to improve on these elements.
Conclusion
In conclusion, the Smart-home study is a feasible and acceptable intervention for adults from 18 or above to take smartphone photos of their home environment for occupational therapists to access whether any environmental risks appeared. Findings from this study are promising for the use of virtual reality and photos to assist with occupational therapists’ home visits in future clinical practice and research, however long-term studies and improvement in technology use are needed to determine the clinical effectiveness, safety, and sustainability before wider implementation of the program is appropriate.
Footnotes
Acknowledgements
The authors express gratitude to the occupational therapists who assessed the smartphone photos.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
