Abstract
Introduction
Successful adoption of POCTs (Point-of-Care tests) for COVID-19 in care homes requires the identification of ideal use cases and a full understanding of the contextual and usability factors that affect test results and minimise biosafety risks. This paper presents a scoping-usability and test performance study of a microfluidic immunofluorescence assay for COVID-19 in care homes.
Methods
A mixed-methods evaluation was conducted in four UK care homes to scope usability and to assess the agreement with qRT-PCR. A dry run with luminescent dye was conducted to explore biosafety issues.
Results
The agreement analysis was conducted on 227 asymptomatic participants (159 staff and 68 residents) and 14 symptomatic participants (5 staff and 9 residents). Asymptomatic specimens showed 50% (95% CI:1.3%−98.7%) positive agreement and 96% (95% CI: 92.5%−98.1%) negative agreement with overall prevalence and bias-adjusted Kappa (PABAK) of 0.911 (95% CI: 0.857−0.965). Symptomatic specimens showed 83.3% (95% CI: 35.9%−99.6%) positive agreement and 100% (95% CI: 63.1%−100%) negative agreement with overall prevalence and bias-adjusted Kappa (PABAK) of 0.857 (95% CI: 0.549−1). The dry run highlighted four main sources of contamination that led to the modification of the standard operating procedures. Simulation post-modification showed no further evidence of contamination.
Conclusion
Careful consideration of biosafety issues and contextual factors associated with care home are mandatory for safe use the POCT. Whilst POCT may have some utility for ruling out COVID-19, further diagnostic accuracy evaluations are needed to promote effective adoption.
Background
Protecting care homes from COVID-19 is complex1–3 and will continue to be challenged by immunosenescence in care home populations 4 and the emergence of new strains, 5 even after widespread roll-out of vaccines. Minimising ingress into homes 6 can reduce the risk of infection but restricting visiting can cause physical and mental deconditioning in residents and may contravene fundamental human rights. 7
Testing regimes may help facilitate safe opening of care homes by enabling early identification of infection through rapid establishment of COVID-19 status for those who are symptomatic, and through regular screening of those who are asymptomatic. Laboratory based reverse-transcriptase polymerase chain reaction (RT-PCR) tests are considered to be highly sensitive and specific but results may take more than 24 h to be received, and sometimes several days to return to care homes, making them suboptimal for outbreak management. 8 Widely deployed lateral flow device (LFD) antigen tests provide results within 30 min but normally have lower sensitivity and specificity than RT-PCR and present other use challenges in a care home context.9,10 Changes in sensitivity of LFDs have been reported as a direct consequence of the context of use and when carried out by untrained healthcare workers.11,12 These findings demonstrate that successful adoption of a diagnostic test requires a fuller understanding of all factors affecting test results, including: user competence assessment; risk assessment of testing kits in the environment in which they will be used; ease-of-use issues; the prevention of errors. We previously described the care home workstream of the COVID-19 National Diagnostic Research and Evaluation Platform in Care Homes (CONDOR-CH), which is a service evaluation project to evaluate novel point of care testing technologies for COVID-19 designed to overcome shortcomings of currently used technologies. 13
The study reported here considers the use, in care homes, of a microfluidic immunofluorescence assay for the direct and qualitative detection of nucleocapsid protein antigen from SARS-CoV-2 (hereafter referred as LumiraDx). The COVID-19 testing strip for LumiraDx is designed to be used with samples collected from the anterior nares or nasopharyngeal site using a swab eluted into a vial of extraction buffer. A single drop of the specimen in extraction buffer is added to the test strip from a vial dropper cap. LumiraDx enacts the test protocol using dried reagents within the test strip. The result is determined from the amount of fluorescence detected within the measurement zone of the test strip. Analyte concentration in the specimen is proportional to the fluorescence detected. Results are displayed on the instrument's touchscreen within 12 min of sample administration. Test strips are single use and disposed of post-assay. The system was trialled and then subsequently deployed in different hospitals across England to inform decision making upon hospital admission.
This study aimed to:
Evaluate diagnostic agreement, usability and biosafety risks of an automated antigen test Generate risk minimisation and implementation strategies to enable test adoption in care homes.
Methods
We conducted a mixed-methods (quantitative/qualitative) 14 evaluation with an agreement study of LumiraDx with qRT-PCR in care homes combined with a qualitative explorations of risks in use and to mitigate biosafety issues using unmoderated remote usability observation.
Homes were recruited using a national care home WhatsApp COVID-19 peer support group, 15 public-facing social media and two national care home organisations: Care England and the National Care Forum. It includes nursing and residential homes, with corporate chain, independent and third-sector providers with between 20−350 residents per home. From this sampling frame we purposively selected four care homes (Appendix I, Table A1): two with nursing and two without nursing (also known as nursing and residential homes respectively); the homes selected were in two regions of the UK, including two independent care homes and two from small chains of ownership. The sample was designed to maximise potential differences in staff training, and organisational configuration that might impact implementation of a point-of-care test The homes had previously been involved with the evaluation of a point-of-care Polymerase Chain Reaction (POC-PCR) test. 13 Their experience of deploying other POCTs was felt to provide useful contextual information for understanding LumiraDx use.
A range of participants were chosen to understand workflows around staff and resident routine testing. This sample enabled us to identify possible errors arising from repeated use of the machine by different staff members, on different days, under different circumstances.
Swabbing and eluting specimens into buffer is undertaken in bedrooms that are remote from the LumiraDx testing equipment raising potential biosafety issues (e.g. from handling COVID-19 positive samples at the bedside, during transfer specimen and upon arrival at the machine) due to risks of sample spillage of a potentially infected specimen. As opposed to standard sample collection techniques for qRT-PCR tests, the swab must be swirled, squeezed and removed from the buffer, and then correctly disposed as a biohazard. This procedure may expose staff and residents to risk (Figure 1). Extracted samples are applied to the test strip by gently pressing the sides of the extraction vial until one whole drop is visible; this procedure may cause contamination of the device and testing area (Figure 2).

The operator removes the patient swab while squeezing the middle of the extraction vial to remove the liquid from the swab. The swab is discarded in biohazard waste.

The operator applies the extracted sample from the extraction vial onto the sample application area of the inserted test strip.
Two randomly selected care homes (hereafter called care home A and care home B) were asked to conduct the simulation with GloGermTM liquid, a mineral oil-based solution containing proprietary powder which fluoresces under ultraviolet light (used in place of buffer whilst following the SOP). After replacing LumiraDx buffer with GloGerm, we asked them to examine their work and care areas under an ultraviolet light for signs of spillage and contamination. Verbalisation and interactions with the device were video recorded. Pictures of the testing area and device were taken at the end of each test Following this, amendments were made to the SOP (Appendix III) and 10 further simulations [one volunteer in each care home (A and B) conducting five tests] were run to assess whether this had rectified the identified risks.
Qualitative data were thematically analysed.16,17 Coding of the responses were performed by one researcher (MM) and codes were agreed following expert review with other human factors experts within the research team. Interviewees consented prior to the study and video interviews were recorded using conferencing platforms (MS Teams, Zoom.) Recordings were transcribed using AI transcription software (Otter.ai). Recording, transcription files and video materials were stored on a password-protected secure server. Enrolled participants were assigned a confidential identification number (IC) on consent, to be used on all corresponding data.
No formal power calculation was undertaken. LumiraDx, in non-care home settings, has a reported sensitivity of 83.8% (95% CI: 76.4−89.2%) and specificity of 98.7% (95% CI 97.2−99.4%) with RT-PCR 20 as the reference method. Our objective was evaluating agreement between LumiraDx and qRT-PCR when the test was deployed in care home settings. All test results, including equivocal results and failures, were reported as per US Food and Drug Administration (FDA) guidance. 21 We calculated positive and negative agreement of LumiraDx with qRT-PCR results, (Cohen) Kappa, and the Brennan and Prediger statistic (equal to the prevalence and biased adjusted kappa (PABAK) with associated 95% confidence intervals. Primary analysis was based on valid results for all tests stratified by symptomatic/asymptomatic participants. Calculations were conducted in Stata/SE 16.1 using sensitivity analysis to examine the impact of equivocal results and test failures.
Results
Biosafety simulations
Four main sources of contamination with GloGerm dye occurred. Table 1 shows UV LED images of operators after conducting the test with the GloGermTM. These data along with interviews, led to SOP modification (Appendix III) to improve the use of personal protective equipment (PPE) and cleaning procedures, and how to handle contaminated material and specimen at the bedside. Simulation after modification showed no further evidence of contamination (Table 2).
Biosafety issues identified using gloGerm dye during the simulation.
Dry-run with gloGerm dye adopting the amended SOP.
Usability
Interview findings focussed on three main areas:
Rapid test turnaround versus test throughput: Fast test turnaround (∼12 min) was valued. This was countered by staff members’ inability to handle more than one specimen at a time, and to carry on with their duties, especially if a large number of tests have to be conducted: “The senior members of staff, the nursing staff, they are not going to be committed that level of time […] you’ve got to go back every 12 min. So, it's taken up more of your time”. (CH3)
Respondents identified specific use cases where LumiraDx was intuitively appealing: notably, sporadic testing such as testing of visitors at entrance to the home. Small changes to workflow were seen as potentially enabling the integration of LumiraDx into home routines, for example by staff testing themselves one at a time on arrival at work.
Biosafety issues and appropriate staff user profile: Spillage was the main biosafety characteristic identified. Participants recognised the mediating effects of safeguard measures and amended SOPs; several staff felt testing was best undertaken by the more senior, and responsible, staff. Seniority was considered to be a function of time working in the home and/or could be denoted by a supervisor role. “I don't trust them as much [junior staff members], you know, to clean up after themselves…to use an expensive piece of equipment to not abuse that, to make sure that they’re writing the documentation down afterwards. So, for me, I would only be comfortable with somebody as a deputy manager or a manager level using the machine”. (CH2)
LumiraDx is marketed as a bedside test for healthcare settings, but staff felt it was inappropriate for a care home setting. This is because of the risk of spillage in areas with carpets and soft furnishing such as resident bedrooms. Further, it was seen as not appropriate in the communal parts of the home because the machine would be likely to attract attention from residents with cognitive impairment. The machine made a noise during testing and the staff felt that residents would find this distressing, further limiting the proposed bedside use.
Training materials: Respondents generally found training materials easy to follow but several reported that the use of appropriate PPE, including visors, needed emphasising. Easily interpretable visual guides and a quick ‘prompt list’ mounted close to the machine would support operators recalling the operational steps, especially when use was infrequent (CH1).
Testing agreement with laboratory RT-PCR
In total 241 tests were run on 227 asymptomatic participants (159 staff; 68 residents) and 14 symptomatic participants (5 staff; 9 residents) (Table 4; and Appendix IV, Table A2 stratified by staff/residents). Formal laboratory results were indeterminate for two specimens, both in asymptomatic participants, so agreement analysis was conducted for 225 and 14 specimens. A Standard for Reporting of Diagnostic Accuracy Studies (STARD) Flow diagram is shown in Figure 3.

STARD diagram of samples acquired and test results.
Full results from lumiraDx and laboratory PCR.
Agreement based on valid measures.
The negative predictive value was 99.5% and 88.9% in asymptomatic and symptomatic participants, whilst positive predictive value was 10% and 100% in asymptomatic and symptomatic participants respectively. A Standard for Reporting of Diagnostic Accuracy Studies (STARD) Flow diagram is shown in Figure 3.
Discussion
Our three main findings were: (i) usability observations are required to adjust SOPs for testing to take account of specific biosafety challenges arising from the care home context, (ii) use (and thus usefulness) is potentially limited if not integrated into workflow, and (iii) training materials need specific adaptations to match the information-literacy of care home staff.
As with other technologies we have evaluated in care homes, 13 the utility of the LumiraDx test will depends on other factors including the care home built environment, staff competencies and workflow. The addition of LumiraDx to the range of tests that could be deployed in care homes, and the specific identification of a use case (e.g. testing of staff and visitors before entering the home), opens up a wider debate about what care homes require as we move into the next stage of the pandemic and beyond. Staff highlighted the potentially restrictive impact of using LumiraDx on workflow because of the existing workload of staff members. We identified issues regarding how testing may influence other aspects of care delivery. Staffing ratios in care homes mean staff sometimes struggle to meet routine care requirements even under normal circumstances. 22 The introduction of time-consuming, and at present statutorily mandated, POCTs risks distracting staff from routine caring tasks. This may compromise the ethos of care in within homes. The care sector should now consider whether these technologies might be purchased and retained in the sector, and how to plan staffing for their deployment without incurring excessive opportunity costs.
This was not a powered diagnostic accuracy study and the resulting negative and positive predictive values are a consequence of the low prevalence of positives in the group. 23 The good negative agreement suggests that LumiraDx may have some utility as a rule-out test It could work well in the context of a threshold test, which is the deployment suggested by care home staff. This would need to be further validated in a cohort study to estimate the test’s negative predictive value in this population. In this context, the false positive rate would mean the need for formal confirmation through formal RT-PCR testing for those testing positive. The reason for the high prevalence of false positives in the sample was not clear. The distribution of these in the dataset suggested that this was a sporadic, rather than systematic issue. It is possible that some user-related issue contributed. For example, when LumiraDx was used in hospital, some false positives were found to result from cross-contamination of users’ gloves. Further evaluation of the technology in real world deployment is required to better understand these issues.
The strengths of this study are that it provides a unique insight into the deployment of a novel POCT in care homes, a setting in which it has not previously been evaluated. Careful consideration of biosafety issues using GloGermTM is, as far as we are aware, a first in this setting and could be replicated in future biosafety assessments in care homes. The in-context data on what is required if the technology is to be successfully adopted for care home deployment, could only have been acquired using the methodologies deployed. Study limitations include the small sample size, and our the findings may not be representative of the 14,000 care homes across the UK. It is recognised that the organisational factors identified in this sample of UK care homes may not be replicated internationally. Furthermore, we have limited knowledge of the physical properties of GloGermTM (e.g. viscosity) as an appropriate surrogate for human specimens. Nevertheless, the cautionary nature of our findings, about the need to evaluate new technologies in context in care homes, will likely to have wide application.
Conclusions
LumiraDx was successfully and safely deployed in care homes following adaption of the standard operating procedure. This took account of the care home environment and staff training. LumiraDx increases the options for testing technologies available to care homes. Future candidate technologies for roll-out in this sector should be subject to similar rigorous mixed-methods evaluation. A consideration of the health economic impact and opportunity costs associated with roll-out of POCT in care homes is required.
Footnotes
Acknowledgements
The authors would like to thank care home managers and staff members who took part in the study and members of the CONDOR platform for their comments: Mr Graham Prestwich, Ms Vale Tate, Dr Charlotte Harden, and Dr David Ashley Price.
Declaration of conflicting interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: RB declares the following: Scientific Advisory Board membership: Creavo, Aptamer Group, LumiraDx; Active consultancy: Siemens, Beckman Coulter, Roche; Recent consultancy: Abbott Point of Care, LumiraDx; Active research grants: Abbott Point of Care, Siemens Healthineers, Beckman Coulter, StreamBio.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by National Institute for Health Research (NIHR), Asthma UK and the British Lung Foundation, as a part of the CONDOR study. MM, PB and PK are supported by the NIHR London In Vitro Diagnostics Co-operative; ALG is funded in part by the NIHR Applied Research Collaboration-East Midlands (ARC-EM). GH is supported by the NIHR Community Healthcare MedTech and IVD Cooperative. AJA is supported by the NIHR Newcastle In Vitro Diagnostics Co-operative. KD is seconded part-time into the technologies Validation Group, within Test and Trace. CT and KS are funded in part by the NIHR Applied Research Collaboration Yorkshire and Humber. DL is funded in part by the NIHR Applied Research Collaboration (ARC) West Midlands and the NIHR Community Healthcare MedTech and IVD Cooperative (MIC) at Oxford Health NHS Foundation Trust RP acknowledges part-funding from the National Institute for Health Research (NIHR Programme Grant for Applied Research), the NIHR Oxford Biomedical Research Centre, the NIHR Oxford and Thames Valley Applied Research Collaborative (ARC), NIHR Oxford Medtech and In-Vitro Diagnostics Co-operative and the Oxford Martin School.
The views expressed are those of the authors and not necessarily those of the funders, the NHS, the NIHR or the Department of Health and Social Care. LumiraDx were loaned, at no cost, by the supplier, LumiraDx UK Ltd. The research team were independent of the manufacturer throughout and LumiraDx have not participated in the research and data analysis.
Ethical approval
This project was approved as a service evaluation by Imperial College Healthcare NHS trust (ICHNT) – registration no. 471. The collation of data from routinely collected specimens taken as part of mandated patient care was determined using the Health Research Authority online toolkit to be a service evaluation.
