Perceived benefits and challenges of using an electronic cancer prediction system for safety netting in primary care: An exploratory study of C the signs

Abstract

Objectives

This paper reports on an exploratory study into the perceived benefits and challenges of using an electronic cancer prediction system, C the Signs, for safety netting within a Primary Care Network (PCN) in the East of England.

Methods

The study involved semi-structured interviews and a qualitative questionnaire with a sample of 15 clinicians and practice administrators within four GP practices in the PCN.

Results

Participants generally perceived benefits of C the Signs for managing and monitoring referrals as part of post-consultation safety netting. Clinicians made little use of the decision support function though, as part of safety netting during the consultation, and referrals were still sent by administrators, rather than directly by clinicians through C the Signs.

Conclusion

Emphasising the benefits of C the Signs for post-consultation safety netting is most likely to gain buy-in to the system from clinicians, and can also be used by administrators for shared visibility of referrals. More evidence is needed on the value of C the Signs for safety netting during the consultation, through better diagnosis of cancer, before this is seen as a valued benefit by clinicians and provides motivation to use the system.

Keywords

cancer referral diagnostic tool implementation primary care safety netting

Background

Urgent referral from primary care to specialist services is recognised as key to achieving early-stage cancer diagnosis, thereby improving treatment efficacy and survival outcomes.^1,2 In the UK, it is estimated that earlier diagnosis could improve the survival chances of 52,000 patients per year.¹ However, the large number of cancer types (over 200), and their varied, ‘non-specific’, or vague symptom presentations^3–5 can make recognition and referral of cancer challenging. In 2015, the National Institute for Health and Care Excellence (NICE) issued new cancer guidance,³ which included lowering the threshold for diagnosis, provision of direct access to diagnostic tests and investigations, and recommendations to improve safety netting. Safety netting aims to ensure patient safety by managing uncertainty in the diagnostic and prognostic process, and in patient management.⁶ Jones et al⁶ propose the following definition:

Safety netting is an essential process to help manage uncertainty in the diagnosis and management of patients by providing information for patients and organising follow-up after contact with a health professional. This aims to empower patients and protect healthcare professionals. Safety netting may be performed at the time of the contact between health professional and patient or may happen after the contact through active monitoring and administrative systems to manage results and referrals (Jones et al.⁶ p. 74).

Based on this definition, we can summarise safety netting for the purpose of this paper as both; (1) procedures/processes that aim to manage uncertainty in the consultation, and (2) procedures/processes that aim for active monitoring and management of referrals and results, post-consultation. Approaches to safety netting have been found to vary widely, depending on a GP’s workload, time pressures, and working patterns, as well as their perceptions of a patient’s reliability to follow advice, and their suspicions about the likely diagnosis.⁷ Safety netting can be perceived as a shared responsibility between GPs and patients, but handing over responsibility to patients after consultations could put them at risk if GPs do not provide adequate information or specify follow-up actions in a clear timeframe.⁸

There is increasing interest in electronic systems and tools to support cancer diagnosis, management, and safety netting. Tools in secondary care have been used to assist diagnosis of specific cancers⁹ and support patients’ decision making regarding treatment options.¹⁰ Tools in primary care can alert GPs to patients’ cancer risk scores, enable symptom checking, and facilitate safety netting.^11,12 There is a lack of consensus though regarding the most important features of an electronic decision-support/safety netting tool, with concerns over additional workload generated by the tool, in some cases, overriding the potential to mitigate risks.¹³ In their study with users of the EMIS safety netting tool, Friedman et al.¹³ found that GPs wanted to have safety netting tools that are simple, flexible, and unobtrusive, and were not keen on having alerts and warning notifications, despite the potential for these to enhance safety netting. Studies on decision support systems in primary care have so far found little evidence of effectiveness in improving outcomes for patients, and there are calls for further research on how GPs interact with the systems, barriers to implementation and uptake, and their clinical effectiveness.¹²

C the signs

C the Signs was founded by two NHS doctors in England in 2017, and is a cancer prediction system which aims to support earlier and faster cancer detection, and improved safety netting of patients with a suspected cancer diagnosis.^14,15 It was piloted across three Clinical Commissioning Groups (CCGs) in the East of England from 2017, and was subsequently introduced in the Primary Care Network (PCN) in this study in June 2021. C the Signs is a cloud-based software which integrates with Electronic Healthcare Record (EHR) systems¹⁵ and it was deployed to all computer terminals in the 36 practices within the PCN. It was not mandatory for GPs or administrators to use C the Signs once it was deployed, and previous methods of referral were still available. The roll-out of the system was supported by awareness building exercises and training events, delivered by members of the C the Signs project team to clinicians and practice administrators. This included face to face training in practices, as well as online sessions, both general and bespoke to an individual practice. Training sessions were not mandatory for GP practices or individual staff, but participation was encouraged by C the Signs and by the CCG.

The system presents the clinician with a dashboard through which they conduct a risk assessment of the patient during the consultation, entering risk factors and symptoms from a pre-populated list. C the Signs then uses an algorithm to determine if cancer is suspected and recommends a pathway, bespoke to the local resources and services. A digital referral form is embedded in the system, and is pre-populated and flagged to the clinician if cancer is suspected, so the clinician can send an electronic referral. The corresponding patient information leaflet is also flagged, so the clinician can send this electronically to the patient.¹⁶ C the Signs is intended to support safety netting by improving diagnosis of suspected cancer, replacing manual creation of referrals by a clinician/administrator (usually completed after a patient consultation) with pre-populated electronic forms, and integrating follow-up processes within the dashboard. The dashboard enables clinicians and administrators to track referrals, and view referral rates and pathways across the practice.

A study by the founders of C the Signs¹⁷ reported an increase in the cancer detection rate among referrals in CCGs where C the Signs was used. However, Black et al.¹⁸ investigated the use of C the Signs in primary care services across five boroughs in North-East London, and found it was unlikely to enable earlier diagnosis of cancer. Black et al. identified that clinicians did not see the benefit of the decision-support tool, so used it infrequently. They did find though that the autocompletion of referral forms effectively managed the referral process, and that the ability to track the patient’s progress through secondary care was also valuable.

The authors were commissioned by the CCG in the East of England to carry out an exploratory study in 2022-2023 on the use of C the Signs within a PCN in the East of England. The study sought to understand the perceived benefits and challenges of using C the Signs for safety netting in primary care, from the perspective of users of the system.

Methods

The study involved qualitative research, carried out via interview or questionnaire, with clinicians and practice administrators in four GP surgeries using C the Signs within the PCN. The research was approved by the Health Research Authority and Health and Care Research Wales and the Norfolk and Suffolk Primary and Community Care Research Office in October 2021. Participants were provided with written information about the study prior to the start of the interview/questionnaire, and their written consent was obtained. Participants were able to withdraw from the study within 2 weeks of the interview taking place, or up until they submitted the questionnaire.

Four GP practices were purposively selected to take part in research between November 2022 and February 2023. The inclusion criteria set by the study commissioner was that participants should be using C the Signs to make or follow-up on referrals, to provide insight into the perceived benefits and challenges of the system, from a user perspective. 13 out of 36 practices in the PCN were using C the Signs, and we selected the four highest usage practices, based on the number of referrals they had made using C the Signs. After initial contact with the highest usage practice, clinical and administrative staff responsible for referrals and safety netting were invited via email to participate in the study, and a total of five people were interviewed. A semi-structured interview schedule was used to guide the discussion (see Supplemental Information 1). Interviews were conducted virtually using Microsoft Teams, lasting approximately 60 min, and were recorded and transcribed via the platform. Initial contact with the other three practices received the response that staff did not have capacity to engage in interviews, therefore the interview schedule was adapted into an online, qualitative questionnaire, using open ended questions, which was sent to the other three practices via email (see Supplemental Information 2). There were 10 responses to the questionnaire. The interview schedule was pilot tested with a clinician in the PCN, who was part of the project team on the roll-out of C the Signs, which confirmed the face validity of the questions. As the questionnaire replicated the interview schedule, the pilot test was not repeated for the questionnaire.

The qualitative data from the interviews and questionnaire were coded and ordered into categories using a Framework method.¹⁹ This method is part of a thematic analysis approach, and involved the following process; (1) transcription; (2) familiarisation; (3) coding; (4) developing a working analytical framework; (5) applying the analytical framework; (6) charting data into the framework matrix; (7) interpreting the data.¹⁹ We created the analytical framework in Excel, and summarised the data from each transcript/online questionnaire into the framework, with the participants listed vertically, and categories listed horizontally (for example; ‘perceived benefits', ‘perceived challenges’, ‘safety netting’). Although we adopted a predominantly deductive approach, with predefined codes identified from the literature, the semi-structured format also allowed participants flexibility to speak or write about their experiences outside the main pre-defined codes, and so inductive coding was also required where unexpected themes were identified. Both deductive and inductive codes were incorporated into the analytical framework. The two authors coded the data separately and then measured the inter-coder reliability level. According to Neuendorf,²⁰ reliability coefficients of 0.80 or greater are acceptable, and the coded data met this requirement, with an overall score of 0.90.

Results

Results have been combined from the interviews and qualitative questionnaire, to provide anonymity for individuals and practices due to the small sample size. Twelve of the 15 participants were clinical staff and 3 were administrators (see Table 1). Participants have been assigned codes as CL (Clinician) or AD (Administrator). Participants had between 1.5 and 38 years’ experience in general practice, and all either made referrals as a decision maker, or were responsible for sending or following up on referrals.

Table 1.

Overview of participants.

Role	No of participants
Clinician	12
GP	9
Locum GP	1
Paramedic	1
Practice nurse	1
Administrator	3
Total	15

Safety netting practices before/outside C the signs

We asked participants to describe safety netting in their practice before/outside of the C the Signs system, as a precursor to then understanding how C the Signs was used for safety netting. Safety netting activities described by participants could be categorised according to who was responsible for the safety netting (clinician/patient/practice administrator), and examples are provided in Table 2.

Table 2.

Safety netting before/outside C the Signs.

Responsibility for safety netting	Examples
Clinician delegates responsibility to patients within the consultation	I always say ‘if you’ve not heard anything in 2 weeks, you must call us back'. (CL3) Ask patients to contact referral point if they have not received communication. (CL6) Ask patients to contact the surgery within a specified time if they have not received communication from referral point. (CL13)
Clinician keeps responsibility after the consultation	I used to always task myself to check where is the patient, what’s going on afterwards. (CL1) We have monthly clinical meetings, where we generally review any near misses or significant events. (CL4)
Clinician delegates responsibility to practice administrators after the consultation	‘Task’ the medical secretary to send electronic referral on their system. Medical secretaries keep a list where they keep an eye on all the 2 week waits. (CL1) We [clinicians] always back up with task to the referrals team in-surgery. (CL2)
Practice administrators take responsibility after the consultation	Manual search of surgery clinical system to check if patients have been seen at the referral point. (AD1) [calling patients to check on referral] It’s quite a lot of effort involved […] leaving voicemails and sometimes we don’t always get an answer. It can be like several weeks before we get a reply […] It’s not very efficient. (AD1) Keep spreadsheet with dates of referrals and where it was sent to. Work through the spreadsheet periodically to ensure a clinic letter has been received. (AD3) [The referral team] at the end of the day, they search for any forms that have been generated and see if there’s any urgent that the clinicians have forgotten to do a task about or alert them to. (CL4) Secretaries keep lists of referrals and monitor patient attendances with task to GP to review again, after 4 weeks. (CL7)

The potential for delays or missed referrals was acknowledged, particularly in the two-stage process whereby the clinician created an intended referral, and the administrator then sent the referral:

A few times, I think it has been that someone forgot to do the task [to send the referral], and then later on the referrals team see a form just sitting in [the system] [...] the team have come back saying ‘there's a two week wait urology referral on the system…is that supposed to be sent?’ (CL4)

It was also evident from the range of examples given that there was no agreed way for safety netting to be managed across the PCN, and that the clinicians and administrators often used separate processes/systems to manage and track referrals.

Perceived benefits of C the signs

C the Signs was noted by some participants as offering an improved interface compared to the previous system (which we will call here ‘System A’) and additional functionality with the auto-population of the referral form. Participants noted: “It’s very simple. […] [System A] just looked so much wordier and I think that then can overwhelm you [...] I like the layout” (CL3), and “I think the auto population of the referral form is helpful” (CL4). Another commented:

C the Signs seems clearer set out. It’s just nicer to look at. And user wise, it’s friendlier. I think it’s very clean to look at, it’s straightforward, it’s less fussy than the other systems and easy to navigate. (CL4)

Three clinicians noted the usefulness of prompts and suggestions offered by C the Signs, in supporting their clinical decisions. For example:

It was coming up with the several different options […] sometimes that makes me think, oh yeah, [...] I need to do that, so for example, a blood test that I hadn’t thought of. (CL1)

C the Signs definitely does sometimes suggest different pathways that you could go down, like haematology or gastro or nonspecific. And then I sometimes think oh, actually, you know – are they a bit more non-specific? Is that really gastro? (CL4)

One participant also appreciated the automatic send of follow-up information to patients; “I like the fact that it sort of notifies the patient because sometimes I used to forget to give the patient the literature about the two week wait, so it does that automatically” (CL1).

Clinicians most widely noted the benefits of C the Signs for monitoring and auditing referrals. For example, two participants commented that C the Signs was “effective at monitoring those with confirmed cancer” (CL8), and “a lot better for tracing referrals and attendance” (CL10). Other participants noted: “I like that I can go in and see all the referrals that have been made, where they’re sitting on the dashboard” (CL1), and “there are [advantages] in terms of monitoring the figures within the practice” (CL2). This provided clinicians with visibility of referrals, and supported the clinician’s responsibility for safety netting after the consultation had finished.

Whilst there were benefits of C the Signs identified by some participants, they all also commented on challenges in using the system.

Perceived challenges of using C the signs

In contrast to the positive views from some clinicians on the system functionality, the requirement to input signs into the system led to one participant describing that it “takes longer and more information required” (CL7). The predictive and auto-population features were also perceived as a hinderance by some clinicians, with one concluding that “C the Signs just gets in the way” (CL9). Subsequently, some clinicians used C the Signs to generate referral forms, but did not utilise the decision support capability:

Initially, it was time consuming because I was trying to go in through searching the system, which was then coming up with lots of different referral options […] But then what I did was, I cut the corner and started doing the direct referral myself. […] I know the NICE criteria, I know what I’m sort of thinking after examining the patient, I’m still doing it kind of in an old-fashioned way, still relying on what I think it is rather than the artificial intelligence side. (CL1)

Normally by the time I’m generating the form, I’ve done those investigation. […] It’s rarely C the Signs telling me, oh, you know, do this, do that, even for things like fit tests [...] I tend to have already talked to the patient and then I’m just generating the form afterwards. (CL4)

It’s notable that these are the same clinicians (CL1 and CL4) who perceived benefits in using the system to support decision-making. Clinicians did note that lack of understanding of the system hindered its potential use, and noted the time pressures again:

I think we’re all probably doing the basics [...] but because of the time pressures […] I have cut corners to keep to time […] If you don’t understand, you will avoid some features. (CL1)

[C the Signs] has massively more potential than what we’re all using it for. And if we could have protected time to really navigate it and then have a, you know, a review on it, then I think that might be a way of getting, you know, colleagues to use it more. [...] I don’t feel we use it optimally, but that’s a time capacity issue. (CL3)

Related to this lack of understanding, was confusion about the referral process through C the Signs:

The only thing I don't understand which I'd like to know a bit more about, when I actually do the referral I send it, but then I think I have to task the medical secretary still, because they send it on a different system. (CL1)

I don't know how [referral process is] working […] when I generate a C the Signs referral, I still send a task in [System A] saying send a two week wait. And I'm not sure what [administrators] do with that or how they check it. (CL4)

C the Signs intends for clinicians to submit referrals directly through the system, removing the need for an additional referral task after the consultation. However, administrators were still sending the referrals once these had been generated by clinicians, with one administrator noting that they sent “4-5 [referrals] daily” (AD2) through C the Signs, and another (AD3) estimating that they had made 43 referrals so far using the system. This indicates a gap in understanding of the system, on the part of both clinicians and administrators, and a missed opportunity to streamline the referral process.

Similarly, whilst the dashboard was intended to facilitate monitoring and auditing of referrals within the system, administrators still maintained their separate processes for monitoring referrals, outside C the Signs. The administrators explained this as follows;

GP safety nets using C the signs leaflets - also internal safety netting by secretaries sending out safety net letter and a scheduled task set up internally to review referral status after [2 weeks] and then again after [4 weeks]. (AD2)

We do a search every month [in System A] to check patients have been seen [by the referral clinic]. (AD3)

Therefore, whilst C the Signs was providing benefits for clinicians in safety netting after the consultation, the potential benefits of the system for administrators to monitor referrals had not been realised.

Overall, there was general scepticism of the value of C the Signs compared to the previous system, for example, “I think [C the Signs and System A] are quite similar. … The general feeling is there’s not a lot of difference with things you have to do” (CL2), “this has been extensively discussed in-house and clinicians do not see where the value-added proposition is” (CL8). This is unsurprising, given that the decision-making, referral, and monitoring functions were not being used to their full ability, or often at all.

The challenges identified by participants impacted the ability of C the Signs to improve safety netting in the practices, as we will now discuss.

Discussion

Clinicians mainly perceived the benefits of C the Signs for monitoring referrals, which is in line with the findings of the study by Black et al.¹⁸ Participants’ examples of safety netting activities before/outside of C the Signs indicated that responsibility for safety netting is perceived to be shared across clinicians, practice administrators, and patients. Whilst shared responsibility is intended to mitigate against errors by involving more people in safety netting, there is potential for falsely assuming that tasks have been actioned or followed up by other members of the team, and the processes used by clinicians and administrators were generally separate (such as electronic ‘tasks’ sent to an individual or team, or spreadsheets maintained by an administrator). C the Signs offers an opportunity to standardise these safety netting activities post-consultation, as recommended by Fleming et al,¹¹ and to make them visible to both clinicians and administrators, by using the dashboard to monitor referrals, attendance, and results. However, this had not been realised in the practices, as administrators were still using separate processes to monitor referrals. Administrators were also still sending the referrals as a separate task, rather than these being sent directly by clinicians through C the Signs. This meant that there was still potential for missed or delayed referrals between the clinician and administrator.

Shared responsibility for safety netting with patients was discussed by participants, which, before/outside C the Signs, involved verbal instructions to patients as part of the consultation. C the Signs has the potential to support this shared responsibility, by flagging the relevant patient information leaflet, and enabling the clinician to send this electronically to the patient, as part of the consultation. Providing patients with specific, written information on when and how best to re-consult, is recommended as part of safety netting,²¹ however this feature was only mentioned by one clinician as a benefit of C the Signs. This indicates an opportunity to raise the awareness both of the importance of providing patients with written information, and the ability of C the Signs to facilitate this.

Clinicians were generally making little use of the decision support function of C the Signs, which is intended to help manage uncertainty in the consultation, by assisting in early diagnosis of cancer. Even clinicians who had tried out this feature had then decided to skip it and go straight to the form; the benefits of the decision support function were therefore outweighed by the need to save time, and by the confidence that clinicians had in their own knowledge and judgements. Clinicians did acknowledge that they were not using C the Signs to its full potential, but motivation to become more familiar with the tool, and to use the decision support functions, was affected by time constraints and the perception that they were already familiar with cancer signs and pathways, and so this aspect of the system was not necessary. Bradley et al.²¹ advise that clinical decision support tools should be emphasised as complementary to, rather than a replacement for, clinical acumen, to gain buy in from clinicians. However, the time pressures of GP consultations mean that C the Signs is not sufficiently valued to use even as a complementary tool. For C the Signs to be used more fully for safety netting in consultations, there needs to be more evidence of its effectiveness in recognising cancer, beyond what would usually be identified by clinicians.

Limitations

The study was conducted with a small sample of staff from GP practices using C the Signs. Gaining a larger sample was not possible due to challenges within the PCN, with lack of capacity in practices due to Covid-19 surges during the data collection period, and overall under-resourcing. A larger sample would have provided greater insight into benefits and challenges of C the Signs within primary care. Although outside the scope of this study, it would also have been valuable to collect data from the 23 out of 36 practices in the PCN who had not adopted C the Signs, to understand the barriers to adoption.

Conclusion and recommendations

C the Signs (and other similar systems) are currently most valued by clinicians for safety netting after the consultation, through managing and monitoring referrals. Emphasising these benefits is therefore most likely to gain buy-in from clinicians in using the tool.

The system also supports shared responsibility for safety netting between clinicians and practice administrators, enabling users of the system to view and audit referrals. Shared responsibility for safety netting is already common in GP practices, but clinicians and administrators often use separate processes. Promoting the shared visibility of the referral and the patient’s subsequent attendance and results as a benefit of C the Signs also has potential to gain support from clinicians and administrators in using the system, to better enable shared responsibility.

There is some evidence that C the Signs can support safety netting during the consultation, when the clinician uses the system to send patient information leaflets and referral letters electronically to the patient as part of the consultation. C the Signs enables clinicians to submit referrals directly through the system, avoiding the potential for missed or delayed referrals by sending them via an administrator. This feature was under-utilised in the practices however, with administrators still sending referrals separately. Although training was available to practice staff upon the roll-out of C the Signs, this aspect of the system needs to be emphasised in both initial and ongoing training. It is also essential for practices to commit to protected time for training, to ensure that both clinical and administrative staff have a shared understanding of the system and agree on the new approach to making referrals.

Finally, more evidence is needed on the value of C the Signs for improving safety netting during the consultation through better diagnosis of cancer, in order for this to be seen as a benefit by clinicians, and encourage their use of this aspect of the system. This evidence may come from further studies comparing cancer detection rates by clinicians using the decision support function within C the Signs, compared to those using clinical acumen alone.

Supplemental Material

Supplemental Material - Substance Use Treatment Completion Among Unhoused Young Adults: A Predictive Modeling Approach

Supplemental Material for Perceived benefits and challenges of using an electronic cancer prediction system for safety netting in primary care: An exploratory study of C the signs by Sara Spear and Pamela Knight-Davidson in Health Informatics Journal

Supplemental Material

Supplemental Material - Substance Use Treatment Completion Among Unhoused Young Adults: A Predictive Modeling Approach

Footnotes

Authors’ note

Some of this research was undertaken whilst SS was at Anglia Ruskin University, Cambridge, England.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Eastern Academic Health Science Network.

Ethical statement

ORCID iD

Sara Spear

Supplemental material

Supplemental material for this article is available online.

References

Behavioural Insights Team . Improving appropriate urgent GP cancer referrals: a final report by the behavioural insights team. 2018. Available at: https://www.greatermanchester-ca.gov.uk/media/1877/gp-referrals.pdf (accessed 13/03/23).

NHS digital cancer survival in England, cancers diagnosed 2016 to 2020, followed up to 2021, 2022; Available at: https://digital.nhs.uk/data-and-information/publications/statistical/cancer-survival-in-england/cancers-diagnosed-2015-to-2019-followed-up-to-2020/cancer-survival-by-stage (accessed 27/04/22).

NICE Guideline . Suspected cancer recognition and referral guidelines. 2015. Available at: https://www.nice.org.uk/guidance/ng12 (accessed 31/03/23).

Jensen

Kristensen

Bjerglund

, et al. The pathway and characteristics of patients with non-specific symptoms of cancer: a systematic review. BMC Cancer 2022; 22: 574.

Erridge

. Rapid Diagnostic Centres and early cancer diagnosis. Br J Gen Pract 2021. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8544161/pdf/bjgpnov-2021-71-712-487.pdf (accessed 27/04/23).

Jones

Dunn

Watt

, et al. Safety netting for primary care: evidence from a literature review. Br J Gen Pract 2019; 69(678): 70–e79.

Evans

Ziebland

MacArtney

, et al. GPs’ understanding and practice of safety netting for potential cancer presentations: a qualitative study in primary care. Br J Gen Pract 2018; 68: e505–e511. DOI: 10.3399/bjgp18X696233.

Evans

Macartney

Bankhead

, et al. How do GPs and patients share the responsibility for cancer safety netting follow-up actions? A qualitative interview study of GPs and patients in Oxfordshire, UK. BMJ Open 2019; 9: e029316.

Eminaga

Al-Hamad

Boegemann

, et al. Combination possibility and deep learning model as clinical decision-aided approach for prostate cancer. Health Inf J 2020; 26(2): 945–962.

10.

Cuypers

Lamers

RED

Kil

PJM

, et al. Uptake and usage of an online prostate cancer treatment decision aid in Dutch clinical practice: a quantitative analysis from the Prostate Cancer Patient Centered Care trial. Health Inf J 2019; 25(4): 1498–1510.

11.

Fleming

Nicholson

Bhuiya

, et al. CASNET2: evaluation of an electronic safety netting cancer toolkit for the primary care electronic health record: protocol for a pragmatic stepped-wedge RCT. BMJ Open 2020; 10: e038562.

12.

Medina-Lara

Grigore

Lewis

, et al. Cancer diagnostic tools to aid decision-making in primary care: mixed-methods systematic reviews and cost-effectiveness analysis. Health Technol Assess 2020; 24(66): 1–332.

13.

Friedemann Smith

Duncombe

Fleming

, et al. Electronic safety-netting tool features considered important by UK general practice staff: an interview and Delphi consensus study. BJGP Open 2023; 7(3).

14.

C the Signs . 2024. Available at: https://www.cthesigns.com/technology (accessed 05/05/2024).

15.

C the Signs . 2024. Available at: https://www.applytosupply.digitalmarketplace.service.gov.uk/g-cloud/services/477844579899862 (accessed 05/05/2024).

16.

Hanlon

Setters

Payling

, et al. Accelerating early identification of cancer in primary care using an artificial intelligence driven solution. York Health Economics Consortium. Available from: https://www.nhsx.nhs.uk/media/documents/York_Health_Evaluation.pdf (accessed 05/05/2024).

17.

Black

Machen

Parker-Deeks

, et al. Using an electronic safety netting tool designed to improve safety with respect to cancer referral in primary care: a qualitative service evaluation using rapid appraisal methods. BMJ Open Qual 2023; 12: e002354.

18.

Gale

Heath

Cameron

, et al. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013; 13: 117.

19.

Neuendorf

. The content analysis guidebook. London: Sage, 2002.

20.

Nicholson

Mant

Bankhead

. Can safety-netting improve cancer detection in patients with vague symptoms? BMJ 2016; 355: i5515.

21.

Bradley

Hall

Maniatopoulos

, et al. Factors shaping the implementation and use of Clinical Cancer Decision Tools by GPs in primary care: a qualitative framework synthesis. BMJ Open 2021; 11: e043338.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.40 MB