Sage Journals: Discover world-class research

Abstract

People with diabetes are at risk for diabetic retinopathy (DR), a leading cause of blindness. Early treatment can preserve sight; however, screening rates are low. We utilized psychological theories of motivation in tandem with human factors tools such as heuristic evaluation and task analysis to develop interventions to improve screening rates at a single clinic. Interventions addressed the system of screening, from the patients and their choices to clinic workers, device usability, and the clinic environment. Findings showed potential solutions to the screening issue situated within the theory of planned behavior. Future steps are to refine interventions and measure effectiveness.

Keywords

medical device theory of planned behavior medical systems diabetic retinopathy

High blood sugar from poorly managed diabetes mellitus causes progressive damage to retinal vasculature, resulting in diabetic retinopathy (DR). DR is the leading cause of blindness in those younger than 65 years, and about 28% of individuals with diabetes will develop a form of DR in their lifetime (Zhang et al., 2010). Annual screenings are recommended as symptoms often do not appear until late in the disease. This recommendation is difficult to follow due to various internal and external barriers (Call et al., 2006; Phillips et al., 2000). Internal barriers to screening include misinformation and anxiety about screening (Strutton et al., 2016) and a lack of awareness concerning the disease and its prevention (Van Eijk et al., 2012). External barriers include inability to physically access health care and inability to pay for health care (Chou et al., 2014), with racial minorities disproportionately at risk (Shi et al., 2014). Prevalence of the most severe form of the disease is three times higher in Black Americans and two times higher in Mexican Americans compared with White Americans (Centers for Disease Control and Prevention, 2020). A recent study showed that 73% of adults with DR in the state of North Carolina were not aware that they had the disease (Skaggs et al., 2017).

The Retinal Care–DR (RCDR) program (Retinal Care Inc., Oklahoma City, Oklahoma) was designed to reduce cost and transportation barriers by screening for DR during patients’ primary care visits. Once the RCDR screening test was administered, results were transmitted to an ophthalmologist who returned a diagnosis to the clinic, negating the need for an additional visit to an ophthalmologist. The main differences between the RCDR program and traditional screening were that the RCDR screenings occurred in a primary care setting rather than ophthalmology and that the screening did not require dilation via eye drops. Dilation was avoided through the use of a handheld electroretinography device (RETeval, LKC Technologies, Gaithersburg, MD) in lieu of traditional retinal photography or slit-lamp examination of the retina. Locating screening at primary care and eliminating dilation were thought to be key in increasing uptake of the screening as lack of transportation to specialized clinics, fear of dilation, and the wish to avoid the after-effects of dilation are often reported barriers to DR screening (Graham-Rowe et al., 2018, Kashim et al., 2018). However, Retinal Care informed us that after being introduced more broadly in 2018, screening rates remained low in clinics offering the RCDR program.

The problem brought to us by the chief science officer for Retinal Care was that screening rates at the clinics were much lower than the number of eligible patients at those clinics. Retinal Care thought part of the issue might be the reliance on medical assistants (MAs) to perform the screening. The original RCDR screening process was designed for one to two “super-users” at the clinic whose job would be educating about DR and performing the screening test, but this allocation of staff was not possible in the small clinics that adopted the screening system. However, any number of other issues might contribute to the low uptake, including the well-known barriers to DR screening. Because of the many variables, from system-level issues with the health care system to the reliance on MAs to the design of the screening device, it was not clear what interventions should increase screening rates.

We used Ajzen’s theory of planned behavior (TPB) along with traditional human factors methods as a framework to analyze what deterred screenings and then proposed interventions and design changes at the process, provider, patient, and screening device levels. We provide our experiences as an example of pairing common human factors and usability methods within the framework of a well-established model of decision making (i.e., the TPB). We used human factors methods to prototype and test one of these proposed changes, culminating in guidelines that should increase screening for DR. Our analysis and intervention targeted an endocrinology clinic in Oklahoma City, Oklahoma.

Theory of Planned Behavior as a Framework for Human Factors Analysis

In the TPB, intentions and behavior are based on three attributes: perceived behavioral control (the person’s perceived ability to enact behavior), subjective social norms (how the individual believes community around them perceive the value of the behavior), and attitudes (the individual’s own views about the behavior and it’s desirability; Ajzen, 1991). A later addition to the TPB supplied actual behavioral control, which not only affects perceived behavioral control but also directly affects behavior, bypassing intention to act. This is because actual behavioral control is often situated in the environment – a classic example came from a study of health care workers and hand hygiene (Bischoff et al., 2000; Kaplan & McGuckin, 1986). The number of sinks available had a strong effect on hygiene, as even the strongest intention will not result in handwashing with no sink present. As human factors psychologists, we wanted to leverage the beliefs within the TBP that were internal to those involved in the task (MAs and the patients who need screening), but acknowledge the power of external forces (such as actual behavioral control) and external barriers in the environment when devising solutions to the screening problem. Thus, we included not only the patients in our consideration but also the MAs at the clinic integral to the screening process.

We examined intention and behavior at multiple levels: (1) the MA choosing to inform patients of the screening option and the benefits connected to it, (2) the decision of the patient whether or not to be screened during a visit, and (3) the MA choosing to follow the protocol until the screening data were successfully transmitted to an ophthalmologist. All these functioned within the environment and systems of the clinic. We considered these two groups in the system of screening, as patient safety and health care worker well-being are correlated (Carayon et al., 2014). In our analysis, we considered the three attributes of the TPB for MAs and patients and how they were influenced by human capabilities and limitations. This was a way of using theory to combine human factors methods that tend to focus on the perceptual, cognitive, and movement abilities of humans with the beliefs that relate to motivation and intention. Organizing our analysis via the TPB focused our search for the sources of undesirable behaviors and, once identified, offered directions for solutions to change behavior.

User Groups

The targeted groups for this study included the MAs and patients at an endocrinology clinic. Six MAs were responsible for informing and screening patients for DR. This was in addition to their other responsibilities, such as moving patients about the office for the doctors, weighing patients, and measuring vitals. Through interviews with Retinal Care, we learned the MAs were taught about DR, the screening device, and the RCDR program by other MAs rather than a formal training program. Patients eligible for DR screening at the clinic tended to be older than 60 years, racial minorities, and uninsured. All had diabetes, type 1 or type 2.

Task Analysis

Task analysis is a useful form of formal task decomposition that can be flexibly focused on interactions with technology and the work environment (Gillan, 2012). We performed a hierarchical task analysis to understand the steps and order of steps currently performed for DR screening (Shepherd, 2003). The task analysis focused on the main goal of the RCDR: for a MA to conduct a DR screening with a qualifying patient. The steps needed to do this are listed in the task analysis in the order in which they were meant to occur.

Through a prior Retinal Care survey of MAs, semistructured interviews and observations, the lead author collected the information necessary for the task analysis of the RCDR screening process (Table 1). Interviewees included the RCDR coordinator, Retinal Care’s chief medical officer, and Retinal Care’s chief science officer. The protocol for interviews was to have the interviewee step through the processes while showing any manuals, guides, or other materials provided to the MAs or patients during screening. During the interviews, we asked clarification questions and whether there were known issues with each step. Retinal Care previously surveyed five MAs who worked at the clinic on the screening process, asking them questions such as “What seems to take most of your time during the RC [Retinal Care] service?” and provided us with the results. Once the task analysis was complete, potential errors at each step were recorded and coded by the authors into categories according to Reason’s (1990) taxonomy of unintended actions (including slips, lapses, and mistakes) and intended actions (some mistakes and violations). Using these categories informed our strategies, as slips, lapses, or violations prompt different interventions.

Table 1.

Sample From the Task Analysis of Diabetic Retinopathy Screening

Task no.	Task	Description of potential error	Error type(s)	Probability	Visibility	Consequence	Potential remediation	TPB
3.0	Collect screening authorization forms	Screening forms must be accessed and given to the patient. Because there are no identifiers on the form, if it is filled out illegibly there may be no link between the patient record and the form. The most likely errors come from misplacement or disconnection between illegible completed forms, patient records, and screening results.	Lapse	High. RCI estimates about 80% of forms were not readable or not sent in with screening data	High. Screening forms are not kept in a visible location in the clinic due to their rarity of use.	High. Without the proper forms the ophthalmologist is not able to give feedback on the readings. Three signed authorization forms allow RCI to bill the professional component of the test, medical record release for their eye care provider, release of information to next of kin, and permission to use their data in the future.	Provide an electronic form already linked to the patient record and DR screening. If an electronic form is not possible or usable for a patient, confirm that the name on the form is legible and if not, add a note to identify the patient who filled out the form. Include instructions to do this in the cognitive aid given to MAs. Investigate whether forms could be combined, requiring a single signature.	Attitudes: Most likely affected by feedback. Do the MAs know that misplaced forms are common and mean screening results can’t be paired to the patient? Is the same MA responsible for collecting the forms and transmitting them? Social Norms: The forms are supposed to be kept in a secure location. Social norms are to adhere to HIPAA rules in the clinic. Perceived behavioral control: It is likely the MAs do not feel they have much control over the handwriting of the patients. Actual behavioral control: If the forms are put in a location that does not cue their later transmission, memory failures prevent transmission.
3.1	Provide patient with three forms	Forms must be accessed from their location in the clinic.	Lapse or violation	Medium. Because the forms are physical and the rest of the screening electronic, MA must remember to retrieve them.	Low. Forms are not kept in a salient space during the appointment, as DR screening is a low-likelihood task for MAs.	High. Without the proper forms the ophthalmologist is not able to give feedback on the readings.
3.1.1	Explain each form	The first form authorizes RCI to access their health information, the second authorizes RCI to contact them or a designated parson, the third concerns billing of insurance. This information is in the forms themselves and in the RCI manual regarding screening. MAs may choose not to verbally explain the forms, leaving the patient to read them.	Lapse or violation	Medium. Because the forms are physical and the rest of the screening electronic, MA must remember to retrieve them.	Low. Forms are not kept in a salient space during the appointment, as DR screening is a low-likelihood task for MAs.	High. Without the proper forms the ophthalmologist is not able to give feedback on the readings.
3.1.2	Wait for patient to read forms, ask questions, and sign	MA may have to answer questions regarding the forms or direct patients where to sign. MAs indicated that they felt unsure about answering these questions and had trepidation about being asked a question they could not answer. Patients with low vision may require the MA to read the forms to the patient.	Lapse or slip	Medium. Patient may have trouble reading the forms if they have vision issues. Patient’s handwriting may be unreadable.	High	Low. It is unknown how much MA answers to patient questions affect screening once agreed to.	Standardized training of MAs could give them more comfort in answering questions about DR. Including a FAQ in their cognitive aid could also help them answer and give them assurance that they could find an answer to patient questions.
3.1.3	Collect forms	Forms must be kept unstapled but together so they can be faxed or scanned to email.	Slip	Low	High	High. Without the proper forms the ophthalmologist is not able to give feedback on the readings.
3.2	Store forms in HIPAA approved location	This is a new location that must conform to HIPAA standards and takes up space in the clinic.	Lapse or slip	Medium. Because the forms are collected but not immediately sent, they must be placed somewhere while the screening occurs. The proper location for storing these forms may not fit into the clinic flow for appointments and DR screening.	Low	High. Without the proper forms the ophthalmologist is not able to give feedback on the readings.
3.3	Fax or email forms to Retinal Care	Forms are generally all processed at the end of the work day. This could be long after the patient was screened. Prospective memory is needed to perform this task. Screening is rare, meaning the need to remember this task is rare.	Lapse	High. Forms must be remembered to be retrieved from secure location and then sent via fax or email.	Low. There is no indication in the process for MAs to know whether this task was completed for any one patient.	High. Without the proper forms the ophthalmologist is not able to give feedback on the readings.	Provide a cue “in the world” to remind MAs to send the forms at the end of the day. This could be via a secure storage location that is salient and needs to be empty by the end of the day (meaning forms were sent and then filed into patient records). If a report of patients being screened that day is created at the start of the day, this form could have a way to record that the forms for each were sent by the end of the day. Last, having electronic forms that are sent immediately on completion will eliminate this error.

Note: Task analysis focused on the MAs role in the screening process. Sample provided is for the task of collecting required authorization forms for screening. The full task analysis, with over 100 steps involved in screening patients, is available in the online Supplemental Appendix. Red indicates the most serious level for an error classification, yellow mid level, and green a low level. This was to allow quick identification of steps that were red all the way across the error classifications. The most problematic steps were informing eligible patients they could be screened, presenting and transmitting authorization forms, and eye sensor strip placement. TPB = theory of planned behavior; RCI = reliable change index; DR = diabetic retinopathy; MA = medical assistant; HIPAA = Health Insurance Portability and Accountability Act; FAQ = frequently asked question.

Each step was analyzed for error probability (based on the interview information as well as our own knowledge of likely usability errors), error visibility (chance the error will be noticed, this included both visual and cognitive “visibility”), and error consequence. We included potential remediations for the error when possible and connected the larger steps to the TBP. The final task analysis was shown to the interviewees for their approval of its accuracy and completeness and changes made according to their comments (Table 1).

Task Analysis Findings

From the RC survey, we discovered that the MAs were uncomfortable doing the screenings, citing time pressure, difficulty using the device, and feeling unequipped and uncomfortable teaching about DR. Interviews revealed that often patients were not informed that the new screening test does not require pupil dilation, a barrier to DR screening. Finally, we discovered from interviews that the clinic had trouble keeping track of the authorization forms required for the ophthalmologist to remotely interpret the screening tests while remaining compliant with health care privacy and other regulations, adding more time to the screening and sometimes rendering it useless. These findings were integrated into the task analysis in the steps that would be affected.

Three themes appeared in the task analysis: (1) Screenings were not incentivized nor integrated into the daily procedures of the clinic, resulting in likely memory errors to prepare for or offer screening and violations when screenings might be judged as not worth the time required of the MAs. Patients were often not given education on the value of a DR screening before deciding whether to be screened. This reduced the number of screenings, but also the workload of the MAs. This theme most closely matched with the attitudes and social norms that related to behavioral intention in the TBP. (2) The processing of authorization forms was a bottleneck for the screening process and also resulted in lost or wasted screenings. This theme matched with actual behavioral control in the TPB. (3) Correct sensor strip placement was a challenging task in use of the RETeval device, inviting mistakes in diagnosing why the sensor strips were not working and skill slips in applying the strips. In this, all four attributes of the TPB could be assumed to affect behavioral intentions and actual behavior. Overall, screening was time consuming, adding around 30 minutes to a visit.

We hypothesized that reducing the time MAs spent tracking down forms and educating patients about DR and about the screening process would result in more screenings because it would reduce their workload (a usability criterion for patient safety; Carayon et al., 2014), improve their attitudes on screening and their perceived social norms. Better training and examples of eye sensor strip placement should increase both MA perceived behavioral control and actual behavioral control. On the patient side, improving education should change perceived social norms and attitudes about being screened.

Heuristic Evaluation

Our task analysis identified use of the RETeval device as a point of difficulty for the MAs. Although we could potentially influence the training and support offered for use of the device, it was not possible to change the device itself. Four trained usability experts conducted an evaluation of the RETeval device using Nielsen’s 10 Heuristics (Nielsen, 1994; Figure 1 and Table 2). The experts discussed their evaluations and came to consensus.

Figure 1.

The screening test device. (1) Docked RETeval screening device docked ready for use with packaged sensor strips and connection cord. (2) Opened sensor strips with the connector attached. (3) DR screening device in use. A–C show the device and the user interface: (A) Shows the menu when initiating screening. (B) shows the interface for entering medical IDs. (C) shows the handle for the RETeval device, with the small connector for the sensor strip lead visible on the underside of the device handle.

Table 2.

Heuristic Analysis of Usability Issues Related to the RETeval Device

Screening device attribute	Description of issue	Heuristic violations
Using scrolling input device	The device is operated by using a small trackball. This ball makes the system difficult and frustrating to operate because inputting values requires continuous manual scrolling, choosing inputs, and scrolling again.	User control, flexibility/efficiency, aesthetic/minimalist design
Entering patient birth date	If a barcode is not generated or scanned, participant information and birthdate must be entered manually. Patient birth date must be entered for the screening device to function. Using the roller ball as an input device, this takes time and this time is increased by scrolling back to the birth years of older patients. If MAs do not dock the device immediately after testing a patient, the records must be matched later by birth date.	User control, error prevention, system visibility, flexibility/efficiency, documentation
Applying and scanning sensor strips	Placement of the eye strips varies between patients. While there is a graphic used to display appropriate positioning (Figure 2), it still leads to errors due to the lack of feedback regarding proper placement. About 2% of the population lacks enough skin conductance for the strips to work, even when properly placed, but it is not possible to tell if the test is failing due to placement or patient skin conductance, or a number of other patient-related variables	User control, error prevention, error recovery, consistency
Patient discomfort	In the whole process, patients must have strips placed near their eyes and hold their eyes open to bright bursts of light. If patients close their eye for an extended period of time during the test, they must be retested. This process must then be repeated for the patient’s other eye. Evident discomfort in a voluntary procedure is demotivating for the MAs and can lead to issues with results, procedure completion, and whether the patient agrees to be screened the following year.	User control, error prevention, error recovery, consistency
Transmitting results	MAs must remember to dock the device to upload data as well as scan and send the paper authorization forms. After this, it can be days before they receive test results because they must wait for an ophthalmologist to review the data. This means that they have extremely delayed or no feedback on whether they screened the patient correctly. Issues either with the retinal screening or the authorization form could prevent the MAs from sending the patient data to the ophthalmologist.	System visibility, flexibility/efficiency, documentation

Note. MA = medical assistant.

The most consistent heuristic violation was a lack of user control, which aligned with our hypothesis regarding the MAs’ integration of screening into their daily routine. User control and freedom describes the user’s ease and ability to undo mistakes they make in the system. If MAs believe they cannot fix their mistakes easily and effectively, they are less likely to use the system. An example of this was placing the eye sensor strips (Figure 2), which required precision, differed slightly depending on the person and caused failures if corrective actions were not taken. Struggles with strip placement likely influenced MAs perceived behavioral control as well as their actual behavioral control. Overall, as can be seen in Table 2, there were a number of heuristic violations.

Figure 2.

Image from the RETeval manual provided to MAs showing correct and incorrect placement of the sensor strips.

Applying the TPB in a Human Factors Process

The TBP offered us a formalized framework to guide our thinking as we applied the traditional human factors tools of task analysis and heuristic evaluation. In the same way a heuristic analysis guides an human factors expert to formally consider system visibility and error prevention, the TPB guided us to consider the attitudes, social norms, and perceived/actual control the MAs had over the screening process. This was helpful as the human factors tools were biased toward human capability and limitation, while the TBP focused on belief and motivation. The most overlap occurred in the TBP attribute of actual behavioral control, but in this our human factors–centered considerations included a broader range control than in traditional TBP studies. Usually actual behavioral control in the TBP is linked closely to the environment, as in the aforementioned example of the number of sinks. We considered not only similar environmental constraints but also the lack of actual control we know that humans have over their memories, attention, and other limited cognitive resources (not traditionally considered in studies using the TPB).

Any intervention would have to address not only improved usability but also change the system that drives social norms, attitudes, and demonstrate to the MAs that they have control over the DR screening process. In Figure 3, we provided a method for linking potential interventions to the beliefs and actions within the TPB they were designed to affect.

Figure 3.

Description of the beliefs of the medical assistants situated within the theory of planned behavior as it applied to providing DR screening to patients. Beliefs present in the theory are linked to issues found in the task analysis and heuristic evaluation (yellow). These are then linked with potential interventions that fall under the general categories of usability, training, and system change (purple). Actual behavioral control, not a belief, has special status as it affects both perceived control and behavior directly. Interventions that would affect actual behavioral control are repeated in a light gray font for perceived behavioral control to show that they affect both.

Addressing the beliefs, intentions, and behaviors of the patients was a more complicated process, made more difficult by our lack of access to actual patients. Thus, to hypothesize interventions aimed at the patients, we relied on human factors expertise in usability and age-related differences. For the patient, screening could be more far reaching than a simple test result – it may give them “bad news” and create new anxieties along with the positive benefits of discovering a condition early enough to prevent blindness. Thus, helping patients form an intention to be screened would likely be influenced by their beliefs about their own disease management along with their beliefs about being screened. Unfortunately, as our task analysis uncovered, patients were not well-educated regarding DR before being asked to make a decision about screening. We combined the effects of screening information on patients, both good and bad, into our TPB/human factors diagram, as they both led to similar intervention solutions (Figure 4). The interventions linked to patient beliefs involved education with two foci. First, patients needed education about why DR screening would be beneficial – this included an explanation of DR, who is susceptible, the likelihood of vision loss, and the success of treatment options and their side effects. Second, patients needed education on the DR screening process – how long it would take, what would be involved, when to expect results, and the accuracy of screening findings. This would, in part, be addressed by empowering the MAs in providing information and answering patient questions. Patients needed this information explained to them in simple terms prior to being asked about screening.

Figure 4.

Description of the beliefs of the patients, as situated within the TPB, for influences on the intention patients have to receive DR screening. In comparison with the interventions proposed for the MAs in Figure 3, the interventions for patients focus on education, for informed decision making, and changes to the MA system to promote beneficial beliefs regarding DR screening (and potential subsequent treatment).

Patient Education Interventions

Because the patients make the ultimate decision regarding DR screening, we chose to implement an educational intervention for patients that supported the clinic workflow. Currently, when given the choice for screening, the patient has had no education about DR or the screening process from the clinic. To reduce the work burden of the MAs, we produced automated forms of education. We developed three methods of pre-visit education: a waiting room screen, waiting room commercial, and referral to the RCDR program’s own educational video that could be viewed on a mobile device. A sample of ten usability participants were recruited through research team contacts. Mean age was 33.5 years (SD = 14.3). Nine participants reported having a loved one with diabetes.

Waiting Room Screen

A slide was designed to be shown on a video screen, intended to rotate with other informational graphics. The selection of the text color and background were chosen to ensure ease in reading and comprehension (Figure 5, left). We showed this design to the nine participants who had a close contact with diabetes. We were unable to test this intervention in the clinic waiting room due to human subjects concerns over presenting in vitro diagnostic medical device information.

Figure 5.

Waiting room slides. Static screen display provides information and a way to access a video without typing or memory for a URL. Left panel shows the original design. Right panel shows iterated design after usability evaluations.

Results

Our questions gauged the accessibility of the text and use of the QR code and how useful it was in encouraging DR screening (Table 3). Our most important measure was the question “What do you know about DR now that you didn’t before?” Participants were able to provide concrete answers after reading the slide (Table 4). We were most interested in their use of the QR code to access more information and found that every participant was able to use the camera app on their smartphone rather than any specialized QR code apps. Most participants also reported that, if they had diabetes, they would be very likely to ask their doctor about DR screening after reading the educational information. Eight out of nine participants found the slide easy to read. All participants recognized the purpose of the QR code and were able to scan it. The majority of participants felt that if they had diabetes they would ask about retinal screening after observing the slide (Figure 6).

Table 3.

Educational Slide Survey Questions

Survey question	M		SD
How old are you? (years)	30.5		14.5
	Frequencies
	Yes		No
	n	%	n	%
Do you have any visual impairments (including glasses)	7	70	3	30
Do you or a loved one have diabetes?	10	100	0	0
Was this graphic easy to read?	9	90	1	10
Was it easy to scan the QR code?	10	100	0	0
	I was prompted		Scanned on my own
Did you use the QR code on your own? Or were you prompted to scan the QR code?	7	70	3	30

Table 4.

Information Learned Through the Educational Slide

Quotes from participants
You can get tested for [DR] quickly at a doctor’s visit
There are treatments available. Ways to prevent it are healthy diet, exercise, check ups
That there are few obvious symptoms until it’s almost too late
I didn’t know what the condition looked like. It was helpful and interesting for me to see visual of the retinal imaging
DR can lead to glaucoma and cataracts, laser surgery can shrink growth of blood vessels, yearly screenings are recommended
DR screenings are fast, pupils do not need to be dilated, diabetes can cause blindness
[DR] can be prevented
[DR] can cause blindness and is preventable
[A] quick exam is available

Note. DR = diabetic retinopathy.

Figure 6.

Likelihood of asking about diabetic retinopathy (DR) screening after observing the educational slide.

Participants suggested making the title more salient and creating a call to action. With those changes in mind, we redesigned the slide (Figure 5, right). The background of the title was outlined in a white rectangle instead of its original blue color. It was also suggested that the “Did you know?” text within the title was not a call to action for the users. Changing the text to “Prevent diabetes related blindness!” informed the user that diabetes can lead to blindness and encourages the user to read further to learn methods to prevent blindness. Last, research suggests that narrative videos are more memorable and persuasive than informational ones, thus having the video tell one patient’s story could enhance effectiveness (Braverman, 2008, Kreuter et al., 2010).

Waiting Room Video

We created a video for the waiting room with information about DR and the screening process (available in the online Supplemental Appendix). Providing something that the patients actively watch and listen to, along with the more emotive nature of the video, could be more attention grabbing and persuasive. Significant aspects of the video included a brief description of DR, awareness of testing without dilation, and availability of the test in the clinic. Information was framed as a gain (maintaining sight) as this has been shown to promote higher rates of screening for other diseases (Apanovitch et al., 2003). The video was less than a minute and covered information key to TPB beliefs.

Results

We administered a brief survey before and after participants viewed the video. All but one of the participants reported an improvement in knowledge after the video (Figure 7). Most found the video effective at teaching and stated they would be willing to be screened after viewing it (Figure 8).

Figure 7.

Knowledge about diabetic retinopathy (DR) before and after the informational video.

Figure 8.

Participants overall reported they would likely ask about screening after watching the video.

Overall, our survey suggested the short video was an effective tool to educate patients on DR and screening. Suggested improvements were to have a louder voice-over volume, end the video with a call to action via a patient talking with their doctor and subsequently getting screened for DR, and the final clip should include a more aesthetically pleasing tone. The final video was provided to Retinal Care as a prototype for one they could create and distribute to clinics.

Discussion

Through evaluation of the system, it became apparent that there were multiple aspects of the patient visit that were contributing to the lack of screenings. We applied traditional human factors methods (interviews, task analysis, heuristic evaluation) to understand the issues, then positioned solutions framed in TPB. This ensured our solutions were not piecemeal – they were linked to the attitudes, social norms, perceived, and actual behavioral control that should be expected to change the screening behaviors of the MAs and the patients. This also ensured our interventions were inspired by psychological theory, generally found to be more successful than atheoretical interventions (Hilliard et al., 2018; Joseph et al., 2016; Steinmetz et al., 2016). We designed one intervention for patient education, chosen because we expected targeting patient intentions would ease the workload of the MAs while also increasing the likelihood of screening.

Future Directions

An important next step in evaluating these two interventions would be to implement them in multiple clinics and measure change in screening rates. Surveys, while helpful in this formative stage, will not be enough of a measure after an intervention stage. While screening rates would be the gold standard of intervention success, we also suggest measuring the time an MA spends with each patient, the number of patients seen each day, and the wait time for a patient after arrival at the clinic. We suggest measuring the subjective workload of the MAs pre- and postintervention, as their perceptions of their work may not be reflected in performance measures (e.g., performance measures do not change, but perceptions in workload are reduced). Last, we need more information on patient knowledge and needs regarding DR, particularly patients with diabetes at different ages, to potentially tailor the education and messaging provided in an intervention study.

Limitations

Since this study was conducted, the screening environment has changed due to the COVID-19 pandemic. This rare event affected our follow up of intervention effectiveness, as clinics first closed, and then reopened in limited capacities. Furthermore, patients scheduled fewer appointments, meaning fewer screening opportunities at the clinics. However, we also believe that the pandemic underlined the need for a smooth screening process at primary care clinics. Patients can be screened for DR in a single visit, rather than scheduling multiple appointments with primary care and ophthalmologists. Because the conditions that bring them to primary care are often more visible symptom-wise than DR (e.g., diabetes), it may be less likely they would delay these appointments.

Another limitation of our work was the lack of interaction we were allowed with MAs and patients. We relied on the extensive experience of Retinal Care employees, particularly the RCDR coordinator who had been embedded in the clinic, training the first MAs, providing some screenings to patients, observing new MA hires trained by other MAs, and who collected the data on patient eligibility from electronic health records versus the actual screening rate of the clinic. In an ideal study we would have been able to interview MAs/patients and perform contextual inquiry at the clinic.

A last limitation was that we investigated DR screening for a single clinic. From the RCDR coordinator, we learned of idiosyncrasies between clinics that would have affected our analysis and the deployment of some interventions. Though these idiosyncrasies limit the generalizability of our particular interventions, we wish to emphasize that it is the process by which we gained the knowledge to create these interventions that is generalizable: traditional human factors methods paired with guidance from the TPB on how belief and motivation affect behavior.

Conclusion

In closing, human factors psychology is an interdisciplinary field. We draw information, tools, and methods from a wide span of disciplines. In the current project this afforded us the unique position of incorporating theories of behavior with task analysis and usability methods to gain insight into the system of MAs, patients, and the environment of the clinic to reduce the barriers of DR screening while increasing the motivation to be screened.

The Theory of Planned Behavior and the Integrated Behavior Model

The theory of planned behavior (TBP) included the traits, states, and other beliefs that predicted intention, and thus indirectly, behavior. However, the TPB did not address other attributes of a person and their environment or their interaction – it assumes that a behavior is possible, and thus the person’s intent is a good predictor of eventual behavior. An improved and updated model for behavior was the integrated belief model (IBM), which shared the attitude and belief characteristics of the TPB, but added four important influences (Montaño & Kasprzyk, 2015). First, the actual level of behavioral control, now called environmental constraints, was added. Second, the knowledge and skills needed for a behavior needed to be present. Third, habit or automatic behaviors affected behavior. Fourth, the obviousness that a behavior is needed or desired (salience) affected behavior. All of these affected behavior directly, bypassing intention.

Although models such as the TBP and IBM were designed with health behaviors in mind, such as seat belt use, sun protection, and smoking cessation, they can help focus user experience studies. For example, the elicitation questions used for the IBM include “What things make it hard for you to insert behavior?” “If you want to do insert behavior, how certain are you that you can?” As with many tools used in human factors studies, the set of questions help formalize measurement and ensure important information is included. This can be combined with human factors expertise in “environmental constraints,” “habit” (i.e., automaticity), “salience,” and assessment of “knowledge and skill.”

Understanding the intersection of humans, technology, and the environment is a specialty of human factors. Often, human factors studies consider the effects of systems and environment on behavior, such as affordances and perceptual biases, rather than the internal beliefs such as social norms included in the TBP and IBM. Thus, combining knowledge from human factors and theories of behavior creates a stronger model for expected choices and behavior. Combining these considerations in the usability process should result in technology and interfaces that are easy to understand, able to be used, and motivate use.

Supplemental Material

sj-pdf-1-erg-10.1177_10648046211039911 – Supplemental material for A Human Factors Evaluation of a Screening System for Diabetic Retinopathy Centered in Psychological Theory

Supplemental material, sj-pdf-1-erg-10.1177_10648046211039911 for A Human Factors Evaluation of a Screening System for Diabetic Retinopathy Centered in Psychological Theory by Alexander T. Casimir, Adrian M. Pavone, Abby Faircloth, Amina Khan, Ommar Khawaja, Landon Grace, Stephen Fransen and Anne Collins McLaughlin in Ergonomics in Design: The Quarterly of Human Factors Applications

Footnotes

Alexander T. Casimir was a master’s student of physiology at North Carolina State University at the time of this research. He has since been accepted to medical school at the University of North Carolina at Chapel Hill. He plans to apply his behavioral and biological science backgrounds to cultivate a more effective culture of medical practice.

Adrian M. Pavone is a psychology graduate from North Carolina State University in Raleigh. While receiving his bachelor’s degree he studied as a research assistant in the Learning, Aging, and Cognitive Ergonomics Lab. He will be pursuing a PhD in community psychology.

Abby Faircloth was a master’s student at North Carolina State University during the time of this research. She received her master’s of physiology from North Carolina State University in 2020.

Amina Khan was a master’s student in the physiology program at North Carolina State University at the time of this research. She received her BA in exercise and sports science with a focus in athletic training from the University of North Carolina at Chapel Hill in 2018.

Ommar Khawaja received his master of physiology degree from North Carolina State University in 2020. He received his undergraduate degree in exercise and sport science from the University of North Carolina at Chapel Hill in 2019.

Landon Grace is an assistant professor in the mechanical and aerospace engineering department at North Carolina State University. He earned his PhD in mechanical engineering in 2012 from the University of Oklahoma.

Stephen Fransen is an associate professor at the Dean McKgee Eye Institute with a focus on retina and vitreous diseases. He earned his MD from the University of Manitoba Medical School, Winnipeg, Manitoba, Canada.

Anne Collins McLaughlin is a professor of psychology at North Carolina State University in Raleigh, North Carolina. She earned her PhD from the Georgia Institute of Technology in 2007. At North Carolina State University, she directs the Learning, Aging, and Cognitive Ergonomics Lab (www.lacelab.org). ORCID iD: .

References

Ajzen

(1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-T

Apanovitch

A. M.

McCarthy

Salovey

(2003). Using message framing to motivate HIV testing among low-income, ethnic minority women. Health Psychology, 22(1), 60–67. https://doi.org/10.1037/0278-6133.22.1.60

Bischoff

W. E.

Reynolds

T. M.

Sessler

C. N.

Edmond

M. B.

Wenzel

R. P.

(2000). Handwashing compliance by health care workers: The impact of introducing an accessible, alcohol-based hand antiseptic. Archives of Internal Medicine, 160(7), 1017–1021. https://doi.org/10.1001/archinte.160.7.1017

Braverman

(2008). Testimonials versus informational persuasive messages: The moderating effect of delivery mode and personal involvement. Communication Research, 35(5), 666–694. https://doi.org/10.1177/0093650208321785

Call

K. T.

McAlpine

D. D.

Johnson

P. J.

Beebe

T. J.

McRae

J. A.

Song

(2006). Barriers to care among American Indians in public health care programs. Medical Care, 44(6), 595–600. https://doi.org/10.1097/01.mlr.0000215901.37144.94

Carayon

Xie

Kianfar

(2014). Human factors and ergonomics as a patient safety practice. BMJ Quality and Safety, 23(3), 196–205. https://doi.org/10.1136/bmjqs-2013-001812

Centers for Disease Control and Prevention. (2020). Diabetic retinopathy factsheet. https://www.cdc.gov/visionhealth/pdf/factsheet.pdf

Chou

C. F.

Sherrod

C. E.

Zhang

Barker

L. E.

Bullard

K. M.

Crews

J. E.

Saaddine

J. B.

(2014). Barriers to eye care among people aged 40 years and older with diagnosed diabetes, 2006–2010. Diabetes Care, 37(1), 180–188. https://doi.org/10.2337/dc13-1507

Gillan

D. J.

(2012). Five questions concerning task analysis. In Wilson

M. A.

Bennett

W. R.

Gibson

S. G.

Alliger

G. M.

(Eds.), The handbook of work analysis: Methods, systems, applications and science of work measurement in organizations (pp. 201–213). Routledge/Taylor & Francis.

10.

Graham-Rowe

Lorencatto

Lawrenson

J. G.

Burr

J. M.

Grimshaw

J. M.

Ivers

N. M.

Presseau

Vale

Peto

C. B.

Francis

(2018). Barriers to and enablers of diabetic retinopathy screening attendance: A systematic review of published and grey literature. Diabetic Medicine, 35(10), 1308–1319. https://doi.org/10.1111/dme.13686

11.

Hilliard

M. E.

Riekert

K. A.

Ockene

J. K.

Pbert

(Eds.). (2018). The handbook of health behavior change. Springer. https://doi.org/10.1891/9780826180148

12.

Joseph

R. P.

Daniel

C. L.

Thind

Benitez

T. J.

Pekmezi

(2016). Applying psychological theories to promote long-term maintenance of health behaviors. American Journal of Lifestyle Medicine, 10(6), 356–368. https://doi.org/10.1177/1559827614554594

13.

Kaplan

L. M.

McGuckin

(1986). Increasing handwashing compliance with more accessible sinks. Infection Control & Hospital Epidemiology, 7(8), 408–410. https://doi.org/10.1017/S019594170006464X

14.

Kashim

R. M.

Newton

Ojo

(2018). Diabetic retinopathy screening: A systematic review on patients’ non-attendance. International Journal of Environmental Research and Public Health, 15(1), 157–169. https://doi.org/10.3390/ijerph15010157

15.

Kreuter

M. W.

Holmes

Alcaraz

Kalesan

Rath

Richert

McQueen

Caito

Robinson

Clark

E. M.

(2010). Comparing narrative and informational videos to increase mammography in low-income African American women. Patient Education and Counseling, 81(Suppl. 1), 6–14. https://doi.org/10.1016/j.pec.2010.09.008

16.

Montaño

D. E.

Kasprzyk

(2015). Theory of reasoned action, theory of planned behavior, and the integrated behavioral model. In Glanz

Rimer

B. K.

Viswanath

(Eds.), Health behavior: Theory, research and practice, (pp. 95–124). John Wiley.

17.

Nielsen

(1994). 10 Usability heuristics for user interface design. https://www.nngroup.com/articles/ten-usability-heuristics/

18.

Phillips

K. A.

Mayer

M. L.

Aday

L. A.

(2000). Barriers to care among racial/ethnic groups under managed care: Ethnic minorities continue to encounter barriers to care in the current managed care–dominated US health care system. Health Affairs, 19(4), 65–75. https://doi.org/10.1377/hlthaff.19.4.65

19.

Reason

(1990). Human error. Cambridge University Press.

20.

Shepherd

(2003). Hierarchical task analysis. CRC Press/Taylor & Francis Group. https://doi.org/10.1201/9781482289206

21.

Shi

Zhao

Fonseca

Krousel-Wood

Shi

(2014). Racial disparity of eye examinations among the US working-age population with diabetes: 2002–2009. Diabetes Care, 37(5), 1321–1328. https://doi.org/10.2337/dc13-1038

22.

Skaggs

J. B.

Zhang

Olson

D. J.

Garg

Davis

R. M.

(2017). Screening for diabetic retinopathy: Strategies for improving patient follow-up. North Carolina Medical Journal, 78(2), 121–123. https://doi.org/10.18043/ncm.78.2.121

23.

Steinmetz

Knappstein

Ajzen

Schmidt

Kabst

(2016). How effective are behavior change interventions based on the theory of planned behavior?. Zeitschrift für Psychologie, 224(3), 216–233. https://doi.org/10.1027/2151-2604/a000255

24.

Strutton

Du Chemin

Stratton

I. M.

Forster

A. S.

(2016). System-level and patient-level explanations for non-attendance at diabetic retinopathy screening in Sutton and Merton (London, UK): A qualitative analysis of a service evaluation. BMJ Open, 6(5). https://doi.org/10.1136/bmjopen-2015-010952

25.

Van Eijk

K. N. D.

Blom

J. W.

Gussekloo

Polak

B. C. P.

Groeneveld

. (2012). Diabetic retinopathy screening in patients with diabetes mellitus in primary care: Incentives and barriers to screening attendance. Diabetes Research and Clinical Practice, 96(1), 10–16. https://doi.org/10.1016/j.diabres.2011.11.003

26.

Zhang

Saaddine

J. B.

Chou

Cotch

M. F.

Cheng

Y. J.

Geiss

L. S.

Gregg

E. W.

Albright

A. L.

Klein

(2010). Prevalence of diabetic retinopathy in the United States, 2005-2008. JAMA Journal of the American Medical Association, 304(6), 649–656. https://doi.org/10.1001/jama.2010.1111

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.12 MB