Sage Journals: Discover world-class research

Abstract

Background. The Gambler II is a web-based utility assessment tool supporting visual analogue scale (VAS), standard gamble (SG), and time trade-off (TTO) utility assessments. It contains novel features, including an easy to use project development authoring tool and use of multimedia clips for health state descriptions. Objectives. Evaluate the usability and understandability of the patient-facing side of The Gambler. Investigate the feasibility of using The Gambler and evaluate its impact on patient knowledge regarding the relevant health states. Materials and Methods. We used The Gambler to assess utilities on a convenience sample of 55 users for common long-term complications of type 2 diabetes mellitus: diabetic neuropathy, diabetic retinopathy, and diabetic foot infection requiring transmetatarsal amputation. Using VAS, SG, and TTO, we collected metadata, such as time spent on each assessment and the entire assessment process. We evaluated usability with an adaptation of the System Usability Scale survey and understandability. We evaluated impact on knowledge gained through knowledge assessments about these complications before and after use of The Gambler. Results. Overall satisfaction with The Gambler was high, 4.02 on a 5-point scale. Usability rated highly at 84.93 on a normalized scale between 0 and 100. Knowledge scores increased significantly following use of The Gambler from pretest mean of 68% to posttest mean of 76% (P < 0.01). Average time using the software: ∼7½ minutes. Conclusions. The Gambler is an easy to use and understand computer-based tool for utility assessment. It is feasible to use within clinical encounters to support shared decision making, and it has unique features that make it a powerful tool for investigators interested in research on health utilities.

Keywords

decision making decision theory user interface health utilities quality of life

Background and Significance

A health utility is a quality of life weighting factor, measuring patients’ values and preferences for health states.^1,2 Utilities in aggregate are critical components of health economic evaluations, such as cost-effectiveness analyses, through metrics such as the quality-adjusted life year. They can also support decision making at the individual patient level.^3,4 The theoretical underpinnings of utility assessment derive from von Neumann-Morgenstern utility theory.² The resulting health utility weights range from zero to one: one representing full health and zero representing the worst outcome, generally death.

Multiple methods and approaches exist for assessing patients’ utilities including the visual analogue scale (VAS), standard reference gamble (SG), and time tradeoff (TTO).^5–8 There have been many attempts to standardize the approach to the assessment process to avoid operator bias and variability, and to simplify the process to avoid repetitious and laborious work on the part of those assessing utilities. Earlier attempts used a variety of paper-and-pencil tools along with adjustable pie charts to represent gambles and TTOs.⁹ More recently, computerized tools have been developed.¹⁰ The Interactive Multimedia Preference Assessment Construction Tool is one of the earlier tools that has gone through several iterations and revisions with the most recent version using Adobe Flash and XML to build health utility assessment projects.¹¹ The Program to Survey Preferences by Evaluating Quality of Life Tradeoffs tool uses standard web platforms (XML, HTML) and contains features including demographic-adjusted information and three utility assessment methods, VAS, SG, and TTO.¹² Other tools are disease specific, or may only utilize a single assessment method.¹³ One such example is a Microsoft PowerPoint-based standard gamble tool developed to assess asthma-specific health states in Malaysia.¹⁴ Still some utility assessment methods are platform specific with the aforementioned PowerPoint SG tool or a TTO tool dependent on RedCap.¹⁵

The Gambler II springs from an earlier version developed in 1992.¹⁶ The Gambler was a generalized utility assessment tool that allowed authors to construct projects to assess health states for any disease process, utilizing any or all of a variety of assessment methods including the VAS, SG, and TTO. Life expectancy for the TTO was based on patient age, drawn from life tables published by the Centers for Disease Control and Prevention. The Gambler was coded in Visual Basic and does not run on newer personal computer platforms.

Since the development of The Gambler, many advancements and innovations in computer and software technologies have occurred. HTML and JavaScript matured enough to allow the development of sophisticated web applications.^17,18 Increased internet bandwidth along with features such as multimedia clip playback, dynamic content generation, and improved server and graphics technologies supported the development of desktop-grade applications on the web. Database technologies now support storage and retrieval of information with protection on secure servers. These advances coincide with the movement within the clinical informatics world to develop tools that are easier to use and geared toward patients.^19–21 The Gambler II is a merited response to this goal.

Understanding the limitations of prior computerized health utility assessment tools and taking advantage of contemporary web-based client-server technologies, our goal was to develop The Gambler II (hereafter referred to as The Gambler) and evaluate its usability, ease of understanding, and feasibility of use in time-sensitive environments. The Gambler is composed of multiple components including the utility assessment authoring tool and the patient-user interface. This article describes the development of the patient-user interface and early feasibility and usability testing of said aspect.

Methods

Gambler: Technical Architecture

The Gambler uses multiple technologies to create a fully functional health utility assessment tool. The web application uses a common architecture known as a LAMP (Linux, Apache, MySQL, and Python) stack to implement web services.²² Python provides the application backbone for The Gambler, performing the numerical calculations needed to convert SG and TTO results into utility weights.²² It also serves as the interface between multiple technologies used to store and retrieve information from patients or study subjects. The Gambler uses MySQL as the database repository supporting simple query exchanges from the database to the application with aid from Python. Bootstrap, JavaScript, and Hypertext Markup Language 5 (HTML 5) support the graphic user interface (GUI), allowing web page development that supports the use of multimedia clip, popups, buttons, sliders, and drag and drop features.^17,23

Health Utility Assessment Methods

The Gambler supports the use of the following utility assessment methods: VAS, SG, and TTO. The VAS is the simplest and fastest assessment method, but some argue it is not a true utility measure. VAS cannot explicitly distinguish a health state rated at 80 (on a 0 to 100 scale) as twice as good as one rated at 40. SG may be most appropriate for decisions under uncertainty that entail risk, such as the choice between medical therapy and a surgical procedure with a significant risk of death. SG captures risk attitude holistically in its assessment. On the other hand, for decisions that do not entail significant risk, the TTO may be best. All of these methods can be used to assess utilities for either current health states or hypothetical, yet unexperienced health states.

The VAS asks the question, “On a scale where 0 represents death and 100 represents perfect health, what number best represents the health state being assessed?” The Gambler implements the VAS by having the patient move a slider to the point on the 0 to 100 scale to indicate the quality of life for the health state being assessed (see Figure 1).

Figure 1

Visual analogue scale. Patients performing the visual analogue scale assessment click on health state icons and drag them to an appropriate value. The numerical value displayed next to the icons is the utility on a 0 to 100 scale. Hovering over an icon will display a text description of the health state.

Of the three assessment methods, the SG is the most firmly grounded in expected utility theory. The SG assesses a health state by determining the risk of death one would accept to improve quality of life over the health state being assessed.²⁴ The most common form of the SG asks the respondent to make a choice between a health state with less than perfect quality of life (the health state for which the utility is being assessed), and a gamble with outcomes of perfect health or death. The utility of this intermediate health state is determined by assessing the probability of death at which the respondent is indifferent to the gamble and the intermediate health state. If, for example, the respondent is willing to accept a 15% chance of dying to be in perfect health, the expected value of the gamble would be 0.85.^5,24

The Gambler uses the analogy of a pill bottle to represent the gamble (see Figure 2). The bottle contains a certain number of pills that will cure or completely alleviate symptoms of the intermediate health state. However, the bottle also contains some number of “poison” pills that will lead to death. The patient need only take a single pill to be cured. The Gambler ping-pongs back and forth between gambles with higher or lower risk of death, depending on the user’s responses. The goal is to find the point at which the user is indifferent between the health state being assessed and the gamble. At this point, the health state utility can be calculated as follows: Utility = 1 − (Number of poison pills/Total number of pills in the bottle).

Figure 2

Standard gamble. The standard gamble presents users with a visualization of a choice between an intermediate health state, in this case diabetic retinopathy (left side of figure), or a gamble (right side of figure), represented by a pill bottle with varying numbers of cure and “poison” pills. The user selects alternative A (the intermediate health state) or alternative B (the gamble) until they are ambivalent between the two choices, at which point they click on “Equal.”

The TTO assesses utility by asking how much time a patient would be willing to give up to be in a better state of health?^24,25 This is accomplished by asking the patient to choose between a set length of life (The Gambler uses the patient’s calculated life expectancy) in a less than perfect state of health and a shorter length of life in a perfect state of health (see Figure 3). The Gambler presents the patient with a series of choices, varying the length of life in perfect health until the patient is ambivalent about the choice. As with the SG above, The Gambler ping-pongs between longer or shorter periods of time in perfect health, depending on responses from the patient. For example, if a patient is willing to accept 10 years of life in perfect health rather than 15 years in an intermediate health state, their utility for the intermediate health state would be 0.67 (10/15). The TTO utility is calculated as follows: Utility = (Time in best health state)/(Total life expectancy).

Figure 3

Time tradeoff. The user is asked to make a choice between a fixed length of life (based on the user’s estimated life expectancy) in an intermediate state of health, in this case, diabetic neuropathy, shown on the left side of the figure, versus a shorter length of time in perfect health, shown on the right side of the figure. The user selects between alternative A and alternative B until they are ambivalent regarding the time tradeoff, at which point they click on “Equal.”

Testing: Usability and Understandability

In order to evaluate The Gambler we developed a full health utility assessment project. We selected type 2 diabetes mellitus (DM), a common health condition that affects over 9% of the US population, so most users would be familiar with the condition and its complications.²⁶ We developed scenario descriptions for three intermediate health states representing complications of DM, in addition to anchor states of well and dead. These included the following: well with diabetes, diabetic neuropathy, diabetic retinopathy, diabetic foot infection requiring transmetatarsal amputation, and death. We represent each health state with an icon and an accompanying brief text description. In order to make full use of Gambler’s features, we also created video clips using patient actors to describe each health state and the impact it has on their function and mood. We developed four sets of multimedia clips for each health state, one for each of the following demographic groups: White men, White women, Black men, and Black women. Thus, for example, a Black female using The Gambler would view multimedia clips of Black women describing each of the health states representing complications of DM. We asked users to assume the role of a patient with DM.

Our evaluation goal was to enroll a minimum convenience sample of 50 subjects, with varying demographic characteristics as previous efforts have demonstrated acceptable usability sample sizes of 10 or fewer.^15,27 Participants came from different parts of the United States, including Ohio, Washington DC, and New Jersey.

We collected a minimal amount of demographic information (age, gender, race), along with highest educational level attained. We also administered a short knowledge survey developed by one of the researchers (MHE), consisting of 10 multiple choice questions (Appendix B) before and after using The Gambler to see whether the process of utility assessment and presentation of information about diabetic complications improved users’ knowledge about these complications. We also asked users to fill out a brief feedback survey at the end of their visit and had an investigator (AAA) conduct a semistructured interview about their experience using The Gambler—problems, what they liked, and suggestions for improving the tool (see Appendix B for questions).

We used an adapted version of the System Usability Scale survey, which uses a 5-point Likert-type scale to evaluate the usability and feasibility of The Gambler (see Appendix B).²⁸ If responses were missing for any data elements, we simply noted this and factored it in our statistical evaluations.

Tools

We used Apple QuickTime X 10.4-10.5 on an Apple MacBook Pro to record video clips for multiple demographic categories. We evaluated the software using multiple physical and software configurations, including different internet browsers (Apple’s Safari 12.0.0–12.0.2 and Google Chrome OS 68.0.3440–71.0.3578). Many participants used their own computers with a variety of operating systems including Microsoft Windows 10 OS, Google Chrome OS, and Apple Mac OS. A significant number (>25) of users used an Apple MacBook Pro with Mac OS 10.13.6–10.14.2 and a Hewett-Packard Mouse.

Institutional Review Board

The research was submitted to the University of Cincinnati’s Institutional Review Board (IRB) for project approval. The IRB approved the project as nonhuman subjects.

Results

User Characteristics

We enrolled 55 users for this study. Ages ranged between 19 and 71 years, with mean age of 40 years. Locations for testing included Ohio, New Jersey, and Washington DC. Highest educational level attained ranged from high school to doctoral level, with master’s-level education having a plurality. We had a diverse representation of races as shown in Table 1.

Table 1

User Characteristics

Age (years), mean (range)	40 (19–71)
Male (%)	58.5
Education, n
High school/GED	9
Some college	8
Associate’s	2
Bachelor’s	11
Master’s	20
Doctoral	2
Professional	3
Race, n
African American	16
Asian	14
Middle Eastern/North American	1
White	24
Total	55

Usability and Understandability

We used nonparametric tests of significance (Mann-Whitney) in our analyses. As shown in Table 2, 79% of the 55 users were satisfied with The Gambler, with scores of 4 or higher. Users understood the utility assessment process. Health state descriptions were clear and understandable for most users. Eighty-three percent and 81% of users, respectively, gave a rating of 4.0 or higher for the SG and the TTO. The majority of users found the pill bottle and the “time bar” useful as representations for the SG and TTO, with 89% and 86% of them affirming that the visuals were easy to understand. Cronbach’s alpha for the usability and understandability questionnaire for all questions excluding the Health State Video and Video instructions for completeness was 0.93.

Table 2

Understandability and Usability

	Mean (SD)	Median (IQR)	n
Overall satisfaction with The Gambler^a	4.02 (0.71)	4.00 (4.0–5.0)	55
How well did you understand and were able to complete the . . .
Demographics page	4.38 (1.07)	4.0 (4.0–5.0)	55
Ordinal ranking assessment	4.23 (0.94)	4.0 (4.0–5.0)	55
Visual analogue scale	4.09 (0.88)	4.0 (4.0–5.0)	53
Standard gamble	4.15 (0.87)	4.0 (4.0–5.0)	55
Time tradeoff	4.09 (1.06)	4.0 (3.0–5.0)	55
How clear and understandable were the . . .
Health state information	4.40 (0.99)	5.0 (4.0–5.0)	55
Icons	4.45 (0.99)	5.0 (4.0–5.0)	55
Text description of health states	4.42 (0.83)	5.0 (4.0–5.0)	54
Health state videos	4.09 (1.06)	4.5 (3.75–5.0)	44
Video instructions	4.11 (1.08)	4.50 (3.0–5.0)	44
Total usability score^b	84.93 (14.92)	88.00 (78.00–96.00)	55
Total utility assessment time (minutes)	7.14 (4.23)	6.25 (3.85–9.29)	55
Cronbach’s alpha	0.93		55
Was the pill bottle representation for the standard gamble understandable (% yes)	89		55
Were the graphics for the time tradeoff understandable (% yes)	86		55

IQR, interquartile range.

Likert-type scale range of 1 to 5.

Normalized to a range of 0 to 100.

A supermajority of users found health state information easy to understand, with 63% giving the health state information a score of 5. Fifty-seven percent of those who read the text descriptions of health states gave a score of 5. Of those who viewed the multimedia clips describing health states, or the instructional videos, more than half (52%) gave a rating of 5. The total usability score, normalized to a 0 to 100 scale, was 84.9 for total population.

Knowledge Assessment

Fifty-five users completed pre- and postknowledge assessments about the three diabetic complications described in this Gambler project. As shown in Table 3, knowledge scores increased significantly following use of The Gambler. The mean pretest score for the full cohort was 68%, while the posttest mean was 76% (P = 0.005), resulting in an increase of almost 8%. Most subgroups demonstrated statistically significant gains in knowledge following use of The Gambler; knowledge gain between pre- and postassessments were not statistically significant between subgroups. A trend exists suggesting a greater impact on knowledge gain among respondents with less than a bachelor’s degree compared with more highly educated respondents, 10.5 versus 6.1, respectively.

Table 3

Diabetic Complications Knowledge Assessment

	Pretest Mean, % (SD)	Pretest Median, % (IQR)	Posttest Mean, % (SD)	Posttest Median, % (IQR)	Mean Change Score, % (post v. pre) (SD)	n	P Value
Full cohort	68.36 (16.19)	70.0 (60.00–80.0)	76 (14.35)	80.0 (70.00–90.0)	7.64 (16.33)	55	0.005
Education^a
Bachelor’s degree or higher	73.06 (14.7)	70.0 (60.0–82.5)	79.17 (11.8)	80.0 (70.0–90.0)	6.11 (14.20)	36	0.036
Less than bachelor’s degree	59.47 (15.45)	60.0 (50.0–70.0)	70.00 (17.0)	70 (60.0–80.0)	10.53 (19.85)	19	0.032
Gender^b
Female	68.64 (16.99)	70.0 (60.0–80.0)	75.91 (13.33)	75.0 (70.0–90.0)	7.27 (14.86)	22	0.076
Male	68.18 (15.90)	70.0 (60.0–80.0)	76.06 (15.19)	80.0 (70.0–90.0)	8.39 (17.91)	31	0.010
Age^c
<40 years of age	66.00 (16.53)	70.0 (60.0–77.5)	74.67 (12.79)	70.0 (70–87.5)	8.67 (8.66)	30	0.014
≥40 years of age	71.20 (15.63)	70.0 (60.0–80.0)	77.6 (16.15)	80.0 (70.0–90.0)	6.40 (18.00)	25	0.064

IQR, interquartile range.

P value for change score bachelor’s degree or higher versus less than bachelor’s degree—0.11.

P value for change score female versus male—0.24.

P value for change score age <40 years versus age ≥40 years—0.25.

Test Study for Diabetic Health State Utilities

We assessed utilities for three health states representing complications of DM. As shown in Figure 4, utilities for each health state varied dramatically across users. Utility scores for each respective health state were lowest using the VAS (see Table 4). Utilities using SG assessments were generally higher than either VAS or TTO, with the exception of diabetic retinopathy where the highest mean ratings were from the TTO. As expected, standard deviations around the mean utility scores for each health state are large, likely reflecting large patient-to-patient variation.

Figure 4

Results of health state utility assessments. Radar plots of health utilities for diabetic neuropathy (blue), diabetic retinopathy (orange), and diabetic foot infection requiring transmetatarsal amputation (gray). Each panel depicts results using a different assessment method. Panel A—Visual analogue scale. Panel B—Standard gamble. Panel C—Time tradeoff.

Table 4

Full Cohort Average Utility for Utility Assessments

	Neuropathy		Retinopathy		Foot Infection		n
	Mean (SD)	Median (IQR)	Mean (SD)	Median (IQR)	Mean (SD)	Median (IQR)	n
Visual analogue scale	60.33 (25.75)	65.0 (42.0–80.0)	46.65 (26.87)	50.0 (25.0–59.0)	37.75 (32.83)	25.0 (15.0–55.0)	55
Standard gamble	75.27 (24.31)	85.0 (52.5–96.0)	71.64 (21.52)	75.0 (50.0–93.0)	71.24 (25.86)	75.0 (50.0–96.0)	55
Time tradeoff	72.33 (24.39)	82.0 (50.0–90.5)	72.07 (24.27)	81.0 (50.0–92.0)	70.56 (23.77)	72.0 (50.0–93.50)	55

IQR, interquartile range.

We examined differences in utilities across user subgroups. Patients 40 years of age and older had higher utilities exclusively with SG than their younger counterparts (P = 0.048; see Appendix A). We noted a similar pattern among respondents with less than a bachelor’s degree compared with those holding a bachelor’s degree or higher, with more highly educated individuals demonstrating a nonsignificant trend toward higher SG ratings for all health states. We found no significant trend in utilities between Black and non-Black users in our analysis. Finally, women demonstrated a nonsignificant trend toward higher health states ratings than men in SG assessments, but there was no consistent pattern in the VAS and TTO assessments. To investigate the possibility of confounders, we performed linear regression analyses on multiple characteristics including age, race, gender, and educational status. We found no statistical significance in usability for users with various differences in characteristics.

Discussion

We developed a usable and easy to understand computer-based utility assessment tool. In our evaluation, for each health utility assessment task, users scored the ease of completing the task at 4 (out of 5). Building on prior efforts, it takes advantage of the latest web technologies to support both the development of health utility assessment projects and the utility assessment process itself. Key features include an interactive user interface that provides graphic visualizations for utility assessments, along with multimedia clips that support both user instructions and health state descriptions using media clips that are demographically matched with users.

A strength of this article, improving the generalizability of our results, is the diversity of participants from different parts of the United States including Ohio, Washington DC, and New Jersey, different age ranges, genders, race, and educational attainments.^29,30 Most users regardless of educational status, race, age, or gender found the software easy to use and understandable.

In our review of the literature, we found a number of Gambler’s features to be unique. While other utility assessment tools have used videos as means of communicating information, to our knowledge, none implemented multimedia instructions for patients or used demographically matched video clips to personalize information about health states.^11,31 This Gambler feature is based on exemplar theory.³² In studies examining the influence of narrative health communication on behavior change, exemplar theory proposes that the use of exemplars that more closely match individual patient characteristics engage patients more fully and have a greater impact on behavior change.³³ In unstructured comments, many users stated the availability of multimedia clips was one of the best parts of The Gambler, allowing them to understand health states and utility assessment instructions better (Appendix D).

A key feasibility factor affecting the use of these tools in practice is the time it takes for patients to complete the assessment process.^34–36 In particular, if utility assessment tools are to be part of a shared decision-making experience, they must be quick and easy to use. In comments and interviews, users felt the assessment process took a reasonable amount of time to complete. We used metadata to determine the amount of time users spent in total and for each activity. Users spent an average of 2.5 minutes completing the SG, 1.3 minutes completing the TTO, and 1.4 minutes completing the VAS (Appendix Table 2). The total time taken to complete all assessments was roughly 7 minutes (Appendix Table 2). The median time patients spend with their physicians in general practice settings is between 15 and 18 minutes.^37,38 Seven minutes added to the beginning of a visit to perform utility assessments is not unreasonable, and in fact appreciated if incorporated into a shared decision-making visit.

As a beneficial side effect of describing health states that patients might face as a consequence of their underlying illness or chronic disease, we hypothesized that use of The Gambler would improve patients’ understanding and knowledge regarding these health states. Indeed, we found that patients’ interactions with The Gambler improved knowledge regarding complications of DM. Median scores on a 10-item knowledge assessment improved from 68.4% to 76.0% (P = 0.005) following interaction with The Gambler. In subgroup analyses all groups except for females and users 40 years of age or older had a statistically significant gain in knowledge after use of The Gambler (Table 4). Interesting trends in change scores existed between subgroups. For example, knowledge assessment scores for users with less than a bachelor’s degree actually improved more (10.5%) than scores for those with a bachelor’s degree or higher (6.1%), suggesting that the educational benefit of The Gambler might be more pronounced for subjects with a lower level of educational experience.

Our review of the literature did not find other studies examining the impact of utility assessment tools on patient education (Appendix B). Thus, in addition to eliciting utilities, The Gambler may serve as an educational adjunct to further support health communication as part of the shared decision-making process. Little work has been done to explore the use of exemplars in the health utility assessment process.³⁹ This is particularly important as we try to confront racial disparities in health care and health education.^40,41 While efforts to combat racial disparities through technology are ongoing, there has been little research in this area in the clinical informatics literature.^31,42,43

We used complications of a common health condition, DM, as our test case for The Gambler evaluation. In general, utilities we found for these health states were similar to those described in other studies.^44,45 In addition, we found systematic differences in utilities assessed with different methods. In particular, we found that utilities for health states were generally higher with the SG than with the VAS or TTO.^3,5,24,46,47 The holistic incorporation of risk attitude in SG assessments influences the utility assessment process. Thus, as most people are risk averse, SG assessments are higher because people are less willing to accept a risk of dying in the SG to avoid the intermediate health state.^48–50 We also examined whether there were differences in median utilities for each health state across subgroups as has been described in other studies.^51,52 Most dramatic differences were for SG assessments comparing users <40 years of age with users ≥40 years of age (see Appendix Table 1). Older respondents consistently rated all health states higher than younger respondents. This is consistent with other studies showing that older individuals are more risk averse.⁵³

We also found that users with lower educational attainment had lower SG utilities for all health states than their more highly educated peers, suggesting increased risk aversion compared with their more highly educated peers. This was statistically significant for the health states of diabetic neuropathy and retinopathy, but not for diabetic foot infection leading to need for transmetatarsal amputation. This is also consistent with other studies suggesting that more highly educated patients are generally more willing to accept risks to improve health.⁵⁴ While women consistently had higher SG utilities than men, suggesting a higher degree of risk aversion in women, this was only statistically significant for diabetic neuropathy. Similar trends have been noted in other studies.^46,54–56

What are the implications of such differences in utilities and risk attitude across sociodemographic groups? Does risk attitude contribute to practice variation and the underutilization of appropriate health care services we sometimes see among women or underrepresented minorities? Indeed, other investigators have found that when faced with a risky medical procedure, such as carotid endarterectomy, Blacks would accept higher risks of stroke to avoid the risky and invasive procedure.⁵⁷ Such disparities may be compounded by potential biases triggered by inadvertent mismatching of demographic factors such as race, between patient actors seen in video clips describing health states and patients.³⁹

In our assessment of utilities for complications of DM, we assumed anchor states of well and dead to be the best and worst outcomes, and then assessed the utility of the diabetes complications, assuming they all had intermediate values falling between the absolute utilities (0 and 1) of these anchor states. Some patients may find certain health states to be worse than death. The Gambler is capable of using health states other than Dead as the anchor state for the worst outcome. However, if the investigator or clinician expected this might be the case for some of the health states being evaluated, they would need to develop a second version of the utility assessment project that used such a state as the anchor for the worst outcome.

Individual patient utilities can be used to inform shared decision making through the use of personalized decision analyses for a variety of clinical disorders.^3,4,58–60 Most recently, we examined the feasibility of performing real-time utility assessments and personalized decision analyses to facilitate patient visits with their cardiologists to discuss anticoagulation options to prevent atrial fibrillation-related stroke.⁵⁹ One could envision a future in which many preference sensitive decisions are facilitated by such an approach.

The Gambler has limitations. While there is full support for direct utility assessment techniques, the software does not support use of indirect utility assessment methods such as European Quality of Life 5D (EuroQol) or the SF-6.^61–64 These indirect methods map multidimensional measures of health status onto utilities, most often obtained through TTO assessments. When evaluating a tool such as The Gambler, it is difficult to separate usability and understandability of the software tool itself from understandability of the health state descriptions and the complexity of the clinical disorder being evaluated. In order to truly demonstrate that usability and understandability are generalizable across a wide range of clinical disorders and their relevant health states, a much more comprehensive evaluation would need to be performed. We chose a common disorder, DM, so users would not be confused or distracted by the complexity of the health states. We believe this provided results that were most representative of the software tool itself.

Conclusions

We demonstrated the feasibility of using The Gambler as an efficient means of collecting utilities. Users satisfied with their interactions found the tool easy to use and easy to understand. We found educational benefits as a positive side effect of the utility assessment process with The Gambler. The Gambler adds a powerful research tool to the armamentarium of health services and outcomes researchers. The Gambler can be a useful adjunct supporting real-time personalized decision analyses as part of shared decision-making approach to patient care.

Supplemental Material

GamblerHUAT_02262020_Appendix.rjf_online_supp – Supplemental material for Automated Tool for Health Utility Assessments: The Gambler II

Supplemental material, GamblerHUAT_02262020_Appendix.rjf_online_supp for Automated Tool for Health Utility Assessments: The Gambler II by Adeboye A. Adejare and Mark H. Eckman in MDM Policy & Practice

Footnotes

Acknowledgements

We would like to acknowledge Robert Ireton for technical support for The Gambler. We would also like to acknowledge the University of Cincinnati Department of Biomedical Informatics and Cincinnati Children’s Hospital Medical Center for technical resources.

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Adeboye A. Adejare

Mark H. Eckman

Supplemental Material

Supplementary material for this article is available on the Medical Decision Making Policy & Practice website at .

References

Szende

Schaefer

A Taxonomy of health utility assessment methods and the role for uncertainty analysis. Eur J Health Econ. 2006;7(2):147–51.

Morgenstern

Von Neumann

Theory of Games and Economic Behavior. Princeton: Princeton University Press; 1953.

Eckman

Wise

Naylor

, et al. Developing an Atrial Fibrillation Guideline Support Tool (AFGuST) for shared decision making. Curr Med Res Opin. 2015;31(4):603–14.

Eckman

Kopras

Montag-Leifling

, et al. Shared decision-making tool for self-management of home therapies for patients with cystic fibrosis. MDM Policy Pract. 2017;2(1):238146831771562.

Gafni

The standard gamble method: what is being measured and how it is interpreted. Health Serv Res. 1994;29(2):207–24.

Martin

Glasziou

Simes

Lumley

A comparison of standard gamble, time trade-off, and adjusted time trade-off scores. Int J Technol Assess Health Care. 2000;16(1):137–47.

Torrance

GW.

Measurement of health state utilities for economic appraisal: a review. J Health Econ. 1986;5(1):1–30. doi:10.1016/0167-6296(86)90020-2

Dolan

Gudex

Kind

Williams

Valuing health states: a comparison of methods. J Health Econ. 1996;15(2):209–31.

Littenberg

Partilo

Licata

Kattan

MW.

Paper standard gamble: the reliability of a paper questionnaire to assess utility. Med Decis Making. 2003;23(6):480–8.

10.

Ruland

CM.

Improving patient safety through informatics tools for shared decision making and risk communication. Int J Med Inform. 2004;73(7–8):551–7.

11.

Bansod

Skoczen

Lenert

LA.

iMPACT4: a framework for rapid, modular construction of web-based patient decision support systems and preference measurement tools. AMIA Annu Symp Proc. 2003;2003:782.

12.

Bayoumi

Dale

ProSPEQT: a new program for computer-assisted utility elicitations. Poster presented at: The 26th Annual Meeting of the Society for Medical Decision Making; October 17–20, 2004; Atlanta.

13.

Zeballos-Palacios

Hargraves

Noseworthy

, et al. Developing a conversation aid to support shared decision making: reflections on designing anticoagulation choice. Mayo Clin Proc. 2019;94(4):686–96.

14.

Yong

Shafie

AA.

Development, feasibility, and validity of a computer-based utility assessment tool for measuring asthma-specific health utilities in Malaysia using the standard gamble method. J Asthma. 2016;53(8):835–42.

15.

Oremus

Sharafoddini

Morgano

Jin

Xie

A computer-assisted personal interview app in Research Electronic Data Capture for administering time trade-off surveys (REDCap): development and pretest. JMIR Form Res. 2018;2:e3. doi:10.2196/formative.8202

16.

Gonzalez

Eckman

Pauker

SG.

Gambler: a computer workstation for patient utility assessment. Med Decis Making. 1992;12:350.

17.

Reimers

Stewart

Presentation and response timing accuracy in Adobe Flash and HTML5/JavaScript web experiments. Behav Res Methods. 2015;47(2):309–27.

18.

Chen

Tan

Kou

, et al. Enrichr: interactive and collaborative HTML5 gene list enrichment analysis tool. BMC Bioinformatics. 2013;14:128.

19.

Tang

Lorenzi

Harle

Zhou

Chen

Interactive systems for patient-centered care to enhance patient engagement. J Am Med Inform Assoc. 2016;23(1):2–4.

20.

Collins

Rozenblum

Leung

, et al. Acute care patient portals: a qualitative study of stakeholder perspectives on current practices. J Am Med Inform Assoc. 2016;24(e1):e9–e17.

21.

Woollen

Prey

Wilcox

, et al. Patient experiences using an inpatient personal health record. Appl Clin Inform. 2016;7(2):446–60.

22.

Lawton

LAMP lights enterprise development efforts. Computer. 2005;38(9):18–20. doi:10.1109/MC.2005.304

23.

Thornton

Otto

Bootstrap [cited April 17, 2018]. Available from: https://getbootstrap.com/

24.

Bleichrodt

A new explanation for the difference between time trade-off utilities and standard gamble utilities. Health Econ. 2002;11(5):447–56.

25.

Attema

Edelaar-Peeters

Versteegh

Stolk

EA.

Time trade-off: one methodology, different methods. Eur J Health Econ. 2013;14(suppl 1):S53–S64.

26.

Centers for Disease Control and Prevention. National diabetes statistics report 2020: estimates of diabetes and its burden in the United States [cited March 7, 2020]. Available from: https://www.cdc.gov/diabetes/pdfs/data/statistics/national-diabetes-statistics-report.pdf

27.

Faulkner

Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput. 2003;35(3):379–83.

28.

Brooke

SUS—a quick and dirty usability scale. Usability Eval Ind. 1996;189:4–7. Available from: https://hell.meiert.org/core/pdf/sus.pdf

29.

Czaja

Zarcadoolas

Vaughon

Lee

Rockoff

Levy

The usability of electronic personal health record systems for an underserved adult population. Hum Factors. 2015;57(3):491–506.

30.

González-Palau

Franco

Toribio

, et al. Designing a computer-based rehabilitation solution for older adults: the importance of testing usability. PsychNology J. 2013;11(2):119–36.

31.

Dehlendorf

Ruskin

Grumbach

, et al. Recommendations for intrauterine contraception: a randomized trial of the effects of patients’ race/ethnicity and socioeconomic status. Am J Obstet Gynecol. 2010;203(4):319.e1–e8.

32.

Hinyard

Kreuter

MW.

Using narrative communication as a tool for health behavior change: a conceptual, theoretical, and empirical overview. Health Educ Behav. 2007;34(5):777–92.

33.

Kim

Bigman

Leader

Lerman

Cappella

JN.

Narrative health communication and behavior change: the influence of exemplars in the news on intention to quit smoking. J Commun. 2012;62(3):473–92.

34.

Hawley

ST.

Challenges to measuring and achieving shared decision-making in practice. Health Expect. 2018;21(1):1–2.

35.

Légaré

Thompson-Leduc

Twelve myths about shared decision making. Patient Educ Couns. 2014;96(3):281–6.

36.

Coulter

Shared decision making: everyone wants it, so why isn’t it happening?

World Psychiatry. 2017;16(2):117–8.

37.

Young

Burge

Kumar

Wilson

Ortiz

DF.

A time-motion study of primary care physicians’ work in the electronic health record era. Fam Med. 2018;50(2):91–9.

38.

Tai-Seale

McGuire

Zhang

Time allocation in primary care office visits. Health Serv Res. 2007;42(5):1871–94.

39.

Lenert

Ziegler

Lee

Unfred

Mahmoud

The risks of multimedia methods: effects of actor’s race and gender on preferences for health states. J Am Med Inform Assoc. 2000;7(2):177–85.

40.

Schlesinger

O’Hara

Taylor

AS.

Let’s talk about race: identity, chatbots, and AI. In: CHI ’18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems; April 2018; Paper No. 315. https://doi.org/10.1145/3173574.3173889

41.

Hankerson

Marshall

Booker

, et al. Does technology have race? In: CHI EA ’16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems; May 2016. https://doi.org/10.1145/2851581.2892578

42.

Buller

Bettinghaus

Fluharty

, et al. Improving health communication with photographic images that increase identification in three minority populations. Health Educ Res. 2019;34(2):145–58.

43.

Freimuth

Quinn

SC.

The contributions of health communication to eliminating health disparities. Am J Public Health. 2004;94(12):2053–5.

44.

Brown

Sharma

Shah

Utility values and diabetic retinopathy. Am J Ophthalmol. 1999;128(3):324–30.

45.

Beaudet

Clegg

Thuresson

Lloyd

McEwan

Review of utility values for economic modeling in type 2 diabetes. Value Health. 2014;17(4):462–70.

46.

van Osch

SMC

Stiggelbout

AM.

The construction of standard gamble utilities. Health Econ. 2008;17(1):31–40.

47.

Torrance

Boyle

Horwood

SP.

Application of multi-attribute utility theory to measure social preferences for health states. Oper Res. 1982;30(6):1043–69.

48.

Finnell

SME

Carroll

Downs

. The utility assessment method order influences measurement of parents’ risk attitude. Value Health. 2012;15(6):926–32.

49.

Ference

Stubbs

Lidder

, et al. Measurement and comparison of health utility assessments in chronic rhinosinusitis. Int Forum Allergy Rhinol. 2015;5(10):929–36.

50.

Wang

Hsieh

Zhu

LJ.

Comparative assessment of different health utility measures in systemic lupus erythematosus [cited January 12, 2019]. Available from: http://www.nature.com/articles/srep13297

51.

Goodacre

Wilson

Bradburn

Santarelli

Nicholl

JP.

Health utility after emergency medical admission: a cross-sectional survey. Health Qual Life Outcomes. 2012;10:20.

52.

Khanna

Jariwala

Bentley

JP.

Health utility assessment using EQ-5D among caregivers of children with autism. Value Health. 2013;16(5):778–88.

53.

Maart-Noelck

Musshoff

Measuring the risk attitude of decision-makers: are there differences between groups of methods and persons?

Aust J Agric Resour Econ. 2014;58(3):336–52. doi:10.1111/j.1467-8489.2012.00620.x

54.

Rosen

Tsai

Downs

SM.

Variations in risk attitude across race, gender, and education. Med Decis Making. 2003;23(6):511–7.

55.

Obaidi

Mahlich

A potential gender bias in assessing quality of life—a standard gamble experiment among university students. Clincoecon Outcomes Res. 2015;7:227–33.

56.

Jariwala

Toh

Shum

, et al. The association between asthma-related emergency department visits and pollen and mold spore concentrations in the Bronx, 2001–2008. J Asthma. 2014;51(1):79–83.

57.

Oddone

Horner

Diers

, et al. Understanding racial variation in the use of carotid endarterectomy: the role of aversion to surgery. J Natl Med Assoc. 1998;90(1):25–33.

58.

Eckman

Alonso-Coello

Guyatt

, et al. Women’s values and preferences for thromboprophylaxis during pregnancy: a comparison of direct-choice and decision analysis using patient specific utilities. Thromb Res. 2015;136(2):341–7.

59.

Eckman

Costea

Attari

, et al. Shared decision-making tool for thromboprophylaxis in atrial fibrillation—a feasibility study. Am Heart J. 2018;199:13–21.

60.

Wess

Saleem

Tsevat

, et al. Usability of an atrial fibrillation anticoagulation decision-support tool. J Prim Care Community Health. 2011;2(2):100–6.

61.

Arnold

Girling

Stevens

Lilford

Comparison of direct and indirect methods of estimating health state utilities for resource allocation: review and empirical analysis. BMJ. 2009;339:b2688.

62.

Brazier

Usherwood

Harper

Thomas

Deriving a preference-based single index from the UK SF-36 Health Survey. J Clin Epidemiol. 1998;51(11):1115–28.

63.

Fryback

Dunham

Palta

, et al. US norms for six generic health-related quality-of-life indexes from the National Health Measurement study. Med Care. 2007;45(12):1162–70.

64.

Hanmer

Lawrence

Anderson

Kaplan

Fryback

DG.

Report of nationally representative values for the noninstitutionalized US adult population for 7 health-related quality-of-life scores. Med Decis Making. 2006;26(4):391–400.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB