Sage Journals: Discover world-class research

Abstract

The digital anger thermometer is a prototype for a mobile application (app) for use with adults in anger management treatment. The digital anger thermometer incorporates standards of software development in addition to anger management resources from the Substance Abuse and Mental Health Services Administration. The digital anger thermometer underwent a usability study conducted by five expert reviewers. The results indicate that it is easy to learn, efficient, and ergonomically sound. However, it does not offer support features or user-error tolerance. The digital anger thermometer prototype requires additional usability studies and comparative research in order for it to become an actual mental health app.

Keywords

databases and data mining e-health evidence-based practice health information on the web mobile health

The digital anger thermometer (DAT) is a prototype for an anger management application (app). The purpose of this article is to review the features and usability of the DAT. Despite the availability of numerous mental health apps, few appear to consider usability evaluations in their development.^1,2 Such an evaluation is an important precondition for the usefulness and efficiency of apps.³ It saves time and money in the developmental phases of a project and reduces overall system errors.^1,4 Consequently, the DAT mobile app prototype underwent an expert review to inform design revisions so that it may eventually become a useful tool in anger management.

With advances in mobile device technology, clients now have access to a wide range of therapy apps to supplement their treatment.⁵ Mobile apps typically include tools for tracking moods, managing daily stressors, and improving overall mental health.⁶ Apps are also capable of monitoring physical symptoms, providing biofeedback, guiding the user through therapeutic scripts, and offering additional self-help resources. There are an increasing number of individuals using mobile technology, and mental health apps allow them to participate actively in their own care.⁷ At the same time, there is growing demand for expansive and cost-effective mental health services.⁸ The portable and cost-effective nature of therapy apps could extend treatment to individuals with limited access to mental health resources.^9,10

Therapy apps address a wide range of issues such as mood disorders, anxiety, post-traumatic stress disorder, schizophrenia, and anger. In a mixed-methods field study using a mood-tracking app, participants reported lower mean scores on a series of single-dimension instruments for anger, anxiety, and sadness scales (p ⩽ .01 for all three) following its use.¹¹ Apps assist in modulating emotions through self-regulation. Self-regulation reduces the severity of mental health symptoms, improves interpersonal relationship, and helps individuals live and work effectively.¹² Anger problems affect relationships and physical health,¹³ and anger management requires self-regulation and awareness.^14,15

Studies show that mood-tracking mobile devices reduced anger and stress levels. Of 78 trait-anxious participants, researchers noted lower scores in both stress reactivity and anxiety following the use of an app addressing cognitive biases.^11,16 Similarly, other researchers¹⁷ reported an overall decrease in state anxiety while using the Mobile Stress Management app (F(1, 28) = 71.365, p ⩽ .001). Although the DAT requires future trials to assess its potential, it models some of the features of currently studied apps by tracking anger levels, cues, and patterns. The interactive nature of the DAT tools could increase client participation in treatment.¹⁸ Apps similar to the DAT include the Mobile Stress and Anger Management Tool¹⁹ (M-SAT) and Mobile Therapy app.¹¹ The M-SAT is a support suite used conjointly by therapist and client. The M-SAT provides real-time support and cognitive behavioral therapy tools and tracks stress and anger levels through physiological sensors and an electronic anger meter (EAM). The interval scale on the M-SAT meter is the same as the meter within the DAT. However, unlike the M-SAT, the DAT directly links the user to online anger management resources and activities based on selected anger meter levels.

The mood-mapping app, known as Mobile Therapy, shares several qualities with the DAT. For one, it is an adjunctive treatment tool requiring weekly interviews between therapists and clients in order to work through anger levels and cues. The Mobile Therapy app’s real-time anger management resources resemble those found in the DAT. The Mobile Therapy app employs a selectable mood scale similar to the one featured in the DAT. The original version of the DAT was a hand-held device known as the EAM.¹⁹ The EAM featured an analog scale of illuminated light-emitting diodes (LEDs), each representing a level of anger. Results of testing the EAM with participants revealed that subsequent designs would benefit from digitization to fit the growing trend of portable device usage.²⁰ While the DAT is largely a digital version of the EAM, it improves upon it by incorporating software development protocol and additional resources on anger management.^21–25

The eventual purpose of the DAT is anger assessment. Individuals experience anger through physical symptoms, cognitions, and behaviors.^26,27 Anger is traditionally assessed through self-report inventories. State (anger episodes in real-time) and trait (pervasive patterns) anger are often included in anger inventories.²⁸ When attempting to measure anger objectively, it is important to consider spectrums of anger that take into account the intensity, frequency, and manner of the episodes.^28–30 An anger spectrum might also include affective experiences from mild irritability to actual assault.³¹ In order to measure anger with mobile devices, developers are beginning to incorporate spectrums of affect into their app designs.^11,30 The development of the DAT aligns itself with this practice by including an anger meter informed by sources while undergoing app development guidelines.

While the contribution of this article is to show the initial development of the DAT, completion of the app requires several stages beyond the usability evaluation. These include issuing additional evaluations to target users, field testing, deployment, and maintenance.³² It is important to use participants that the app intends to serve during early phases of development. Testing for a majority of commercial apps occurs with participants for which the app is intended.³³ Field testing commences when testers control for as many variables as possible in the user’s environment.³⁴ After testing, deployment and maintenance involve configuring networks and adjusting security for true mobile use.³⁵ Prior to any of these steps, basic design and usability must occur.³⁶

The section “Features of the DAT” outlines the features of the DAT and the body of literature that informs its design. The section “Materials and methods” outlines the usability evaluation. The section “Results” discusses the implications and further work required for developing the DAT.

Features of the DAT

Every app consists of a series of displays and functions known as a graphical user interface (GUI).³⁷ A GUI is an interfacing program containing a series of controls and displays that allow a user to interact with a computer.³⁷ The DAT interface was designed for Microsoft Surface Pro tablets and emerging Windows-based mobile systems. An overview of the main DAT form is shown in Figure 1. The following section outlines the tools involved in constructing the prototype, followed by the layout. The layout includes a login frame, a database, main GUI, dialog boxes, and graphing tools.

Figure 1.

Overview of the DAT app.

Development tools

Developing the DAT required an integrated development environment (IDE), which is a set of programming tools used to develop software.³⁸ Programming occurred on an ASUS X75A powered by an Intel Core i3 processor with a 4-GB RAM. SQLlite open-source software provided the framework for constructing the DAT database.

Login screen and security

The issue of security in mental health apps is ongoing.³⁹ Security standards for the ubiquitous use of mobile mental health applications are lacking.⁴⁰ Even now, the Health Insurance Portability and Accountability Act of 1996 (HIPAA) does not adequately address the security issues surrounding telehealth.

Security breaches of programs and pages are a major concern and increasingly make news headlines, such as the case of the Heartbleed bug, which affected mobile applications last year.⁴¹ The SQLite database that holds user login information includes the SQLCipher encryption extension, a security component that actively defends against hacker intrusions. This extension works concurrently with the database to encrypt information further and is a standard choice for mobile applications. Additionally, the DAT login page allows a user to select a password up to 14 characters in length, offering adequate unpredictability.⁴² The DAT also requires that the user enter both characters and numbers in the password.

Database

A database is a collection of digitally retrievable information.⁴³ Mobile app design standards recommend that mobile applications with multiple storage requirements use a shared database.⁴⁴ The DAT employs the open-source database known as SQLite.⁴⁵ The SQLite software library uses an SQL database engine capable of lightweight data storage, making it ideal for most mobile apps.

Main form

DAT thermometer

The main form consists of several panels, and the main panel contains a thermometer graphic controlled by user input on a slider button (see Figure 2). Inspiration for the thermometer design came from the 10-scale anger meter featured in the Substance Abuse and Mental Health Services Administration (SAMHSA) anger management manual.^20,24 The thermometer currently provides a means for self-report with the hope that someday it can be used for more direct measurement of anger.

Figure 2.

The user selects an anger level, which populates the explanation and suggestions text fields.

The DAT uses an interactive 0–10 scale on all items and the default meter level is zero, signifying no anger, represented by a blue graphic color of the “mercury” at the bulbous base of the thermometer graphic. This incremental measure of arousal is an important component of assessing anger.^30,46 Starting at one, the information on anger levels begins to appear and the mercury color changes to green. As it progresses, the colors change to warmer yellow and red tones. This color scheme primarily follows temperature thermometers that monitor heat levels, popularly found in graphic anger meters.⁴⁷ The thermometer connects two other panels, one for explaining the levels and the other for recording anger incidents.

Explanations panel

The explanations panel contains the results of each selected level. It explains the symptoms of anger on various levels of a spectrum.^23,31 The top end of the panel features a combination box containing suggestions for possible interventions tailored to each level of anger. Typical interventions include cognitive behavioral therapy, relaxation scripts, and progressive muscle relaxation. Previous anger management app research revealed the necessity for users to add their own interventions;^11,48 consequently, this panel offers an editable text field prompt for entering a personalized intervention.

After choosing an intervention, clickable hyperlinks (shortcuts between web pages) to online anger management resources appear (see Figure 3). Effective anger management requires a psychoeducational component.⁴⁹ The sites offer information on cognitive behavioral therapy,⁵⁰ mindfulness,⁵¹ deep breathing,⁵² and general relaxation.⁵³ Because additional self-help resources reinforce anger intervention effectiveness,^54,55 a button on the bottom of the panel links the user to further reading on anger management.

Figure 3.

The user selects a hyperlink for improving communication skills. With weekly levels filled out, clicking the graph button produces a pie chart of the data.

Weekly levels panel

The weekly levels panel contains tools for monitoring and recording the highest level of anger throughout the week, an important component of anger management.^23,56 The weekly levels panel can record behavioral information across a designated time span for informing treatment,⁵⁷ and the panel includes spaces known as text fields for recording the highest daily level of anger, along with fields for adding user notes regarding the level. With the ability to review anger levels over time, a user can record possible antecedents to the anger or further describe the anger incident for use in clinical sessions.²⁵ The weekly levels panel also offers a means for the user to illustrate the data through a pie chart or bar chart, which provides users with quick visual representation of the anger data, although no evidence currently supports the use of such a graphic. The panel also features a save function for the charts so that users can review information over time.

Materials and methods

Usability evaluations help identify strengths and weaknesses of applications while offering feedback for improving their usability.¹ They also offer a glimpse into actual user experiences prior to further development of software models so that developers can streamline the applications and reduce system errors.⁵⁸ The DAT underwent a usability evaluation conducted by subject matter experts. The evaluation consisted of a questionnaire issued to the experts in a controlled setting. The scored instruments revealed information pertaining to the design and functions of the DAT.

Questionnaire

This study employed the IsoMetrics questionnaire⁵⁹ (for the evaluation of DAT GUI). The IsoMetrics inventory is a user-oriented instrument based on part 10 of the ISO 9241. It presents with an appreciable amount of literature backing its use in the evaluation of health-based applications.^1,60–63 Validity studies of the instrument resulted from comparing the scale means of five major software systems.⁵⁹ The questionnaire consists of 75 items, each ratable on a 5-point Likert-style scale within seven test dimensions: Suitability for the task, Self-descriptiveness, Controllability, Conformity with user expectations, Error tolerance, Suitability for individualization, and Suitability for learning.

Participants and procedure

The expert evaluation occurred between July and August 2014. Probabilistic models show that five raters are sufficient for usability testing.⁶⁴ The minimum requirements for participation included an advanced degree in a computer field and at least 2 years of software development. Candidates included three app developers, a website designer, and a computer scientist. Mean age of these participants was 32.2 years (standard deviation (SD) = 2.32, range: 29–36).

The IsoMetrics manual⁶⁵ guided the testing procedure. Each expert received an interview regarding their general experience and familiarity with systems similar to the DAT prior to receiving the test. Any questions that arose were addressed prior to handing out the questionnaire. The experts then received a brief review of the app and a paper version of the IsoMetrics questionnaire, followed by a reading of the text on the instructions statement.

There was a high selection rate on the IsoMetrics test. Scoring involved use of the mean to calculate the response averages. Negatively formulated items underwent a transformation prior to calculation. An analysis of the Suitability for the task subscale further illustrated positive and negative points of the DAT.

Results

The results of the questionnaire are in Table 1. The means and corresponding SD scores were calculated from the responses of the five raters. The standard cutoff score for the IsoMetrics usability evaluation is 3, which denotes a moderate score.¹ Any rating higher than 3 is therefore considerable, but should be assessed depending on the context of use.⁶⁵ The highest scores are in the Conformity with user expectations (M = 3.68, SD = 0.47) and Suitability for learning (M = 3.80, SD = 0.57) dimensions. The lowest scores are in the Error tolerance (M = 2.75, SD = 0.26) dimension, of which there is high agreement among the raters.

Table 1.

Usability evaluation scores of the DAT.

Dimension	M	SD
Suitability for the task	3.41	0.46
Self-descriptiveness	3.38	0.41
Controllability	3.28	0.26
Conformity with user expectations	3.68	0.47
Error tolerance	2.75	0.26
Suitability for individualization	3.40	0.75
Suitability for learning	3.80	0.57

All scores from n = 5 raters.

Individual evaluation of the items yielded additional information. The items with the highest scores included S.5, ease of retrieving about a certain entry field (4.00), and E.1, anticipating which screen will appear in next processing sequence (4.20). The lowest scored items pertained to S.3, understanding immediately what is meant by the messages displayed by the software (2.40), and F.6, entries are checked for correctness before further processing is initiated (1.80).

The Suitability for the task dimension covers the overall effectiveness and efficiency of software features. Delineation of this area reveals detailed information about the usability of the DAT. The results are given in Figure 4. Item descriptions with the corresponding means and SDs are as follows.

Figure 4.

Mean values for individual items in the Suitability for the task dimension.

Positively rated items include the following:

A.1. The software forces me to perform tasks that are not related to my actual work (M = 4.60, SD = 0.89) (reverse-scored for statistical analysis).

A.8. Too many different steps need to be performed to deal with a given task (M = 4.20, SD = 1.30) (reverse-scored for statistical analysis).

Negatively rated items include the following:

A.3. The software lets me completely perform entire work routines (M = 2.20, SD = 0.54).

A.4. The functions implemented in the software support me in performing my work (M = 2.40, SD = 0.89).

The positively rated items show a measure of efficiency in performing the tasks (above a rating of 3), although there appears to be an outlier in at least one of the responses of A.8 where the rating deviated significantly from the other respondents. The negatively rated items indicate incompleteness of the program and lack of support features. Considering these results, the DAT app could be improved as discussed below.

Discussion

A usability evaluation and anger management resources informed the development of the DAT. The results indicate that the overall ergonomic quality of the DAT is appreciable, and it presents with some measure of efficiency, but the expert sample size was low and the program is far from complete. The highest dimension score is in the Suitability for learning subscale. This indicates that it does not require much time to learn the DAT program and that some features of the software assist the user in becoming familiar with the layout. The lowest score is in Error tolerance. This essentially means that the program is not accommodating mistakes made by the user, and small user errors are yielding larger issues within the program. During testing of the DAT, several crashes occurred in the programming when users attempted to correct their information. Analysis of the individual items in the Suitability for the task dimension indicated a lack of support features when these events occurred. This affirms that the DAT requires debugging in this area and that the DAT can benefit from a user support component currently absent in its design. In addition to reworking the software aspect of the DAT, its ability to work as a mental health app requires further exploration.

Despite the growing popularity of mental health apps, there are numerous drawbacks to their use, including effectiveness, device limitations, security, and reimbursement.⁶⁶ There is a lack of studies in app effectiveness, and newly formed committees are only now beginning to determine standards to ensure measures of reliability and validity.⁶⁷ Although some practitioners are beginning to use apps as therapeutic tools, no long-term studies on their effectiveness in treatment have been done.⁵ In a comprehensive 5-year literature search of 5464 abstracts, researchers found little evidence for the effectiveness of mental health apps.⁹ Although there is some evidence for their effectiveness in use with engaging younger individuals in therapy, gaps in research remain to address app feasibility, cross-cultural considerations, technical literacy, and ethical issues.⁶⁸

Concerning device limitations, limited memories of many portable devices are unable to hold large amounts of data.⁶⁹ Small features can deter disabled and elderly individuals from using apps.⁷⁰ Small device size forces mobile apps to run with reduced graphics and formats and it makes these devices easy to lose. Smaller devices are vulnerable to prying eyes and hackers, which pose a threat to the security of sensitive information contained in the devices.⁷¹

Cost is a significant barrier to the widespread adoption of mental health apps. The use of mobile technology in treatment sometimes requires dialing costs incurred by clinician and client.⁷² A large gap exists in reimbursement for healthcare technology: there is no billing code for the use of mobile apps in therapy.⁴⁶ In time, with growing initiatives for technology and participatory care, this could change and mobile apps could be used for billable expenses. Until then, clients, clinicians, and researchers must cover the costs of app technology.

The DAT, like other mental health apps, requires controlled studies to determine true effectiveness.¹¹ Subsequent versions of the DAT should incorporate the results of this study during further development. The next stage of usability engineering involves iterative development.^73,74 This includes a process of continuously evaluating and changing features of the DAT in order to identify major bugs in the software. In addition, a second usability study using non-expert users will help to identify additional issues with software ergonomics by offering a population resembling intended users of the DAT.^4,75

After refining the user interface of the DAT, its ability to measure anger requires examination. To assess effectiveness, a follow-up study might include a negative affect measure such as the State-Trait Anger Expression Inventory²⁷ used in conjunction with trials of the DAT in a pre- and post-study of participant scores. If the DAT app undergoes such refinements, it could become an additional tool for use in anger management treatment.

Conclusion

Development of the DAT app included sources on anger management and a usability evaluation. A digital variation of the SAMHSA anger meter scale is at the core of its user interface. The main body of the DAT contains tools derived from established anger management interventions. Although the DAT includes some standards and sources in its design, the usability evaluation pointed to incomplete areas of the DAT that require further study before becoming a shareable app. Additionally, the DAT requires experimental studies with target users in order to assess its effectiveness.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

References

Hamborg

Vehse

Bludau

. Questionnaire based usability evaluation of hospital information systems. Electron J Inform Syst Eval 2004; 7, http://www.ejise.com/volume7/issue1

Landauer

. The trouble with computers: usefulness, usability, and productivity. Cambridge, MA: MIT Press, 1995.

Nielsen

. Usability engineering. Boston, MA: AP Professional, 1993.

Dumas

Redish

. A practical guide to usability testing. Wilmington, NC: Intellect Books, 1999.

Price

Yuen

Goetter

. mHealth: a mechanism to deliver more accessible, more effective mental health care. Clin Psychol Psychother 2014; 21: 427–436.

Trudeau

. Mental health apps: like a “therapist in your pocket.” National Public Radio, 2010, http://www.npr.org/templates/story/story.php?storyId=127081326

Morris

Aguilera

. Mobile, social, and wearable computing and the evolution of psychological practice. Prof Psychol Res Pr 2012; 43: 622–626.

Eaton

McCay

Semrau

. Scale up of services for mental health in low-income and middle-income countries. Lancet 2011; 378: 1592–1603.

Donker

Petrie

Proudfoot

. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013; 15: e247.

10.

Handel

. mHealth (mobile health)-using apps for health and wellness. Explore 2011; 7: 256–261.

11.

Morris

Kathawala

Leen

. Mobile therapy: case study evaluations of a cell phone application for emotional self-awareness. J Med Internet Res 2010; 12: e10.

12.

Lyubomirsky

King

Diener

. The benefits of frequent positive affect: does happiness lead to success? Psychol Bull 2005; 131(6): 803–855.

13.

Smith

Glazer

Ruiz

. Hostility, anger, aggressiveness, and coronary heart disease: an interpersonal perspective on personality, emotion, and health. J Pers 2004; 72: 1217–1270.

14.

Kassinove

Tafrate

. Anger management: the complete treatment guidebook for practitioners. Atascadero, CA: Impact, 2002.

15.

Puff

Seghers

. The everything guide to anger management: proven techniques to understand and control anger. Avon, MA: F+W Media, 2014.

16.

Dennis

O’Toole

. Mental health on the go: effects of a Gamified attention-bias modification mobile application in trait-anxious adults. Clin Psychol Sci 2014; 2: 576–590.

17.

Villani

Grassi

Cognetta

. Self-help stress management training through mobile phones: an experience with oncology nurses. Psychol Serv 2013; 3: 315–322.

18.

Kim

Lin

Sung

. To app or not to app: engaging consumers via branded mobile apps. J Interact Advert 2013; 13: 53–65.

19.

Mattson

. The Electronic Anger Meter (EAM): a device for gauging child anger. Br J Play Ther 2013; 2013(9): 4–14.

20.

. Math that moves: schools embrace the iPad. The New York Times, http://www.nytimes.com/2011/01/05/education/05tablets.html?pagewanted=all&module=Search&mabReward=relbias%3Ar%2C%7B%222%22%3A%22RI%3A15%22%7D&_r=0 (accessed 4 January 2011).

21.

DeFoore

. Anger: deal with it, heal with it, stop it from killing you. Deerfield Beach, FL: Health Communications, 2004.

22.

DiGiuseppe

Tafrate

. Anger treatment for adults: a meta-analytic review. Clin Psychol: Sci Pr 2003; 10: 70–84.

23.

Marcus

(ed.). Design, user experience, and usability: theory, methods, tools and practice. New York: Springer Verlag, 2010.

24.

Reilly

Shopshire

. Anger management for substance abuse and mental health clients: a cognitive behavioral therapy manual. Rockville, MD: Center for Substance Abuse Treatment, Substance Abuse and Mental Health Service Administration, 2007.

25.

Ruci

Karcanaj

. Three main pain points from today’s Smartphones. J Contr Theor Informat 2013; 3(6): 17–24, http://www.iiste.org/Journals/index.php/CTI/issue/view/877

26.

Novaco

. The Novaco Anger scale and Provocation Inventory (NAS-PI). Los Angeles, CA: Western Psychological Services, 2003.

27.

Sukhodolsky

Golub

Cromwell

. Development and validation of the anger rumination scale. Pers Indiv Differ 2001; 31: 689–700.

28.

Spielberger

. Manual for the State-Trait Anger Expression Inventory (STAXI). Odessa, FL: Psychological Assessment Resources, 1988.

29.

Novaco

Renwick

. Anger predictors of the assaultiveness of forensic hospital patients. In: Sanario

(ed.) Behavior and cognitive therapy today: essays in honor of Hans J. Eysenck. Oxford: Pergamon, 1998, pp. 199–208.

30.

Pollak

Adams

Gay

. PAM: a photographic affect meter for frequent, in situ measurement of affect. In: Proceedings of the SIGCHI conference on human factors in computing systems, Vancouver, BC, Canada, 7–12 May 2011, pp. 725–734. New York: ACM.

31.

Bus

Durkee

. An inventory for assessing different kinds of hostility. J Consult Psychol 1957; 21: 343–349.

32.

Vithani

Kumar

. Modeling the mobile application development lifecycle. In: Proceedings of the international multi-conference of engineers and computer scientists, Hong Kong, China, 12–14 March 2014.

33.

Uhler

Mehta

(eds). Mobile computing, applications, and services. Seattle, WA: Springer, 2013.

34.

Singh

Yadav

. Field testing of mobile applications: an effective approach. Int J Comput Appl 2013; 21: 1–7.

35.

Bartz

. Mobile computing deployment and management: real world skills for CompTIA mobility+ certification and beyond. Indianapolis, IN: John Wiley & Sons, 2015.

36.

Zhang

Adipat

. Challenges, methodologies, and issues in the usability testing of mobile applications. Int J Hum: Comput Int 2005; 18: 293–308.

37.

Ericson

. Concise encyclopedia of system safety: definition of terms and concepts. Hoboken, NJ: John Wiley & Sons, 2011.

38.

Niemeyer

Knudsen

. Learning Java. Sebastopol, CA: O’Reilly Media, 2005.

39.

Luxton

McCann

Bush

. mHealth for mental health: integrating smartphone technology in behavioral healthcare. Prof Psychol: Res Pr 2011; 42: 505–512.

40.

Luxton

Kayl

Mishkind

. mHealth data security: the need for HIPAA-compliant standardization. Telemed J E Health 2012; 18: 284–288.

41.

O’Connor

. Google services updated to address OpenSSL CVE-2014-0160 (the Heartbleed bug), 2014, http://googleonlinesecurity.blogspot.com/2014/04/google-services-updated-to-address.html

42.

Bidwell

. Hack proofing your identity in the information age. Boston, MA: Syngress, 2002.

43.

Powell

. Beginning database design. Indianapolis, IN: Wiley, 2006.

44.

Natan

. Implementing database security and auditing. Burlington, MA: Elsevier Digital Press, 2005.

45.

Lee

. Creating and using databases for android applications. Int J Database Theory Appl 2012; 5: 99–106, http://www.sersc.org/journals/IJDTA/vol5_no2/8.pdf

46.

Russell

. Circumplex model of affect. J Pers Soc Psychol 1980; 39: 1161–1178.

47.

Schopenhauer

. On vision and colors. New York: Princeton Architectural, 2010.

48.

Jones

Dechmerowski

. Improving the efficiency and effectiveness of cognitive behavioral therapy: the utility of a provider portal, 2013, http://ftp.rta.nato.int/public/PubFullText/RTO/MP/STO-MP-HFM-231/MP-HFM-231-03.pdf

49.

Taylor

. A review of the assessment and treatment of anger and aggression in offenders with intellectual disability. J Intell Disabil Res 2002; 46: 57–73.

50.

Beck

Fernandez

. Cognitive-behavioral therapy in the treatment of anger: a meta-analysis. Cognitive Ther Res 1998; 22: 63–74.

51.

Wright

Day

Howells

. Mindfulness and the treatment of anger problems. Aggress Violent Beh 2009; 14: 396–401.

52.

Edwards

. A psychology of breathing methods. Int J Ment Health Promot 2005; 7: 30–36.

53.

Davison

Williams

Nezami

. Relaxation, reduction in angry articulated thoughts, and improvements in borderline hypertension and heart rate. J Behav Med 1991; 14: 453–468.

54.

Campbell

Smith

. Integrating self-help books into psychotherapy. J Clin Psychol 2003; 59: 177–186.

55.

Thomas

. Teaching healthy anger management. Perspect Psychiatr Care 2001; 37: 41–48.

56.

Donovan

. Dealing with your anger: self-help solutions for men. Alameda, CA: Hunter House, 2001.

57.

Bush

Skopp

Smolenski

. Behavioral screening measures delivered with a smartphone app: psychometric properties and user preference. J Nerv Ment Dis 2013; 201: 991–995.

58.

Kushniruk

Triola

Boryeki

. Technology induced error and usability: the relationship between usability problems and prescription errors when using a handheld application. Int J Med Inform 2005; 74: 519–526.

59.

Gediga

Hamborg

Düntsch

. The IsoMetrics usability inventory: an operationalization of ISO 9241-10 supporting summative and formative evaluation of software systems. Behav Inform Technol 1999; 18: 151–164.

60.

Bevan

. International standards for HCI and usability. Int J Hum: Comput St 2001; 55: 533–552.

61.

Bundschuh

Majeed

Burkle

. Quality of human-computer interaction: results of a national usability survey of hospital-IT in Germany. BMC Med Inform Decis 2011; 69: 1472–1477.

62.

Gediga

Hamborg

. IsoMetrics: an usability inventory supporting summative and formative evaluation of software systems. In: Proceedings of the 8th international conference on human-computer interaction: ergonomics and user interfaces, Munich, 22–26 August 1999, vol. 18, pp. 1018–1022. London: Lawrence Erlbaum Associates.

63.

Safdari

Dargahi

Shahmoradi

. Comparing four softwares based on ISO 9241 part 10. J Med Syst 2012; 36: 2787–2793.

64.

Virzi

. Refining the test phase of usability evaluation: how many subjects is enough? Hum Factors 1992; 34: 457–468.

65.

Gediga

Hamborg

Willumeit

. The IsoMetrics manual (version 1.15). Osnabrück: Universität Osnabrück, 1998.

66.

Boulos

MNK

Wheeler

Tavares

. How smartphones are changing the face of mobile and participatory healthcare: an overview, with example from eCAALYX. Biomed Eng 2011; 10: 1–14.

67.

Grady

Myers

Nelson

. Evidence-based practice for telemental health. Telemed J E Health 2011; 17: 131–148.

68.

Seko

Kidd

Wiljer

. Youth mental health interventions via mobile phones: a scoping review. Cyberpsychol Behav Soc Netw 2014; 17(9): 591–602.

69.

Lin

Vassar

. Mobile healthcare computing devices for enterprise-wide patient data delivery. Int J Mob Commun 2004; 4: 343–353, http://www.inderscience.com/jhome.php?jcode=ijmc

70.

Mori

Harada

. Is learning a family matter?: Experimental study of the influence of social environment on learning by older adults in the use of mobile phones. Jpn Psychol Res 2010; 52: 244–255.

71.

Ongtang

McLaughlin

Enck

. Semantically rich application-centric security in Android. Secur Comm Network 2012; 5: 658–673.

72.

Krisberg

. Cell phone popularity a barrier for public health data collection: more Americans forgoing phone landlines. The Nation’s Health, 2009, vol. 39, pp. 1–12, http://thenationshealth.aphapublications.org/content/39/7/1.2.short

73.

Gediga

Hamborg

Düntsch

. Evaluation of software systems. In: Kent

Williams

(eds) Encyclopedia of library and information science, vol. 72. Boca Raton, FL: CRC Press, 2002, pp. 166–192.

74.

Hahn

. iPhone application development: strategies for efficient mobile design and delivery. Oxford: Chandos, 2011.

75.

Carroll

Human-computer interaction: psychology as a science of design. Int J Hum: Comput St 1997; 46: 501–522.

Usability evaluation of the digital anger thermometer app

Abstract

Keywords

Features of the DAT

Development tools

Login screen and security

Database

Main form

DAT thermometer

Explanations panel

Weekly levels panel

Materials and methods

Questionnaire

Participants and procedure

Results

Discussion

Conclusion

Footnotes

Declaration of conflicting interests

Funding

References