Sage Journals: Discover world-class research

Abstract

Self-service technology is widely used in the financial sector to ensure consumers can easily conduct financial transactions. For example, banks have successfully applied cardless withdrawal services to ATMs. However, many factors still exist as to why cardless withdrawals cannot be widely promoted. In the preliminary investigation work of this study, we investigated the user experience design of cardless cash withdrawals of the top three banks in Taiwan’s ATM market share. After summarizing many problems, we optimized the design to include two sets of typography and guidance. We used the task-oriented cognitive walkthrough method to survey 36 participants and adopted post-task and post-session self-report assessments, such as SEQ, NASA-TLX, satisfaction, preference, NPS, etc. Finally, it is supplemented by semi-structured interviews to understand users’ performance, thoughts, and problems when completing relevant tasks. It was found that users with different experiences have different needs for the interface, which are reflected in different workload items. In addition, the correlation between the scales was discovered, and suggestions were provided for future industry workers who want to practice data-driven UX.

Plain language summary

Using task-oriented cognitive walkthrough and post-task/post-session self-report to evaluate bank ATM cardless service interface and analyze the impact of visual hierarchy design on user experience

Why was the study done? Modern ATMs prioritize usability as financial services shift towards user experience. While cardless services offer enhanced security by allowing app verifications reducing password theft risks, their longer operation times hinder widespread adoption. This study examines the ATM cardless system with smartphone verification to assess user interactions and understanding. What did the researchers do? Initially, a situational survey was executed. We assessed cardless interfaces of the top three banks based on ATM count, using a task-oriented cognitive walkthrough and feedback from 30 participants. Selecting one bank with superior outcomes, we designed two typography and guidance versions. Subsequently, similar research steps from the initial phase were repeated. Experiments used four material combinations, with two tasks: “applying for cardless service” and “cardless cash withdrawal.” Participants were unfamiliar with cardless transactions. What did the researchers find? We collected data from 36 participants across four groups with diverse ATM experience. Metrics included operation times, errors, and scores from SEQ, NASA-TLX, satisfaction, preference, SUS, and “willingness to use and NPS.” Interviews provided insights into issues and participants’ opinions. High and low-experience users differed in interface usage and workload. We analyzed correlations in data-driven UX, discussed information architecture, visual hierarchy, and attention allocation, and offered UX recommendations. Our optimized design greatly surpassed the bank’s original performance and user ratings. What do the findings mean? The optimized design outperformed the original bank interface. We refined the visual design by employing heuristic evaluation and visual hierarchy theories. Beyond task performance metrics, we understood the varied experiences between seasoned and new users. Conclusively, we offer industry insights on user experience evaluation.

Keywords

touch-based properties and capabilities of the user user/machine systems visual hierarchy data-driven UX methods

Introduction

According to the latest statistics from Financial Supervisory Commission R.O.C.—Banking Bureau (2023), there are 32,913 ATMs in Taiwan as of the end of August 2023, with the density ranking highest globally. ATMs not only help emerging and mature markets move towards urbanization and digitalization but also reduce the frequency of human error among physical bank employees, thereby ensuring efficient operations and saving consumers time in queues. Gradually, ATMs will no longer be just automated devices for withdrawing or deposits; in the future, banking services will be ubiquitous and consumer-led (King, 2012, 2018). NCR, an Australian ATM development company, said that today’s ATMs have become an established and familiar part of the banking system, and in the future, improving the system’s usability will be the primary development focus (Cluckey, 2020; NCR, 2021). The Financial Supervisory Commission launched the “Building Digital Financial Environment Plan 3.0” in 2015 (Financial Supervisory Commission R.O.C, 2013), which promoted the broader application of ATMs than in the past, thereby bringing more convenience and security to meet consumer demand for transactions. The service model of financial enterprises in the future is more user-experience-oriented. On the other hand, the form and function of ATM also gradually refer to the smartphone user experience; for example, users are familiar with the touch screen, finger pinching, mobile authentication, and other operation methods on the smartphone, it is recommended that the above operation experience can also be added to the ATM (NCR, 2021).

Coventry et al. (2003) found that the first successful operation of a task on an ATM increases user satisfaction and, thus, the willingness to use the feature. However, the needs and expectations of bank customers for services are also expanding, and the quality of financial services should be characterized by independence, flexibility, freedom, and flexibility to meet these needs (Khalfan & Alshawaf, 2014; Lustsik, 2004). Customer satisfaction ultimately leads to the use process being linked to brand loyalty (Churchill & Surprenant, 1982; Jamal, 2007), and satisfaction occurs when customers compare their perception of actual product/service performance to their expectations (Oliver, 1980). Many studies have also shown that the quality of e-banking services significantly impacts customer satisfaction (Asiyanbi & Ishola, 2018; Bei & Chiao, 2006; Hammoud et al., 2018; Ranaweera & Neely, 2003; Zhou, 2004). Das et al. (1996) surveyed customer satisfaction with Canadian public services and found that user satisfaction is affected by user experience and consumer demand.

IBM believes that security is an important factor in operating ATMs, and authentication is defined as four elements: what the consumer knows (password), what he has (bank card), what he has (fingerprint), and what can be done (OTP, appointment number, QR code; Kessem, 2018; Lin, 2015). Among them, cardless services have the advantage of security; consumers must verify their identity through mobile apps so they do not worry about password theft (C. Gordon, 2016). In addition to contactless transactions through smartphones, today’s ATMs have also developed card-not-present transaction services with the help of mobile device authentication technology, thereby replacing bank card identity verification. Forgetting to carry a bank card is the main motivation for consumers to operate cardless services for the first time (Chang, 2019). Card-not-present services offer a variety of use cases to meet consumer demand for cash (CommBank, 2014).

Currently, many banks provide different cardless cash withdrawal methods, including iris recognition, fingerprint recognition, face recognition, third-party application binding, NFC sensing, and smartphone verification. However, people’s doubts and distrust of biometrics and the limitations of smartphones in sensing functions do not apply to most users. Among the various cardless withdrawal methods, NFC cardless services can reduce the transaction time from 30 to 40 s to about 10 s. Speed is an important part of the consumer experience and can provide more convenient services than traditional ATMs (Walker, 2017). De Luca and Frauendienst (2008) evaluated the function of ATM password authentication and found that the operation time and steps will also affect the user’s satisfaction with the system and suggested that the time taken by identity authentication should be maintained within 10% of the total operation time and proposed that the public tends to use password as identity authentication. Seifert et al. (2012) set the operation of the ATM part on the smartphone and found that the hybrid operation of the ATM and smartphone is faster and more comfortable. Currently, cardless service is not a common way to operate ATMs. The reason is that cardless withdrawals take more time than traditional transaction operations. As a result, banks are still unable to popularize cardless services despite continuous promotion (Chenlin, 2020).

Therefore, this study will investigate the ATM cardless service that uses smartphone identity verification to understand users’ cognition and operating behavior of the information on the interface. In addition to in-depth quantification of users’ operating performance, it can be conducted through users’ self-reports, including SEQ, NASA-TLX, satisfaction scale and preference, SUS, “willingness to use,” and NPS method. Evaluate. We hope to achieve three goals:

(1) Optimize the interface visual design to shorten the cardless withdrawal operation time and provide a good user experience.

(2) Through various user experience survey methods, we can understand different users’ pain points when operating cardless services on ATMs.

(3) Proposing an ATM cardless service interface design scheme that conforms to user experience as a reference for future related interface design.

Literature Review

Evaluate the Core Concepts of UX

According to the ISO 9241-210 (International Standard, 2019) definition, “User experience is user’s perceptions and responses that result from the use and/or anticipated use of a system, product or service.…users’ perceptions and responses include the users’ emotions, beliefs, preferences, perceptions, comfort, behaviors, and accomplishments that occur before, during and after use.” Based on this definition, this study adopts some core concepts and indicators of user experience, among which the information architecture of the interface needs to be examined. Using websites as an example, Garrett (2002) proposes that the order of “Elements of UX” from abstract to concrete is Strategy, Scope, Structure, Skeleton, and Surface. In a nutshell, the surface layer must map the structure layer to user needs and corporate goals. Rosenfeld et al. (2015) proposed that users, content, and context be taken as the core concepts of information architecture and applied to user experience. Camilli et al. (2011) found that simplifying the information architecture reduces user error rates in the operating system while effectively increasing user satisfaction. Visual hierarchy can be key in information architecture planning (Kingston, 2020). The “Usability Heuristic” is a concrete and effective approach when information architecture corresponds to Skeleton and Surface layers. Nielsen’s (1994)“Usability Heuristics” is still applicable to user interface design, where experts evaluate the “visibility of system status,”“user control and freedom,”“consistency,”“error prevention and recognition,”“flexibility and efficiency,”“aesthetic and minimalist design,”“providing user help,” etc. Therefore, when we review the existing ATM human interface, reviewing its information architecture will be one of the critical points, and we will use “Usability Heuristics” to evaluate design conditions. In addition, information architect Morville (2004) further proposed seven indicators of user experience honeycomb: usable, useful, desirable, findable, valuable, accessibility, and credibility as UX goals.

Lee and Allaway (2002) found that enhancing the customer’s sense of personal control over the service experience reduces perceived risk—financial, performance, social, psychological, safety, and time/convenience losses (Dowling, 1986; Peter & Tarpey, 1975) and accelerates user willingness to use the new Self-Service Technology. Event predictability, controllability, and outcome desirability will allow users to perceive personal control (Averill, 1973). This echoes the Usability Heuristics, where user control is one of the items that evaluate interface design. Years of practice and academic research in display interface design, information architecture, visual design, and user experience between the inseparability.

Visual Hierarchy and User’s Attention

The visual hierarchy controls the delivery of the experience; if the user does not know where to pay attention, the layout likely lacks a clear visual hierarchy (K. Gordon, 2020). Considering visual design principles and being aesthetically pleasing, it can improve usability, trigger positive emotions, and further strengthen brand recognition (Lupton, 2015; Poulin, 2018). Perhaps this explains why so many UX practitioners have transitioned from visual designers.

Faraday’s visual hierarchy guidelines (Faraday, 2000) divide the visual hierarchy model into two phases; the first stage is the “search phase,” which guides the user to significant “entry points” through motion, size, image, and color. The visual characteristics of text style and position guide the user from strongest to least impactful, and the second stage is the “scan phase,” when the user extracts information from nearby elements. This is achieved by following grouping principles and reading conventions. However, Still’s (2018) study found that the entry point of visual attention is best predicted by position, color, or text style rather than size or image. However, attention is driven not solely by stimulus-driven, triggered by visual features.

In the early dichotomy (bottom-up vs. top-down), attention drive developed into a trichotomy, in which Theeuwes (2019) proposed three factors: stimulus-driven, goal-driven, and history-driven selection. His findings suggest that lingering biases of previous selection episodes significantly influence attentional selection. Therefore, history-driven selection can also be used at the visual level to guide the user’s attention; for example, we can use the “center stage” and “Z-layout” or “F-layout” in the general user design pattern of the ATM interface.

Task-Oriented Cognition Walkthrough

The “Cognitive Walkthrough” method is suitable for assessing the ease of learning the system from the perspective of a new user (Salazar, 2022). It is a low-cost, fast, and effective method for evaluating ease of use in humans (Lyon et al., 2021). C. Lewis et al. (1990) developed cognitive walkthroughs to evaluate ready-to-use interfaces such as kiosks and ATMs. In contrast to heuristic evaluation and usability testing, it focuses on the user’s cognitive activities, specifically their goals, when performing specific tasks (Mahatody et al., 2010). Cognitive walkthroughs are an influential method designers use to solve cognitive challenges in design (Ning et al., 2019).

The design and implementation process includes determining the task conditions, layering the task analysis, prioritizing the task, converting the task into a test scenario, grouping the test subject, and finally identifying and classifying usability problems. The implementation allows the test subject to go through each task step, allowing the designer to identify what problems exist in the interface design (C. Lewis & Wharton, 1997). Observers will observe four critical questions in the cognitive exercise methodology (Blackmon et al., 2002): (1) Will users try to achieve the right result? (2) Will users notice that the correct action is available? (3) Will users associate the correct action with the result they are trying to achieve? (4) After the action is performed, will users see that progress is made toward the goal? It is also necessary to record the time each task was completed and the number of errors, note the timing of the potential problems hidden in the subject’s behavior and conduct interviews afterward.

User Self-Report Evaluation

In addition to observing the user’s key questions and performance in the cognitive exercise, the post-task self-report assessment can measure the overall perceived usability and compare the existing problems with the observations (Laubheimer, 2018). This study used two types of questions: post-task questionnaires, completed immediately after completing the task, and capture participants’’ impressions of the task, which in turn collects many subjective answers. The other is post-session questionnaires, which reflect the overall experience usability reported by users because of the peak-end effect (Cockburn et al., 2015; Kahneman et al., 1993). Post-task questionnaires include Single Ease Question (SEQ), NASA-TLX, satisfaction and preference, and post-session questionnaires include SUS, willingness to use score, and net promoter score (NPS).

SEQ assesses “ease” on a seven-point scale with a single item (Sauro & Dumas, 2009). Albert and Dixon (2003) think it’s more important to compare post-task evaluation to pre-task scores, known as expectation measurement. The task evaluation results are based on the average expectation score and the average experience score as the X and Y axes, respectively, which are “Fix it fast,”“Don’t touch it,”“Promote It,” and “Big Opportunity.”

NASA-TLX (Task Load Index) is a standard questionnaire used in many human factors and ergonomics studies. NASA-TLX assesses user stress during operation based on a weighted average of six indicators: mental Demand (MD), physiological demand (PD), temporal demand (TD), performance (OP), effort (EF), and frustration (FR). This scale will be divided into two steps. First, the user compares six metrics in pairs to determine which ones are more important for the user to operate the system. Users are then asked to rate six metrics on a scale of 0 to 100 and then weigh the scores for that metric to get the final score (Bianchi et al., 2010; Hart, 2006; Hart & Staveland, 1988; Laubheimer, 2018). After 35 years of development, Helton et al. (2022) argue that Physical Demand should be removed because it may not be meaningfully combinable with cognitive demands.

Perceived usability predicts users’ willingness to adopt mobile banking (Agyei et al., 2020). System Usability Scale (SUS) is considered an inexpensive and effective tool for evaluating the ease of use of a system (Tassabehji & Kamala, 2012). The mechanism is the most widely used questionnaire in academia and industry for perceived usability. Studies have shown that SUS has high reliability validity and is suitable for assessment in different contexts (Peres et al., 2013). Since its development in 2008, SUS’s position as an assessment tool for perceived ease of use is still considered robust and stable (J. R. Lewis, 2018).

The Net Promoter Score (NPS) was translated into a new metric by Reichheld (2003) with a single question, “How likely are you to recommend [OOO] to a friend or colleague?” This metric predicts future business growth better than previously used customer satisfaction or loyalty metrics. Since then, NPS has been widely used in different industries, including consumer marketing, IT services, healthcare, and more, and as a powerful indicator of customer loyalty. Its strength lies in its ease of management and understanding, and it helps to understand customer satisfaction with a service or product (Owen, 2019). Baehre et al. (2022) have demonstrated NPS has the greatest predictive value when predicting sales growth. Participants will answer “Willingness to introduce this product to your family or friends in the future” based on their experience and are scored on a 10-point scale from very reluctant (0) to very willing (10). The results are based on the score, and the participants are divided into Detractors (0–6 points), Passives (7/8 points), and Promoters (9/10 points). Past studies found correlations between ease of use and NPS (Friedman & Flaounas, 2018; Sauro, 2010). User experience variables contribute 32%–40% to 32%–40% of the likelihood that users will recommend a product (Bradner & Sauro, 2012).

Summary

Based on the above discussion, this study will start from visual layer design to the UX improvement of ATM cardless service. Cognitive walkthroughs and post-task/post-session user self-reports were used to gain insight into the problems to be solved to achieve the objectives of this study.

Related Work

The number of ATMs in Taiwan exceeded 30,000, and the three banks with the largest number were selected as the research objects in the early stage of this study, namely Bank A (6,023), Bank B (4,191), and Bank C (3,647). We collated and analyzed the User Flow of these three ATMs, including the application process of cardless service and the smartphone cardless withdrawal process. The investigation process at this stage is the same as in the following research method (refer to Figure 2). The purpose is to find out the problems from the interface design of these three companies, select the interface of one ATM, and sort out the design direction and conditions that need to focus on optimization as the basis for experimental design.

There were 30 participants in this stage and participants aged 20 to 39 who had not used cardless services. The result is that Bank B has the best performance among the three banks, so we choose B to optimize further and modify the user experience process and interface components of the ATM cardless cash interface and propose two proposals for different ATM operation experiences, which are used as materials for later experiments. In this study, we knew from the willingness to use an assessment that the willingness to use cardless withdrawal after the operation of the three bank participants significantly decreased by 30% to 60%. The ATM cardless withdrawal interfaces of the three banks cannot fully meet the operational needs of the test participants, and there is still room for improvement.

In the survey results of this stage, we synthesized the three problems and proposed solutions and UX goals presented in Table 1.

Table 1.

The Solution to the Current Problem, the Heuristic Evaluation, and the UX Goal.

UX Dimension	Problem	Solution	Usability heuristics	UX goals
Information zarchitecture	ATM cardless services require an application portal.	First, distinguish the functions provided by “Customers of our bank” and “Customers of other banks,” arrange the “Cardless Service Application Function” on the homepage, and present this one-time “Application Function” with a special button. In addition, the entrance to “Application Service” is also provided on the “Cardless Withdrawal Function” page.	—	Usable Findable
	Tips and guidelines for applying are required.		Help and documentation
	The application service function is arranged in the fourth layer of the information architecture sub-function and is not easily discovered.		Flexibility and efficiency of use
	The sequence of steps is not as expected, for example, Choose “Smartphone Withdrawal” before being asked if he/she is a customer of the bank or another bank.		Error prevention
	The naming of function buttons is not easy to understand and even misunderstood.	Rename the button to match the function (renamed “Smartphone Withdrawal” to “Cardless Smartphone Withdrawal”; “Smartphone Serial Number” to “Cardless Smartphone Serial Number”). Classify buttons into primary, secondary, Call-to-Action (CTA), and special buttons and give them different colors.	Match between the system and the real world
Visual hierarchy	The arrangement of button colors distracts the user’s visual focus from the task.		Aesthetic and minimalist design	Desirable
	The serial number of the smartphone cardless withdrawal is a lengthy random code, resulting in a large burden of memorizing and repeatedly confirming the serial number in the process of entering the serial number.	Use “chunking” to divide the 8-code withdrawal serial number into two units.	Error preventionRecognition rather than recall	Desirable
	The visual hierarchy of page information is complex and flat, and the readability is low, resulting in users not understanding.	Readjust visual hierarchy with color, size, font weight, and gestalt principles.	Aesthetic and minimalist design	DesirableUsable
	Smartphones and ATMs are not aligned in visual elements.	Unify icons and naming between smartphones and ATMs.	Consistency and standards	DesirableUsable
	The user’s sense of control is low, and, e.g., returning to the previous page should be fixed in the same position on each page.	Pin “Back to Previous Page” in the upper left corner.	User control and freedomConsistency and standards	Usable
Guidance and assistance	During the operation, it is not possible to confirm that the operation has been performed with the remaining steps.	Provide a “progress bar” for users to confirm the steps that have been completed so far and those that have not been completed.	Visibility of system status	Findable
	Lack of guidance prompts and multiple attempts are required to obtain the mobile cardless withdrawal serial number.	A “How to Obtain Serial Number” button is provided at the bottom left of the withdrawal page to allow users to obtain assistance.	Help and documentation	FindableUsable
	The timing of guidance needs to be well organized.		Help and documentation	Findable

Materials

Based on the above-mentioned survey, the ATM of one of the best-performing banks was selected. The user experience process and interface components in the ATM cardless withdrawal interface were further optimized and corrected, and two proposed designs were proposed for subsequent experimental verification.

The bank’s smartphone app interface encountered fewer problems during the experiment but more problems with the ATM interface. There are two main problems: first, the hierarchical structure of the entry configuration of the cardless service application affects the time of the subject’s operation task, and the other is the lack of appropriate guidance instructions when switching the carrier (ATM and smartphone). Not only did the participants not know how to complete the task, but they also spent more time completing it. Therefore, we optimized the material design based on the survey results (please refer to Table 1). For details on the new design and the arrangement of the two sets of task pages, please refer to Figure 1.

Figure 1.

Design and process of two sets of tasks.

Typography A continues the traditional form of interface information. The operation interface of ATM adopts the “Center Stage” pattern, maintaining the principle that one interface focuses on one thing, and the user’s operation matters are displayed in the center of the interface. On the cardless withdrawal homepage, the Z-Layout pattern maintains the reading order from top to bottom and places the application entry in the lower right corner. On the Cardless Withdrawal Verification page, the way the user can choose to authenticate for the Cardless Withdrawal is listed.

Typography B is divided into two sections. Typography B establishes the structure of the publication surface by the order of the left-to-right visual hierarchy and uses the size to determine the visual weight. According to the importance and relevance of the information, the information is divided into chunks to highlight the main information on the right. For example, applying for a cardless service is usually a one-time operation, classified as secondary information. On the cardless withdrawal homepage, the application entry is configured on the left side of the interface, and the cardless withdrawal-related operations are configured on the right side. On the cardless withdrawal verification page, provide the cardless withdrawal authentication method the user wants to choose, then proceed to the authentication operation.

Guidance a (Ga) uses a numerical list to present the six steps while bolding the font and using darker color blocks to highlight important keywords. GA is equipped with a small icon with the same serial number as the smartphone for users’ reference and use, hoping to help users quickly capture the guide, continue to operate on the smartphone app and obtain the smartphone cardless withdrawal serial number.

Guidance b (Gb) is mainly icon-based, with the same large icon as the serial number of the smartphone, presenting six steps in the way of focusing on one step on one slide page, and users can tap the left and right arrow buttons on the interface to switch between the front and back pages. At the same time, important keywords are highlighted in bold font for users’’ reference and use, hoping to help users follow the guidance confidently and without panic, continue to operate on mobile online banking, and obtain the mobile cardless withdrawal serial number.

Method

The experiment was divided into four phases: the first stage was the participant’s personal background and ATM/cardless cash withdrawal-related experience questionnaire; the second stage was task-oriented cognitive walkthroughs; the third stage was post-mission assessment, including SEQ, NASA-TLX, satisfaction, and preference; the fourth stage is to evaluate overall usability with SUS, willingness to use score and NPS, and semi-structured interviews are conducted after completion of the assessment (Figure 2).

Figure 2.

Experiment progress.

Task Design for Cognitive Walkthroughs

The task design of the cognitive walkthrough was mainly divided into two tasks: Task 1 was the cardless service application, and Task 2 was the cardless smartphone cash withdrawal. A researcher was responsible for chairing, observing, and documenting. The exercise provided two scenarios with the following objectives and task objectives:

Task 1 was to apply for cardless services for the first time. The scenario was “You have recently learned about the trend of cardless services as fintech. To experience the new technology, try applying for a cardless service in case you need it.” The goal of the task was to complete the service activation of “Cardless smartphone Cash Withdrawal” and see the “Service Successfully Activate” screen.

Task 2 was to complete cardless withdrawals. The situation was, “You’re about to go out for lunch, and when you get to the restaurant, you find that your wallet was left at home. To purchase lunch smoothly, please withdraw 1,000 dollars via smartphone serial number without a card.” The goal was to obtain the withdrawal serial number from the smartphone interface and enter it in the ATM interface.

Since the material contained two different typography (A/B) and two guidance (Ga/Gb) designs, it was divided into four groups: A-Ga, A-Gb, B-Ga, and B-Gb. Each participant only performed one of these sets.

Participants

A total of 36 participants, aged between 20 and 29, were recruited, none of whom had experience in cardless services. Based on the results of the ATM-related experience survey in the first stage, we divided the participants into three groups of low, medium, and high experience and evenly distributed them into four groups; that is, each group had nine participants, three each with low experience, medium experience, and high experience.

Equipment and Location

This study was affected by the global epidemic of COVID-19, and it was impossible to conduct physical experiments with participants, so it was verified through remote experiments. The interactive interface was designed in Figma. We used Google Meet online conferencing software for remote calls. The researcher shared the experimental operation interface through the computer remote control software AnyDesk and granted the participants’ screen control permissions. Before the experiment began, researchers confirmed that the ATM screen display window width was maintained at 1,024 × 768 pixels, and the smartphone interface simulated the screen size of the iPhone 11 Pro. All scale assessments were completed in electronic questionnaires after the session.

Design of the Scale

SEQ is based on the Likert 7-point scale: one is extremely easy, and seven is extremely difficult. After the researcher announced each task to the subject, the participant was asked, “How easy is it that you expect to complete Task 1/Task 2?” After each task was completed, the test participant was asked again, “After completing the task, how do you think the difficulty of Task 1/Task 2 is?” The satisfaction question was “How satisfied are you with the ATM layout/guide screen” and the preference question was the multiple-choice question “What is your preference for the layout/guide” The participants only sew all the designs at this time, and must choose one from each of A and B, Ga and Gb. Please refer to Appendix 1 for the SUS Scale (10 questions). NASA-TLX is available on NASA’s official website (https://humansystems.arc.nasa.gov/groups/tlx/). The question of willingness to use is “Based on your experience just now, how do you want to use cardless withdrawals in the future.” The NPS scaled from 1 to 10 and asked the subject, “Will you introduce this feature to your family or friends in the future.”

Results

In this experiment, four groups were proposed: A-Ga, A-Gb, B-Ga, and B-Gb, and 36 participants were obtained.

Cognitive Walkthrough

Before performing the cognitive walkthrough, participants estimated that the average time it would take to complete Task 1, “Cardless Service Application,” was 3 min and 43 s. The results of Task 1 show that the average operation time of Typography A is 64 s, while the average operation time of Typography B is 59 s. However, compared with A, although B’s average operation time is shorter, B’s number of errors (seven times) is higher than A’s (six times). However, statistics show no significant difference between A and B. In A and B, we provide two application portals in the information structure of the ATM cardless withdrawal service, one on the cardless withdrawal first page and the “?” button on the cardless withdrawal verification page. In A, 13 participants applied on the home page, while five chose to apply from the cardless withdrawal verification page. In B, 17 participants applied on the homepage, and only one chose to apply from the cardless withdrawal verification page, which is why the average task operation time is lower than A.

Before Task 2, participants estimated that the average time to complete a withdrawal was 2 min and 36 s. The statistics of the results of Task 2 showed that the average operation time of Ga was 65 s, and the average operation time of Gb was 90 s. Using t-test analysis, it was found that there was a significant difference between them (p = .002 < .01). After interviews, it was learned that the Gb page focuses on one-step guiding, which requires more interaction and time cost for participants to understand the method of obtaining the withdrawal serial number, which is the reason why the operation time of Gb is significantly higher than that of Ga.

In terms of the number of errors, Ga had a total of five errors, while Gb had four errors. The results show that although Ga has more errors, participants have a shorter operation time for Ga’s cardless withdrawal task. In addition, two participants from Gb encountered touch performance due to hardware issues, which caused them to tap the next step arrow multiple times without noticing, while one participant from Ga made an error because they missed a step. However, the statistical analysis results show no significant difference between Ga and Gb regarding the number of errors.

We further explored the time it takes to use the guidance and calculate the time after participants click the “How to get the withdrawal serial number” button, finish reading the guidance, and click the “I Got It” button at the bottom of the interface (Table 2). The statistical results found that participants spent 20 s on Ga and 41 s on Gb. Using t-test analysis, it was found that there was a significant difference between them (p = .000 < .001).

Table 2.

Task Performance: Operation Time and Number of Errors.

Record items	Task1: A	Task1: B	Task2: Ga	Task2: Gb	Reading guide: Ga	Reading guide: Gb
Ave. operating time (unit: second)	64	59	65**	90**	20***	41***
Error (unit: times)	6	7	5	4

Note. N = 36.

p < .01. ***p < .001.

Post-Task Self-Report Assessment

SEQ Scale

The SEQ questionnaire at this stage explored the ease of participants in operating two tasks. Participants were asked to fill in the estimated difficulty of the task before the task and evaluate it again after the task. Finally, calculated its growth rate.

The statistical results show that, as shown in Table 3, in Task 1, the average task difficulty before the A task was 3.3 points, the average task difficulty after the task was 1.4 points, and the task difficulty was reduced by 57.6%. B’s average task difficulty before the task was 2.8 points, B’s estimated average task difficulty before the task was 1.5 points, and the post-task difficulty was reduced by 46.4%. It can be seen from this that the task difficulty of the two proposals is easier than participants imagined. However, in Task 1, B is slightly more difficult for participants.

Table 3.

Experiment 2: Task 1: Average Score and Growth Rate of SEQ Single Task Difficulty for Each Page Proposal.

Group	Average estimated difficulty	Average assessment difficulty	Growth rate
A	3.3	1.4	−57.6%
B	2.8	1.5	−46.4%
Ga	3.2	1.6	−50%
Gb	2.7	1.9	−29.6%

Note. N = 36; 7-point Likert scale, ranging from very easy at 1 point to very difficult at 7 points.

NASA-TLX Workload Assessment

This section first analyzes participants to evaluate the weights of the five workloads. The calculation results show that, as shown in Table 4, Mental Demand (MD), Temporal Demand (TD), Frustration (FR), and participants are the three main indicators that they pay attention to when using ATM.

Table 4.

Weights for NASA-TLX Workload.

	Total weight	Average weight	Ranking
Mental demand	120	3.3	3
Time demand	137	3.8	1
Performance	92	2.6	4
Effort	53	1.5	5
Frustration	130	3.6	2

In Task 1, cardless service application, the average workload of 18 participants for A is 16.81, while the average workload of the other 18 participants for B is 21.52. From this result, the workload of A is lower than that of B, and the average load of MD, OP, EF, and FR, respectively, is lower than that of B. Under the independent sample t-test, there is a significant difference between A and B in the MD index (p = .038 < .05). This study learned from interviews that B differs from the previous ATM interface layout. Low-experience participants are less accustomed to the sequential layout of the left-to-right visual hierarchy and spend more brainpower understanding the new interface.

In Task 2, cardless cash withdrawal, the average value of the Ga-weighted workload of the 18 participants is 19.31, and the average value of the Gb workload of the 18 participants is 21.18. This result shows that the workload of Ga is lower than that of Gb, and the average load of TD and OP is lower than that of Gb, respectively, but it is not statistically significant. It was learned from the interviews that the higher interaction cost of Gb increases the time required for participants to operate tasks and is reflected in the performance of operational tasks (Table 5).

Table 5.

NASA-TLX Workload Weighted Average Scores for Tasks 1 and 2.

	Task 1: A	Task 1: B	Task 2: Ga	Task 2: Gb
Mental demand	56.67*	100.56*	49.44	33.61
Time demand	87.22	72.22	97.22	130.56
Performance	37.50	39.44	33.33	58.89
Effort	15.83	26.67	23.33	16.39
Frustration	52.06	90.31	88.13	73.61
Overall score	16.81	21.52	19.31	21.18

Note. N = 36.

p < .05.

Satisfaction and Preference Assessment

Participants were asked to rate their satisfaction with their assigned design proposal and operation process. Satisfaction was scored using a 7-point Likert scale, with one being strongly dissatisfied and seven being strongly satisfied. When the preference is a multiple-choice question, let participants look at all interfaces simultaneously during the assessment (that is, participants only saw another set of unused designs at this time), and choose one from A and B; choose one from Ga and Gb.

We can find from Table 6 that participants have higher satisfaction and preference for the use of A. Through interviews, it was found that participants are more accustomed to A’s operating behavior and reading order, and using an interface that focuses on one thing makes them feel more comfortable and at ease. However, some participants in B mentioned that the interface presentation is innovative and fresh due to the usage habits of this layout and that they can successfully complete tasks after becoming familiar with the operations.

Table 6.

Satisfaction Score and Preferred Number of People for Each Design.

	Task 1: A	Task 1: B	Task 2: Ga	Task2: Gb
Ave. satisfaction	5.9	5.4	5.5	5.4
No. of participants	22 (12)	14 (8)	24 (13)	12 (7)

Note. N = 36; 7-point Likert scale, ranging from strongly dissatisfied at 1 point to strongly satisfied at 7 points.

It can be found from Table 6 that participants have higher satisfaction and preference for the use of Ga. It was learned from the interviews that 67% (12) Ga participants believed that the bold fonts and darker color blocks of the guidance instructions, combined with the icons, effectively highlighted important keywords and helped participants quickly understand the information of the guidance instructions. However, some Gb participants believed that the interaction cost of constantly checking back and forth between ATMs and mobile online banking increased their burden.

The Difference Between High and Low Experience

This section mainly presents the impact of typography and guidance proposals on participants’ task performance with different ATM usage experiences. We conducted a t-test analysis on the results of NASA-TLX and added the opinions and data from interviews with participants after the mission for reference.

Comparison of Typography A and B

The overall mean value of low-experienced participants in Typography A (M = 20.17) is lower than that of B (M = 17.08). Among them, the mean score of A (M = 77.50) in Mental Demand is lower than that of B (M = 185.83), and there is a significant difference (p = .008 < .01). Low-experienced participants of B said that B is different from the previous ATM interface. Because the participants are not used to the reading order of this page, they need to spend more mental energy to understand the new interface. They further stated that they were worried that making mistakes during the operation of this layout would cause their bank cards to be withdrawn, so they did not have the confidence to successfully complete the task under unfamiliar B.

The average workload of high-experience participants in B (M = 9.25) is lower than that in A (M = 15.62). The mean value of Temporal Demand (TD; M = 35.83) is lower than that in A (M = 97.50), and there is a significant difference. (*p = .012 < .05). B’s participants particularly emphasized that the progress bar on the left side of the interface during the cardless service application process allows them to clearly confirm the completed steps and effectively confirm their own progress, which helps them evaluate the time it takes to complete the subsequent steps. In addition, six participants said that the information presented by B is more distinct on a visual level, thus helping experienced users to retrieve necessary information quickly and effectively from left to right of the ATM interface. It was also learned from the interviews that five A’s high-experience participants needed to switch between a physical numeric keypad and a virtual numeric keypad during the operation process, which increased their mental load during the operation task.

Comparison of Ga and Gb

The mean value of the workload of low-experience participants in Ga (M = 29.54) is higher than that in Gb (M = 22.29), but there is no significant difference between them. During the interview, four of Ga’s low-experience participants stated that the amount of information arranged on one page with all the guidance information was relatively large, so they were worried about accidentally omitting one of the steps during the operation and causing errors, and they needed to operate more carefully.

The mean value of workload of high-experience participants in Ga (M = 11.43) is lower than that in Gb (M = 18.42), where the mean value of TD in Ga (M = 49.17) is lower than that in Gb (M = 147.50) with a significant difference (**p = .009 < .01). Nine high-experience participants believe that Gb’s one-page with a one-step guide made them think that the operation process is more complicated and rigorous, so they read each step carefully to ensure that no step is missed. However, most high-experience participants felt that the steps were easier than expected after completing the operation, making them feel that the time it took to complete the task could have been faster and shorter. What is more worth mentioning is that five high-experience subjects in Gb did not continue to understand the instructions at the ATM after completing step one and successfully completing the task. In addition, after completing step one, some high-experience participants returned to the ATM to quickly understand the remaining steps before returning to their mobile phones to obtain the serial number. They further explained that the steps to obtain the serial number were relatively simple, so this design increased the interaction cost for them to constantly check back and forth between the ATM and their smartphones.

Correlation Between Task Performance and Scales

Spearman correlation test of task performance with NASA-TLX, satisfaction, preference, willingness to use, usability, and NPS. We found that operation time was positively correlated with workload and negatively correlated with preference in both tasks. But we can also see that satisfaction, preference, and ease of use are not correlated with NPS in either task (Table 7).

Table 7.

Spearman Correlation Test of Task Performance with NASA-TLX, Satisfaction, Preference, Willingness to Use, Usability, and NPS.

	M	SD	1	2	3	4	5	6	7	8
Task 1: Apply service
1. Time	61.61	17.60	—
2. Error times	0.36	0.49	.635**	—
3. NASA-TLX ave.	19.17	10.93	.668**	.415*	—
4. Satisfaction	5.69	1.17	−.344*	−.12	−.515**	—
5. Preference	1.42	0.50	−.638**	−.635**	−.646**	−.037	—
6. SUS	91.18	8.14	−.595**	−.338*	−.459**	.184	.540**	—
7. Willing to use	6.11	0.71	−.16	−.046	−.353*	.595**	−.044	.199	—
8. NPS	8.69	0.95	−.158	−.234	−.231	.031	.322	.232	−.02	—
Task 2: Cash withdrawal
1. Time	79.44	25.35	—
2. Error times	0.25	0.44	.359*	—
3. NASA-TLX ave.	20.25	9.05	.393*	.183	—
4. Satisfaction	5.53	1.50	−.21	.141	−.595**	—
5. Preference	1.33	0.48	.544**	.136	.607**	−.491**	—
6. SUS	91.18	8.14	−.276	−.505**	−.330*	.187	−.104	—
7. Willing to use	6.11	0.71	−.03	.092	−.152	.266	−.281	.026	—
8. NPS	8.69	0.95	−.315	−.428**	.026	−.124	.105	.242	−.033	—

Note. Using Spearman correlation testing, sample size N = 36; M is mean; SD is standard deviation; 1 to 8 is rho value.

p < .05 (two-tailed). **p < .01 (two-tailed).

Comparison of the Original Interface with Two Sets of Interfaces Optimized for the Design

Comparison of Task Performance with SEQ

This section compares the participants’ average operation time and the number of errors in operating the two tasks between the new designs and the bank’s original interface. And use one-way ANOVA variation statistical analysis to confirm whether there is a statistically significant difference between them.

In Task 1, A and B are better than the original interface regarding operation time and number of errors. Moving the cardless service application entrance to the first page can reduce the time and number of errors required to complete the task, allowing participants to complete the task on the first page confidently.

We further used one-way ANOVA variation statistical analysis to confirm whether there is a statistically significant difference in the operation time and number of errors between the original interface and new designs. The results showed that the operation time (***p = .000 < .001) and the number of errors (**p = .002 < .01).

In addition to the operation time of Gb being higher than the bank’s existing interface, the number of errors of Ga and Gb is also higher than that of the bank’s existing interface. Some participants in Gb encountered unresponsiveness in the interface, causing participants to touch accidentally. Therefore, the number of errors caused by the two guidance proposals is higher than that of the original interface. In addition to Ga being better than the original interface in terms of operation time, the SEQ scores of Ga and Gb are also better than the existing interface. This study infers that guidance instructions help participants effectively obtain the withdrawal serial number. Therefore, even if the Ga has many errors, it can effectively reduce participants’ time to complete the task (Table 8).

Table 8.

The Average Operation Time, Number of Errors, and Average Scores of SEQ of Individual SEQ Tasks in the Original Interface and Proposals.

	Task 1			Task 2
	Original interface	A	B	Original interface	Ga	Gb
Ave. operating time (unit: second)	117***	64***	59***	88**	67**	92**
Error (unit: times)	14**	6**	7**	2	6	4
SEQ	1.9	1.4	1.5	2.1	1.5	1.6

Note. One-way ANOVA was used; N = 36; M is mean; SD is standard deviation.

p < .01 (two-tailed). ***p < .001 (two-tailed).

On the other hand, after one-way ANOVA analysis of task operation time, number of errors, and SEQ, the results showed that task operation time (**p = .000 < .001) and number of errors (p = .002 < .01). All are statistically different, but not different from SEQ scores (p = .258 > .05). Task 2 has a significant difference only in operation time (p = .009 < .01).

Post-Session Self-Report Assessment

SUS

The questionnaire consists of 10 questions; each question is a sentence describing the system being evaluated and uses a 5-point Likert scale based on the user’s subjective feelings, ranging from strongly disagree (one point) to strongly agree (five points). Through a specific algorithm, the score range will fall between 0 and 100 (Brooke, 1996). Past research found that SUS can be used as a scoring standard through the Curved Grading Scale (CGS; Sauro & Lewis, 2011).

The average SUS scores of A-Ga, A-Gb, B-Ga, and B-Gb are shown in Figure 3. The SUS scores of the four groups of proposals are ranked as B-Ga > B-Gb > A-Gb > A-Ga. The results showed that the SUS scores of the four groups of proposals after optimizing the cardless withdrawal interface all fall into the A Grade. Among them, the average SUS score of proposal A-Ga is 90.56, the average SUS score of proposal A-Gb is 90.83, the average SUS score of proposal B-Ga is 92.22, and the average SUS score of proposal B-Gb is 91.11.

Figure 3.

SUS results of original design and four proposals.

The average score for Q5 in the SUS questionnaire (I think the functions of the cardless service are well integrated) is as high as 4.7 points. Most participants said that the interface successfully provided guidance instructions at appropriate steps, and the switching process between the ATM and mobile online banking was smooth, helping them to complete the task smoothly. In addition, the SUS questionnaire’s average score for Q9 (I am confident that I can use cardless services) is as high as 4.9 points. Through interviews, this study learned that progress bars helped recall and confirm completed steps or progress and evaluated the time required to complete subsequent steps, which increased participants’ confidence in operating tasks.

After the optimization, the overall average usability score of the ATM cardless withdrawal interface has been significantly improved compared to the original interface, and the SUS level has been improved from B Grade to A Grade. This study further analyzed the individual scores of the subjects. Four participants rated the original interface SUS as A Grade, and as many as 29 participants rated the optimized interface SUS as A Grade. Therefore, it can be explained that the optimized ATM cardless withdrawal interface optimization is more user-friendly regarding usability and user’s operating behavior and needs.

“Willingness to Use.”

The statistical results show that the usage intention of the four groups of proposals has significantly improved after operating the interface. The average willingness score ranking after use is A-Ga > A-Gb > B-Gb > B-Ga, and the average willingness growth rate ranking is B-Ga > B-Gb > A-Gb > A-Ga. The average scores and growth rates of the four groups can be found in Table 9. It can be seen from the results that A-Ga’s willingness to use after interface operation is the highest among the four groups of proposals. However, although B-Ga’s usage intention after interface operation is the lowest among the four groups of proposals, it has the highest growth rate.

Table 9.

Compare the Willingness and Growth Rate of the Bank’s Existing Interfaces and the Four Group.

	Willingness to use before experimentation	Willingness to use after experimentation	Growth rate
Original interface	4.4	2.9	−34.1%
A-Ga	3.4	6.4	+87.1%
A-Gb	3.3	6.1	+83.3%
B-Ga	2.8	5.9	+120.8%
B-Gb	3.0	6.0	+100%

Note. 7-point Likert scale, ranging from strongly disagree at 1 point to strongly agree at 7 points.

The willingness to use the optimized interface after operation has been significantly improved compared to the original interface. The intention to use the original interface dropped by 34.1%, while the intention to use the optimized interface increased by 94.6%. Therefore, it can be explained that the optimized interface will make users more willing to use cardless withdrawals in the future.

NPS

NPS scores are divided into three broad categories: Promoters (9–10), Passives (7–8), and Detractors (0–6). The NPS total partition range is from −100% to +100%. NPS > 0 indicates more Promoters than Detractors, while NPS < 0 indicates poor satisfaction, with more Detractors than Promoters (Melnic, 2016). The NPS calculation formula is as follows. Table 10 compares the data from the original interface with the four sets of proposals.

Table 10.

The Comparison of NPS and the Number of Each Category Between the Original Interface and Four Proposals.

	NPS	Detractors	Passives	Promoters
Original interface	20	2	4	4
A-Ga	45	1	3	5
A-Gb	78	0	2	7
B-Ga	78	0	2	7
B-Gb	56	0	4	5

Note. NPS = Promoters %—Detractors %.

The overall NPS of the optimized interface has been significantly improved compared to the original interface. Further analysis of the NPS levels of participants shows that the original interface and the optimized interface “Promoter” have 4 and 24 participants, respectively, accounting for 40% and 66.7% in total. Therefore, it can be explained that the optimized interface makes users more willing to recommend this cardless cash withdrawal service to their relatives and friends.

In addition, compare the average NPS of the original and optimized interfaces. The results show that the average NPS score of the original interface is 7.8 points, while the average NPS score of the optimized interface is 8.7 points. After independent sample t-test analysis, the confirmation results showed a significant difference between the two (p = .037 < .05).

Discussion

This study selected the bank with the best performance from the three banks with the largest ATMs in Taiwan as the optimization target. We adjusted the information structure, used visual hierarchy design methods, and added assistance mechanisms to optimize the bank’s ATM cardless withdrawal problem. Below, we discussed the impact of visual hierarchy design on the distribution of participants’ attention, the differences between people with high and low experience, and data-driven user experience design.

The Impact of Visual Hierarchy Design on Attention Allocation

Unlike web pages, the ATM interface operates in a real environment under the pressure of queuing, safety considerations, and no real human assistance. How to reduce users’ operation time and error rate tests the effective allocation and guidance of ATM designers to user attention in visual design. The optimized proposals of this study use a visual hierarchical design to influence the distribution and control of the participant’s attention. In task-oriented cognitive walkthroughs, current goals (goal-driven) are undoubtedly the main deployment of attention distribution in tasks. Therefore, the flattening of the information architecture hierarchy is not a disadvantage in the functional characteristics of the ATM interface. For example, we have moved the cardless withdrawal request portal to the home page. Dramatically reduce the time and error rate users spend searching for this feature. Even if the information architecture is flat, we still use color to design the difference in visual search and recognition on the same page, which can help users distinguish the main function, Call to Action, and other secondary function buttons at the stage of perception. This virtually reduces the user’s time and workload (Figure 4).

Figure 4.

The color design of the buttons expresses the information architecture; on the left is the existing interface of the bank, and on the right is the optimized design.

In addition, the “consistency” of design reduces the gap between the participants learning the new interface. For example, in the B, the Z-Layout is designed to meet the visual search characteristics of the regular user browsing the digital layout. From the title at the top of the page, in the order of reading in Z-shaped order, to the “Call to Action” button in the lower right corner. This is to “history-driven” attention distribution to make the test subject more intuitive when scanning (Figure 5).

Figure 5.

Z-layout reading order.

However, we even moved the “Apply Now” button to the first page, giving it a special button color; it was originally expected to be able to deploy physical salience (stimulus-driven) attention. Still, it turned out that participants were affected by history-driven attention, triggering the “Banner Blindness” phenomenon in 39% (7) participants. The application service areas (the content of the blue box in Figure 6) of A and B were misjudged as advertising areas. How to attract attention without being affected by “Banner Blindness” may be discussed in more depth in future visual design research.

Figure 6.

The area in the blue box that causes “Banner Blindness.”

In addition to the design of buttons, using the visual design “chunking” skill for information is a way to solve cognitive load. Recoding smaller units of information into larger units (Thalmann et al., 2019) adjusts the user’s visual order in terms of visual perception and cognitively reduces working memory (Allen et al., 2021; Farooqui et al., 2023; Norris & Kalm, 2021). Designers use the visual hierarchy of “chunking” to guide the user’s attention, visual search order, and information processing. This practice showed its effectiveness in the eight-digit withdrawal serial number that requires the user’s short-term memory during the task and the need to operate back and forth between the smartphone and the ATM. 39% (14 participants) of participants proposed “chunking,” which helped them memorize and enter serial numbers quickly and rhythmically, reducing the load they needed to memorize them back and forth (refer to Figure 7).

Figure 7.

Numbers are easy to remember when chunking.

In addition, 50% (nine participants) said they ignored icons and focused mainly on text. Past studies (Kozak et al., 2021) have found that the indirect regulation of human visual memory, in the competition between text memory and picture memory with shared resources, the instruction to remember text will reduce the intensity of picture memory. However, these participants said in interviews that they chose only to read words under the dual pressure of queuing and financial risk.

Human-Centered Design (the Difference Between Low-Experienced and High-Experienced)

One of the interesting findings of this study is that people with high and low experience in using ATMs have different performances on different designs. The “center stage” design of the B layout allows low-experienced participants to reduce learning and focus on the main operations on each page. The difference in analysis results is reflected in their mental demand. The average operation time of the low-experienced person in A layout is shorter, and the mental demand is less, resulting in their preference for the A layout design, higher satisfaction, and higher evaluation of usability. This is also reflected in the design of the guide page in Task 2, which is a one-by-one step-by-step operation, and although it took longer for low-experience participants, they are one score more satisfied with Gb than Ga, and twice as many people prefer Gb as Ga, although they are not statistically significant.

Conversely, for highly experienced participants, “time” seemed what they cared about most. In Task 1, layout B’s average operation time is less than layout A’s. But even an eight-second gap is reflected in their temporal demand and has a significant difference (A: M = 97.5, B: M = 35.83, p = .012 < .05). The temporal demand of highly experienced people is the only significant difference in other load items. Similarly, the time spent by highly experienced people operating the guidance page in Task 2 was statistically significant. The Gb six steps are distributed to six pages, adding to their temporal demand (Ga: M = 49.17, Gb: M = 147.50, p = .009 < .01). Instead, Ga listed six steps on the same page, using visual hierarchy design techniques to enhance the clarity of information, so that the satisfaction of high-experienced people with Ga (M = 6.5) was 1.5 higher than that of Gb. It is directly reflected in the preferred choice: Ga has 11 people, and Gb only one person.

From the above discussion, we can also find that Gb uses the “Slide” pattern to change pages. The design pattern of progressive disclosure (Carroll & Carrithers, 1984; Nielsen, 1994; Spillers, 2004) is gradually exposed, providing a better experience for low-experience and even novices. It is worth mentioning that the “progress bar” designed by the progressive disclosure mode is used. This is the biggest difference between the optimization design of this study and the original bank interface. 69% (25) of the participants believed that the progress bar helped recall and confirm the completed steps or progress and assess the time it took to complete the next steps, which increased participants’ sense of security during the task operation. 25% (nine participants) believed that the progress bar helped them understand the steps that needed to be performed, and they could judge whether to continue according to the current situation, helping to reduce the participants’ stress during the operation.

Based on the combined results, B-Ga suits high-experience and A-Gb for low-experience users. Interface design can be individualized to align with human-centered design concepts, although not practiced on current systems. However, regarding the depth of personal data collection by banks, coupled with the development and application of artificial intelligence, it is just around the corner to practice interfaces that can meet personal experiences and habits in the future.

Data-Driven UX Practical Advice

This study adopted a commonly used scale in the field of user experience and collected self-reported data from users. Regarding cognitive walkthroughs, objectively observing the correlation between operation time and error rate and post-task load, satisfaction, preference, and usability are worthy of designers’ in-depth understanding and further from the interviews to gain insights. However, judging from the analysis results of this study, we do not recommend that only a single assessment of “usage intention” and “net promoter score” are often seen in the industry to examine customer loyalty. Customer loyalty does not come purely from user experience but may consider factors unknown to other researchers (J. R. Lewis, 2018), such as customer welfare, security, law, personal information privacy protection, etc. (Ryu, 2018). On the other hand, relying solely on these two scales has its limitations, and it is impossible to obtain differences among users with different experiences. Therefore, we suggest that when the industry is engaged in user experience surveys, they should timely introduce different experience surveys at different stages of interface design to finetune the design. Based on the comparison before and after this study’s design, in the interface’s visual design, corporate designers can aim to improve the usability of products/services to enhance brand loyalty.

Conclusion

After investigating the current situation of ATM cardless services, this study attempts to optimize the ATM interface design with better performance to obtain a more perfect user experience. The verification results prove that the optimized design performs better than the bank’s original interface on all data. We considered heuristic evaluation to improve interface design and Faraday’s Visual Hierarchy Model to optimize the design of the visual hierarchy. The user experience survey used task-oriented cognitive walkthroughs, NASA-TLX, SEQ, satisfaction, preference, SUS, “willingness to use” scale, and NPS. In addition to obtaining task performance and self-reported data and understanding the relationship between the data on each scale, it also understands the differences between high and low-experienced users. Finally, we provide advice to the industry on user experience surveys.

Limitations

The experimental phase of this study coincided with the COVID-19 epidemic, so all remote experiments were adopted, and the situation of ATMs in the actual environment could not be reproduced in the scenario simulation of cognitive walkthrough. In addition, due to the influence of the online experiment under the test, most participants recruited are between 20 and 39 years old, and it is suggested that it can be extended to other age groups in the future to understand their operation situation, willingness, and ideas, and apply it to design and help improve the versatility and future development of ATM cardless services.

Footnotes

Appendix 1 Authors’ Contributions

Methodology, writing - review and editing, resources, and supervision: Meng-Cong Zheng. Conceptualization, investigation: Joon Ming Nigell Lay. Methodology, formal analysis, writing - original draft preparation, visualization: Ching-I Chen.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Meng-Cong Zheng

Ching-I Chen

Data Availability Statement

The data supporting this study’s findings are not openly available and are available from the corresponding author upon reasonable request.

References

Agyei

Sun

Abrokwah

Penney

E. K.

Ofori-Boafo

(2020). Mobile banking adoption: Examining the role of personality traits. SAGE Open, 10(2), 1–15. https://doi.org/10.1177/2158244020932918/ASSET/IMAGES/LARGE/10.1177_2158244020932918-FIG3.JPEG

Albert

Dixon

(2003). Is this what you expected? The use of expectation measures in usability testing [Conference session]. Proceedings of the Usability Professionals Association 2003 Conference.

Allen

M. G.

Destefano

Brady

T. F.

(2021). Chunks are not “content-free”: Hierarchical representations preserve perceptual detail within chunks [Conference session]. Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 43).

Asiyanbi

H. B.

Ishola

A. A.

(2018). E-banking services impact and customer satisfaction in selected bank branches in Ibadan metropolis, Oyo state, Nigeria. Accounting, 4(4), 153–160. https://doi.org/10.5267/J.AC.2018.3.001

Averill

J. R.

(1973). Personal control over aversive stimuli and its relationship to stress. Psychological Bulletin, 80(4), 286–303. https://doi.org/10.1037/H0034845

Baehre

O’Dwyer

O’Malley

Lee

(2022). The use of net promoter score (NPS) to predict sales growth: Insights from an empirical investigation. Journal of the Academy of Marketing Science, 50(1), 67–84. https://doi.org/10.1007/S11747-021-00790-2/TABLES/8

Bei

L. T.

Chiao

Y. C.

(2006). The determinants of customer loyalty: An analysis of intangibile factors in three service industries. International Journal of Commerce and Management, 16(3–4), 162–177. https://doi.org/10.1108/10569210680000215/FULL/PDF

Bianchi

Oakley

Lee

J. K.

Kwon

D. S.

(2010). The haptic wheel [Conference session]. CHI ‘10 Extended Abstracts on Human Factors in Computing Systems (pp. 3625–3630). https://doi.org/10.1145/1753846.1754029

Blackmon

M. H.

Polson

P. G.

Kitajima

Lewis

(2002). Cognitive walkthrough for the web [Conference session]. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 463–470). https://doi.org/10.1145/503376.503459

10.

Bradner

Sauro

(2012). Software user experience and likelihood to recommend: Linking UX and NPS [Conference session]. Experience Professionals’ Association International Conference.

11.

Brooke

(1996). SUS: A “quick and dirty” usability scale. In Jordan

P. W.

Thomas

McClelland

I. L.

Weerdmeester

(Eds.), Usability evaluation in industry (pp. 207–212). CRC Press. https://doi.org/10.1201/9781498710411-35

12.

Camilli

Dibitonto

Vona

Medaglia

C. M.

Di Nocera

(2011). User-centered design approach for interactive kiosks [Conference session]. Proceedings of the 9th ACM SIGCHI Italian Chapter International Conference on Computer-Human Interaction: Facing Complexity (pp. 85–91). ACM Press. https://doi.org/10.1145/2037296.2037319

13.

Carroll

J. M.

Carrithers

(1984). Training wheels in a user interface. Communications of the ACM, 27(8), 800–806.

14.

Chang

(2019). ATM cardless customer experience in banking digitalization, taking Cathay United Bank, CTBC Bank, Taishin International Bank For Example. National Chengchi University. https://ndltd.ncl.edu.tw/cgi-bin/gs32/gsweb.cgi/login?o=dnclcdr&s=id=%22107NCCU5735107%22.&searchmode=basic

15.

Chenlin

S.-H.

(2020, December). Can cardless withdrawals work? China Bank will challenge the use of territory in 2021. Radio Taiwan International. https://www.rti.org.tw/news/view/id/2088013

16.

Churchill

G. A.

Jr. Surprenant

(1982). An investigation into the determinants of customer satisfaction. Journal of Marketing Research, 19(4), 491–504. https://doi.org/10.1177/002224378201900410

17.

Cluckey

(2020). 2020 ATM & self-service software trends. https://www.atmmarketplace.com/resources/2020-atm-self-service-software-trends/

18.

Cockburn

Quinn

Gutwin

(2015). Examining the peak-end effects of subjective experience [Conference session]. Conference on Human Factors in Computing Systems—Proceedings, 2015-April (pp. 357–366). https://doi.org/10.1145/2702123.2702139

19.

CommBank. (2014, April). CommBank launches Australian-first innovations to continue its lead in the mobile banking and payments space. CommBank. https://www.commbank.com.au/about-us/news/media-releases/2014/commbank-launches-australian-first-innovations-to-continue-its-lead-in-the-mobile-banking-and-payments-space.html

20.

Coventry

De Angeli

Johnson

(2003). Usability and biometric verification at the ATM interface [Conference session]. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 153–160). https://doi.org/10.1145/642611.642639

21.

Das

Mackenzie

(1996). Satisfaction with town services: Do user experience and satisfaction count? The Service Industries Journal, 16(2), 191–205. https://doi.org/10.1080/02642069600000021

22.

De Luca

Frauendienst

. (2008). A privacy-respectful input method for public terminals. Proceedings of the 5th Nordic Conference on Human-Computer Interaction: Building Bridges, 358, 455–458. https://doi.org/10.1145/1463160.1463218

23.

Dowling

G. R.

(1986). Perceived risk: The concept and its measurement. Psychology & Marketing, 3(3), 193–210. https://doi.org/10.1002/MAR.4220030307

24.

Faraday

(2000). Visually critiquing web pages [Conference session]. Multimedia’99: Proceedings of the Eurographics Workshop in Milano, Italy, September 7–8 (pp. 155–166). Springer Vienna. https://doi.org/10.1007/978-3-7091-6771-7_17

25.

Farooqui

A. A.

Gezici

Manly

(2023). Chunking of control: An unrecognized aspect of cognitive resource limits. Journal of Cognition, 6(1), 25. https://doi.org/10.5334/joc.275

26.

Financial Supervisory Commission R.O.C. (2013, January 13). “Building digital financial environment 3.0” launched. Financial Supervisory Commission R.O.C. https://www.fsc.gov.tw/ch/home.jsp?id=96&parentpath=0,2&mcustomize=news_view.jsp&dataserno=201501130003&toolsflag=Y&dtable=News

27.

Financial Supervisory Commission R.O.C.—Banking Bureau. (2023, August). Financial Statistics—Banking Bureau. FSC R.O.C.—Banking Bureau. https://www.banking.gov.tw/ch/home.jsp?id=157&parentpath=0,4&mcustomize=bstatistics_view.jsp&serno=201105120012

28.

Friedman

Flaounas

(2018). The right metric for the right stakeholder: A case study of improving product usability [Conference session]. ACM International Conference Proceeding Series (pp. 602–606). ACM Press. https://doi.org/10.1145/3292147.3292224

29.

Garrett

J. J.

(2002). The elements of user experience: User-centered design for the web. Elements, 10, 208. http://www.amazon.com/dp/0735712026

30.

Gordon

(2016, May 31). How cardless solutions help transform the ATM. NCR. https://www.ncr.com/company/blogs/financial/how-cardless-solutions-help-transform-the-atm

31.

Gordon

(2020, March 1). 5 principles of visual design in UX. Nielsen Norman Group. https://www.nngroup.com/articles/principles-visual-design/

32.

Hammoud

Bizri

R. M.

El Baba

(2018). The impact of e-banking service quality on customer satisfaction: Evidence from the Lebanese banking sector. SAGE Open, 8(3), 1–12. https://doi.org/10.1177/2158244018790633/ASSET/IMAGES/LARGE/10.1177_2158244018790633-FIG2.JPEG

33.

Hart

S. G.

(2006). Nasa-Task Load Index (NASA-TLX); 20 years later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 50(9), 904–908. https://doi.org/10.1177/154193120605000909

34.

Hart

S. G.

Staveland

L. E.

(1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in Psychology, 52(C), 139–183. https://doi.org/10.1016/S0166-4115(08)62386-9

35.

Helton

W. S.

Jackson

K. M.

Näswall

Humphrey

(2022). The National Aviation and Space Agency Task Load Index (NASA-TLX): Does it need updating? Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 66(1), 1245–1249. https://doi.org/10.1177/1071181322661370

36.

International Standard. (2019, July). ISO 9241-210:2019(en) Ergonomics of human-system interaction—Part 210: Human-centred design for interactive systems. https://www.iso.org/obp/ui/en/#iso:std:iso:9241:-210:ed-2:v1:en

37.

Jamal

(2007). Retail banking and customer behaviour: A study of self concept, satisfaction and technology usage. The International Review of Retail, Distribution and Consumer Research, 14(3), 357–379. https://doi.org/10.1080/09593960410001678381

38.

Kahneman

Fredrickson

B. L.

Schreiber

C. A.

Redelmeier

D. A.

(1993). When more pain is preferred to less: Adding a better end. Psychological Science, 4(6), 401–405. https://doi.org/10.1111/J.1467-9280.1993.TB00589.X

39.

Kessem

(2018). IBM security: Future of identity study. IBM.

40.

Khalfan

A. M.

Alshawaf

(2014). Adoption and implementation problems of e-banking: A study of the managerial perspective of the banking industry in Oman. Journal of Global Information Technology Management, 7(1), 47–64. https://doi.org/10.1080/1097198X.2004.10856366

41.

King

(2012). Bank 3.0: Why banking is no longer somewhere you go, but something you do. Marshall Cavendish International Asia Pte Ltd. https://books.google.com/books/about/Bank_3_0.html?hl=zh-TW&id=tWKJAAAAQBAJ

42.

King

(2018). Bank 4. 0: Banking everywhere, never at a bank (1st ed.). Wiley.

43.

Kingston

(2020, March 17). Visual hierarchy in design: Key principles & examples. Adobe XD Ideas. Adobe. https://xd.adobe.com/ideas/process/information-architecture/visual-hierarchy-principles-examples/

44.

Kozak

Herz

Bar-Haim

Censor

(2021). Indirect modulation of human visual memory. Scientific Reports, 11(1), 1–7. https://doi.org/10.1038/s41598-021-86550-2

45.

Laubheimer

(2018, February 11). Beyond the NPS: Measuring perceived usability with the SUS, NASA-TLX, and the single ease question after tasks and usability tests. Nielsen Norman Group. https://www.nngroup.com/articles/measuring-perceived-usability/

46.

Lee

Allaway

(2002). Effects of personal control on adoption of self-service technology innovations. Journal of Services Marketing, 16(6), 553–572. https://doi.org/10.1108/08876040210443418/FULL/PDF

47.

Lewis

Polson

P. G.

Wharton

Rieman

(1990). Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces [Conference session]. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Empowering People—CHI ‘90 (pp. 235–242). https://doi.org/10.1145/97243.97279

48.

Lewis

Wharton

(1997). Cognitive walkthroughs. In Helander

M. G.

Landauer

T. K.

Prabhu

P. V.

(Eds.), Handbook of human-computer interaction (pp. 717–732). Elsevier. https://doi.org/10.1016/B978-044481862-1.50096-0

49.

Lewis

J. R.

(2018). The system usability scale: Past, present, and future. International Journal of Human–Computer Interaction, 34(7), 577–590. https://doi.org/10.1080/10447318.2018.1455307

50.

Lin

(2015). Research on the development trend of automated service equipment. Springer.

51.

Lupton

(2015). Graphic design: The new basics. Princeton Architectural Press.

52.

Lustsik

(2004). Can e-banking services be profitable? SSRN Electronic Journal. Working Paper 30-2004. University of Tartu Economics and Business Administration. https://doi.org/10.2139/SSRN.612762

53.

Lyon

A. R.

Coifman

Cook

McRee

Liu

F. F.

Ludwig

Dorsey

Koerner

Munson

S. A.

McCauley

(2021). The cognitive walkthrough for implementation strategies (CWIS): A pragmatic method for assessing implementation strategy usability. Implementation Science Communications, 2(1), 1–16. https://doi.org/10.1186/S43058-021-00183-0/TABLES/4

54.

Mahatody

Sagar

Kolski

(2010). State of the art on the cognitive walkthrough method, its variants and evolutions. International Journal of Human–Computer Interaction, 26(8), 741–785. https://doi.org/10.1080/10447311003781409

55.

Melnic

E. L.

(2016). The science of customer satisfaction in the retail banking system—A critical comparison between the two international indexes: Net promoter score (NPS) and American Customer Satisfaction Index (ACSI). Journal of Emerging Trends in Marketing and Management, 1(1), 215–223. https://econpapers.repec.org/article/aesjetimm/v_3a1_3ay_3a2016_3ai_3a1_3ap_3a215-223.htm

56.

Morville

(2004). User experience design. http://semanticstudios.com/user_experience_design/

57.

NCR. (2021, January). A history of ATM innovation. NCR. https://www.ncr.com/blogs/banking/history-atm-innovation

58.

Nielsen

(1994). Usability inspection methods [Conference session]. Conference on Human Factors in Computing Systems—Proceedings, 1994-April (pp. 413–414). https://doi.org/10.1145/259963.260531

59.

Ning

Goodman-Deane

Clarkson

P. J.

(2019). Addressing cognitive challenges in design—A review on existing approaches. Proceedings of the Design Society: International Conference on Engineering Design, 1(1), 2775–2784. https://doi.org/10.1017/DSI.2019.284

60.

Norris

Kalm

(2021). Chunking and data compression in verbal short-term memory. Cognition, 208, 104534. https://doi.org/10.1016/j.cognition.2020.104534

61.

Oliver

R. L.

(1980). A cognitive model of the antecedents and consequences of satisfaction decisions. Journal of Marketing Research, 17(4), 460–469. https://doi.org/10.1177/002224378001700405

62.

Owen

(2019). Net promoter score and its successful application. In Kompella

(Ed.), Marketing wisdom (pp. 17–29). Springer. https://doi.org/10.1007/978-981-10-7724-1_2

63.

Peres

S. C.

Pham

Phillips

(2013). Validation of the system usability scale (SUS). Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 57(1), 192–196. https://doi.org/10.1177/1541931213571043

64.

Peter

J. P.

Tarpey

L. X.

Sr . (1975). A comparative analysis of three consumer decision strategies. Journal of Consumer Research, 2(1), 29. https://doi.org/10.1086/208613

65.

Poulin

(2018). The language of graphic design: An illustrated handbook for understanding fundamental design principles. Rockport Publishers.

66.

Ranaweera

Neely

(2003). Some moderating effects on the service quality-customer retention link. International Journal of Operations and Production Management, 23(2), 230–248. https://doi.org/10.1108/01443570310458474/FULL/PDF

67.

Reichheld

F. F.

(2003, December). The one number you need to grow. Harvard Business Review, 81(12), 46–55. https://hbr.org/2003/12/the-one-number-you-need-to-grow

68.

Rosenfeld

Morville

Arango

(2015). Information architecture: For the web and beyond (4th ed.). O’Reilly Media. https://www.amazon.com/-/zh_TW/Louis-Rosenfeld-ebook/dp/B015D78JV6?ref_=ast_author_mpb

69.

Ryu

H. S.

(2018). What makes users willing or hesitant to use Fintech?: The moderating effect of user type. Industrial Management and Data Systems, 118(3), 541–569. https://doi.org/10.1108/IMDS-07-2017-0325

70.

Salazar

(2022, February 13). Evaluate interface learnability with cognitive walkthroughs. Nielsen Norman Group. https://www.nngroup.com/articles/cognitive-walkthroughs/

71.

Sauro

(2010, January 7). Does better usability increase customer loyalty?Measuring U. https://measuringu.com/usability-loyalty/

72.

Sauro

Dumas

J. S.

(2009). Comparison of three one-question, post-task usability questionnaires [Conference session]. Conference on Human Factors in Computing Systems—Proceedings (pp. 1599–1608). https://doi.org/10.1145/1518701.1518946

73.

Sauro

Lewis

J. R.

(2011). When designing usability questionnaires, does it hurt to be positive? [Conference session] Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2215–2224). https://doi.org/10.1145/1978942.1979266

74.

Seifert

De Luca

Rukzio

(2012). Don’t queue up! [Conference session] Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia (pp. 1–4). https://doi.org/10.1145/2406367.2406422

75.

Still

J. D.

(2018). Web page visual hierarchy: Examining Faraday’s guidelines for entry points. Computers in Human Behavior, 84, 352–359. https://doi.org/10.1016/J.CHB.2018.03.014

76.

Spillers

(2004). Emotion as a cognitive artifact and the design implications for products that are perceived as pleasurable. Experience Dynamics, 1–14.

77.

Tassabehji

Kamala

M. A.

(2012). Evaluating biometrics for online banking: The case for usability. International Journal of Information Management, 32(5), 489–494. https://doi.org/10.1016/j.ijinfomgt.2012.07.001

78.

Thalmann

Souza

A. S.

Oberauer

(2019). How does chunking help working memory? Journal of Experimental Psychology: Learning Memory and Cognition, 45(1), 37–55. https://doi.org/10.1037/XLM0000578

79.

Theeuwes

(2019). Goal-driven, stimulus-driven, and history-driven selection. Current Opinion in Psychology, 29, 97–101. https://doi.org/10.1016/J.COPSYC.2018.12.024

80.

Walker

(2017, June). Banks rush to bring old-fashioned ATMs into digital age. Financial Times. https://www.ft.com/content/e1c0bc66-5510-11e7-9fed-c19e2700005f

81.

Zhou

(2004). A dimension-specific analysis of performance-only measurement of service quality and satisfaction in China’s retail banking. Journal of Services Marketing, 18(7), 534–546. https://doi.org/10.1108/08876040410561866/FULL/PDF

Pursuing Flawless Experience Starts with the Adjustment of the Visual Layout: ATM Cardless Cash Withdrawal Services Study

Abstract

Plain language summary

Keywords

Introduction

Literature Review

Evaluate the Core Concepts of UX

Visual Hierarchy and User’s Attention

Task-Oriented Cognition Walkthrough

User Self-Report Evaluation

Summary

Related Work

Materials

Method

Task Design for Cognitive Walkthroughs

Participants

Equipment and Location

Design of the Scale

Results

Cognitive Walkthrough

Post-Task Self-Report Assessment

SEQ Scale

NASA-TLX Workload Assessment

Satisfaction and Preference Assessment

The Difference Between High and Low Experience

Comparison of Typography A and B

Comparison of Ga and Gb

Correlation Between Task Performance and Scales

Comparison of the Original Interface with Two Sets of Interfaces Optimized for the Design

Comparison of Task Performance with SEQ

Post-Session Self-Report Assessment

SUS

“Willingness to Use.”

NPS

Discussion

The Impact of Visual Hierarchy Design on Attention Allocation

Human-Centered Design (the Difference Between Low-Experienced and High-Experienced)

Data-Driven UX Practical Advice

Conclusion

Limitations

Footnotes

Appendix 1

Authors’ Contributions

Declaration of Conflicting Interests

Funding

ORCID iDs

Data Availability Statement

References