Sage Journals: Discover world-class research

Abstract

To provide timely and appropriate assistance, robots must have the capability of proactively understanding a user’s personal needs, the so-called human intention inference. In human–human interaction, humans have a natural and implicit way to infer others’ intentions by selecting correlated context features and interpreting these features based on their life experience. However, robots do not have this capability and it is not realistic to build an explicit formula to associate human intentions with context. In this article, a novel fuzzy context-specific intention inference method is developed for human-like implicit human intention inference. With a fuzzy manner, context features are converted into discrete context statuses, which are similar to human subjective feelings. An intention-centered common sense database is developed consisting of correlated fuzzy context statuses, object affordances, and their relationship with human intentions. With this database, a Fuzzy Naïve Bayesian Network algorithm is adopted for implicit intention inference. Home scenario results validated the fuzzy context-specific intention inference methods reliability and lab scenario results validated the fuzzy context-specific intention inference methods effectiveness and robustness. This work is expected to develop intuitive and effective human–robot interaction, consequently enhancing the adoption of assistive technologies and improving the independence of the disabled and elderly in activities of daily living.

Keywords

Human–robot interaction fuzzy context intention inference fuzzy Naïve Bayesian context–intention correlation

Introduction

Robotic elderly caregiving

The population aging trend currently is widespread across the world.^1.2 A large fraction of the elderly population is suffering from different degrees of physical impairments.³ At the same time, the whole society is encountering a shortage of caregivers.² Given this situation, assistance robots that help the elderly with activities of daily living (ADLs) have been introduced to improve the elderly’s quality of life.^4,5,6 Typical applications include navigation robots,⁷ rehabilitation robots,^1,8 and smart reminders.^9,10

Robotic caregiving requires robots to be sufficiently intelligent in order to implicitly and naturally understand a user’s intentions with minimal human physical involvements.^3,7,11,12 By implicitly and naturally understanding human intentions, a robot could proactively and timely assist a user with his individual needs, which will improve the effectiveness and acceptance of human–robot interaction (HRI).^11,13,14 Research about human intention inference occurs mainly in two categories: action-specific intention inference and context-specific intention inference. Action-specific intention inference uses human activity/action patterns to infer human intentions including situation-assisted human intention inference,^15,16 online human utility-based human intention inference,^17,18 and hierarchical action modeling human intention inference^19,20 Context-specific intention inference involves context to assist human intention inference including environment-based most-desired intention identification,^21,22 context-assisted potential intention generation,^23,24 and spatial constraint intention inference^25,26

Context-specific intention inference

Using context in human intention inference has several advantages. First, to conduct a natural and reliable human intention inference, context-specific intention inference exploits the valuable intention-related information embedded in the surrounding environment thus increasing inference reliability.^1,23,25 Second, the involvement of context simplifies inference procedures.²⁷ Third, context-specific intention inference is an effortless, natural, and intuitive way for the elderly with severe motion disability to interact with assistive robots.^1.3 It is unrealistic for this user group to create explicit action inputs.²⁸ Thus, human involvement should be minimized. Ideally, human intention inference should be performed implicitly based on existing clues. This creates a strong need for a robot to maximally and effectively exploit the useful information from the surrounding environment.

Although context-specific intention inference has those advantages, its implementation is still a long way to go. First, a human’s subjective feelings toward context are ignored.^29,30 For example, an individual feels “comfortable” with a temperature between 19°C and 28°C and feels “hot” above 26°C. The overlapping temperature range 26–28°C, which is subjectively considered as both comfortable and hot in some degree, is often simply classified as either hot or comfortable. Second, environmental measurements are continuous, not discrete. It is unrealistic to build an explicit formula for human intentions with continuous measurements. Last, context information is useful yet redundant. It is hard to get a trade-off between exploitation of useful information and elimination of redundant information.³¹.³²

Some efforts have been made to partially address the context information processing problem in intention inference. Thresholds have been set for distinguishing context feature statuses. Only when the value was over the fixed thresholds would some statuses be considered in intention inference processes.³³ Statuses have been predefined and were compared with the measured context values to determine the current status as relatively high/normal/low.³⁴ In study by Nakano et al. and Hong et al.,^35,36 context features were simplified by discarding “unnecessary context.” Both the threshold method and the discarding method filtered out some useful information hiding in the context values near the thresholds or in the discarded context. In addition, the contributions of context values over the thresholds were not processed discriminately. Furthermore, the threshold method discarded the fact that a person’s feelings toward context features related to an intention are subjective and there should be overlapping between context statuses.

In this article, we propose a novel fuzzy context-specific intention inference (FCII) method to implicitly infer human intentions in ADL using fuzzy context statuses. This method could naturally and effectively exploit intention-related information embedded in the daily environment. Combined with a fuzzy context processing method and the common sense knowledge extracted from daily life experience, the intention-related context features are selected and the fuzzy context–intention correlations are established. Then based on a Fuzzy Naïve Bayesian network (FNBN) algorithm, the most likely human intention is inferred.

Our main contributions in this article are as follows: (1) Context feature statuses are defined in a fuzzy way to reflect the subjective nature of human perception to the surroundings. In addition, this fuzzy way allows the full range of a context feature to be considered and the contributions of different statuses for this context feature can be treated discriminately. (2) Based on a FNBN algorithm, fuzzy context–intention correlations are established to perform reliable intention inference in ADLs. (3) It allows a robot to infer a human’s intentions based on object affordances and other context features, which is more implicit and better exploits the intention-related context information compared with other intention inference methods.^19,37 In our method as shown in Figure 1, we generate a set of potential intentions from objects that a person attends to; then other context information is used to infer the most likely intention from those potential intentions. (4) An intention-centered common sense database for ADLs is created based on people’s daily life experience. Context feature selection is performed to simplify the inference processes by selecting the most intention-related features and removing redundant context (RC).

Figure 1.

Fuzzy context-specific intention inference.

Related work

Existing FCII applications

FCII has been frequently applied in inferring a driver’s intention using traffic-related context information, such as topological/geometrical characteristics of the intersection, traffic light status, and current/previous vehicle’s position/distance/angle with respect to the intersection. Driver intention inference was used for improving safety applications such as navigation systems. The involved fuzzy methods varied from one to another. Spatial information such as street network and location has been used to establish the fuzzy correlation with specific driver behavior patterns.^38,39 Combining this with a driver’s driving data, fuzzy logical algorithms were applied to predict a driver’s turning/following behaviors.^40,41 Based on the time sequence of driver operation behaviors, a fuzzy C means algorithm was used to calculate the similarities between acceleration features for recognizing a driver’s acceleration intention.⁴² Besides driver intention inference, the fuzzy context-aware method was also used for detecting suspicious behaviors,⁴³ specific environment-assisted gesture intention estimation,⁴⁴ and so on.

While FCII has been used in the aforementioned applications, FCII in the robotic caregiving field is still uncommon. Previous research in driver intention, gesture intention, and suspicious behavior detection mainly focused on the context such as vehicle acceleration,^40,41,42 lane position,^38,39 action position,⁴³ and object distributions.⁴⁴ The context features in our study focus on those closely related to indoor ADLs such as room temperature, noise level, light intensity, and object statuses. The categorization of these feature statuses is guided by human subjective cognitions toward the surrounding environment.

FCII in elderly caregiving

For robotic caregiving, a robot should smartly and implicitly understand human intentions with minimal human physical requirements.^45,46,47 Therefore, robots should fully explore the intention-related information embedded in the surrounding environment for better intention inference performances.^48
–50

The methods described in the study by Papageorgiou et al. and Iakovidis and Papageorgiou^37,51 defined fuzzy relations among the disease-related symptoms such as cough and fever, measurement parameters such as blood pressure and radiologic evidences, and medical informatics such as age and sputum culture, to infer the disease severity for the elderly patients. Even though both their work and our work aimed to use human-related context to assist a human, however our context types and the application goals are different. They used the disease-related context to identify a human’s physical statuses, while our work aimed to use the environment-related context to infer a human’s mind statuses (desired activities).

In the study by Zhou et al.,⁵² based on volunteers’ subjective judgments, fuzzy features of human activities, such as watching TV and sleeping, were defined with both user interrupt levels, which denoted the human-willingness degree to be interrupted, and the alert urgent levels, which denoted the alert-urgent degree to be triggered. Based on these two kinds of fuzzy activity features, a reminder was decided whether or not to take the action “interrupting a user” to perform new tasks such as taking a phone call. In both their work and our work, the environmental context such as a human’s location was used and a human’s activity was assisted. However, fuzzy reasoning is at different levels. Their work used fuzzy features of an activity to determine the optimal reminding action (action level), while our work used fuzzy environmental context to infer human activities (activity level).

Using fuzzy context to infer a human’s indoor activities (intentions) has rarely been done before. Therefore, in this article, our main contribution is to introduce and validate the FCII method for inferring human intentions in the indoor elderly caregiving scenarios.

FCII methodology

Context processing

Fuzzy definition for context

Our research is based on a smart home environment, where context features (such as temperature, light density, humidity, and audio/video, shown in Figure 2) are measured to monitor the environment and the user. Usually, a person’s intention is connected with the surrounding environment. For example, when the outside is noisy, people are more likely to close the windows; when the weather is nice, people are more willing to walk outside. Therefore, it is crucial to decode context in a specific situation to estimate human intentions. Context is defined as any information that characterizes the intention-related situations.²³ It could be any environmental feature or entity involved in an intention-related situation, such as humidity, location, person, or objects. One type of context measurement is defined as a context feature C_p(the p_th context in a specific situation and p = 1, 2, …, P, where P is the number of context features).

Figure 2.

Different context features.

Different values of the same context feature C_p would trigger different human intentions. For example, when people feel cold they would put on more clothes. When they feel hot, they would take off some clothes. Actually hot and “cold” are human subjective feelings toward temperature. We define this subjective feelings for a specific context feature C_p as the feature status s_pk(the k_th status of the p_th context and k = 1, 2, …, K_p, where K_p is the number of statuses for the context feature C_p).

A context feature C_p can be converted into several feature statuses s_pk, which are defined by fuzzy membership functions based on people’s daily life experience. Human potential intentions I_g(g = 1, 2, …, G) are only connected with certain feature statuses s_pk, not directly connected with the context feature C_p. For a context feature C_p, the fuzzy memberships for its statuses s_pk are m_pk correspondingly. The trapezoidal membership function is considered for a specific status s_pk of feature C_p with a value range of μ₁–μ₄. μ₂ and μ₃ are the values between μ₁ and μ₄ for intermediate feature statuses, and their relation is μ₁ < μ₂ < μ₃ < μ₄, shown in equations (1) and (2). A fuzzy factor ∂% decides the percentage of the fuzzy range in the whole range μ₁–μ₄. The fuzzy function m_pk for C_p consists of several fuzzy status memberships M_pk(C_p), shown in equations (3) and (4) and Figure 3. The values of μ₁, μ₂, μ₃, and μ₄ are defined by volunteers’ subjective judgments, shown in equations (1) to (4). For a single feature status ranging μ₁–μ₄ identified by all the volunteers, the top ∂% of the data ranging μ₃–μ₄ and the bottom ∂% of the data ranging μ₁–μ₂ are with linearly changed fuzzy membership values. The remaining (1–2 ∂%) of the data ranging μ₂–μ₃ is with a fixed fuzzy membership value 1. The complete context feature processing process is shown in Figure 4. An environmental sensor net generates various kinds of context feature measurements. Then fuzzy statuses are assigned to each specific feature measurement. Last, the related context–intention correlations are represented as

Figure 3.

Fuzzy memberships for context feature statuses.

Figure 4.

Context–intention correlation framework.

(μ_{2} - μ_{1}) = \partial (μ_{4} - μ_{1})

(μ_{4} - μ_{3}) = \partial (μ_{4} - μ_{1})

m_{p k} = {\begin{matrix} \dots \\ M_{p k} (C_{p}), μ_{1} < C_{p} < μ_{4} \\ \dots \end{matrix}

M_{p k} (C_{p}) = {\begin{matrix} (C_{p} - μ_{1}) / (μ_{2} - μ_{1}) & μ_{1} < C_{p} < μ_{2} \\ 1 & , μ_{2} < C_{p} < μ_{3} \\ (μ_{4} - C_{p}) / (μ_{3} - μ_{4}) & , μ_{3} < C_{p} < μ_{4} \end{matrix}

Feature selection

Context information is usually redundant, and if all context features are considered, it would increase the computational cost and decrease the inference accuracy and efficiency. Therefore, feature selection is needed. Feature selection is performed at the context feature level and the criteria are based on the correlation strength f(S_pk|I_g) between a context feature status s_pk and an intention I_g. The correlation strength f(S_pk|I_g) can be also understood as the conditional probability of s_pk if the intention I_g occurs. For a context feature C_p, if all $f (s_{p k} | I_{g}), k = 1, 2, \dots, K_{p}$ given a specific intention I_g, are lower than a threshold θ, we define the context C_p as a RC features, shown in equation (5). We also define s_pk with the correlation strength f(S_pk|I_g) higher than a threshold β is the strongly correlated context status, which imposes a key influence on I_g. The RC feature statuses would be eliminated in the feature selection process. Given strongly correlated statuses (SCS) decide the accuracy of intention recognition, the goal in feature selection is to collect all the SCS. The statuses, which are neither SCS nor redundant feature statuses, are also meaningful for intention inference. The feature selection method can involve these meaningful statuses in a weighted manner to improve intention inference accuracy

RC = {C_{p} | f (s_{p k} | I_{g}) < θ, k = 1, 2, \dots, K_{p}}

Common sense knowledge representation

Common sense is the knowledge accumulation of daily life experience. It represents the empirical relations between activities, objects, and situations, such as what kind of objects is often placed in a specific location³⁷ and what the functions of an object are and how to use it. This knowledge is obvious to humans, but to a robot, it is often too implicit to understand or even completely missing.⁵³ The lack of common sense creates obstacles for HRI. To make robots smart to proactively understand humans and perform timely elderly caregiving, it is necessary to endow robots with the common sense knowledge. Based on the “life experience,” a robot can reason human intentions in a more effective and natural way. Therefore, the common sense knowledge database, representing the intention–context correlations, is developed to be intention centered based on daily life scenarios.

Objects, a critical part of context information, are always closely connected with human intentions. For instance, when a person wants to leave, he/she would frequently look at the watch. When a person is thirsty, he/she would subconsciously look at a cup or a water machine. Therefore, objects are an essential part in the common sense knowledge. In this study, objects that the user attends to O_h(the h_th object, h = 1, 2, …, H) are treated as a separate category from other context features C_p. The object–intention correlations are established based on an affordance database.

The common sense knowledge includes object–intention correlationf(O_h|I_g), context status–intention correlation f(S_pk|I_g), and intention possibility f(I_g) (the possibility that an intention happens in daily living). The knowledge structure is shown in Figure 5. The intention-centered common sense knowledge describes the intention-related context and their correlation strengths with specific conditional possibility values. When an intention happens, the specific situation could be decomposed into context–intention correlations and object–intention correlations.

Figure 5.

Common sense knowledge representation.

For the general common sense structure of a single intention, we define context feature statuses and objects as “Node (Nn)” and the context–intention correlations as “Edge.” The common sense knowledge scale E from the perspective of the edge number is shown in equation (6), where H denotes the object number, $\sum_{p = 1}^{P} K_{p}$ denotes the total status number for a context feature, and G denotes the total intention number in a database. Given that for a certain number of intentions in a database, the edge number E increases as the Nn number (including both objects and statuses) increases, the dependence among the Nn number and edge number is positively linear, shown in Figure 6. Therefore, to scale-up the FCII method to a large number of ADLs is feasible

Figure 6.

Common sense database scale estimation.

Knowledge scale = E = (H + \sum_{p = 1}^{P} K_{p}) G

Human intention inference

Fuzzy context–intention correlation

As the fuzzy definition toward the context, a specific value of the context feature C_p may simultaneously belong to different statuses s_pk(k = 1, 2, …, K_p) with corresponding memberships m_pk (k = 1, 2, …, K_p). Different context statuses s_pk are correlated to the intention I_g with different strengths, shown in Figure 7. The fuzzy context–intention correlation is described by equation (9). The value of the context feature C_p is decomposed into several feature statuses s_pk. The conditional correlation strengths f(S_pk|I_g) (k = 1, 2, …,K_p) between different feature statuses and the intention I_g are accumulated to calculate the comprehensive context–intention correlation f_fuzzy (C_p|I_g)

Figure 7.

Context–intention correlation.

f_{fuzzy} C_{p} | I_{g} = \sum_{k = 1}^{K_{p}} m_{p k} f (s_{p k} | I_{g})

CSII based on FNBN algorithm

Based on the results of context processing and the establishment of context–intention correlation, human intention can be recognized if given a specific set of context features. A FNBN algorithm is used to conduct CSII. A NBN algorithm is a machine learning algorithm that is usually used for pattern recognition. NBN is good at reasoning the certainty based on complex uncertainties.⁵⁴ In an ambiguous situation with noise and incomplete information, an NBN algorithm has reliable classification ability, assigning the appropriate labels onto some cases with several clues.⁵⁵ Since the context–intention correlations are fuzzy and the context features could be considered as mutually independent, NBN is appropriate for context-specific intention inference, as shown in equation 8

I_{g max} = \underset{I_{g} \in {I_{1}, . ., I_{G}}}{\arg \max} f (I_{g}) \prod_{p = 1}^{P} f (C_{p} | I_{g})

where C is a specific set of context features given C_p(p = 1, 2, …, P). Among the intention candidates $I_{g} (g = 1, 2, \dots, G)$ , I_{g max} is the most likely intention in a scenario with the context C. Combining with the fuzzy context statuses and common sense, NBN is improved as the FNBN algorithm, as shown in equation (9).

I_{g \max} = \underset{I_{g} \in {I_{1}, . ., I_{G}}}{\arg max} f (I_{g}) \prod_{p = 1}^{P} f_{fuzzy} (C_{p} | I_{g}) = \underset{I_{g} \in {I_{1}, . ., I_{G}}}{\arg max} f (I_{g}) \prod_{p = 1}^{P} (\sum_{k = 1}^{K_{p}} m_{p k} f (s_{p k} | I_{g}))

The overall FCII method is shown in Figure 8. The method mainly includes two parts: context processing and intention inferring. First, according to the human attended objects {O₁, …, O_H}, affordance database, and common sense, the human potential intentions {I₁, …} are generated. Then based on the common sense knowledge, feature selection is conducted to extract the strongly correlated context features {C₁, …, C_P} from the initial context features {C₁, …, $C_{P_{0}}$ } and then according to their values each context feature status is assigned with a specific membership { $s_{p k_{1}}$ ( $m_{p k_{1}}$ ), …, $s_{p K_{p}} (m_{p K_{p}})$ }. Based on the FNBN algorithm, the fuzzy context–intention correlation $f_{fuzzy}$ ( $C_{p} | I_{g}$ ) is established and the most likely human intention $I_{g max}$ is inferred.

Figure 8.

The FCII method. FCII: fuzzy context-specific intention inference.

Experiments

Two scenario types are designed, home scenarios, which are set as volunteers’ home environments in which volunteers performed the required activities, and lab scenarios, which are set as designed lab environments in which volunteers performed the required activities. Different situations in home scenarios contain different context features that influence humans’ intentions. By performing intention inferences in these home scenarios, we aimed to validate FCII’s reliability. Lab scenarios could be intentionally designed to be specific types of intention-related situations. By performing intention inferences in lab scenarios, we aimed to validate FCII’s effectiveness and robustness. Two types of intention-related scenarios were designed in lab scenarios, “intention-dominant scenarios” in which human intentions are obvious with relatively less object and context overlap and the “ambiguous scenarios” in which human intentions are ambiguous with relatively more object and context overlap. A humanoid robot NAO was required to perform intention inference in these scenarios to validate FCII’s effectiveness and robustness.

As Figure 9 shows, intention candidates {“wash the cup,” “drink using the cup”} are related to the object “cup.” Context features available are {“time,” “cup location,” “room temperature,” “water taking times,” “noise level,” and “light intensity level”}. The experiments included three sections. In “Introduction” section, knowledge about fuzzy context and common sense was generated by processing data from the first set of questionnaires. In “Related work” section, using the second set of questionnaires, FCII for distinguishing the intentions {wash, drink} was performed in home scenarios to validate its reliability. In “FCII methodology” section, with the NAO robot, FCII for distinguishing the intentions {drink, wash} was performed in three different controlled scenarios to evaluate the effectiveness and robustness of the presented method.

Figure 9.

FCII for distinguishing intentions {drink, wash} in controlled scenarios. FCII: fuzzy context-specific intention inference.

Context processing and common sense knowledge building

The subjective statuses of context features and potential intentions are defined by two volunteers, as shown in Table 1. Their relations with intentions were subjectively evaluated by 100 volunteers ranging in age 19–60 and then recorded by questionnaires according to their daily life experiences, including activity participations and observations. For the fuzzy context status definitions in questionnaires, questions were set as “Water Taking Times. Enough: above __times of drinking for a day; Not Enough: less than __ times of drinking for a day.” The fuzzy factor ∂% was set as 15%.

Table 1.

Context feature status.^a

Context features	Statuses
Time	Near mealtime
Time	Not near mealtime
Cup location	Near sink
	Not near sink
	Hot
Room temperature	Comfortable
Room temperature	Cold
Water taking times	Enough
Water taking times	Not enough
Noise level	Noisy
	Quiet
	Bright
Light intensity	Moderate
Light intensity	Dark

^aTime denotes the time relative to meal time, including time is near meal time (near meal time) and time is not near mealtime (not near mealtime). Cup location denotes the location of a cup relative to a sink, including cup is near a sink (near sink) and a cup is not near a sink (not near sink). Room temperature denotes a human’s feeling toward the room temperature, including hot (Hot), comfortable (Comfortable), and cold (Cold). Water taking time denotes the frequency of water drinking for a human, including a person has taken enough times of water (Enough) and a person has not taken enough times of water (not enough). Noise level denotes a human’s feeling toward the environment’s noise level, including noisy (Noisy) and quiet (Quiet). Light intensity denotes the human’s feeling toward the environment light intensity, including bright (Bright), moderate (Moderate), and dark (Dark).

The common sense knowledge representations were wash/drink centered. For extraction of the context–intention common sense knowledge, questions were set as “Water Taking Times. Enough: □Wash □Drink □None; Not Enough: □Wash □Drink □None.” By statistical analysis of the questionnaire results, the intention probability $f (I_{g})$ and the conditional probability $f (s_{p k} | I_{g})$ (correlation strength) were calculated by equations (10) and (11). In this experiment, the object–intention correlation $f (O_{h} | I_{g})$ was equal to the intention probability $f (I_{g})$ . The related context features and objects were connected to the intentions {wash, drink} with specific correlation strengths to form a knowledge structure, as shown in Figure 5.

(I_{g}) = n_{g} / N

f (s_{p k} | I_{g}) = n_{p k} / n_{g}

where N is the total number of scenarios, $n_{g}$ is the number of a specific intention I_g involved in these scenarios, and n_pk is the number of the feature status s_pk involved in the $n_{g}$ scenarios.

The context features C_p that have all status–intention correlations $f (s_{p k} | I_{g})$ (k = 1, 2, …, $K_{p}, I_{g}$ = “wash” or “drink”) lower than 0.1 were eliminated as RC.

Reliability validation in home scenarios

Situations of performing activities in home scenarios were recorded using a new set of questionnaires, which were finished by 20 new volunteers. In each questionnaire, a volunteer was required to write down the “close-to-truth” environmental context conditions under which his/her intentions {wash, drink} usually occurred. The environmental context conditions were expressed by context feature statuses according to volunteers’ subjective feelings toward their surrounding environments. The FCII method was performed in these scenarios. For example, in a scenario {close to mealtime, cup location close to sink, temperature comfortable, quiet}, a volunteer indicated his intention as wash. The features related to {wash, drink} were first selected. Then by the FNBN algorithm (equation (9)), the probabilities of {wash, drink}, $f (I_{wash} | C)$ and $f (I_{drink} | C)$ , were calculated respectively and the most likely intention was identified. Then the inferred intention was compared with the indicated intention by the volunteer to evaluate its correctness.

Effectiveness and robustness verification in lab scenarios

Scenario 1 was a “wash-dominating scenario” with more strongly correlated context features of wash than those of drink {time to meal: 26 min, location to sink: 0.55 m, drinking times: 4, light intensity: 509 lux}. Scenario 2 was a “drink-dominating scenario” with more strongly correlated context features of drink {time to meal: 10 min, temperature: 27°C, enough drinking times: 2, noise level: 60 dB}. Scenario 3 was an ambiguous scenario with equal numbers of SCS for both wash and drink and also with more RCs, {time to meal: 50 min, location to sink: 0.6 m, temperature: 29°C, drinking times: 5, noise level: 40 dB, light intensity: 300 lux}. In all three scenarios, different kinds and degrees of RC information were added intentionally. The human intention in each of these scenarios was inferred using FCII method.

Results

Results for context processing

According to the first set of questionnaires, fuzzy context definitions are shown in Figure 10. Statuses {14–68 min to the meal time, 0.5–1.4 m distance between a cup and the sink, 10–16°C and 23–30°C of room temperature, 3–8 drinking times per day, 50–65 dB noise level, 70 160 lux and 500–550 lux light intensity} were the fuzzy range in which a feature value belongs to two statuses. For example, the cup–sink distance 0.8 m was categorized into two statuses, with the membership 0.33 for “close” and 0.4 for “far.” This fuzzy categorization for the cup-sink distance is reasonable. When the cup-sink distance is between 0.5 m and 1.4 m, it is hard to classify as absolutely close or far. We can only subjectively think some distance values are closer to close or far.

Figure 10.

Fuzzy context C_p.

Context status–intention correlations $f (s_{p k} | I_{g})$ (Figure 11) characterize the probability relation between the human intention I_g and a subjective context status s_pk. From the questionnaire results, the possibility values $f (I_{g})$ for {wash, drink, none} in our daily life were {0.2036, 0.3071, 0.4839} respectively, as shown in Figure 12. That is, among the cup-related intentions, drink (0.3071) occurs more frequent than wash (0.2036). We define that the feature statuses with the top 20% highest strength values $f (s_{p k} | I_{g})$ are strongly correlated feature statuses, and the statuses with the strength values less than 30% of the maximum strength value are RC context feature statuses. The value 20% and 30% could be adjusted according to application needs. Feature selection results are shown in Table 2. These results were consistent with our daily life experience. For example, noise/light has little influence on {wash, drink} intentions. When time is near to the meal time, we are more likely to wash the cup for daily use and when a cup is placed close to sink it indicates we intend to wash the cup. Based on these results, the wash/drink-centered common sense knowledge is represented as Figure 13.

Figure 11.

Context status–intention correlations f(S_pk|I_g).

Figure 12.

Probability f(I_g) for cup-related intentions.

Figure 13.

Wash/drink-centered common sense knowledge representation.

Table 2.

SCS and RC.

SCS/RC	Feature statuses
SCS—wash	Close to mealtime, close to sink
SCS—drink	Hot temperature, drinking not enough
RC features	Noise level, light intensity

SCS: strongly correlated statuses; RC: redundant context.

Reliability validation in home scenarios

Figure 14 shows the detailed process of FCII in one of the designated scenarios. In this scenario, the designated context included {time close to meal, cup location close to sink, comfortable temperature, high noise level}. As volunteers were asked to directly designate context statuses, the designated scenarios were then all ideal situations. That is, each context feature had only one status with the membership 1, and all its other statuses with the memberships 0. The inferred intention was wash which was consistent with the volunteer’s designated intention. Among the 20 questionnaires, 90% of the wash intention was recognized successfully, and 95% of the drink intention was recognized successfully. The inferred intentions were highly consistent with the volunteers’ designated intentions. Therefore, the FCII method based on the empirical common sense knowledge is reliable to get satisfactory inference performances.

Figure 14.

Detailed FCII process in a home scenario. FCII: fuzzy context-specific intention inference.

Effectiveness and robustness validation in lab scenarios

For each scenario, the memberships of context statuses for each context feature were first calculated, as shown in Table. 3. Based on equation (9), {wash, drink} intentions in these scenarios were recognized, as shown in Figure 15. For the wash-dominating (shown in Figure 16) and drink-dominating scenarios, the inferred intentions were the same as their dominating intentions, respectively. The good performance of resisting the RC disturbances in scenarios 1 and 2 verified the robustness of the FCII method. In scenario 3, the inferred probabilities of both intentions were too close to infer the actual intention. This result was reasonable because even humans could not infer others’ intentions in a scenario with highly mixed context features. In this situation, more clues would be needed to perform intention inference.

Figure 15.

FCII results in three scenarios. FCII: fuzzy context-specific intention inference.

Figure 16.

FCII process for scenario 1. FCII: fuzzy context-specific intention inference.

Table 3.

Interpretation of context features to fuzzy context statuses for three scenarios.

		Scenario 1		Scenario 2		Scenario3
		fea	Mem	fea	Mem	fea	Mem
Time (min)	Close to mealtime	26	0.7391	10	1	50	0.2174
Time (min)	Far from mealtime	26	0.0227	10	0	50	0.5682
Location (m)	Close to sink	0.55	0.9	–	–	0.6	0.8
Location (m)	Far from sink	0.55	0.05	–	\	0.6	0.11
Temperature (°C)	Hot	\	\	27	0.8	29	1
	Comfort		\		0.75		0.25
	Cold		\		0		0
Drinking times	Enough	4	0.25	2	0	5	0.5
Drinking times	Not enough	4	0.8	2	1	5	0.6
Noise level (dB)	Noisy	\	\	60	1	40	0
Noise level (dB)	Quiet	\	\	60	0.5	40	1
Light intensity (lux)	Bright	509	0.81	\	\	300	0
	Comfort		0.82		\		1
	Dark		0		\		0

fea: sensor values for environmental feature statuses; Mem: membership for the environmental statuses.

Discussion

This fuzzy context research is meaningful in developing practical applications. Existing research such as study by Liu and Zhang and Tahboub^6,12 used the sensory data for intention inference directly. However, given the knowledge embedded in data is more important than data value itself, direct usage of sensory data is not enough for getting satisfactory intention inference performance. With the FCII method, objective sensory data is mapped into humans’ subjective feeling/knowledge. Given an intention is a subjective decision of a human, using humans’ subjective feeling/knowledge to decode subjective decisions is meaningful.

Moreover, to make the FCII method more capable of solving practical problems in general real-world situations, efforts have been put in our paper on improving the method’s general applicability, summarized as follows (“Discussion” section). (a) The fuzzy method is developed for generating context feature statuses from different types of sensory data, as shown in equations (1) to (4). With statistical learning methods in equations (10) and (11), the sensory data was interpreted into intention-related knowledge by measuring the involvements and influences of these statuses given a human intention. (b) The feature selection method was developed for selecting the dominant feature statuses related to a human intent, as shown in equation (5). With this method, the interpreted knowledge is truly useful for understanding practical human intention–related situations. (c) We developed a fuzzy context-based intention inference method, as shown in equations (8) and (9). With this method as well as the generated knowledge, human intentions could be inferred in various environment conditions. (d) To further improve the method’s general applicability, we designed the knowledge representation with an intention-centered manner, which enables a linear relationship between the number of intentions and the number of dominant features. This linear relationship reduces the scale of the knowledge and leaves potential to easily generalize the method to a larger number of intentions.

Conclusion and future work

In this article, we presented a novel FCII method to allow robots to implicitly infer the elderly’s intentions in ADLs. Based on human daily life experience, we defined statuses of context features involved in ADLs in a fuzzy manner. Each context feature owns several fuzzy statuses, similar to human subjective feelings toward the surroundings. We also established the intention-centered common sense knowledge to endow robots with the knowledge of general context status–intention correlations. With the fuzzy context statuses and the intention-centered common sense knowledge, an FNBN algorithm was used to infer the human desired intention.

Simulations and experiments were conducted using questionnaires and the humanoid robot NAO. Fuzzy context memberships and common sense knowledge were generated from volunteers’ daily life experience through the form of questionnaires. Under the volunteer-designated scenarios defined, the high reliability of the FCII method was validated by the high success inference rate. In addition, the effectiveness of the FCII method was validated by its good performance of translating initial raw context measurements to fuzzy context statuses and inferring intentions that are reasonably consistent with a human’s performance.

In the future work, we will improve the applicability of our FCII method by involving more objects/context and constructing a more complete knowledge structure with the consideration of intentions’ interaction. With these improvements, we aim to use the FCII method to infer a larger number of ADL-related intentions. Moreover, advanced experimental systems would be developed to perform practical experiments to evaluate FCII’s performances. Last, our future plan includes using FCII to develop a knowledge-supported robot, which will be endowed with a large amount of knowledge for intelligent decision-making.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Pollack

. Intelligent technology for an aging population: The use of AI to assist elders with cognitive impairment. AI Mag 2005; 26(2): 9–24.

Montemerlo

Pineau

Roy

. Experiences with a mobile robotic guide for the elderly. In: AAAI Conference on Artificial Intelligence (AAAI), Alberta Canada, July 28– August 1 2002, pp. 587–592. AAAI.

Roy

Baltus

Fox

. Towards personal service robots for the elderly. In: Workshop on Interactive Robots and Entertainment, Pittsburgh, Pennsylvania, USA, vol. 25, p. 184, 2000. AAAI.

Liu

Zhang

Webb

. Context-specific intention awareness through web query in robotic caregiving. In: IEEE International Conference on Robotics and Automation (ICRA) (ed Bajcsy

Ruzena

), Seattle, Washington, USA, WA, 26–30 May 2015, pp. 1962–1967. IEEE.

Liu

Zhang

. Use context to understand user’s implicit intentions in Activities of Daily living. In: IEEE International Conference on Mechatronics and Automation (ICMA) (ed Tarn

), Tianjin, Beijing, China, 3–6 August 2014, pp. 1214–1219. IEEE.

Liu

Zhang

. Understanding human behaviors with an object functional role perspective for robotics. IEEE Trans Cognit Dev Syst 2015; 8(2): 115–127. DOI: 10.1109/TAMD.2015.2504919.

Morris

Donamukkala

Kapuria

. A robotic walker that provides guidance. In: IEEE International Conference on Robotics and Automation (ICRA), Taiwan, Taipei, 14–19 September 2003, vol. 1, pp. 25–30. Hoboken, NJ, USA: IEEE.

Loconsole

Banno

Frisoli

. A new Kinect-based guidance mode for upper limb robot-aided neuro rehabilitation. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, 7–12 October 2012, pp. 1037–1042. Hoboken, NJ, USA: IEEE.

Pineau

Montemerlo

Pollack

. Towards robotic assistants in nursing homes: challenges and results. Robot Auton Syst 2003; 42(3): 271–281.

10.

Pollack

Brown

Colbry

. Autominder: an intelligent cognitive orthotic system for people with memory impairment. Robot Auton Syst 2003; 44(3): 273–282.

11.

Trafton

Cassimatis

Bugajska

. Enabling effective human-robot interaction using perspective-taking in robots. IEEE Trans Syst Man Cybernet A 2005; 35(4): 460–470.

12.

Tahboub

. Intelligent human-machine interaction based on dynamic bayesian networks probabilistic intention recognition. J Intell Robot Syst 2006; 45(1): 31–52.

13.

Murphy

Nomura

Billard

. Human–robot interaction. Robot Automat Mag IEEE 2010; 17(2): 85–89.

14.

Baxter

Belpaeme

Canamero

. Long-term human-robot interaction with young users. In: IEEE/ACM Human-Robot Interaction Conference Robots with Children Workshop, Lausanne, Switzerland, 6–9 March 2011, pp. 1–4.

15.

Pereira

Han

. Elder care via intention recognition and evolution prospection. In: Salavador

Abreu

Dietmar

Seipel

(eds) Applications of Declarative Programming and Knowledge Management. Heidelberg, Germany: Springer Berlin Heidelberg, 2011, pp. 170–187.

16.

Dindo

Zambuto

Pezzulo

. Motor simulation via coupled internal models using sequential Monte Carlo. In: Proceedings-International Joint Conference on Artificial Intelligence, AAAI: Barcelona, Catalonia, Spain, Vol. 22, pp. 2113.

17.

Mao

Gratch

. A utility-based approach to intention recognition. In: AAMAS Workshop on Agent Tracking: Modeling Other Agents from Observations (ed Avrahami

Dorit

), NY, USA, 19 July 2004, pp. 1–6. Ramat Gan, Israel: Bar Ilan University.

18.

Dominey

Warneken

. The basis of shared intentions in human and robot cognition. New Ideas Psychol 2011; 29(3): 260–274.

19.

Krauthausen

Hanebeck

. Situation-specific intention recognition for human-robot cooperation. In: Dillmann

Rüdiger

Beyerer

Jürgen

Hanebeck

Uwe D.

Schultz

Tanja

(eds) KI 2010 Advances in Artificial Intelligence. Berlin, Germany: Springer Berlin Heidelberg, 2010, pp. 418–425.

20.

Pezzulo

Dindo

. What should I do next? Using shared representations to solve interaction problems. Exp Brain Res 2011; 211(3–4): 613–630.

21.

Saffiotti

Broxvall

. PEIS ecologies: Ambient intelligence meets autonomous robotics. In: Proceedings of the 2005 Joint Conference on Smart Objects and Ambient Intelligence: Innovative Context-Aware Services: Usages and Technologies, Grenoble, France, 12–14 October 2005, pp. 277–281. New York, NY: ACM.

22.

Schrempf

Albrecht

Hanebeck

. Tractable probabilistic models for intention recognition based on expert knowledge. In: 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA, 29 October–2 November 2007, pp. 1429–1434. Hoboken, NJ: IEEE.

23.

Dey

Abowd

. Towards a better understanding of context and context-awareness. In: Gellersen

Hans-W.

(eds) Handheld and Ubiquitous Computing. Heidelberg, Germany: Springer Berlin Heidelberg, 1999, pp. 304–307.

24.

Brass

Schmitt

Spengler

. Investigating action understanding: inferential processes versus action simulation. Curr Biol 2007; 17(24): 2117–2121.

25.

Kiefer

Schlieder

. Exploring context-sensitivity in spatial intention recognition. Behav Monitor Interpretat 2007; 296: 102–116.

26.

Tahboub

. Natural and manmade shared-control systems: an overview. In: The International Conference on Robotics and Automation, Seoul Korea, 21–26 May 2011, pp. 2655–2660.

27.

Cole

. Building environmental assessment methods: redefining intentions and roles. Build Res Inform 2005; 33(5): 455–467.

28.

Ouden

Frith

. Thinking about intentions. Neuroimage 2005; 28(4): 787–796.

29.

Mäntyjärvi

Seppanen

. Adapting applications in handheld devices using fuzzy context information. Interact Comput 2003; 15(4): 521–538.

30.

Imura

Takagi

Yamaguchi

. Intention recognition using conceptual fuzzy sets. In: Fuzzy Systems, Second IEEE International Conference, San Francisco, CA, USA, 28 March–1 April 1993, pp. 762–767.

31.

Wang

Hwang

Ting

. UbiPaPaGo: context-aware path planning. Expert Syst Appl 2001; 38(4): 4150–4161.

32.

Park

Cho

. A modular design of Bayesian networks using expert knowledge: context-aware home service robot. Expert Syst Appl 2012; 39(3): 2629–2642.

33.

Siegel

Stiefelhagen

. Sensor fusion using dempster-shafer. In: IEEE Instrumentation and Measurement Technology Conference, Anchorage, AK, USA, 21–23 May 2002, vol. 1, pp. 7–12.

34.

Kruse

Pandey

Alami

. Human-aware robot navigation: a survey. Robot Auton Syst 2013; 61(12): 1726–1743.

35.

Nakano

Miyazaki

Hirasawa

. Understanding unsegmented user utterances in real-time spoken dialogue systems. In: the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, College Park, Maryland, USA, 20–26 June 1999, pp. 200–207. OMIPRESS INC.

36.

Hong

Cho

Lee

. Developing a context-aware system for providing intelligent robot services. In: Havinga

Paul

Lijding

Maria

Meratnia

Nirvana

Wegdam

Maarten

(eds) Smart Sensing and Context. Heidelberg, Germany: Springer Berlin Heidelberg, 2006, pp. 174–189.

37.

Papageorgiou

Papandrianos

Karagianni

. A fuzzy cognitive map based tool for prediction of infectious diseases. In: IEEE Fuzzy Systems, 2009, pp. 2094–2099.

38.

Kiefer

Stein

. A Framework for Mobile Intention Recognition in Spatially Structured Environments. BMI, 2008.

39.

Lefèvre

Laugier

Javier

. Exploiting map information for driver intention estimation at road intersections. In: IEEE Intelligent Vehicles Symposium (IV), 2011. pp. 583–588.

40.

Kumagai

Sakaguchi

Okuwa

. Prediction of driving behavior through probabilistic inference. In: 8th International Conference on Engineering Applications of Neural Networks, St. Louis, MO, 3–7 October 2003, pp. 117–123. Springer, New Mexico: Springer.

41.

Lidstrom

Larsson

. Model-based estimation of driver intentions using particle filtering. In: Yanqin

Gao

(ed) 11th IEEE Intelligent Transportation Systems, Beijing, China, 12–15 October 2008, pp. 1177–1182. Hoboken, NJ: IEEE.

42.

Zhang

Yan

. A recognition model for acceleration intention of automobile drivers based on fuzzy clustering: Reston, VA: ASCEProceedings of the First International Conference on Transportation Information and Safety, American Society of Civil Engineers, Wuhan Hubei China, 30 June–2 July 2011, pp. 1938–1947.

43.

Miyanokoshi

Sato

Yamaguchi

. Suspicious behavior detection based on case-based reasoning using face direction. In: SICE-ICASE, International Joint Conference IEEE (ed Yang

Hyun Chang

), Busan, Korea, 18–21 October 2006, pp. 5429–5432. IEEE.

44.

Sato

Yamaguchi

Harashima

. Natural interface using pointing behavior for human–robot gestural interaction. IEEE Trans Indus Electron 2007; 54(2): 1105–1112.

45.

Millard

Hallett

Luker

. Nurse–patient interaction and decision-making in care: patient involvement in community nursing. J Adv Nurs 2006; 55(2): 142–150.

46.

Viani

Robol

Polo

. Wireless architectures for heterogeneous sensing in smart home applications: Concepts and real implementation. Proc IEEE 2013; 101(11): 1–16.

47.

Schank

Abelson

RP.

Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures. Hove, United Kingdom: Psychology Press, 2013.

48.

Ruiter

Cummins

A model of intentional communication: AIRBUS (Asymmetric intention recognition with bayesian updating of signals). In: Proceedings of SemDial, Paris, France, 19–21 September 2012, pp. 149–50. Université Paris-Didero.

49.

Awais

Henrich

. Human-robot collaboration by intention recognition using probabilistic state machines. In: RAAD, IEEE 19th International Workshop, Budapest, 24–26 June 2010, pp. 75–80. Hoboken, NJ, USA: IEEE.

50.

Schrempf

Albrech

Hanebeck

. Tractable probabilistic models for intention recognition based on expert knowledge. In: IROS, IEEE/RSJ, San Diego, CA, 29 October–2 November 2007, pp. 1429–1434. Hoboken, NJ: IEEE.

51.

Iakovidis

Papageorgiou

. Intuitionistic fuzzy cognitive maps for medical decision making. IEEE Trans Inform Technol Biomed 2011; 15(1): 100–107.

52.

Zhou

Chu

. A context-aware reminder system for elders based on fuzzy linguistic approach. Exp Syst Appl 2012; 39(10): 9411–9419.

53.

Tenorth

Klank

Pangercic

. Web-enabled robots. IEEE Robot Autom Mag 2011; 18(2): 58–68.

54.

. The information content of forward-looking statements in corporate filings—A naïve Bayesian machine learning approach. J Acc Res 2010; 48(5): 1049–1102.

55.

Murphy

KP.

Machine Learning: A Probabilistic Perspective. Boston, MA: MIT press, 2012.

Fuzzy context-specific intention inference for robotic caregiving

Abstract

Keywords

Introduction

Robotic elderly caregiving

Context-specific intention inference

Related work

Existing FCII applications

FCII in elderly caregiving

FCII methodology

Context processing

Fuzzy definition for context

Feature selection

Common sense knowledge representation

Human intention inference

Fuzzy context–intention correlation

CSII based on FNBN algorithm

Experiments

Context processing and common sense knowledge building

Reliability validation in home scenarios

Effectiveness and robustness verification in lab scenarios

Results

Results for context processing

Reliability validation in home scenarios

Effectiveness and robustness validation in lab scenarios

Discussion

Conclusion and future work

Footnotes

Declaration of conflicting interests

Funding

References