Methodology & Themes of Human-Robot Interaction: A Growing Research Field

Abstract

This article discusses challenges of Human-Robot Interaction, which is a highly inter- and multidisciplinary area. Themes that are important in current research in this lively and growing field are identified and selected work relevant to these themes is discussed.

1. Introduction: Challenges of Human-Robot Interaction

Human-Robot Interaction (HRI) is a challenging research field at the intersection of psychology, cognitive science, social sciences, artificial intelligence, computer science, robotics, engineering and human-computer interaction. A primary goal of research in this area has been to investigate ‘natural’ means by which a human can interact and communicate with a robot. Due to the embodied nature of this interaction, where robots and humans need to coordinate their activities in time and space in real-time, often ‘face-to-face’, the quality of these interactions is related to, but different from e.g. human-computer interaction (HCI). Researchers in the area are increasingly aware that new methods, methodologies, and in general a theoretical and conceptual basis needs to be formed, if HRI is to establish itself as a research field along-side e.g. HCI or psychology. As a research field, HRI is still in its infancy. Numerous studies have been performed over the past 15 years, but one of the hallmarks of science has yet to be achieved: results at present are hardly ever reproducible by other research groups. This criterion is vital, since it is a necessary requirement for making progress and creating a foundation of knowledge and insights that other researchers can refer to and built upon. The lack of reproducibility of results is due to various factors, described in more detail below, in addition to other challenges of HRI research.

1) (Why to avoid) Methodological battles: Due to the widely innovative nature of HRI studies, researchers cannot design their experiments according to agreed upon methodologies, and relating it to a series of previous works, as is common practice e.g. in psychology. Note, even fields like psychology do not have ONE agreed upon methodology in how to carry out research, e.g. whether to apply quantitative, statistical methods requiring large-scale experiments, i.e. involving large sample sizes of participants, or to pursue a qualitative approach focusing on case studies and in-depth analysis. Similarly in HRI we find complementary quantitative as well as qualitative approaches towards studying HRI. Similar to discussions within psychology on what the “right approach” is, HRI often involves discussions on quantitative, large scale studies versus qualitative small-scale studies based on case studies. In order to advance the field of HRI it is important to ‘make peace’ between proponents of one or the other methodology, acknowledging that there are several paths one can take in order to illuminate the issues under investigation. It would certainly be damaging to the young field of HRI to have a dogmatic attitude and define once and for all what the ‘correct way’ is to pursue HRI studies. Note, sometimes a ‘story well told’ can have more impact, also in terms of reaching a wider audience, than strictly adhering to textbook knowledge, cf. the seminal publications of Oliver Sacks where case studies illuminate the fascinating world of ‘other minds’, e.g. (Sacks, O. 1985).

It is at present unclear how a foundation of HRI methods and methodologies would look like. However, it is important to realize that the nature of human-robot interaction is related to but different from human-human or human computer interaction, thus, instead of simply following e.g. social sciences textbook knowledge, adaptations and new developments of methods are asked for. It is important to be precise about the methodological approaches used in HRI studies, but at the same time one needs to be aware that there is is no ‘once-and-for-all’ solution applicable across HRI.

2) Robots are not people. It is a misconception that results from human-human interactions can be directly applied to human-robot interaction. In order to back up this attitude the work by Reeves and Nass (1996) and others is often cited who have shown that people behave towards artifacts, i.e. computers, in many ways socially, e.g. attribute personality to them, reciprocate favours etc. However, it is important to realize that these results are not as suprising as they might seem: human intelligence has strong routes in human social intelligence. Ethologists and others even argue for the social origins of human intelligence, a proposal called the ‘Social Intelligence Hypothesis’ ¹ . Regardless of academic discussions on the exact factors that lead to human intelligence, it is widely accepted that human cognitive development is socially situated. Consequences of this are the strong links of any areas of our lives with ‘sociality’, reaching deeply into our minds and the language we use to communicate with the world around us, even when discussing abstract concepts: we are ‘fighting with deadlines’, ‘struggling with a problem’ etc. Human anthropomorphize not only nature, but basically anything around them: we might talk to our broken coffee machine, praise our car if it starts on a very cold morning, or pretend that a stuffed horse is a living animal when playing with our children. It is the very nature of human beings to be social. We can see the severe consequences of people who lack genuine social understanding, which might inhibit them from being a full member of our society and culture, e.g. (Baron-Cohen, S. 1997).

Thus, the fact that we behave socially towards artifacts reveals a lot about ourselves, and often specific features that the artifact possesses can be used to trigger social responses. For example, a ‘cute robot’ might trigger nurturing instincts in us, e.g. (Breazeal, C. 2002). However, it does not mean that we mistake robots for people, or more generally, that we treat robots and living things exactly alike, e.g. it has been shown that while children interact with an Aibo robot similarly to a real dog, they do not view it exactly as a living dog, i.e. with respect to moral standing (Kahn, P. H., et al. 2006).

Consequently, while it is exciting to build robots that elicit social behaviour in people in order to develop and regulate interactions and to provide a ‘natural interface’, it is scientifically perhaps more fruitful to investigate situations where people do NOT treat robots socially, in particular revealing the aspects of a robot's appearance and behaviour that might ‘break the illusion’, and how one might recover from such situations. After all, as long as robots and human are distinguishable, and this will still be the case for many years to come, people are likely to not treat them identically to human beings. Once people can no longer distinguish a robot from a person, a goal that is being persued in the field of Androids (MacDorman, K. F. & Ishiguro, H. 2006), then people will treat them like humans.

3) The importance of being creative: While we argued above for the need of HRI studies to replicate other researchers' results, the search space of robot behaviour, appearance and functionality is huge. It is very hard to do ‘exhaustive search’ and systematically explore this design space, in particular since the search space might not be static due to robots showing learning and adaptability. In computer science various search techniques are used to address the problem of huge search spaces. In particular in cases where the search space is ‘rough’, i.e. where it is difficult to predict what will happen if once pursues one specific dimension in the search space, evolutionary techniques (Holland, J. H. 1975) are often successful which explore large areas of the landscape in parallel, and exploit in more depth good solutions found. A similar situation is currently characteristic of the HRI research landscape: different research labs in parallel explore the design space of HRI robots. It is hoped that mapping out this design space will contribute to a common foundation of knowledge on how robots should look like and behave for a particular HRI task or application domain. Note, the solutions will change due to technological advancements, but also due to an increasing exposure of people to robots which will impact on their expectations of and attitudes towards robots.

4) Experiments with Humanoids. HRI benefits from the availability of increasingly sophisticated humanoid robotic platforms such as QRIO (Sony), Wakamaru (http://www.mhi.co.jp/kobe/wakamaru/english/), Asimo (Honda), and others. Human-sized robots can afford interactions apparently at ‘eye level’. Using humanoids researchers are investigating how people respond to different appearances or behaviours of robots in order to inform further robot development. Such studies closely resemble those in psychology and social sciences into human behaviour. However, conducting and evaluating interaction studies that meet the requirements and standards of human-human interaction studies (e.g. as conducted in related areas such as psychology or social sciences) is still a big challenge. In human-human interaction studies, methods often involve naive (test) participants, in addition to a trained and pre-instructed person (a confederate experimenter) who, unknown to the participant proper, exhibits behaviours related to different experimental conditions. Can a robot be a confederate experimenter? Adopting this approach in the area of HRI, in order to elicit certain behaviours in other people, is non-trivial. It implies that the robot needs to be equipped with several non-trivial behaviours that support interactions and communication with humans. Programming robots so that they can exhibit these behaviours reliably, robustly, safely, while readily and in real-time responding to often subtle and highly dynamic behaviours of the human interaction partner are vital requirements in order to meet specific experimental conditions. But even a robot that has been programmed with a variety of interactive behaviours is unlikely to be able to cope with the whole range of erratic, idiosyncratic, or otherwise unusual behaviour that humans are likely to perform in any such studies. Such requirements pose big technological challenges, i.e. they demand sophisticated programming and engineering skills, and the development process will take an extended period of time. Thus, compared to instructing a human in human-human interaction studies the effort to realize interaction studies with robots is immense and poses significant obstacles to the advancement of research in the field. The Wizard-of-Oz (WoZ) technique, as a rapid prototyping method, is a widely used evaluation technique in HCI and, more recently, HRI research (e.g. Green, A., et al. 2004), that can result in proof-of-concept evaluation data with a prototype version. It involves a human who is (unknown to the test subjects) controlling the behaviour of the system, ranging from full teleoperation to partial control of ‘higher level’ decision making processes. However, if the human ”wizard“ faces a complex behaviour arbitration problem then the cognitive load on the human can be substantial. Other methods involve the use of a Theatrical Robot (Robins, B. et al. 2006), a life-size, embodied, simulated robot which allows one to investigate requirements of robot design even prior to any hardware and software development. This paradigm can be applied most usefully in very early phase of the robot design. The Theatrical Robot consists of a human instructed to behave and/or appear like a robot. The human is a professional such as a mime or person trained to perform pre-scripted behaviours, as needed for experimental protocols, reliably and with high precision.

Figure 1 shows a sketch of a typical development time line of HRI robots. In an initial phase of planning and specification, mock-up models might be used before hardware and software development commences (Bartneck, C. & Jun, H. 2002). Once a system's main components have been implemented, a WoZ study is applicable, and/or video based methods can be applied (Woods S., et al. 2006). As soon as working prototypes exist that also conform with safety requirements, interaction studies can be conducted. The Theatrical Robot paradigm allows one to conduct interaction studies even from the early phase of planning and specification onwards throughout the whole development of the robot. Once working prototypes exist mock-up models or the Theatrical Robot are likely to become less useful since interaction studies can be run with the ‘real system’ rather than its embodied simulation. However, the Theatrical Robot can also be used as a stand-alone-method, without necessarily being linked to a robot development project. For example, the Theatrical Robot paradigm has been used successfully in a study investigating how children with autism respond to a robot with different appearances (Robins, B. et al. 2006).

Figure 1:

Sketch of a typical development time line of HRI robots.

Figure 2:

HRI Experimental Paradigms: The Theatrical Robot used to evaluate interactions of children with autism with a humanoid robot of either ‘robotic’ (reduced features) or ‘human-like’ (full human features) appearance, while behaving identically (Robins, B. et al. 2006).

5) Projecting into the future: If forecasts become true, more and more service and other robots will enter our daily lives as assistants or companions. Present day service robots still show limited (social) intelligence and interaction abilities. Thus, rather than limiting HRI studies to those robots that are now available and can autonomously function in an HRI scenario, many researchers pursue the WoZ approach discussed above.

In this way higher-level cognitive and social skills can be simulated, and results can be gained from HRI studies that can then further direct the development of controllers that might implement these functions. However, the value of this approach goes beyond its utility as a prototyping technique. It is a means to ‘project into the future’, to envisage and ‘enact’ HRI scenarios that are beyond current state of the art robotics, engineering and artificial intelligence technology. The benefit here is that we can create scenarios that can stimulate a discussion not only among the HRI/robotics community, but including the wider public, on ethical, social and other issues surrounding the issue of robots sharing our lives with us. It ensures that the future will not only be created in research labs, but that a dialogue can begin in order to identify desirable as well as undesirable directions. It also allows the wider public to get accustomed to ‘what robots can become’, in a more realistic way, i.e. grounded in HRI/robotics technology, rather than solely relying on science ficition movies and novels to shape the minds of non-experts. This dialogue with the public on HRI and other robotics activites is crucial to empower people rather than seeing robots emerging from labs, being ‘let loose’ on the public and consequently causing often unpredictable reactions. An example for a public engagement activity in United Kingdom is Walking with Robots (http://www.walkingwithrobots.org/).

6) The added value. For still some time to come, versatile, sophisticated, e.g. learning robots, will be expensive. Also, robots cannot replace human contact, interactions with robots are still mechanical in nature, and it is unclear to what extent they are able to provide the emotional and social satisfaction that human contact can provide (Dautenhahn, K. in press). Last but not least, full-sized humanoid robots do pose a safety issue. There might be advantages to using robots, but it is advisable to point out clearly the added value and to justify the use of robots compared to interactions with other people (e.g. in care situations), animals (e.g. in therapy scenarios), non-robotic toys (e.g. in play or education applications), computers (e.g. in education or entertainment applications), or other biological or artificial entities that might serve a similar function depending on the application area. Thus, comparative studies exposing people to the robot in one condition, but e.g. to other comparable artifacts in another condition (or several other conditions) can illuminate the added value of a robot, e.g. (Kahn, P. H., et al. 2006), (Dautenhahn, K. & Werry, I. 2004), and thus provide a justification for HRI research in this domain that goes beyond scientific curiosity or technological interests.

2. Selected Themes in HRI Research

There are many highly challenging research issues that can be investigated in HRI research. Just to name a few:

a) Long-term interactions: first impressions (or encounters at ‘zero acquaintance’ as it is termed in psychology) are important in HRI, and suitable for many applications where human-robot encounters will be brief, and non-repeated. However, many other application areas study scenarios that involve repeated, long-term interactions. Preferences and attitudes at zero acquaintance are likely to change, and novelty effects will wear out. Carrying out long-term interaction studies is labour-, time- and equipment- intensive, but crucial in order to address situations where e.g. robots will co-habitate with humans in their homes, workplaces etc.

b) Robots in education, therapy, rehabilitation and supporting the elderly: Assistive robotics is a growing application domain for service robots. It involves critical safety and ethical issues, e.g. when robots take one roles of assisting vulnerable people or people with special needs. Robots that support the elderly are studied in many countries that are facing the challenge of an aging society. Helping elderly people to live independantly for longer is certainly a very worthwhile goal, if the research agenda is clear about avoiding the replacement of human contact.

c) Multimodal interactions, expressiveness, and conversational skills in interactions: Research aiming at providing robots with human-like features and qualities is expanding, including studies into gesture communication, interaction kinesics, posture, and social spaces that are important in human-human interaction and are increasingly applied to human-robot interactions, facial expressions that are meant to provide emotional expressiveness in conjunction with other non-verbal and verbal cues, linguistic communication and dialogue with robots which is hoped to provide a ‘natural interface’ (more natural than e.g. a keyboard or push buttons) in applications requiring direct communication between between humans and robots, and, last but not least research into providing robots with personalities that people can relate to.

d) Social learning and skill acquisition via teaching and imitation: This theme involves research on robots that can adapt to changing environments and requirements, that can ‘grow’ with increasing levels of skills and knowledge they acquire, and that can be programmed indirectly by demonstrating tasks. Once aquired, the robot could also transfer the newly learnt skills to other robots, scenarios that might give rise to an (as yet futuristic) ‘robot culture’. Nevertheless, robots that only carry out what they have been hardwired to do might be suitable for predictable and highly structured manufacturing environments, but will fail dramatically in any human-enhabited environments, such as people's homes, offices etc.

e) Cooperation and collaboration in human-robot teams: In this area work focuses on applications where robots and humans do not only live side by side, but they also need to work hand in hand, e.g. in space applications where human astronauts and humanoids need to collaboratively assemble parts etc. Such work is highly influenced by collaboration in human teams, but faces particular challenges, not least because people use mental models and expectations in order to make predictions about the future behavior of their team partners, which is difficult in situations when confronted with a robot where it is unclear what skills the robot possesses and does not possess.

f) Detecting and understanding human activity: Robot perception is still an open problem. This ranges from perceiving objects to be grasped or landmarks used for navigation to recognizing what humans are doing in order to coordinate human-robot interactions; from problems like recognizing that the human is sitting on the sofa and reading a newspaper to recognizing cues and gestures (e.g. waving, pointing, eye gaze etc.) that are important in interaction. In human-human interaction often subtle and widely unconsciously produced and perceived cues are used to regulate interaction. It is still unclear to what extend the subtleties of recognizing such cues can be achieved with robot perception systems. Also, human-human interactions, in particular in scenarios with long-term and repeated interaction, crucially depend on interaction histories, a theme that has thus far attracted little attention in HRI research, but cf. (Nehaniv, C. L. et al. 2006).

3. Examples of State of the Art HRI research

HRI research is currently being presented at numerous international related conferences, e.g. CHI, AAAI, IJCAI, AAMAS, EpiRob, ICDL, IROS, ICRA etc., as well as two conferences dedicated to HRI and Human and Robot Interactive Communication, namely IEEE RO-MAN (since 1991) and ACM/IEEE HRI (since 2006). In order to exemplify some of the themes listed in Section 2, the following paragraphs briefly discuss selected research presented at the most recent HRI related conference, namely IEEE RO-MAN 2006, held at University of Hertfordshire, Hafield, UK, 6–8 September 2006. Note, these are selected works and not meant to be an exhaustive review of HRI research.

Giusti and Marti (2006) give an an example of HRI research addressing the themes Long-term interactions and Robots supporting the elderly. They present a case study where a group of elderly people interacted twice a week over one month with a robot. Interactions took place in a naturalistic setting of a nursing home. Results emphasize that the specific context where the (human-robot and human-human) interactions happen is crucial to the application of the robot as an assistive robot for people with dementia. A related long-term study over two months with the same robot (Paro) was conducted by Wada et al. (2006). Generally both studies emphasize the importance of long-term studies for robots being placed in people's homes as companions in the context of robot-assisted therapy. The papers also demonstrate the need for quantitative as well as qualitative evaluation techniques.

Long-term interactions also play an important role in the work of Tanaka and Movellan (2006) who study interactions of a small humanoid robot with children in a classroom over three months. Their evaluation technique uses the sensor record of the robot's touch sensor in order to evaluate interactions. Note, this technique ² shows that robots can provide unique possibilities for new evaluation techniques: one cannot easily track proprioceptive sensor information in human-human interaction, while it is very straightforward to do so for a robot.

Detecting and understanding human activity is an important ingredient in many HRI studies. Maas et al. (2006) introduce a topic tracker developed for human-robot dialogue that uses a variety of sensor data (vision, laser scanner) in order to assist the tracking of a topic. While this is work in progress, it indicates the necessity for a robot to show more human-like understanding of situations in order to improve interaction. Sisbot et al. (2006) describe a human-aware navigation planner. Different from traditional navigation planners that mostly treat humans as obstacles, issues such as a safety and visibility are taken into account.

Detecting, understanding and responding appropriately also play a key role in human-robot collaboration. Collaborative scenarios between humans and robots can take many forms. Crick et al. (2006) investigate human-robot collaborative drumming. Visual, auditory and proprioceptive data needs to be integrated for the robot to drum in concert with human performers. While in this case the robot needed to get right the dynamics of the interaction, Kim and Hinds (2006) showed that in the context of a joint task, a robot's level of autonomy (to make decisions) and transparency (in the sense of explaining its behaviour) influenced people's attributions of credit or blame to the robot. Thus, while the drumming task primarily addressed low-level dynamics of sensorimotor behaviour, the Kim and Hinds study addressed higher-level psychological issues of attribution, indicating the range of different HRI research questions that are relevant for human-robot collaborative scenarios.

Psychological issues play an important part in HRI. Research ranges from the assessment of people's anxiety towards robots (Nomura et al. 2006), to investigations of people's expectations of service robots (Oestreicher L., & Severinson Eklundh, K., 2006), to relationships of people's personality traits and preferences they express towards robot behaviour (Syrdal, D. S. et al., 2006), or the relationship between a robot's perceived personality and level of user control (Meerbeek, B., et al., 2006).

4. Conclusion

Human-Robot Interaction involves many exciting challenges both with respect to the technical challenges, as well as with respect to the human-centred aspects involved. The latter includes issues such as people's expectations of, attitudes towards and perceptions of robots, multi-modal interaction modalities, acceptability and believability of interaction, robot behavior that is comfortable and acceptable to humans, etc. These are complemented by robot-cognition centred issues of levels of robot (social) intelligence, as well as robot-centred issues whereby robots are viewed as autonomous beings that need to satisfy their own emotional and social needs and pursue their own goals, see further discussion in (Dautenhahn, K. in press). It is hoped that HRI will continue to develop and grow, building foundational knowledge that will benefit future generations of HRI researchers for in this exciting research field.