Introduction to the thematic issue on Natural Interaction in Intelligent Environments

1. Introduction

Nowadays, all kinds of smart devices have been networked together and communicated with each other, thereby become an intelligent environment where interaction with the computing devices has become necessity and it happens anywhere anytime. For that reason, human-computer interaction (HCI) has been a lively field of research in the last few years. The aim of HCI research is to provide natural interface when we interact with the intelligent environment. “Natural” here refers to a goal in the user experience – that the interaction can be as natural as interaction with human being, while interacting with the technology, the interface can be used without previous learning.

In the past decade, the post-“Windows Icons Menu Pointer” era, many efforts have been devoted in the research of natural interaction. New devices are invented such as interactive walls, muti-touch screen, tablets, smart phones and Xbox Kinect. The growth of embedded and networked devices collectively termed as the ‘Internet of Things’ (IoT) has become an enabler for facilitating richer context awareness and more natural interaction in the intelligent environment. New interaction technologies are explored, such as speech recognition, computer vision, affective interfaces, silent speech, air-writing, gesture-based interaction, Virtual Reality (VR), brain computer interfaces (BCI), and other physiological interfaces such as electrocardiography (ECG), electromyography (EMG), electro-oculography, galvanic skin response (GSR), force, accelerometry, and eye gaze, among many others. These state-of-the-art technologies together with mobile and wearable intelligent devices open up many new research opportunities focusing on human centred design with a particular focus on extending the contextual sensing and natural interaction of ubiquitous systems and intelligent environments. Consequentially there will be need for researchers and industry to develop more unobtrusive and natural communication between computing artefacts and users to make pervasive systems more aware and reactive to their feelings, moods and desires.

This thematic issue on natural interaction in Intelligent Environments (IEs) aims to showcase recent research within the HCI that focuses on how recent natural interaction technologies can be used to develop personalized novel systems in context of user behaviors and interaction in IEs. IEs are distributed network of intelligent devices that are sensitive, adaptive and responsive to the presence of people. Such environments are as varied as the places people inhabit encompassing settings such as homes, classrooms, offices, factories, transport, cities and online virtual worlds. Papers in this thematic issue try to answer the questions such as: How to provide multi-modal interactions in IE? Which modalities are best suited for different devices and tasks? How the system adapts with the intuition and preferences of its users? How the system supports interactivity based on the continuous interpretation and processing of tasks, activities and contexts? How to provide responsiveness to the user? How can we use user-centered design in IEs? How to improve user experience?

The guest editors of this JAISE thematic issue, Liping Shen, Andrés Muñoz and Tongzhen Zhang have endeavored to make this a relevant, timely and interesting collection of articles for the ambient intelligence and smart environments community. The thematic issue received a total of 17 submissions from which five high quality papers were selected based on double blind review process. This is a rapidly growing area of research and this issue only shows a selected spectrum of the work being done in this area. We look forward to hearing from readers about other areas and problems that need to be addressed for the natural interaction in IEs.

2. In this thematic issue

Interaction with robots is one of the main research lines related to natural interactions. Not only are the synergies between a robot and the intelligent environment to be considered, but also the communication between human beings and the robot is a crucial aspect. Sandygulova, Dragone and O’Hare propose in their paper entitled ‘PRIveT – a portable ubiquitous robotics testbed for adaptive human-robot interaction’ a testbed aimed at supporting interaction studies among human users and IEs. An interactive and autonomous robotic system is presented to conduct the study. This robotic system is able to collaborate with users by reacting to their activities and even helping them to understand the basic concepts beneath the robotic system. Their work offers appealing reflections on testbed features for this kind of robotic systems. Egerton and Lakshantha focus in their paper ‘A diagrammatic framework for intuitive human robot interaction’ on an augmented reality-based framework to allow expert and non-expert users to control and program robots in an intuitive manner. It provides a diagrammatic service as a visual form of two-way interactions with robots including several types of markers including marker-less objects for augmented reality. Their results show greater situational awareness and improved task completion times when compared to conventional HCI methods.

In order to represent natural interactions in IEs some type of knowledge model must be adopted. Caballero, García-Valverde, Pereñíguez and Botía address in their paper ‘Activity recommendation in intelligent campus environments based on the Eduroam federation’ a model for a recommendation system within a campus environment based on the users’ interactions with the secure federated network Eduroam. This system stores users’ previous interactions in a transparent manner to offer recommendations without a prior configuration even in other Eduroam-federation universities different from the user’s home one. Ni, Pau de la Cruz and García Hernando introduce in the paper entitled ‘A foundational ontology-based model for human activity representation in smart homes’ a model for human activity in Smart Homes based on the NeON methodology. This activity model is composed of a network of ontologies grouped in three categories: user ontologies, Smart Home context ontologies and Activity of Daily Living ontologies. The aim of this proposal is to facilitate a generic application development for Smart Home systems despite of the different types of sensor devices or human activities. To achieve this goal, authors integrate the aforementioned ontologies in an upper ontology known as DOLCE+DnS Ultralite.

Finally, an architecture to address users’ needs in interactive context-aware applications is introduced by Preuveneers, Berbers and Joosen in their paper ‘SAMURAI: A batch and streaming context architecture for large-scale intelligent applications and environments’. Their architecture integrates and exposes well-known components for complex event processing, machine learning, and knowledge representation. It is built around key concepts of the Lambda architecture (including batch, speed and service layers) and leveraged big data enabling technologies to achieve horizontal scalability and responsive interaction with its users. Applications of the SAMURAI architecture are shown in the healthcare domain and context-aware authentication.