Abstract
Web of Objects uses a semantic ontology to create and provide knowledge-based Internet of Things services in terms of virtual objects and composite virtual objects. The objects are created through virtualization with the use of semantic ontology to form the virtual objects, where multiple virtual objects are combined to form the composite virtual objects to offer the services. Beacon-enabled Web of Objects is the extension of the existing web that allows the beacon to broadcast the uniform resource identifier of the real-world object to the nearer mobile devices. Based on the user selection and request, the service is handled and offered by the Web of Objects platform; thus, an architecture on the beacon-enabled Web of Objects has been proposed in this article. To offer the knowledge-based services, a knowledge creation model has been presented. To realize the knowledge-based service features on the proposed architecture, a use case scenario has been presented and a conceptual semantic ontology model has been designed. Finally, to assess the features, a prototype has been implemented and demonstrated on the use case scenario.
Introduction
It is a new trend to provide an intelligent brain in Internet of Things (IoT) ecosystem to create knowledge and analytics to invoke decisions, actions, and business outcomes. The IoT ecosystem combines intelligence at the edge, enterprise level, or both. Within the wide diversity of available IoT objects and service features, user faces problem of searching and identifying the right service functions to support user’s demand. Considering these requirements, we have identified two key factors in distributed IoT environments:
How the information regarding the most appropriate IoT objects will be delivered to the user to choose from the multiple objects;
How the objects will be defined, communicated, and combined to create intelligent services to optimize the available resources that enhances the existing services.
To support these two key factors, an architecture on beacon-enabled Web of Objects (WoO) 1 has been proposed in this article that integrates the beacon technology with the WoO platform, so that
The first factor will be supported by a beacon that broadcasts uniform resource identifier (URI) of the real-world object to the mobile devices, and user will choose and tap on a link that refers to the concerned objects;
The second factor will be supported by a WoO platform that supports knowledge and intelligence through the use of semantic ontology.
In the beacon-enabled WoO platform, a beacon is used to broadcast the URI of the real-world object to the nearer mobile device through a standard protocol, once user taps on a link of the object, then all the processes starting from the service request analysis till the service execution are handled in the WoO platform. Using the beacon-enabled WoO, a new era of IoT services will be emerged in the WoO platform. Thus, to describe the beacon-enabled WoO platform, this article focuses on the following issues:
How a beacon will be used for the identification of the real-world objects in the platform;
How and why the object will be virtualized in the web environment;
How the knowledge-based service will be offered in terms of composite virtual object (CVO) in the WoO;
How the knowledge will be created about the available objects.
Focusing on these issues, the proposed architecture supports advertisement of available objects and hides the complexity of the virtual object (VO) in the distributed application domain. It provides approximation and reuse of VOs and CVOs using cognitive functionalities, and updates the knowledge through the context awareness of the available objects.
Section “Architecture on the beacon-enabled WoO” proposes and discusses the beacon-enabled WoO architecture focusing on the identified issues. Section “WoO platform in the proposed architecture” discusses the object virtualization, virtualization process, creation of CVO, and knowledge creation in the WoO platform, which are the core parts in the proposed architecture. Section “Instantiation of beacon-enabled WoO platform” presents and discusses the instantiated architecture of the beacon-enabled WoO platform and the knowledge-based service creation. Section “Use case and prototype implementation” discusses the implementation aspects of a prototype on a use case scenario and its demonstration. Finally, this article concludes in section “Conclusion.”
Related works
Research works have been going on regarding the identified issues, such as object virtualization and knowledge-based service creation. As an early stage, research on the Physical Web has been going on that specifies the details of the beacon technology.
Interoperability among devices in WoO plays important role through the object virtualization. Virtualizing the object to form the VO to represent the features and capabilities of the object in the virtual world is possible with the use of semantic ontology. Object virtualization and features of WoO platform have been discussed in Kibria et al., 2 Shamszaman et al., 3 Kibria et al., 4 Shamszaman et al., 5 and Kibria and Chong. 6
Real-world object virtualization and their virtual composition to provide the intelligent services require selecting and combining functionalities of multiple VOs. Cognitive functionalities can select most relevant VOs of an application dynamically. Abstraction of technological heterogeneity derived from vast amount of heterogeneous objects and the development of a cognitive management framework comprising the VO/CVO have been discussed by iCore project in Foteinos et al., 7 Sasidharan et al., 8 and Kelaidonis et al. 9 Ubiquitous service provisioning can be ensured by composing VO/CVO and service policies. To compose a service, the composition architecture, methodology, and algorithm have been discussed in Ara et al. 10 and Kim et al.11,12
In a previous work, 13 objectives of the Do-it-Yourself Smart Experiences (DiYSE Smart Experiences), such as creating awareness, setting up and controlling applications by user own self, and interactive and following experiences in the IoT, have been discussed. The creation of applications for networked tangible objects is also enabled in DiYSE. The semantic ontology model is applied in the sensor network, smart space, and smart location to provide various IoT-related services. Thus, ontology evaluation, ontology alignment, and matching functionalities have been described in previous works.13–15
The knowledge is treated as a meaningful information, which can be expressed using a semantic ontology and acquired through the study, investigation, and observation over the course of time. Acquired knowledge can be organized by indexing and filtering of the contents. In Jurisica et al. 16 and Li et al., 17 the concepts to capture and exploit the meaning of information for representing, organizing, creating, and using the knowledge have been presented. Knowledge can be organized as a subject–action–object manner. In Staab et al., 18 a framework, tools, and methodology for developing ontology-based knowledge management system have been presented.
To create the knowledge as well as to provide user centric services in the ubiquitous IoT environment, the system should be aware of the context of the objects. Context awareness enables the automatic modification of the system behavior with a minimum human intervention. An application can offer context aware service if the application system has the knowledge regarding the context of the objects, location, and user. The context aware services and the application to build better service provisioning have been discussed in BUTLER project. 19 Providing right service at the right time is the goal of a context aware system, which requires comparing and matching the available context information. In Yao and Kumar 20 and Liu et al., 21 context-based services using semantic ontology have been discussed.
Identification of the real-world objects can be done with the help of physical web that can provide lightweight discovery and interaction to the IoT. Physical web connects the real-world objects to the web through the beacon. The open source and technical details on the physical web related project have been provided on GitHub. 22 To broadcast the URI, several protocol (e.g. Eddystone) specifications for defining the Bluetooth low energy (BLE) message format for the beacon message have been provided in previous work. 23
Architecture on the beacon-enabled WoO
As the number of users has been increasing exponentially, the number of user requirements has also been increasing that requires more objects to be connected. Instead of searching from the huge number of connected objects, it would be convenient for the users if the information regarding the real-world objects could be advertised to them. Beacon-enabled WoO provides identification capability of the real-world objects using URIs, where the objects are represented through the VOs in the WoO platform. Figure 1 shows the high-level view of the proposed beacon-enabled WoO architecture. Real-world objects are virtualized in the VO level, their URIs are broadcasted to the mobile devices, and a harmonization among the real-world objects and VOs is performed for provision of intelligent IoT service on the WoO platform.
High-level view on proposed beacon-enabled Web of Objects architecture.
The architectural components of the beacon-enabled WoO consist of the beacon, mobile device, web server, and WoO-based platform for IoT services. The WoO-based platform for IoT services provides a personalized WoO platform that represents the virtual space.
Beacon-enabled WoO allows the interaction between the mobile device and the WoO platform. It creates a system where beacons broadcast URIs over the Bluetooth or WiFi to the nearer mobile devices. In response, mobile devices listen to the nearby beacons and display the notification of the surrounding nearby beacons. When user taps on the notification of beacons, the list of top ranked and filtered URIs is displayed on the mobile devices. On advertising URIs of real-world objects, identified VOs are mapped and collaborated with service functions in WoO platform to offer services at different locations, such as shopping mall, hospital, bus/train station, airport, restaurant, museum, etc.
To broadcast a URI, different protocols including Eddystone, multicast Domain Name System (mDNS), Universal Plug and Play (UPnP), and Simple Service Discovery Protocol (SSDP) are available that run over the Bluetooth for the low-power consumption. Due to the use of BLE, Eddystone is preferred in the proposed architecture, because it supports the format of BLE packet that can be configured in the beacon.
The beacon scanner scans the nearby beacons and displays the associated information, such as the title, description, and URIs on the mobile device. The scanner ranks the URIs and filters spam URIs based on the browsing history and the context of the objects. If user taps on the URI, the request is directed to the server and server responses with the requested service. Figure 2 shows the sequence of interactions on the beacon-enabled WoO platform.
Sequence of interaction on the beacon-enabled WoO platform.
The four issues identified earlier are supported in the beacon-enabled WoO platform. We have already discussed on how the beacon will be used for the identification of the real-world objects. The details on the object virtualization in WoO platform and the knowledge creation have been described in later section.
To control and maintain the real-world objects, to achieve the intelligence, and interoperability among the VOs, a semantic ontology is used to virtualize the objects. Semantic ontology based system provides intelligent services that maximize and optimize the available resources through assessing, accessing, and reusing the related VOs to the applications.
Intelligent IoT services are offered in terms of CVOs that are created by combining the functionalities of multiple VOs and service rules. To execute the knowledge-based service, CVO receives the service execution request that is generated by analyzing the service request and the real-world knowledge. The cognitive functionalities at this level enable the system to iteratively monitor the CVO and update the real-world knowledge.
Knowledge creation and modification is a continuous process that uses machine learning mechanisms. The raw data collected through the VOs are detected, reasoned, and classified to project the context of the objects. To express the knowledge, a semantic ontology is used.
WoO platform in the proposed architecture
Comparing to the existing work, the main advantages of the proposed architecture are support of detection and approximation of the relevant VO and CVO for reusability, and knowledge-based intelligent services. Once the user taps on the link of the object on the mobile device, the request is directed to the web server and the intelligent services are instantiated and executed on the WoO platform and offered to the user. The WoO is an application level platform that facilitates application deployment, maintenance and operations, and integration of isolated information from the different application domains. The main principle of the WoO is the extension of the web into the virtual world, which involves interaction with the real-world objects in the ambient environment. Figure 3 shows the technical view of the proposed beacon-enabled WoO architecture.
Technical view of the beacon-enabled WoO architecture.
Communicating, accessing, and controlling the real-world objects are not always possible directly. Embedding computational capabilities in these objects will provide qualitative services remotely by combining the collected data through the VOs and from web.
To define a VO, domain-specific VO information model is used that contains the information regarding the object. The attributes and properties of the object are described in Resource Description Framework/Extensible Markup Language (RDF/XML) format. Web Ontology Language (OWL) is used to describe the relationship among the VOs. OWL provides the classes, properties, and data values, which can be stored as a semantic web document written in RDF. The defined VOs are stored in the VO repository at the VO level and its data are stored in the database. Each VO, stored in the VO repository includes the VO address information, unique identification, status of the VO, service interface, access rights, literal describing VO meaning, timestamps to monitor the objects, and list of functionalities. The VO execution module provides the interface to the CVO layer.
To maintain the constant flow of data, cognitive functionalities at the VO level continuously monitor the association between the real-world objects and the relevant VOs. In case of error in the association, the cognitive functionalities report to the CVO level that replaces the VOs.
The cognitive capabilities at the CVO level enable the interconnection and interoperability among the VOs and CVOs that lead to intelligent decision through reasoning. Monitoring and collecting the context information, evaluating them, and reasoning based on the threshold values can provide the intelligent decision. The cognition in the WoO can be achieved by inferencing the domain-specific VO and CVO that requires the CVO to be defined in advance. The components in the defined CVO enable the cognition that allows the reuse of the CVO for the efficient and fast service. The CVO has the capability to inherit all the properties and features of the relevant VOs, which combines the functionalities of VOs and maintains the orchestration with other entities within and outside the domain.
Initially, CVO template is created based on the CVO information model, service requirements, policies, and relevant VOs. The created CVO is stored in the CVO template database which can be reused if similar new CVO needs to be instantiated. The CVO template contains the information including the unique identifier of the CVO and VO, and the metadata that describes the functionalities of the CVO. Each CVO also includes access control information to restrict the access to the CVO. Each time a CVO is instantiated from the CVO template, the related VOs and CVOs are referred from the VO and CVO repository.
In WoO, a service is the logical mash-up of CVOs, relevant VOs, and service policies. The service template is created and stored in the service template repository for reusing in case of similar service creation. For executing the requested service, the request parameters are extracted from the service request that generates the service execution request in the service level. Access control mechanism at the service level restricts the user to access and use the service.
Providing the right service at the right time is a challenging issue due to dynamic user context and service requirements. For the fast execution of a service, the context of the service is matched with the stored context of the service in terms of VO, CVO, location, and time. Accurate context matching might not be possible all the time, but a satisfactory percentage of matching allows each level in WoO to select, bind, and execute the service. A matching algorithm to match and compare the context of the service has been developed, where only the location has been considered as a context for the simplicity. Figure 4 shows the matching algorithm.
Matching algorithm that compares and matches service based on location context information.
In WoO, knowledge of the real-world is created and updated by processing the context information, such as the stream of VO collected data. To fulfill the application requirements in dynamic environment, the system should have the ability to automatically update itself that requires the perception of the context. Thus, to generate the context aware service execution request, real-world knowledge is important. In the process of knowledge creation, the data collected through the VOs is aggregated to detect the event, where a time window is added to monitor the objects at a time interval. To detect the current context information from the aggregated data, the information is processed using the event processing technique, such as Esper 24 at the CVO level. The detected context features are then reasoned with the inference engine that recognizes the current context to classify the context status. To update the knowledge, a prediction of the context of the objects is projected by applying machine learning algorithm on the classified context status. At the service level, the knowledge regarding the real-world objects is analyzed with the service request parameters to generate the execution request. Figure 5 shows the knowledge creation process and update mechanism in the WoO platform.
Real-world object knowledge creation to provide intelligent IoT services.
Instantiation of beacon-enabled WoO platform
To provide the knowledge-based IoT services and to address the issues mentioned in section “Introduction,” the proposed architecture on the beacon-enabled WoO has been instantiated. For better understanding, the instantiated architecture has been discussed using top to bottom approach.
Service level provides an interface between the WoO platform and the user. Service request is received through the application program interface (API) and accepted from authorized user. If the request is in natural language form, the natural language processing technique is used to translate into the service request. The requested service parameters are extracted from the service request. The machine learning technique at the service level helps in dynamic service composition by considering the real-world knowledge. Service request parameters and the real-world knowledge are analyzed to generate the request and context parameters. Based on these parameters, the already instantiated services are matched in the service registry with the approximation and reuse function. If similar service is not found, the service request and context parameters are delivered to the CVO level for service composition and execution.
The cognition capabilities at the CVO level enable the detection, approximation, and re/use of relevant CVO that is appropriate for the application domain and service requirements. It is not possible to match the same CVO all the time, thus closest CVO is searched that can provide similar functionalities for the requested CVO. Thus, a correlation matrix is used to calculate the correlation between the functions in the requested CVO and the functions offered by already instantiated CVO. Correlation matrix holds the integer value of 0 and 1; in the matrix 0 means that there is no correlation among the functions and 1 means that the requested function of the CVO can be served with the function in the instantiated CVO. Reusing of existing CVO increases the efficiency of computation and time that leads to resource optimization. If similar CVO is found, then it is instantiated directly from the CVO registry; otherwise, the CVO needs to be created from the CVO template that refers to the relevant VOs in the VO registry. Figure 6 shows the instantiated architecture.
Instantiation of the proposed architecture.
In the similar way, the VO is instantiated if requested VO is available in the VO registry; otherwise, VO is created following the VO information model.
Use case and prototype implementation
Beacon-enabled WoO will expand the scope of the existing IoT services. To realize the features of the WoO mentioned in section “Introduction” and the intelligent IoT services on the proposed architecture, a use case scenario at a train station has been studied. This section demonstrates the use of the proposed architecture and discusses the implementation aspects of a prototype on the use case scenario.
Use case scenario
Beacons deployed at a train station, broadcast information regarding the objects, such as products at the shop, city information, hotel information, different available services, such as ticket counter, wash room, and exit. Besides the deployed beacons, different sensors and actuators are also deployed to monitor the status of the station to take necessary measures.
In case of emergency situation, the system automatically sends an emergency notification and a safe exit path to all smartphones. Passengers are also able to search the safe exit that is nearest to them by tapping the URI that refers to the exit. Based on the passenger’s location, the nearest safe exit including the direction to the exit is displayed on their smartphone.
Proof of concept
We have identified a smart station application domain in the use case scenario. To support the domain, the NormalSituation CVO and the EmergencySituation CVO have been instantiated. NormalSituation CVO is used to classify the normal situation and EmergencySituation CVO is used to classify the emergency situation at the station. Different VOs have been defined to represent the objects, such as passenger, security personnel, product, smartphone, closed-circuit television (CCTV), sensors (e.g. temperature, humidity, and CO), and control center. Multiple instances of the VOs have been instantiated such as TempSensor0001 and TempSensor0002 instance of temperature VO. An additional CVO, called Advertisement CVO, has been instantiated that is mainly triggered to send message if any sale on the products is offered or price is reduced at the station.
The VO collected data is continuously monitored and analyzed based on the predefined rules and threshold values through the inference engine that classifies the current situation as normal or emergency by reasoning NormalSituation or EmergencySituation CVO. If the emergency situation is classified, then the application server (AS) provides the safe exit path to user smartphone.
Focusing on the issues mentioned in section “Introduction,” the following features have been assessed through the proof of concept:
Interconnection among objects;
Discovery of objects;
Access to functional capabilities of connected objects;
Intelligent service features in accordance with user centric, knowledge driven, and collaborating capability.
Prototype implementation
To realize the knowledge-based intelligent services on the proposed architecture and to assess the features, a prototype has been implemented on the use case scenario. Figure 7 shows the implementation architecture that consists of an AS, an ontology server (OS), ontology database, VO and CVO database, and a gateway. Four functional entities have been implemented in the AS including the service manager, control center, emergency service, and context analyzer. An API for the communication between the AS and OS and the VO and CVO manager function have been implemented in the OS. The VO and CVO database have been managed in the OS. A database has been maintained to store the data related to the application.
Implementation architecture on use case scenario.
Among the four functions in AS, service manager function has been implemented as an interface to communicate to the outside world. When a request is received in the form of URI, the service manager decodes the URI and maps to the referred VOs and CVOs. It sends the information to the OS through the API to compose the service by referencing the VOs and CVOs and service rules. Service manager also sends null information if the desired service is not available or the service cannot be composed. Based on the user request, service manager function directly provides the service if it does not require any computation, such as advertising any product, otherwise the function invokes other function to handle the computation.
The emergency service function has been implemented to perform the task at the emergency situation. If emergency situation is detected, then this function sends notification message along with the safe exit path to the user smartphone. It also sends notification message to the control center. The control center function sends message to the security personnel along with additional information, such as density area on the platform. This function enables a panel to display the fire area automatically, where the display area also can be selected manually by selecting specific CCTV.
The context analyzer function has been implemented to update the VOs based on the current context information. It identifies and processes the information related to the user/security personnel location, number of users, and density area that are sent to the emergency service function. Context analyzer function helps in reconfiguration of the safe exit path based on the newly detected fire place and the user location.
The AS sends HTTP GET or POST request to the OS using APIs. The OS provides APIs to AS to perform different operations using Java Servlets. Two APIs including UpdateExec to update the query execution and QueryExec to read or get the inferred result have been implemented for the VO and CVO manager. The VO and CVO manager function provides the interface to the AS to create, read, update, and delete operations on VO and CVO database.
In the prototype environment, sensors have been connected to the gateway through ZigBee. The users have been connected through their smartphone via received signal strength indicator (RSSI). The AS has been developed using Node.js and the OS runs on the Java platform. Java servlet has been used to provide the API to the AS. TDB has been used to store the ontology, VO, and CVO in RDF format. MongoDB has been used to store the data for simplicity.
A semantic ontology model has been designed based on the use case scenario using the Protégé. 25 The model has been represented in OWL in the ontology database. The main goal of the semantic ontology is to create a linked data; thus, the VOs and CVOs have been represented in RDF/XML format in the database. SPARQL has been used to query the VOs and CVOs by the OS. To integrate and reason the huge amount of internal data stored in the local database and the external data available on the web, the linked data is an essential part in the semantic ontology. Among the different datasets, DBPedia is the linked dataset that includes content in the RDF format. For testing the server, Hermit 1.3.7 has been used as the reasoner and Apache Jena has been used as inference engine in the deployment. The semantic ontology model has been shown in Figure 8, where the instantiated CVOs have been highlighted.
Semantic ontology model on use case scenario.
At the run time, the defined VOs in semantic ontology can be created by SPARQL. As mentioned earlier, VO and CVO manager in the OS has been used to manage the defined VOs by different operations, such as create, read, update, and delete operation. Figure 9 presents the “SmartPhone” VO description in OWL individual in the use case semantic ontology, the VO description includes a specific instance—“Phone0001.” The object properties and data properties have been used to make the relationship among the VOs and the instances. Relation among the VOs helps the AS to extract useful information through SPARQL query. The SPARQL query from the AS to OS is sent via HTTP GET request. SPARQL query has been used to update the VOs in the OS.
Snippet of the VO description in OWL.
The CVOs have been defined by combining the relevant VOs and service rules that describe the characteristics of the CVOs. Initially, the CVOs have been defined using the class expression on the Protégé, which have been converted into SPARQL query in the deployment. Figure 10 shows the snippet of defined CVO—NormalSituation in the ontology that has been described in OWL. In the CVO description, threshold values have been set for the reasoning purpose.
Snippet of the CVO description in OWL.
Prototype demonstration
We have demonstrated at our department floor, thus the layout of the user interface has been design based on the floor layout. In the demonstration, three sensors have been installed to collect the temperature, humidity, and CO data. All these three sensors and a smartphone have been connected through the gateway. Figure 11 shows the testbed environment that consists of (a) sensors; (b) gateway, sensors, and monitor that represented the digital signage; and (c) user smartphone displaying the safe exit path.
Testbed environment including (a) sensors, (b) gateway, and (c) user smartphone displaying safe exit path.
For testing the prototype, a fire was created, where three sensors were set earlier (Figure 11(a)) to sense the data. Sensors collected the data regarding the temperature, humidity, and CO and transmitted to the server periodically through the gateway (Figure 11(b)). Due to the increased values in temperature and CO and decreased value in humidity for the created fire, the emergency situation was classified correctly whenever these values crossed the threshold. In the same way, whenever we moved away the fire from the sensors, the normal situation was classified because the sensed values were set within the threshold. Whenever the emergency situation was classified, the server provided the safe exit path on the user smartphone (Figure 11(c)).
Finally, it can be summarized that the features have been assessed through the proof of concept in terms of the following:
Domain-specific object virtualization using the semantic ontology;
Implementation of a function in the OS that discovered the VO and CVO using SPARQL query;
Implementation of four functional entities that handled the service request, monitored the context of objects based on the sensed values, and triggered the action based on the classification of the emergency situation;
Instantiation of three CVOs by combining relevant VOs and service rules, knowledge expression using the semantic ontology that maintained the VOs as the hierarchy of classes and subclasses, and classification of the emergency situation.
Conclusion
Due to exponential growth of Internet users and the connected IoT objects, providing the appropriate services is one of the key challenges. To address the challenge, beacon-enabled WoO platform has been proposed. A beacon is used to broadcast the URI of the real-world object to the nearer mobile device through a standard protocol, once user taps on a link of the object, all the processes starting from the service request analysis till the service execution are handled in the WoO platform.
WoO platform enables virtualization, service composition, and harmonization among multiple VOs and extends the capabilities of open IoT services.
In the ubiquitous IoT environment, knowledge-based service provisioning requires monitoring and identifying the current context of the real-world objects. The processes of context detection, classification, and situation projection that update the real-world knowledge have been discussed here. To realize the knowledge-based service provisioning on the beacon-enabled WoO platform, the proposed architecture has been instantiated. For the proof of concept, a prototype has been implemented and demonstrated. The prototype worked well.
In this article, the performance has not been evaluated. In the future, we aim to enhance the VO composition and classification mechanisms, address scalability issues, and evaluate performance of the system.
Footnotes
Academic Editor: Takeo Fujii
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIP) (No. B0113-15-0002, Development of Self-Learning Smart Ageing Service based on Web Objects).