Two-Dimensional Code Based Spatiotemporal Modeling and Its Application in Object Tracing

Abstract

Space-time consistency is of great importance for mapping things from physical world to cyber world, which could be one of the fundamental issues in constructing cyber-physical system (CPS). A two-dimensional code spatiotemporal modeling method is proposed in this paper. We apply the two-dimensional code as an information carrier to store object's characteristics (object's ID, attributes, instructions, spatiotemporal data, etc.), the smartphone as a mobile device to obtain the object's real-time spatiotemporal data with its system time and GPS module; then the object's spatiotemporal information can be transmitted and stored in a backend server with a database. Meanwhile, the object's spatiotemporal information can also be inquired by smartphone users and other web users. A use case that applied this proposed method to trace object's smartphone information is described later, and the result shows that tracing objects with two-dimensional code has many advantages, such as fast reading and transmitting, low cost, and robust, confidential, and strong readability; furthermore, the proposed modeling method keeps the consistency of object's time and space in CPS.

1. Introduction

Nowadays, two-dimensional code has developed into a very prevalent stage in modern commercial activities, such as product antiforgery, ad serving, web links, data downloading, commodity transaction, positioning and navigation, electronic certificates, information transfer, and Wi-Fi sharing. Being used as the unique ID in many applications or in storing several objects’ basic attributes has been discussed in some literatures. RFID solution can help companies to trace their products throughout the entire lifecycle, but this solution costs a little higher. 2D data matrix codes are a lower cost alternative which owns the merits of high accuracy and low cost [1]. However, these applications need specific read devices to read and decode two-dimensional codes. Usually, the devices are not easy to take. As personal companions, smartphones are used widely in our everyday life, and the equipped cameras on smartphones extremely extend the application scope of two-dimensional code. In [2], the authors examined 6 two-dimensional codes for camera phone applications and discussed the global standards for camera phone applications. Tarjan et al. suggested using a quick response two-dimensional barcode (QR code) to record the key tracing data during the transformation stages of individual food product and analyzed the readability of QR code of variable contents, size, and data error correction levels with smartphones running an android platform [3]. Libraries are enjoying the benefits brought by two-dimensional barcode and smartphones. Ashford discussed several examples of QR code used in libraries in US and he argued that we should take care to apply QR codes where they really make our users’ lives easier [4]. Meanwhile, safety and security are another issue apart from convenience in two-dimensional code applications. Wang et al. proposed a new optical encryption technology based phase retrieval algorithm and QR code, the encryption process of the proposed method is greatly simplified, and the encryption system showed strong invulnerability to various attacks [5].

There is a growing interest in being able to model phenomena that vary both in time and across space. Parent et al. discussed a spatiotemporal conceptual model addresses both spatial and temporal modeling together and covers the requirements at both logical and conceptual level [6]. Furthermore, time and space are the basic attributes of objects in the physical world and are significant during objects mapping from physical world to cyber world [7]. And time space consistency is of great importance to realize cyber-physical system (CPS), which could be one of the fundamental problems in CPS. In this paper, a spatiotemporal model to trace objects with two-dimensional codes printed or pasted on them is proposed. The object's real-time space and time data can be recorded by smartphones and then transmitted and stored in a backend server. Meanwhile, the method provides spatiotemporal data query service. During the modeling procedure, the time and space attributes of objects keep consistent. And the real-time location and object tracing are useful in logistics, health, safety, other location-based services, and so on.

In this paper, the authors take the advantages of two-dimensional code and smartphones to accomplish a spatiotemporal modeling and realize objects tracing. Firstly, Section 2 provides the discussion of the motivation for the modeling. The features of two-dimensional code and smartphones provide many benefits to trace objects. Then in Section 3, the two-dimensional code based spatiotemporal modeling method is introduced in detail, including its system architecture and the specific working procedure. In Section 4, a use case developed with android phone and MySQL is present to test the function of the proposed method. A conclusion is drawn in Section 5.

2. Motivation for Modeling

2.1. Spatiotemporal Modeling and Space-Time Consistency in CPS

Cyber-physical system (CPS) aims to realize a system to combine computing, network, and physical environment, which can provide services such as real-time sensing, dynamic control, and information service by integration and collaboration of computation, communication, and control (3C) technologies.

Objects in physical world can be mapped into cyber world, and then we call them cyber entities as Figure 1 shows. We study and summarize the characteristics, attributes, and relationship of objects in physical world, and these could be constructed in cyber world. We could find some better solutions by studying the characteristics, attributes, and relationship of cyber entities which are mapped from physical world. Time and space are the basic attributes of objects in the physical world and are significant for objects when mapping from physical world to cyber world. The unique and consistent time and space attributes of objects keep confusion away during mapping and interacting between physical and cyber world. So a two-dimensional code based spatiotemporal modeling with the help of smartphones is discussed in this paper, which can help to solve the space-time consistency problem.

Figure 1

The mapping and interaction between physical world and cyber world.

2.2. Two-Dimensional Code

Two-dimensional code is another readable code, which develops from one-dimensional code. The binary data (0 and 1) which is widely used to store data in computer technology can be represented by white and black rectangular pattern and is stored in the two-dimensional code. The stored information will be obtained after being scanned and decoded by the reading devices. The two-dimensional code used as an information carrier to store and transmit data in the proposed spatiotemporal modeling has some unique advantages [8, 9]:

• (i)

informative: the two-dimensional code can store as many as 1850 capitals, or 2710 Arabic numbers, or 1108 byte data, and it can also store more than 500 Chinese;

(ii)

great correction ability: the information in two-dimensional code can still be decoded when the code is partly damaged, and even 50% of the code area is damaged;

(iii)

high reliable decoding: its error rate is less than 1/10,000,000 that is much lower than general code, which is 2/1,000,000;

(iv)

high security: the related special decoding algorithm should be applied when the two-dimensional code is encoded in a special way. Meanwhile, encryption can also be applied to enhance the security. In [10], a method using photon-counting encryption and implemented with phase encoded QR codes to accomplish object authentication is proposed;

(v)

low cost: the two-dimensional code is easy to make and it is much cheaper than IC or RFID cards;

(vi)

easy to read: it can be read by laser and CCD reader. A smartphone with a camera can also act as a reader.

The above features make two-dimensional code a good choice to act as the tag, which stores the unique object's ID and other pieces of information.

2.3. The Smartphone with a Camera and GPS Module

The smartphone acting as a personal assistant has more and more powerful multimedia processing ability and network function. This characteristic makes it adequate to be the key device in the proposed model. The camera on the smartphone can read the two-dimensional code, the system time and GPS module can provide the real-time space and time data, the powerful CPU equips the ability to encode and decode the two-dimensional code and process other tasks, and the network function can keep it always connecting to the Internet and communicating with the server. These features help smartphone accomplish the spatiotemporal modeling with two-dimensional codes on objects and a backend server.

The smartphone is used to replace the traditional two-dimensional code reader in the spatiotemporal modeling. Smartphone is light and easy to take, and it is with us almost all the time. It is more convenient to use the smartphone to read two-dimensional code than to take a traditional reader.

The system time of smartphone can be registered by the Mobile Telecom Carrier at any time. The real-time location of the object can be obtained by the GPS module in the smartphone. All the location data come from the GPS system. The system time and location data are registered all the time. The space-time consistency performs well.

3. Two-Dimensional Code Based Spatiotemporal Modeling

3.1. System Architecture

The proposed modeling system consists of 3 parts: the two-dimensional code printer, the smartphone App, and the server with a database. Figure 2 is the architecture of the model.

Figure 2

The architecture of the spatiotemporal modeling system.

3.1.1. Two-Dimensional Code Printer

The two-dimensional code printer creates the original two-dimensional code, which is printed or pasted on the object to be traced. Here we call the original two-dimensional code $C_0$ . $C_0$ is the unique identifier of the object, and it stores the object's ID and its basic attributes. It will be sent to the server after being created and printed on the object, so that the server can record the object's ID and its attributes. The reserved storage space for the object's spatiotemporal data will be created in the database by the server. Only the IDs of objects from two-dimensional code printer will be accepted by the server. Others will never get service from the server.

3.1.2. The Smartphone App

The smartphone App is the most important part to collect the spatiotemporal data in this modeling. Smartphone becomes more and more popular, and many people keep more than one. The advanced information technology brings more powerful functions for smartphones. The smartphone with a build-in camera can be used as a two-dimensional code reader and generator. The system time module and GPS module in the smartphone can provide the real-time spatiotemporal data. The powerful network function can keep the smartphone connecting to the Internet all the time. This feature can help to transmit the spatiotemporal data to the server and query the object's spatiotemporal trajectory.

The smartphone App has three basic functions: user authentication, spatiotemporal data obtaining and transmitting, and the trajectory query.

(a) User Authentication. Not all spatiotemporal data obtained by a smartphone App can be stored in the server. The server will refuse to serve data from unqualified users. The user's identity should be verified first before data transmitting.

(b) Spatiotemporal Data Obtaining and Transmitting. When the user scans the two-dimensional code ( $C_0$ ) on the object with smartphone App, the App will decode it and get the object's ID. At the same time, the system time and location data can be read by invoking the related function. Then the App encodes the object's ID, the current spatiotemporal data, and the storing instruction in another two-dimensional code, called $C_1$ . After encoding, $C_1$ will be sent to the server via GPRS, 3G, 4G, or Wi-Fi.

(c) Trajectory Query. When the user wants to know the object's spatiotemporal track, a query request will be sent to the server. The request information including object's ID and the query request is also stored and transmitted in a two-dimensional code, we call it $Q_0$ . The ID can be entered by user or scanned from the original two-dimensional code ( $C_0$ ).

3.1.3. The Server with a Database

The server's mainly task is to receive and decode the two-dimensional code including object ID, spatiotemporal data, and instructions from the printer or the smartphone and then respond to the instructions. After obtaining a storing instruction, the server will search the database to match the object's ID and then store the spatiotemporal data in the reserved space. If the server receives the object track query request, the object's ID will be matched and the stored spatiotemporal data will be searched and sent to the user who requested it. The server can proceed with the user authentication procedure as well.

3.2. Spatiotemporal Modeling Procedure

Table 1 is a list of abbreviations used to describe the modeling procedure.

Table 1

A list of abbreviations used.

$$C_{\mathit{0}}$$	Original two-dimensional code, printed or pasted on the object, contains object's ID and its basic attributes
$$C_{\mathit{1}},C_{\mathit{2}},\dots ,C_n$$	n two-dimensional codes generated by smartphone App, they are generated after smartphone scanned $$C_{\mathit{0}}$$ and contain object's ID, spatiotemporal data, and storing instruction
$$Q_{\mathit{0}}$$	Two-dimensional code generated by smartphone App when the user wants to query the object trajectory, it contains object's ID, a query request
$$Q_{\mathit{1}},Q_{\mathit{2}},\dots ,Q_n$$	The response to user query from the server, it stores object's trajectory information
$$C_{\text{pw}}$$	Two-dimensional code generated when the user needs to authentic itself
$$T_{\mathit{1}},T_{\mathit{2}},\dots ,T_n$$	Real-time of the object
$$S_{\mathit{1}},S_{\mathit{2}},\dots ,S_n$$	Real-time location of the object

This proposed modeling system aims to realize the object's real-time tracing and spatiotemporal data query. When $C_0$ printed on object is scanned, the smartphone reads and decodes it, and then the object's ID can be got by the smartphone App. At the same time, the system time and current location data can be read by calling the related API. The smartphone generates $C_1$ to store the object's ID, the current time, and location data, storing instruction, and then transmits $C_1$ to the server. After receiving $C_1$ , the server will decode it and get the object's ID. The server will search the database to match the ID, if it is stored in the database, and then the object's time and location data will be stored in the database in order. Otherwise, the data will be discarded. When another scanning is proceeded by smartphone, another spatiotemporal data and object's ID will be stored and transmitted in another new two-dimensional code, and the server will proceed another decoding and spatiotemporal data storing. After several rounds, the object's whole spatiotemporal trajectory will be stored in the database of the server. When the trajectory query request arrives, the server will identify the object's ID and decide whether to answer the request or refuse it. The response information from the server is also stored and transmitted in two-dimensional code. That is to say, all the communication between the server and smartphone is accomplished by two-dimensional code as an information carrier. This communication method is chosen because of its special privacy mechanism.

3.2.1. User Authentication

It is a very important issue to ensure that the data stored in the database is valid for the system security. The user authentication is needed when the two-dimensional code is created by the printer, when the smartphone transmits the object's real-time spatial-temporal data to the server and when the user intends to query the object's spatiotemporal trajectory. We can verify the user's identity with its name and password. When the user proceed the login process, its name and password will be encoded in another new two-dimensional code: we call it $C_{\text{pw}}$ . $C_{\text{pw}}$ also contains the authentication instruction to ask the server to accomplish the authentication. The server decodes $C_{\text{pw}}$ and checks the user's identity and then decides whether this user is a qualified user. Only the qualified user can accomplish its task with the server. After identity authentication, the server will get ready to allocate storage space in the database or receive the spatiotemporal data from the user or respond to the user's query. For a better description, we suppose that the user authentication has been already done successfully in the modeling procedure.

3.2.2. Printing the Two-Dimensional Code

The original two-dimensional code (i.e., $C_0$ ) printed or pasted on the object stores the object's ID and basic attributes. When $C_0$ is created, it will be transmitted to the server. The server creates a form to store the object's ID and its basic attributes and then allocates space to store the spatiotemporal data in the database:

•  

$C_0$ : object's ID + basic attributes.

3.2.3. Spatiotemporal Data Obtaining and Storing Procedure

(a) Scanning Two-Dimensional and Transmitting the Spatiotemporal Data. This procedure is an important part of the spatiotemporal modeling. The spatiotemporal data will be collected by smartphones and then transmitted to the server. This procedure is divided into 4 steps.

Step 1 (scanning the two-dimensional code on the object).

The smartphone which is operated by the user scans $C_0$ , and then the object's ID will be got after decoding the information in $C_0$ .

Step 2 (obtaining the object's spatiotemporal data).

After getting the object's ID, the smartphone reads its system time ( $T_1$ ) and GPS data ( $S_1$ ). This procedure can be accomplished by invoking the related function provided by the smartphone operating system provider.

Step 3 (create a new two-dimensional code ( $C_1$ )).

Then the object's ID and the spatiotemporal data ( $S_1$ , $T_1$ ) will be encoded and stored in a new two-dimensional code: we call it $C_1$ . Meanwhile, $C_1$ contains an instruction which tells the server to store the spatiotemporal data:

•  

$C_1$ : object's ID + $S_1$ + $T_1$ + storing instruction.

Step 4 (sending $C_1$ to the server).

After $C_1$ is ready, it will be transmitted to the server:

•  

$C_1\to$ server.

(b) Storing the Spatiotemporal Data. When the server receives $C_1$ , the decoding process will be executed. Then the server can get the instruction which tells it to store the spatiotemporal data ( $S_1$ , $T_1$ ) carried by $C_1$ .

As an information carrier, $C_1$ stores the object's ID and spatiotemporal data ( $S_1$ , $T_1$ ). The server decodes $C_1$ and matches the ID in the database and then adds the spatiotemporal data to the space which stores this object's ID:

•  

$S_{\mathrm{1}}$ , $T_{\mathrm{1}}\to$ database.

When the user scans $C_0$ again, another group of spatiotemporal data is created, and Steps 2, 3, and 4 above will be repeated. After n rounds, n groups of spatiotemporal data will be stored in the database of the server. The stored spatiotemporal data represent the object's spatiotemporal track. Table 2 shows the stored data of object's spatial-temporal data.

Table 2

Storing format of object's spatial-temporal data.

Object's ID

Basic attributes

( $$S_{\mathit{1}},T_{\mathit{1}}$$ )

( $$S_{\mathit{2}},T_{\mathit{2}}$$ )

( $$S_{\mathit{3}},T_{\mathit{3}}$$ )

$$\cdots \cdots$$

3.2.4. The Query Process

Step 1 (sending the query request to the server).

When the user intends to query the object's spatiotemporal track, the smartphone App creates another two-dimensional code: we call it $Q_0$ , which stores a query request and the object's ID. The object's ID can be entered by the user or scanned and decoded from $C_0$ . Then $Q_0$ will be sent to the server:

•  

$Q_0$ : object's ID + a query request → server.

Step 2 (server decoding and searching the database).

The server decodes the object's ID from $Q_0$ after receiving it, and then it searches the database to match the object's ID and gets the object's spatiotemporal data.

Step 3 (storing spatiotemporal data in a new two-dimensional code).

After the matched ID and spatiotemporal data are found, the server will send the spatiotemporal data to user. Here, the spatiotemporal data ( $S_n$ , $T_n$ ) and object's ID are stored in new two-dimensional code ( $Q_n$ ). The response of query is also stored and transmitted in two-dimensional codes:

•  

$Q_1$ : ( $S_1$ , $T_1$ ) + object's ID,

 

$Q_2$ : ( $S_2$ , $T_2$ ) + object's ID,

 

⋮

 

$Q_n$ : ( $S_n$ , $T_n$ ) + object's ID.

Step 4 (smartphone App decoding and displaying the tracing).

When the response of query is arrived, the App will decode the two-dimensional codes ( $Q_1,Q_2,\dots ,Q_n$ ) and extract the spatiotemporal data in them. The tracing result is displayed on the screen in front of the user.

4. A Use Case

We develop a smartphone App and a virtual web server to verify the proposed spatiotemporal modeling system. The result shows that the proposed method works well. Meanwhile, it has a couple of advantages, such as high speed, high security level, low cost, and being easy to use.

We choose Android 4.2 as the smartphone OS, because Android is most widely used OS on smartphone. We also choose eclipse 4.2 with the ADT plug-in as development tool and JAVA as development language.

The two-dimensional code printed on object is generated by website. There are many websites providing free two-dimensional code. In this use case, we choose QR code because it is widely used and its encoding and decoding work is easy to accomplish. Figure 3 is the flowchart of smartphone App. We also provide pseudocodes of the smartphone App as shown in Algorithm 1.

Algorithm 1

Entering user ID and password;

If (User is unqualified)

 Refuse to provide service and go to End;

Else

 Waiting for user's instruction;

Switch (User's instruction)

{

 Case Scan:

 {

  Initialize camera and take picture of the two-dimension code;

  Send the image to decode module;

  Obtain object's ID;

  Obtain GPS data and real-time time and date data;

  Encode object's ID, store instruction, space and time data into a new two-dimension code;

  Send the new two-dimension code to the backend server;

  Break to end;

}

Case Query:

{

 If (Entering the object's ID)

  Scanf object's ID;

 Else

 {

  Initialize camera and take picture of the two-dimension code;

  Send the image to decode module;

  Obtain object's ID;

 }

 Encode object's ID, query instruction into a new two-dimension code;

 Send the new two-dimension code to the backend server and wait for response;

 If (Get response data)

  Decode and display the result;

 Else

  Show failure;

  Break to end;

 }

}

End;

Figure 3

The flowchart of smartphone App.

The server consists of Windows 7, Apache, MySQL, and PHP language: we call it WAMP for short. The WAMP is a powerful network solution. Windows 7 is the OS where the web softwares run on it. Apache is one of the most popular web server software programs. MySQL is one of the best relational database management systems for web development. PHP is a general script development language used for web development. Figure 4 shows the flowchart of the web and Algorithm 2 is pseudocodes of the virtual web server.

Algorithm 2

Wait for message from smartphone App;

If (Receive a two-dimension code)

 Decode and get instruction and data in it;

Switch (Instruction)

{

 Case User identification:

 {

  Search the user's ID in the database and match its password;

  If (User is qualified)

   Return identification successful and break to end;

  Else

   Discard data and break to end;

 }

 Case Storing spatial-temporal data:

 {

  Search object's ID in the database;

  If (Not find)

   Regard it as a new ID and store object's ID, basic attributes, and spatial-temporal data;

  Else

   Store the spatial-temporal data in the array;

   Break to end;

 }

 Case Query trajectory:

 {

  Search object's ID in database;

  If (not find)

   Show failure and break to end;

  Else

  {

  Encode object's ID and spatial-temporal data in a new two-dimension code;

  Send the spatial-temporal data to smartphone App;

  }

 }

}

End;

Figure 4

The flowchart of virtual web server.

We pasted the two-dimensional code on an object and scanned it at several places randomly. The system worked well, the spatiotemporal data could be stored in the database, and the trajectory query also succeeded. Figure 5 is the query results by smartphone App; the data was got by smartphone App and stored in the virtual web server. And the benchmark results in Table 3 show that the time matches well but a little deviation in space data between the experiment result and the benchmark results.

Table 3

The benchmark results.

Number	Date time	Longitude	Latitude
1	2014-5-24 14:27:35	116.358114	39.990193
2	2014-5-24 15:27:41	116.356463	39.992318
3	2014-5-25 08:27:16	116.355721	39.987314
4	2014-5-25 09:31:18	116.367145	39.996342

Figure 5

The results of the query operation on smartphone App.

Figure 5 and Table 3 show that the system works well and the results are reliable to some extent. The time data is accurate enough, and the precision of space data could be accepted as well.

As Table 4 shows, traditional object tracing methods usually apply one-dimensional codes and optical reading devices. Compared with the traditional methods, the proposed method shows some advantages in portability, cost, correction ability, safety, and privacy. The traditional optical reading device is larger and heavier than a smartphone. What is more, the smartphones are so widely used that almost every one owns smartphone. They act as personal assistants in our everyday life and we take them everywhere. In a way, the proposed method does not need additional reading devices; the smartphones can take their place. It is the same case when comparing proposed method to traditional means in cost. We only need smartphone equipped with camera and GPS module not to buy specific reading devices. Two-dimensional code shows great correction ability, the information in two-dimension code can still be decoded when the code is partly damaged, and even 50% of the code area is damaged. Safety and privacy are another merit, because two-dimensional code applies specific encoding and decoding algorithms. Encryption algorithms can also be applied to enhance the security. Reading device in traditional method applies GPRS or wired ways to connect to the Internet. GPRS cannot provide high speed data transmission and wired network limits the range of application. Today, more and more smartphones can access Internet by 3G, 4G, and Wi-Fi, and this network access can provide high bandwidth and high speed network service.

Table 4

The comparison between traditional object tracing methods and proposed two-dimensional code based modeling system.

Item	Traditional method	Proposed method
Portability	Not easy to take	Easy to take
Cost	Additional cost	No additional cost
Correction ability	Little	Great
Safety and privacy	Low	High
Network	GPRS or wired	GPRS, 3G, Wi-Fi, and 4G

5. Conclusion and Future Work

In this paper, a spatiotemporal modeling is proposed, which applies two-dimensional code as the object's unique identifier, smartphone as two-dimensional code reader and data transmitter, and a backend server to store spatial-temporal data and provide service for users. The modeling system can trace and query the object's spatiotemporal information. A use case is also introduced to realize the modeling method, and the result shows that the modeling system works well and holds a potential future. Apart from high reliability, high security, and high speed, applying two-dimensional code and smartphone as the main system tools costs less and is easy to realize the modeling system. The proposed modeling system can extended to gain superior safety and privacy of transmission data as well.

The future work aims at developing new encoding and decoding algorithms to make the two-dimensional code modeling system safer and more robust. We will extend the smartphone App and virtual web server and make them more powerful.