Hidden Markov Mined Activity Model for Human Activity Recognition

Abstract

Object-usage-based human activity recognition systems require activity data for learning. Acquiring such data from the real world is expensive and time consuming. To overcome such difficulties, the exploitation of web activity data is gaining popularity. However, due to a lack of much real-world information in such data, existing activity models are not suitable for web data. In this paper, we propose a hidden Markov model- (HMM-) based activity model specially designed to use web activity data for activity recognition. It utilizes a sequence of object-usage information for activity recognition. We also propose a web activity data mining algorithm for this model. It is extremely fast and efficient in comparison with the existing algorithms. We perform three experiments to validate the proposed model. We show that the model can be effectively utilized by an activity recognition system.

1. Introduction

Real-world activity data collection is used for engineering human activity recognition systems [1–6], but the process is cumbersome. Expertise and resources are required to design and install sensors, controllers, network components, and middleware just to perform basic data collections. As a result, very little real sensor activity data has been collected and analyzed, and only rarely is this data made available to the research community [7].

To overcome such difficulties, we need an alternate source of this data that is inexpensive and readily available. One of the most promising sources is the World Wide Web (WWW). Numerous web pages exist on the web that explain how to do activities of daily livings. Each of these pages provides details about activities, such as what objects to use, how to use them, and in what sequence.

Researchers have been working to mine this data to train activity models. For example, Perkowitz et al. introduced the notion of mining generic activity models from the web. They showed that it is possible to convert web data into activity models that can be used in conjunction with RFID tags to recognize activity [8]. Wyatt et al. improved the system by introducing a model that includes idiosyncrasies of the environment in which it will be deployed [9]. Hu and Yang showed how to use web knowledge as a bridge to help link different activity label spaces in transfer learning for activity recognition [10].

Although these systems show an important direction, the activity models are not made for web activity data. These are mostly independent and identically distributed (i.i.d.) object-usage models. The i.i.d. assumption would be sufficient for a model trained using real-world data. However, it will not be the same when using web data because of the lack of real-world information such as time. An activity model should include as much information as possible, because web data can only offer a generic view of an activity. Sarkar et al. used room locations (e.g., kitchen and living room) and objects together to build an activity model [3, 11]. They have shown that the addition of extra information leads to higher recognition accuracy. However, their system would not perform well in a home with only one room, such as a studio apartment.

In this paper, we propose an activity model suitable to use web activity data for training. The model is based on a hidden Markov model (HMM) in which the output of a state depends on the sequence of object usage at a given time. The model relaxes the i.i.d. assumption and exploits the sequential pattern of object usage. This model is not suitable to train with real-world data in a conventional way, as the complexity to determine the object-usage sequence probabilities will be very high. However, this will not be the same for web activity data, since we propose that the model be trained while the system is online and recognizing activity. We propose an algorithm to mine and determine the object-usage sequence probability on demand. We perform three experiments to validate the performance of the proposed model. We show that the accuracy of activity recognition is remarkable when applied to three real-world datasets.

The remainder of the paper is organized as follows. In Section 2, we discuss the advantages and disadvantages of the various previous studies related to this work. In Section 3, we introduce the activity recognition system and discuss the properties of the model the system used. In Section 4, we describe the web activity data mining algorithm. In Section 5, we show the validity of the system with the help of experimental results and discussions. In Section 6, we conclude the paper with a direction of future work.

2. Related Work

Object-usage-based activity recognition has long been a goal of researchers due to its strength to provide support in diverse healthcare applications [1, 2, 12–15]. A variety of activity models have been proposed for this purpose. For example, Tapia et al. [1] proposed a Naive Bayes activity model to recognize activities in a home setting. They showed excellent promise, even though their mechanism suffers from low recognition accuracy. Van Kasteren et al. [2] used similar settings with a hidden Markov model and conditional random field.

Although the parameters of some of these models can be learned from web activity data, the activity recognition accuracy will not be high, since web activity data lacks real-world information. To the best of our knowledge, Perkowitz et al. [8] first introduced a technique for mining generic activity models from the web. They converted natural-language recipes into activity models and used them in conjunction with RFID tags to recognize activity. Their model consists of a sequence of states and is based on a particle filter implementation of Bayesian reasoning. Their model extractor works as follows.

• (i)

Select a set of websites describing activities, such as http://www.ehow.com/ and http://www.epicurious.com/, and understand the HTML structure of such websites.

(ii)

Search each of the pages for the activity direction and extract the direction.

(iii)

Set the label of an activity as the title of the direction.

(iv)

Extract the object phrases from the direction.

(v)

Remove the phrases without noun sense.

(vi)

Determine the object-usage probability as a Google Conditional Probability (GCP):

$$\begin{array}{cc}& \text{GCP}\left(o_i\right)=\frac{\text{hitcount}\hspace{0.17em}\hspace{0.17em}\left(\text{object}\mathrm{\hspace{0.17em}\hspace{0.17em}}\text{activity}\right)}{\text{hitcount}\hspace{0.17em}\hspace{0.17em}\left(\text{activity}\right)},\end{array}$$

(1)

 

where hitcount $(x\hspace{0.17em}y)$ is the number of pages Google returns if we search with x and y.

(vii)

Finally select only the tagged objects (objects with embedded RFID tags) from the phrases.

They use a sequential Monte Carlo (SMC) approximation to infer activities probabilistically. They borrowed the inference engine from a previous study [15]. Despite their good performance in classifying hand-segmented object-use data, they suffered from low accuracy and limited applicability. In addition to this, they used specific web sites whose formats were known before mining the activity models [9].

Wyatt et al. [9] proposed an unsupervised activity recognition system (UARS) using web activity data mining. They developed two algorithms: a document genre classifier that identifies the pages describing an activity and an object identifier that extracts all the objects from a page and calculates the object's weights. Their mining algorithm works as follows.

• (i)

It queries Google with the activity name along with “how to” as the discriminating phrase. Google returns the number of pages it has indexed in its server.

(ii)

It then retrieves P pages as the top z pages within the total pages returned by Google. They did not define the optimal value of z. The efficiency of mining clearly depended on z, with a larger value of z meaning more efficient mining.

(iii)

The algorithm uses the genre classifier to determine $\tilde{P}$ , a subset of P, as the activity pages.

(iv)

Using the object identification technique, for each page p in $\tilde{P}$ , it extracts all the objects mentioned in the page and calculates their weights, $\widehat{w}$ .

(v)

Finally, the algorithm calculates the objects usage probabilities as

$$\begin{array}{cc}& p\left(\text{object}\mid \text{activity}\right)=\frac{\mathrm{1}}{\left|\tilde{P}\right|}{{\displaystyle \sum }}_p‍w_{\text{object},p}.\end{array}$$

(2)

From the mined information they assembled an HMM, M, which has the traditional 3 parameters: $(1)$ prior probabilities for each state π which were set to be uniformly distributed, $(2)$ the transition matrix T of constant probabilities, $(3)$ and observation matrix B, where $B_{ji}=p({\text{object}}_i\mid {\text{activity}}_j)$ .

Although these systems perform well in mining activity models from the web, they take hours or days to do so. Additionally, the accuracy of activity recognition is not satisfactory. This is because the models are only based on object-usage information. As the web provides a general sense of object usage for an activity, using them in the real world where an activity is individual-specific would not provide high-accuracy activity recognition. We need to use as much information as possible to bridge the gap between general view and individual-specific view of an activity.

Sarkar et al. [3, 11] used the location of an activity along with object-usage and showed that the addition of location provides better accuracy. Their model works well in environments with many rooms. However, it would not perform equally well in a home with only one or two rooms, since such a situation location could not offer significant information about an activity.

In this paper, we propose an object-usage-based activity model. The difference from the existing model is that it uses an object-usage sequence instead of treating each of the objects independently. The model is applicable to diverse homes regardless of the number of rooms. The model requires object-usage sequence data for training. We also propose a web activity mining algorithm for extracting such sequential data from the web.

3. Activity Recognition System

3.1. Overview

We consider an environment in which a set of objects (e.g., light, door, and faucet) are embedded with sensors. A sensor is attached to an object in a way such that it is possible to determine the state of the object when used. Given a set of activities to monitor and object names (with embedded sensors), the purpose of the activity recognition system is to recognize the current activity of a person depending on the sequence of objects used at a given time.

The system does not require training before deployment. It will be trained online while a person is doing daily activities. The system determines the probability of a pair of object-usage sequences (e.g., refrigerator and cabinet) each time it observes a new pair. It reduces the system's complexity since it does not need to know every possible pair of object usage. An overview of the system is shown in Figure 1.

Figure 1

Overview of the activity recognition system.

Let $A=\{a_{\mathrm{1}},a_{\mathrm{2}},\dots ,a_m\}$ be the set of activities to monitor, and let $O=\{o_{\mathrm{1}},o_{\mathrm{2}},\dots ,o_l\}$ be the set of objects in the environment, where m and l are the total number of activities and objects, respectively. Let $\mathrm{\Theta }=\{{\theta }_{\mathrm{1}},{\theta }_{\mathrm{2}},\dots ,{\theta }_n\}\in O$ be the set of object-usage sequences at any given time, where n is the total number of object-usage. The goal is to map the observation sequence (i.e., $\mathrm{\Theta }$ ) into predefined activity labels. For each pair of object usages, the activity recognizer checks if it already knows the probability, which it determines from the web with the help of a mining engine if it does not already know. It uses the probabilities to recognize the current activity.

3.2. The Activity Model

The activity model is based on HMM. Each of the states is an activity, and the observation probabilities are the sequence of object-usages. The graphical representation of the model is shown in Figure 2. It consists of a hidden state (i.e., activity), $a^t$ , at time, t, and the observation (i.e., object-usage sequence) ${\mathrm{\Theta }}^t$ on each state. The hidden state at time t depends on the previous state at time $t-\mathrm{1}$ . The observed variable at time t depends on the state at time t. The goal is to find the joint probability distribution,

$$\begin{array}{cc}& P\left(a,\mathrm{\Theta }\right)={\displaystyle \prod }_{t=\mathrm{1}}^T‍P\left(a^t\mid a^{t-\mathrm{1}}\right)P\left({\mathrm{\Theta }}^t\mid a^t\right),\end{array}$$

(3)

where $P(a^{t-\mathrm{1}}\mid a^t)$ is the transition probability from state $a^{t-\mathrm{1}}$ to $a^t$ . If we train an AR system using real-world activity data, we count the number of occurrences of transitions, observations, and states to find the probabilities that maximize the joint probability [2]. However, as we consider the web activity data to train the system, there is no way we can count the transitions because the transitions between activities are highly subject dependent. Therefore, the transition probability matrix, T, is set to a constant probabilities [9]. For T, the duration of any activity is set to, γ, and all the self-transition probabilities, $T_{jj}$ , are set to $\mathrm{1}-\mathrm{1}/\gamma$ . The remaining probability mass is uniformly distributed over all transitions to other activities.

Figure 2

The graphical representation of the activity model.

In (3), $P({\mathrm{\Theta }}^t\mid a^t)$ is the probability of observing ${\mathrm{\Theta }}^t$ in state $a^t$ . We can calculate the probability by ignoring the sequential aspects and treat the observations as i.i.d. However, this approach would fail to exploit the sequential patterns, such as correlations between observations that are close in the sequence [16]. Therefore, in the probabilistic model, we relax the i.i.d. assumption. We use a first-order Markov chain of observations ${\mathrm{\Theta }}^t$ at time t in which the distribution $P({\theta }_n^t\mid {\theta }_{n-\mathrm{1}}^t,a^t)$ of a particular observation ${\theta }_n^t$ is conditioned on the value of the previous observation ${\theta }_{n-\mathrm{1}}^t$ . Without loss of generality, we can use the product rule to express the joint distribution for a sequence of observations in the form

$$\begin{array}{ll}P\left({\mathrm{\Theta }}^t\mid a^t\right)& =P\left({\theta }_{\mathrm{1}}^t,\dots ,{\theta }_n^t\mid a^t\right)\\ & =P\left({\theta }_{\mathrm{1}}^t\mid a^t\right){\displaystyle \prod }_{k=\mathrm{2}}^n‍P\left({\theta }_k^t\mid {\theta }_{k-\mathrm{1}}^t,a^t\right).\end{array}$$

(4)

There are n distinct observation symbols per state. The observation symbols correspond to the physical output of the system being modeled. We consider the object-usage sequence as the observations symbols per state.

During training, we determine the $P({\theta }_{\mathrm{1}}^t\mid a^t)$ as the number of web pages that describe an activity using the object, ${\theta }_{\mathrm{1}}$ , over the number of pages that describe an activity, a. We determine the $P({\theta }_k^t\mid {\theta }_{k-\mathrm{1}}^t,a^t)$ as the number of pages that describe an activity using the two objects, ${\theta }_k^t$ and ${\theta }_{k-\mathrm{1}}$ in a sequence, over the number of pages that describe an activity, a. We describe how to determine the number of pages in Section 4.

During inference, the Viterbi algorithm is used to find the most likely labels for the new observation sequences [2]. This algorithm has been successfully applied with HMM to solve many activity recognition problems.

4. Web Activity Data Mining

As we can see in (4), to train the system we need to know two types of probabilities: the probability of using an object given an activity, that is, $P({\theta }_{\mathrm{1}}^t\mid a^t)$ , and the probability of using an object given another object and an activity, that is, $P({\theta }_k^t\mid {\theta }_{k-\mathrm{1}}^t,a^t)$ . The purpose of web activity data mining is to determine these probabilities. Before describing more details, it would be convenient if we illustrate few important facts about the web pages that describe the activities of daily lives.

4.1. Web Activity Pages

There are two types of activity pages on the web: explicit activity page (EAP) and implicit activity page (IAP).

Definition 1 (explicit activity page).

A web page is called an explicit activity page if it provides detailed instructions about performing an activity. It has a title, which in most cases contains the activity name. It has a text section that provides details of an activity such as what objects to use and their sequence.

For example, the web page [17] shown in Box 1 is an EAP that contains the activity name, “Bathing,” in the title and contains a detailed description of the activity in the body. The text has a set of object names (such as towels and shampoo) and their usage sequence related to “Bathing.”

Box 1: An example explicit web activity page [17].

Bathing

$⋮$

When bathing a person with dementia, …

Prepare the bathroom in advance by:

> Gathering bathing supplies.

 Have large towels (that you can completely wrap around the person for privacy and

warmth), shampoo and soap ready before you tell the person that it's time to bathe.

> Making the room comfortable.

 Pad the shower seat and other cold or uncomfortable surfaces with towels. Check

that the room temperature is pleasant.

> Placing soap, shampoo and other supplies within reach.

$⋮$

> Monitoring water temperature.

$⋮$

Definition 2 (implicit activity page).

A web page is called an implicit activity page if it does not provide explicit instructions about an activity but instead provides information that is implicitly related to an activity.

For example, the web page [18] shown in Box 2 is an IAP that contains the activity name, “Bathing,” in the title and contains implicit information in the text related to an activity. It also refers to a set of object names (such as Door and Bathtub) and their usage sequence related to an activity.

Box 2: An example of an implicit web activity page [18].

Safety Bath Tub FAQs—Seabridge Bathing

$⋮$

Not all safety walk-in tubs are The same. When comparing safety bath tubs from

different manufacturers, here are some of the differences you should know about.

$⋮$

Door System and Operation

$⋮$

Safety Bathtub Fill Time

$⋮$

What is Seabridge Dual Draining?

$⋮$

Why do some walk-in baths have drains on the outside of the bath?

$⋮$

Are safety bathtubs that hold less water better than those that hold more?

$⋮$

4.2. Mining

The goals of mining for a given set of activities are to find EAPs and IAPs and extract object-usage information from them. One way of accomplishing this would be to search for these pages by a search engine (e.g., Google), download the pages, and obtain objects information from them using a natural language processing (NLP) algorithm. However, it will not be feasible for us since downloading the pages could take hours or even days for a single activity. We need an algorithm that extracts the desired information in a real-time without downloading the web pages.

A set of web search engines (e.g., Google and Bing) already have downloaded the pages and stored all the information on their server. The mining will be very fast if we can dig out the desired data from their server. Fortunately, almost all the search engines (SEs) provide special mechanisms for querying the required information. For example, Table 1 provides three query modifiers and operators that can be used along with queries. Our objective is to use these to get the desired information.

Table 1

Search engine (SE) query modifiers and operators.

Name	Description
“ ”	Quotes forces SE to search for exact phrase.
Intitle	Including [intitle:] in a SE's query will return all the EAPs and IAPs containing the word in the title.
*	Use an asterisk *(wildcard) within a phrase search to match any word in that position. Note: Bing does not require*.

Algorithm 1 shows the way of achieving this. It takes the list of activities, A, and the set of object-usage, $\mathrm{\Theta }$ , as input. The outputs are the numbers of web activity pages (WAP) of size $mX\mathrm{1}$ for each of the activities and the numbers of pages of object-usage sequence (POS) of size $mXn$ .

Algorithm 1: MineFromWeb(A, $\mathrm{\Theta }$ ). Web activity mining.

Data: Activity list, A, Object list $\mathrm{\Theta }$

Result: Web Activity Pages (WAP): $mX\mathrm{1}$ matrix, Pages with Object-usage

     Sequences (POS): $mXn$ matrix

(1) for   $i\leftarrow \mathrm{1}$   to   $length(A)$   do

    /* Check in the local database, if not exists get it from web */;

(2)   if LocallyExists( $a_i$ )   then

(3)      $\text{WA}{\text{P}}_i$ = GetLocal( $a_i$ ); /* Get locally   */;

(4)   else

(5)      $\text{WA}{\text{P}}_i$ = SE(“ $intitle$ :“ $a_i$ ” ”); /* SE (Search Engine) would return the number of

       pages indexed by the search engine for the given query */;

(6)     SetLocal( $a_i$ , $\text{AP}{\text{I}}_i$ ); /* Store locally    */;

(7)   if LocallyExists( $a_i$ , ${\theta }_{\mathrm{1}}$ )   then

(8)      $\text{PO}{\text{S}}_{i\mathrm{1}}$ = GetLocal( $a_i$ , ${\theta }_{\mathrm{1}}$ );

(9)   else

(10)     $\text{PO}{\text{S}}_{i1}$ = SE(“ $intitle$ :“ $a_i$ ” “ ${\theta }_{\mathrm{1}}$ ” ”)

(11)    SetLocal( $a_i$ , $\text{PO}{\text{S}}_{i\mathrm{1}}$ ); /* Store locally   */;

(12)   for $k\leftarrow \mathrm{1}$   to  $length(\mathrm{\Theta })-\mathrm{1}$   do

(13)     if LocallyExists( $a_i$ , ${\theta }_{ik}$ , ${\theta }_{i(k+\mathrm{1})}$ )   then

(14)       $\text{PO}{\text{S}}_{i(k+\mathrm{1})}$ = GetLocal( $a_i$ , ${\theta }_k$ , ${\theta }_{k+\mathrm{1}}$ );

(15)     else

(16)      $\text{PO}{\text{S}}_{i(k+\mathrm{1})}$ = SE(“ $intitle$ :“ $a_i$ ” “ ${\theta }_k\mathrm{\hspace{0.17em}\hspace{0.17em}}\mathrm{*}\mathrm{\hspace{0.17em}\hspace{0.17em}}{\theta }_{k+\mathrm{1}}$ ” ”)

(17)     SetLocal( $a_i$ , ${\theta }_k$ , ${\theta }_{k+\mathrm{1}}$ );

For each activity $a_i$ in A, the following tasks are done. $(1)$ The local database is checked for the number of web activity pages, ${\text{WAP}}_i$ , indexed by a search engine (SE) that either explicitly or implicitly describes the activity, $a_i$ . It determines this from the web with the query, intitle: “ $a_i$ ,” if it is not locally available. $(2)$ It then determines the number of pages that mentioned the first object, ${\theta }_{\mathrm{0}}$ . We denote this as, ${\text{POS}}_{i\mathrm{1}}$ . $(3)$ Finally, for each sequence of object-usage pairs, ${\theta }_{i}k$ and ${\theta }_{i(k+\mathrm{1})}$ in $\mathrm{\Theta }$ , it searches and stores the number of pages indexed by the search engine, ${\text{POS}}_{i\mathrm{1}}$ , using the query, “intitle”: “ $a_i$ ” “ ${\theta }_k*{\theta }_{k+\mathrm{1}}$ .”

4.2.1. Number of Queries Required for Mining

As we can see in Algorithm 1, the number of queries needed, r, depends on two factors: $(1)$ the set of activities the system is recognizing, m, and $(2)$ the total number of objects used, $\left|\mathrm{\Theta }\right|$ for an activity. For each of the activities it requires $\left|\mathrm{\Theta }\right|$ number of queries. Therefore, we can write

$$\begin{array}{cc}& r=m*\left|\mathrm{\Theta }\right|.\end{array}$$

(5)

For example, if 3 objects are used for an activity and there are 10 activities to monitor. To mine the model parameters, it would require 30 queries in total.

5. Evaluation and Results

We perform three experiments to evaluate the performance of the proposed system. In the first experiment we evaluate the system's performance in recognizing activities of daily life and compare it with a previous system [3]. In the second experiment, we estimate the time required to mine web activity data.

We use similar settings to those used in another study [19]. We use two popular search engines, Google and Bing, for mining web activity data.

We use three real-world activity datasets gathered by Tapia et al. [1] at the MIT PlaceLab (Placelab 1, Placelab 2), and by Van Kasteren et al. [2] at the Intelligent Systems Lab Amsterdam (ISLA). The same set of activities is considered as in [3]. The γ is set to 5 to set the self-transition probabilities to 80%. It ensures that the object-usage sequences play the central role for transition between states.

5.1. Experiment 1: Activity Recognition Accuracy

In this experiment, we verify the accuracy of activity recognition. Figure 3 summarizes the results for three datasets. The first two bars from left to right of each of the three bars represent the class accuracy of the system when using Google and Bing, respectively. Using Google, the system achieves overall class accuracies of 68.12%, 65.50%, and 79.12%, respectively, for the three datasets. Using Bing, the system achieves overall class accuracies of 69.12%, 67.35%, and 80.46%, respectively. The accuracy of activity recognition is better when using Bing's data for training. This indicates that the Bing activity data is somewhat more organized than that of Google.

Figure 3

The accuracy of activity recognition.

5.2. Experiment 2: Performance Comparison with Other Systems

We compare the system's performance of our system (HMMaM) with two existing systems, a general-purpose activity recognition system (GPARS) [3] and unsupervised activity recognition using automatically mined common sense (UARS) [9].

We compare two versions of GPARS; in the first version it uses a naive Bayesian-based two-layer classifier to classify an activity, and in the second version it uses one-layer (also naive Bayesian-based) classifier. The two-layer classifier works as follows: it first classifies a group of potential activity using a location-and-object-usage based model in the first layer and then classifies an activity from that group using an object-usage based model in the second layer. In the one-layer classifier, although, the GPARS uses location-and-object-based model, however, by setting the parameter, $\alpha =\mathrm{0}$ , we make sure that location does not have an impact on classification. Comparison with one-layer classifier is important since it represents how the system would work in real-world with no specific location.

The comparison results are shown in Figure 4. The second and third bars represent the accuracies of the two versions of GPARS, respectively. Even though the proposed system does not use location information in which an activity is performed, the system performs equally well in comparison with the two-layer GPARS. The system outperforms one-layer GPARS. This is because the proposed system utilizes object-usage sequences which give more realistic information about an activity.

Figure 4

Performance comparison with the existing systems.

The accuracies with UARS are shown in the fourth bar of Figure 4. The proposed system also outperforms URAS in classifying an activity. This is expected, since HMMaM uses robust model for classifying an activity.

5.3. Experiment 3: Time Required for Mining

In this experiment, we evaluate the efficiency of the mining engine in extracting web activity data. We inspect how long it takes to mine data for each day in each of the three testing datasets. Figure 5 shows the time required per day for each of the datasets. The figure represents 10 days of data.

Figure 5

Web activity mining time per day.

The mining time decreases gradually over time. Figure 5 shows that, after 3–5 days, the mining time goes down to nearly zero for all the datasets. This is expected, since the mining engine stores mined data locally and uses them for future reference.

The mining time for a dataset acquired from an environment containing more tagged objects (with embedded sensors) is generally higher than that for other datasets. Figure 5 shows that the mining time for Placelab 1 datasets is higher, since the number of tagged objects is higher. This is a common phenomenon, because in that dataset, there are more objects per activity in general.

5.4. Effect of Constant Self-Transition Probability

We have evaluated the algorithm with different γ values, ranging from 2 to 8. Similar to URAS, [9], the accuracy of activity recognition has not been affected much. The mean accuracy across these values for three datasets was 63.5%, 60.34%, and $\mathrm{78.08}\%$ , respectively. The average performance of the system was good across datasets when the γ was set to 5. In other words, 80% self-transition probabilities show better accuracy on average.

6. Conclusion

We have introduced a novel activity model for human activity recognition in a home setting. This is an HMM-based model in which the transition to the next state at a given time depends on the current state and the observation sequence. The states are the activities, and an observation is an object-usage sequence. The transition from an activity to this activity or another activity depends on the prior probabilities and object-usage sequence probability.

Although the use of an object-usage sequence gives better understanding of an activity, it is very complex and time-consuming to learn sequence probabilities from real-world activity data. A substantial number of objects can be used in an environment, and therefore, the number of possible sequences can be enormous. Instead of using real-world data, we used web activity data and proposed an efficient web mining algorithm to learn the sequence probabilities on demand.

We performed three experiments to verify the activity model and to validate the performance of the mining algorithm. We showed that the model can be applied to recognize the activities of daily life, and the mining algorithm can efficiently mine activity data from the web.