DESIGNING AND SECURING AN EVENT PROCESSING SYSTEM FOR SMART SPACES

ABSTRACT

Smart spaces, or smart environments, represent the next evolutionary development in buildings, banking, homes, hospitals, transportation systems, industries, cities, and government automation. By riding the tide of sensor and event processing technologies, the smart environment captures and processes information about its surroundings as well as its internal settings to support context awareness and intelligent inference. For example, in long term home health monitoring, automatic detection of activity and health status allows elders to receive continuous care at home, thus reducing health care costs, improving quality of life, and enabling independence. Moreover, one promising goal of smart spaces,

“collecting and accessing information everywhere anytime”, triggers the needs for efficient yet secured information sharing among smart spaces. In this dissertation, we explore new approaches to address challenges on processing high-level events and securing information sharing in the context of semantic spaces.

We first explore the event processing solutions in smart spaces. We propose an event processing framework, which includes two sub-processes: stream event processing and semantic event processing. Stream event processing extracts knowledge from sensor streams for each modality. We present a novel model to recognize motions of the objects attached with passive RFID tags. The model applies Hidden Markov Model (HMM) to accurately infer the motion sequence based on the relative variance in a time series of response rates. The model is augmented with an adaptive model, which can be used to dynamically adjust to the changing environment based on the change-point detection algorithm. Moreover, an online Viterbi algorithm is developed to get low output latency.

Then, we turn our attention to inferring high-level semantic events based on the preliminary semantic events from stream event processing. We propose event ontology to enable semantic indexing and detecting of machine-processable events and exchanging event data among heterogeneous engines.

Moreover, we develop an event processing framework driven by the event ontology, OntoCEP, to elaborate event composition and semantic reasoning, which apply event patterns and context reasoning rules into one coherent system. The event composition engine conducts detection of complex patterns of events based on the relationships such as causality, and timing relationships. The semantic reasoning engine integrates context knowledge from which more meaningful events are deduced.

Finally, we investigate how to secure sharing of complex data objects among pervasive information systems. To address the challenges posed by heterogeneous data sources, complex objects and context dynamics, we propose an advanced authorization model that supports specifying and enforcing authorizations in flexible and efficient ways. The model employs semantic web technologies to conceptualize data and explicitly express the relationships among concepts and instances involved in information sharing. Authorizations can be specified at different levels of the predefined concept hierarchies and be propagated to lower-levels. A novel decision propagation model is proposed to enable fast evaluation and updating of concept-level access decisions. To resolve conflicts among policies, we model a policy set as a semilattice, upon which a binary operation is defined to adapt to various requirements. Moreover, enabled by ontology reasoning tools, a flexible specification approach of authorization, namely rule-based policy generation, is developed to encode context dynamics, making the authorization enforcement adaptive to contexts.

CHAPTER ONE

INTRODUCTION

Smart spaces, or smart environments, represent the next evolutionary development in buildings, banking, homes, hospitals, transportation systems, industries, cities, and government automation. Relying on sensory data from the real world, the smart environment captures information about its surroundings as well as its internal settings, thus enabling the development of a variety of services for home living, healthcare and education. For example, in long-term home care, automatically monitoring elder‟s health status allows them to receive continuous care at home, thus reducing health care costs, improving quality of life, and enabling independence.

However, the availability of sensing capabilities cannot satisfy the new expectations of smart spaces accompanied by the new paradigms of information technologies, including information retrieval and Internet technologies. Rather than monitoring services, smart spaces are expected to do more such as complex event detection, context-aware notification, statistical analysis and Internet-wide information sharing. To achieve these goals, many critical research challenges, such as: how to semantically annotate sensor readings; how to automatically infer high-level events by composing facts and events of different contents, formats and modalities; and how to securely deliver information and services over Internet, need to be handled. In this dissertation work, we investigate these challenges in depth, focusing on the infrastructure and event process systems for smart spaces.

1.1. SCENARIO AND HUMANIZATION PROCESS

Let‟s consider a simple motivating scenario: the Greens live in a residence house. The grandfather, Bob Green, has been retired for several years and has heart disease. From Monday to Friday, the other family members go to work, whereas Bob Green stays at home. In order to prevent the occurrence of an emergency, the house is installed with various sensors (e.g., temperature, humidity, light sensor) and converted into an indoor Smart Space. Bob wears an RFID wristband with a temperature and accelerometer sensor so that his identity, temperature, location and motion status can be captured in time. The sensory readings from different modalities are integrated and processed by a smart space middleware, in order to develop comprehensive information about Bob‟s current health status and activities, deliver notification such as medicine reminder messages and to conduct statistical analysis on Bob‟s daily activity patterns. At the same time, the other family members can use smart space clients to access important information about Bob (e.g., healthy status, abnormal activities). For example, when Bob falls down or his blood pressure exceeds 140/90 mmHg, the smart space can immediately detect it and send notification to his family members and healthcare professionals.

From this scenario, some unique features of a smart space could be identified. The important one is that, applications in smart spaces must be “smart” enough so that they can rapidly detect and adapt to context changes. This motivates us to humanize every monitored object in a smart space. A monitored object could be a person, an important asset (e.g., bed, computer) or a zone (e.g., bedroom, kitchen). These objects are required to be able to think, respond and communicate with others. Figure 1.1 shows the sketch of the humanizing process, which compose the following three sub-processes:

Identifying and sensing. The goal of the identification process is to create a virtual entity in the cyberspace for every object of the physical space to be monitored. The information about virtual entities reflects physical reality. Additionally, like the human sensing process, a virtual entity senses environmental changes and activities of the corresponding physical object through various sensors.

Detecting and reasoning. To humanize an object, it is necessary and important to have the corresponding virtual entity computational-intelligent. As we know, human makes decisions by comprehensively analyzing human sensory information (i.e., changes of environment and itself) and exiting facts (drawn from knowledge and experience). Similarly, a virtual entity aggregates and correlates various types of sensory events, refers to the knowledge base and then infers the high-level facts and makes corresponding decisions.

Communicating and presenting. Another key requirement for humanization is to enable the humanized object to communicate with a human. There are two levels of communication ability. The first is that the virtual entity can actively “tell” things to those who have interest and are listening. The second is that the exchange of information is a two-way communication and based on interactions in various forms (e.g., text message, audio). Additionally, every virtual entity is required to communicate with others as an independent communication entity, no matter it is surrogated or not.

1.2. BACKGROUND AND INFRASTRUCTURE

In this section, we present some background information for this dissertation work. After that, a SIP (session initiation protocol) based smart space infrastructure will be introduced as the research platform for the research works in the dissertation.

1.2.1. SMART SPACES AND INTERNET OF THINGS

The smart space technologies, such as near-field communications, real-time localization, and context intelligence enable entities in a smart space to understand and react to their environment. All these capabilities are developed on the basis of an efficient infrastructure for smart spaces. Moreover, a new vision of networked smart spaces, named “Internet of Things” (IoT) (Gershenfeld et al. , 2004), which integrates smart space services and Internet technologies, raises many interesting research issues such as network discovery, data and signal processing (Gershenfeld and Krikorian, 2004) in terms of Internet-scale service infrastructure. In this work, we will cover some challenging research problems for smart spaces and IoT.

Specifically, for the dissertation, we developed an infrastructure named SENSIP for smart spaces. This infrastructure helps to integrate modules of data collection, information processing and Internet-scale information dissemination for entities (people, assets and zones) in smart environments. Therefore, it can be applied into IoT. We introduce the details of the infrastructure in the next section.

1.2.2. A SIP-BASED INFRASTRUCTURE FOR IOT

SENSIP is a SIP-based infrastructure, designed to facilitate in making use of local smart services and sharing smart space information over Internet. SENSIP is developed by following the humanization process we introduced. It implements the data collection functions and employs the information processing model we will investigate, thus efficiently discovering semantic information about the entities in a smart space. The entity positioning and mobility management services are fulfilled by the joint effort of SENSIP middleware and the SIP registrar. The SIP presence protocol is extended to support sharing smart services using both PUSH and PULL modes. Besides the multimedia communication and instance messaging, a real-time map and attribute table are developed to present smart services.

The SENSIP infrastructure can be structurally divided into four modules (shown in Figure 1.2), namely 1) the Sensor devices and entities, 2) the SENSIP middleware, 3) the communication server and database, and 4) the watchers. The four components are detailed below: 

Sensor devices and Entities. A smart space is usually defined as a custom-built condominium instrumented with a variety of ambient and wearable sensors. Examples of the ambient sensor include reed switch sensor, temperature and humidity sensor, light sensor, gas sensor, current flow sensor and water flow sensor. Wearable sensors are attached to people and assets, for example, 3-axis accelerometer sensors, body temperature sensor and blood pressure sensor etc. In this work, we believe some special devices such as RFID, microphones and cameras are important information sources in smart spaces; so we also view them as sensors. Entities are the people, assets and zones in smart environments. They are attached with various sensors and assigned to Uniform Resource Identifiers (URIs) by the system. In the vision of IoT, they are also indicated as things.

SENSIP middleware. The SENSIP middleware plays a key role in managing devices and entities, conducting event processing, and connecting a smart space to the SIP network. SENSIP middleware implements the data capture model, data processing model and communication manager. The data capture model provides a unified operation interface for different types of monitoring devices including RFID and sensors, and captures sensory streams through a set of device drivers provided by vendors. Data processing model processes sensor events and semantic events in order to obtain meaningful and usable knowledge such as status information of monitored objects and alarms. This topic will be the focus of this dissertation. The communication manager manages all the entities in a smart space and fulfills SIP functions for every entity, including registering entities in the SIP registrar, controlling multimedia communication sessions, and publishing entities‟ information to the SIP presence server.

Communication server and database. The communication server implements various SIP services, such as the call router, registration and instant messaging and presence, etc.. This component provides linkages for all the communication entities interacting with each other via a standard SIP interface. Disseminating smart space information, the communication server maintains a database which is updated by information from the SENSIP middleware, and can respond the query from watchers (PULL mode) or send information to watchers according to their subscription status (PUSH mode).

Watcher. A watcher is software or a device by which people can obtain information about monitored objects and access other smart services. Normally, it is a SIP-enabled client, for example, SENSIP watcher and SIP instance messenger, or a SIP-enabled device such as a

PDA or smartphone. By setting up a gateway, we can enable terminals supporting other protocols to obtain information of monitored objects. For example, a jabber gateway can translate SIP messages into XMPP (eXtensible Messaging and Presence Protocol) message so that any XMPP user can receive sensor information from the SENSIP infrastructure.

1.3. MOTIVATION AND RESEARCH FRAMEWORK

In this section, we discuss the research issues and identify the research problems in the field of smart spaces, followed by an overview of our research.

1.3.1. RESEARCH ISSUES

With reference to the humanization process for smart spaces, we can derive a big picture covering most research issues of smart spaces. Figure 1.3 shows a framework, where research issues are organized by the life cycle of smart space information processing: data collection, stream event processing, semantic event processing and information dissemination. Note that we classify issues of event processing into two classes – stream event processing and semantic event processing – based on the characteristics of the events (see details later).

Data Collection. Data collection involves the process of using a variety of sensor devices, such as ambient sensors and biosensors to capture data in smart space. Research issues in this stage mainly concern the use of sensors and sensor networks, for example, radio range of sensors should cover objects being monitored in a space, especially for wireless sensors, radio range is also required to ensure network connectivity (Gao et al., 2006); sensors‟ power consumption should be minimized since their limited energy resource determines their lifetime (Shnayder et al., 2004). In a smart space, sensor deployment establishes an association of sensors with specific entities to be monitored. Basic deployment of sensors in a smart space can be conducted manually according to a certain plan, for example attaching blood pressure sensor to a human. Some ad-hoc methods are developed for wireless sensor network, for instance, throwing nodes from an aircraft into a broad area, or using locomotive sensor nodes to cover an area (Wang et al., 2006b).

Stream Event Processing. Extracting semantic knowledge from raw sensor data is not a trivial task. First of all, considering the rapid generation of sensor streams, which are usually imprecise and noisy, data preprocessing steps (see details later) need to be applied. Many approximation algorithms for sensor data preprocessing have been proposed in recent years (e.g., (GutierrezOsuna and Nagle, 2002, Jeffery et al., 2006b)). Moreover, in many cases, we need to transform sensor data to other meaningful streams; for example, streams from two accelerometers attached on a person are combined and annotated with gesture labels. Machine learning techniques have been widely employed to process sensor data and developed classification models from sensor data, for example, extracting a online prediction model (Bontempi and Le Borgne, 2005, Cohen and Oviatt, 1995) and recognizing human activities (Bao and Intille, 2004, Ben-Arie et al., 2002).

Semantic Event Processing. Semantic events encode context dynamics in terms of entities of interest in smart spaces. Additionally, semantic events are defined independently of sensor modalities where they originally generated. For example, one semantic event of a person is Fell Down. A semantic event can be inferred from other semantic events. To the best of our knowledge, very little works has been done on inferring semantic events in smart spaces. In (Wun et al., 2007), an integrated system that decouples the process of semantic data fusion from application logic based on a content-based publish/subscribe techniques is proposed. Context management and reasoning has been investigated in smart spaces, for example, using Semantic Web technologies to handle context-aware issues (e.g., explicit representation, information querying reasoning) in smart spaces (Wang et al., 2004a).

Information Dissemination. Information dissemination concerns the sharing information among smart space entities. Important issues need to be considered include communication models, entity discovery, mobility management, and information security. Various communication models have been investigated by previous works, for example, the Publish/subscribe coordination model (Xie et al., 2002), the distributed agent system (Coen et al., 1999), and the transport layer middleware (Okoshi et al., 2001). For entity discovery and mobility management, several solutions, applied in different infrastructures, have been discussed, for example, Object Naming Service (ONS) is designed as the standard for object discovery in EPCglobal network (Mealling,

2004), Distributed Hash Table (DHT) is applied to manage RFID resources in peer-to-peer network (Simplot-Ryl), SIP registrar services provide strong support for entity and mobility management in SIP communication networks (Li et al. , 2008). Securing information is a topic in smart spaces, for example, context-aware access control issues are investigated in (Al-Muhtadi et al., 2003, Gupta et al., 2006).

Based upon the above observations, we can summarize the current research issues and status as follows: i) research on wireless sensor network, including hardware design, sensor deployment and power management has been popular in academia and was deeply investigated. ii) Although a standard communication infrastructure has not been specified for smart spaces or IoT, there are many mature candidates, that have been developed and applied in some application fields (e.g., web service architecture, Voice on IP infrastructure). They can be directly applied in Smart Spaces without fundamental research work. iii) There are few research works and software on information processing for smart spaces and IoT; To the best of our knowledge, automatic “thing”-oriented data processing rather than sensor and context oriented processing  has not been explicitly investigated, let alone an integrated model to bridge the gap from sensor readings to high-level semantics about things. And iv) Previous research work on access control models for smart spaces has considered encoding context constraints; however, other features such as thing-oriented data capture and cross-domain sharing are still not paid enough attention.

To sum up, as a new field, smart spaces or IoT still have many research challenges, that are not been well covered by previous research work in related research areas, such as information retrieval and information security; Thus, a framework which can fully encode the features of smart spaces and provide thing-oriented information processing and security is highly desired. This motivates us to examine the following two groups of core research problems for this dissertation work:

Category I: Processing thing-oriented information in smart spaces.

Problem 1: How to automatically and efficiently annotate sensory readings with semantic meanings?

Problem 2: How to effectively infer high-level events of entities in smart spaces?

Problem 3: How to design an event processing framework powered by heterogeneous process engines?

Category II: Securing the dissemination of information over the Internet.

Problem 4: How to support authorization specification for distributed and heterogeneous data?

Problem 5: How does the authorization model support fine-grained authorization for complex data object?

Problem 6: How to capture context dynamics for the authorization model to make appropriate access control decisions?

These problems are labeled in Figure 1.4. For each problem, we point out the corresponding solutions and chapters in our work. In the following sections, we detail the two groups of problems and provide an overview of the solutions.

1.3.2. EVENT PROCESSING FRAMEWORK

Sensor networks produce large amounts of data to react to the dynamics of a smart space. Different functionalities are provided by sensor nodes to measure different attribute information. For example, a temperature sensor and a humidity sensor can be used to detect temperature and humidity at a certain place, respectively. However, the sensing tasks, which people hope to develop, are usually at a high level, such as “reporting an alarm when any object in the house is detected on fire”. To solve these problems, a comprehensive framework is required to extract high-level semantics from sensory readings. In practice, implementing an automated information processing system for smart spaces is a very complicated process. In this work, we envision the involved tasks are either stream-oriented or semanticoriented; therefore we develop an information processing framework (shown in Figure 1.5) which explicitly includes two sub-processes: stream event processing and semantic event processing.

Stream event processing extracts knowledge from raw sensor streams. The raw data must first undergo a series of preprocessing steps to convert them into an appropriate format for subsequent processing. Typically it includes: i) feature extraction, to identify relevant attributes for semantic labeling using such techniques as change detection and feature selection, and feature transformation like normalization, and Fourier or wavelet transforms; ii) data cleaning, to detect and correct data quality issues such as noise, outliers, missing values, and miscalibration errors; iii) data reduction, to improve the processing time or reduce the variability in data by means of techniques such as statistical sampling and data aggregation; and iv) dimension reduction, to transform the data in the high-dimensional space to a space of fewer dimensions with linear (e.g., principal component analysis) and nonlinear dimension reduction techniques (e.g., locally linear embedding).

The next step of stream event processing is semantic annotation, which maps the stream data into a set of atomic semantic events. By these events we mean the current states of objects and environments, such as moving, lifting up hands, speech, etc., which cannot be decomposed or represented straightforwardly in terms of other concepts. This process is usually considered as a supervised machine learning problem in multiple modalities, such as environment measures, audio, body movements, etc. For each modality, we can apply an appropriate machine learning model to detect events types automatically. For example, to detect motion status of the object attached with a passive RFID tag, in Chapter 3 we develop a Hidden Markov Model (HMM) based solution, augmented with an online Viterbi and change point algorithm.

The results of the semantic annotation process are processed by a semantic adaptor. One job of the adaptor is to transform the events with heterogeneous structures into a uniform format, for example, XML. Moreover, by referring to sensor-entity assignments, events in multiple modalities are explicitly

“attached” to a certain entity. For example, the gesture events, sourced from sensor ACC01 and blood pressure events sourced from sensor BP007, are related to the entity Alice. Notice that a semantic event can be related to multiple entities. For instance, the location event can be related to both object and zone.

In the semantic inference step, high-level semantics are inferred from low-level semantic events. In our framework, an event ontology is developed to enable semantic indexing and detecting of events machine-processable and exchanging event data between different processes. Moreover, the proposed OntoCEP system integrates event composition and semantic reasoning engines. The event composition engine detects complex patterns of events based on the relationships defined in event ontology, such as causality, and timing relationships. For example, we can define an event pattern for a person as: “if event SitDown is followed by event TouchBook, a new event Reading is inferred”. The semantic reasoning engine integrates context knowledge and conditions on which more meaningful events are deduced. For example, given an event “a person is still in a spot for a while”, if the system identifies the facts that the person is a ever being heart attack patient and the spot is not an area for sleep, then the system infers an event implying the patient has lost consciousness in a sudden. It is important to note that, other multimodality fusion models, such as Bayesian network (BN), and support vector machine (SVM) can also be applied to discover high-level semantic events.

1.3.3. AUTHORIZATION MODEL FOR SMART SPACES

With the development of pervasive information systems, on the one hand, smart services become more efficiently and ubiquitously accessible; however, on the other hand, new challenges and expectations of access control have arisen accompanying with the new paradigms of IoT. We can investigate these challenges from the “thing” aspect and the “Internet” aspect.

The “things”, which are the objects to be protected, are data-centric rather than document-centric, and usually requires a variety of information. Using a patient‟s information as an example, it may include her/his profile, medical histories, examination reports, radiology images and even the real-time activity information. Dealing with complex objects, the access control system needs to provide fine-grained authorization services, that is, different data records and different portions of the “things” can have different protection requirements. For example, some sensitive elements of the medical records, such as HIV/AIDS diagnosis, should be hid from general information during the sharing process, unless a special treatment option is indicated. Moreover, the authorization model is required to capture context dynamics and enforce right authorizations at the right time. For instance, the patient Alice‟s heart rate has drop to 25 and passing out, the doctors nearby, who in normal situations are not allowed to access the patient‟s electronic health record (EHR) database, can access Alice‟s recent diagnosis.

As the “things” are networked by the “Internet”, how to effectively secure information sharing among multiple data sources in the network becomes a challenging task. First, it is expected that the Internet-wide integration brings the IoT to a situation where a large volume of data records are made available to a large number of users. Second, a complex “thing” is likely to be distributed in multiple data sources. For instance, a good‟s transaction records (e.g., purchase order, invoice, tracking records) are distributed in different organizations over a supply chain. Moreover, the data from various data sources are syntactically and schematically heterogeneous. The typical problem is that, different systems create different schemas and maybe different element names in databases, though some may have the same meaning. Therefore, there is a need for an access control model that allows authorization policies‟ designers to define policies over heterogeneous systems for various organizations, while simultaneously ensuring that the size of the policy repository is always under control.

To solve these challenges, we propose a novel authorization model that can addresses them in a systematical, flexible and efficient way. Relying on semantic web technologies, the model annotates EHR objects and sub-objects with concepts. By doing this, authorizations can be specified over these predefined concepts instead of database records and documents, thus, eliminating structural heterogeneity across data sources and reducing the size of the policy repository. We also propose a decision propagation algorithm, based upon which decision propagation trees are derived from hierarchies of users and resources. Comparing with policy propagation, decision propagation can reuse access decisions of other subject-object pairs, which makes the computing and updating of access decision efficiently. We advocate the computing and storing of concept-level access decisions in an offline manner, since they normally are stable and of small number. Moreover, policy generation rules can be defined to make the specification process flexible. These rules also can capture context dynamics and condition for the enforcements of context-aware authorizations. Finally, our system models the decision computing process as a binary operation over a semilattice (McAlister, 1974). By extending the binary operation, various conflict resolving solutions can be implemented according to different requirements.

1.3.4. CONTRIBUTIONS OF THIS WORK

The contributions of this dissertation are manifold:

We develop a HMM-enabled solution augmented with an adaptive model for passive RFID systems. The solution applies online Viterbi algorithm to infer the motion status sequence based on the relative variance of response rate. Instead of using a dedicated hardware, the system is developed with passive RFID tags, which cost as low as several cents. The system can reuse existing infrastructure and billions of RFID devices (especially in supply chains). Moreover, as a method of non-intrusive monitoring, the possibility of invading people‟s privacy can be reduced.

We propose to use event ontology to index high-level semantics in smart spaces. Semantic indexing and detecting of events with respect to certain ontology makes it machineprocessable, allows for exchanging data between different processes, and improves semantic interoperability for heterogeneous event inference engines. Moreover, indexing semantic events allows users to subscribe and search and mine by specifying requests in terms of a limited vocabulary (keywords).

We develop a model named OntoCEP to coordinate various semantic event detectors in a smart space. As such, different types of concept detectors can interface with others on shared vocabulary and run in a chaining manner. This model makes implementing a universally compliant framework to fit multiple usage environments possible. The current implementation integrates event composition processes and context inference processes into a coherent system.

With semantic web technologies, we conceptualize healthcare data; thereby authorizations can be specified at different levels, and be propagated to lower-level concepts. Thereby, structural heterogeneity across data sources is eliminated and the size of the policy repository might be reduced. Moreover, concept-data matching and mapping can be conducted by local schema designers of data sources, so that policy designers do not need to get familiar with distributed data sources.

We develop a novel decision propagation model and use lookup table to enable fast evaluation and updating of authorization decisions. Moreover, the system resolves the conflict processes as a binary operation over a semilattice, which can be adapted to various requirements. Comparing with traditionally propagation methods, decision propagation can reuse access decisions of other subject-object pairs, which makes efficient the computing and updating of access decision. Moreover, based on different requirements, various conflict resolving solutions can be adopted by extending the binary operation.

Relying on ontology reasoning tools, we apply a rule-based policy generation mechanism to specify and enforce appropriate authorization. These policy generation rules can capture context dynamics and condition to enforce context-aware authorizations. Moreover, the system can define instance-level policies in a flexible way by using policy generation rules to encode the restrictions of entities‟ physical relationships.

The rest of the dissertation is organized as follows. In Chapter 2, we review the research that is related to our work, including background and preliminary technologies that we will adopt in our system design. In Chapter 3, we present the motion detection solution using passive RFID systems and evaluate its performance. In Chapter 4, we introduce an ontology-driven event detection infrastructure for smart spaces. In Chapter 5, a novel access control model for pervasive healthcare systems is presented to secure the system. Finally, in Chapter 6, we conclude our work with a summary, and discuss future research directions.