RESEARCH


Understanding Intent Using a Novel Hidden Markov Model Representation
Understanding intent is an important aspect of communication among people and is an essential component of the human cognitive system. This capability is particularly relevant for situations that involve collaboration among multiple agents or detection of situations that can pose a particular threat. For surveillance or military applications, it is highly important to enable understanding the intent of relevant agents in the environment, from their current actions, before any attack strategies are finalized. The approach relies on a novel formulation of Hidden Markov Models (HMMs), which allows a robot to understand the intent of other agents by virtually assuming their place and detecting their potential intentions based on the current situation. This allows the system to recognize the intent of observed actions before they have been completed, thus enabling preemptive actions for defense. The system's capability to observe and analyze the current scene employs novel vision-based techniques for target detection and tracking, using a non-parametric recursive modeling approach.

Relevant Publications
Support
This work was supported by the Office of Naval Research award N00014-06-1-0611 and by the National Science Foundation EPSCoR Ring True III award EPS0447416.

Segmentation for Videos with Quasi-Stationary Backgrounds - A Non-Parametric Approach

Video segmentation is one of the most important tasks in high-level video processing applications. Background modeling is the key to detection of foreground regions (such as moving objects - e.g., people, cars) in videos where camera is assumed to be stationary. However, possible changes in the background of the video such as waving flags, fluctuating monitors, water surfaces, etc. make it difficult to detect objects of interest in the scene. Due to the diverse nature of video applications it has been a main concern for researchers to design a general, scene-independent system. In this project we first propose a novel adaptive statistical method as a baseline system that addresses this issue. After investigating its performance we introduce a universal statistical technique which aims to overcome the weaknesses of its predecessor in modeling slow changes in the background. Finally, a new analytical technique is proposed in this work that approaches the problem of background modeling in a different direction. This technique is introduced in order to solve the limitations of statistical techniques which are bound to the accuracy of the probability density estimation. The performance of each of the proposed methods is studied and scenarios in which each of them leads to a better performance are investigated.

Relevant Publications
Support
This work was supported in part by a grant from the University of Nevada Junior Faculty Research Grant Fund and by NASA under grant NCC5-583.

Visual Awareness and Long-Term Autonomy for Robotic Assistants
A major challenge in deploying robots into the real world is the design of an architectural framework which can provide long-term, natural and effective interactions with people. Within this framework, key issues that need to be solved relate to the robots' ability to engage in interactions in a natural way, to deal with multiple users, and to be constantly aware of their surroundings. We propose a control architecture that addresses these issues. First, we endow our robot with a visual awareness mechanism, which allows it to detect when people are requesting its attention and try to engage it in interaction. Second, we provide the robot with flexibility in dealing with multiple users, such as to accommodate multiple user requests and task interruptions, over extended periods of time. In support of our robot awareness mechanism, we develop visual capabilities that allow the robot to identify multiple users, with multiple postures, in real-time, in dynamic environments in which both the robot and human users are moving. To enable long-term interaction, we design a control architecture which enables the representation of complex, sequential and hierarchical robot tasks.

Relevant Publications
Support
This work was supported by the Office of Naval Research award N00014-06-1-0611.

An Automatic Framework for Figure-Ground Segmentation in Cluttered Backgrounds

Grouping processes, which "organize" given data by eliminating irrelevant items and sorting the rest into groups, each corresponding to a particular object, can provide reliable pre-processed information to higher level vision functions, such as object detection and recognition. Here we consider the problem of grouping oriented segments in highly cluttered images. We developed a general scheme based on an iterative tensor voting approach which has been shown to improve segmentation results considerably. Segments are represented as second-order tensors and communicate with each other through a voting scheme that incorporates the Gestalt principles of visual perception. The key idea of our approach is to conservatively remove background segments using multi-scale analysis, and re-vote on the retained segments. This process results in better quality segmentations, especially under severe background clutter. Particularly remarkable, our experiments reveal that using our approach as a post-processing step to the boundary detection methods evaluated with the Berkeley dataset improves the results in 84% of the grayscale test images from this benchmark.

Relevant Publications

Efficient Vehicle Tracking and Classification for an Automated Traffic Surveillance System
We propose a new traffic surveillance system that works without prior, explicit camera calibration, and has the ability to perform surveillance tasks in real time. Camera intrinsic parameters and its position with respect to the ground plane are derived using geometric primitives common to any traffic scene. We use optical flow and knowledge of camera parameters to detect the pose of a vehicle in the 3D world. This information is employed by a model-based vehicle detection and classification technique used by our traffic surveillance application. The object (vehicle) classification uses two new techniques - color contour-based matching and gradient-based matching. Our experiments on several real traffic video sequences demonstrate good results for our foreground object detection, tracking, vehicle detection and vehicle speed estimation approaches.

Relevant Publications

Voting-Based Computational Framework for Motion Analysis



Our research addresses the problem of visual motion analysis, by formulating it as a motion layers inference from a noisy and possibly sparse point set in a 4-D space. Our approach is based on a layered 4-D representation of data and a voting scheme for token communication, within a tensor voting computational framework. From a possibly sparse input consisting of identical point tokens in two frames, the image position and potential velocity of each token are encoded into a 4-D tensor. Within this 4-D space, moving regions are conceptually represented as smooth surface layers, and are extracted through a voting process that enforces the smoothness constraint while preserving motion discontinuities.
The key features of this approach are: 1) inference of a dense representation in terms of accurate velocities, motion boundaries and regions, without any a priori knowledge of the motion model, based on the smoothness of motion only; 2) consistent handling of both smooth moving regions and motion discontinuities; 3) integration of motion and monocular (intensity) cues for accurate segmentation; 4) a 4-D layered representation that allows for spatial separation of the points according to both velocities and image coordinates, thus letting tokens from the same layer to strongly support each other, while inhibiting influence from other layers, or from isolated tokens; 5) a non-iterative voting scheme, which does not require initialization and does not suffer from local optima or poor convergence problems, and whose only free parameter is the scale of analysis, an inherent characteristic of human vision. [Details]

Relevant Publications
Support
This research has been funded in part by the Integrated Media Systems Center (IMSC), a National Science Foundation Engineering Research Center, Cooperative Agreement No. EEC-9529152, and by National Science Foundation Grant 9811883.

GlobeAll: Panoramic Video for an Intelligent Room
This project is targeted at a real-time modular system for vision-based intelligent environments. We designed and developed GlobeAll, a modular prototype based on an electronic pan-tilt-zoom camera array. The visual input is acquired by a multiple-camera system, which generates a composite view of the scene with a wide field of view (as a planar mosaic) and a view of the desired region of interest (as an electronically-controlled virtual camera). By maintaining an adaptive background model in mosaic space, the system segments the foreground objects as planar layers. Among them, targets are selected and tracked by redirecting the virtual camera. An interpretation module analyzes the generated models (segmented objects, trajectories), allowing for the detection of simple events.
Compared to other solutions, the key features of our system are: 1) acquisition of a large field of view, while also capturing enough resolution for focusing on a certain region of interest; 2) ability to perform pan-tilt-zoom operations electronically rather than mechanically; 3) better precision and response time in redirecting the region of interest; 4) low cost and high robustness, since it involves a digital solution, instead of using expensive and fragile mechanical or optical components. [Details]

Relevant Publications
Support
This research has been funded in part by the Integrated Media Systems Center (IMSC), a National Science Foundation Engineering Research Center, Cooperative Agreement No. EEC-9529152, with additional support from the Annenberg Center for Communication at the University of Southern California and the California Trade and Commerce Agency. The support of the Philips Multimedia Center is also gratefully acknowledged.

Created by: Mircea NICOLESCU (e-mail: mircea@cse.unr.edu)