Monitoring Human Activity

A project of the Artifical Intelligence, Robotics and Vision Laboratory
University of Minnesota, Department of Computer Science and Engineering

Home

Detection of Events

Real-Time Tracking

Action Recognition

Learning Patterns from Video Sequences

Publications

People

The problem of using computer vision to track and understand the behavior of human beings is a very important one. It has applications in the areas of human-computer interaction, user interface design, robot learning, and surveillance, among others. At its highest level, this problem addresses recognizing human behavior and understanding intent and motive from observations alone. This is a difficult task, even for humans to perform, and misinterpretations are common. We are studying this problem in the areas of human motion recognition, surveillance, tracking, and activity detection.

In the area of surveillance, automated systems to observe pedestrian traffic areas and detect dangerous action are becoming important. Many such areas currently have surveillance cameras in place. However, all of the image understanding and risk detection is left to human security personnel. This type of observation task is not well suited to humans, as it requires careful concentration over long periods of time. Therefore, there is clear motivation to develop automated, intelligent, vision-based monitoring systems that can aid a human user in the process of risk detection and analysis.

Detection of Events


Detection of Abandoned Objects	Detection of Thrown Objects	Detection of Unusual Crowd Activity
Camera Tampering Detection	Perimeter Breach Detection	Detection of Motion in Restricted Areas
Detection of Loitering Individuals

Real-Time Tracking


Tracking People with Probabilistic Occlusion Reasoning	Tracking People Using Pan-Tilt-Zoom Cameras	Tracking People in Simultaneous Video Streams

Action Recognition


Monitoring Crowded Scenes	View-Dependent Human Motion Recognition	Online Motion Recognition
View-Independent Human Motion Recognition	Monitoring Bus Stops	General Activity Recognition

Learning Patterns From Video Sequences


Learning Static Occlusions from a Moving Figure

Publications

E. Ribnick, S. Atev, and N. Papanikolopoulos. "Estimating 3D Positions and Velocities of Projectiles from Monocular Views." IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), to appear 2009.

E. Ribnick and N. Papanikolopoulos. "Estimating 3D Trajectories of Periodic Motions from Stationary Monocular Views." In Proceedings of the European Conference on Computer Vision (ECCV), 2008.

P. Kilambi, E. Ribnick, A. Joshi, O. Masoud, and N. Papanikolopoulos. "Estimating Pedestrian Counts in Groups." Computer Vision and Image Understanding (CVIU), 2008.

Ajay J. Joshi and Nikolaos Papanikolopoulos. "Learning of moving cast shadows for dynamic environments." In Proceedings of the IEEE International Conference on Robotics and Automation, 2008.

Ajay J. Joshi and Nikolaos Papanikolopoulos. "Learning to detect moving shadows in dynamic environments." In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 30, no. 11, pages 2055-2063, 2008.

S. Atev and N. Papanikolopoulos, "Multi-View 3D Vehicle Tracking with a Constrained Filter", In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2008), pp. 2277-2282, May 2008.

K. A Patwardhan, G. Sapiro, and V. Morellas, "Robust Foreground Detection In Video Using Pixel Layers". IEEE Pattern Analysis and Machine Intelligence, PAMI(30), No. 4, April 2008, pp. 746-751.

Ajay J. Joshi, Stefan Atev, Osama Masoud, and Nikolaos Papanikolopoulos. "Moving shadow detection with low- and mid-level reasoning." In Proceedings of the IEEE International Conference on Robotics and Automation, 2007.

E. Ribnick, S. Atev, O. Masoud, N. Papanikolopoulos, and R. Voyles. "Detection of Thrown Objects in Indoor and Outdoor Scenes." IEEE/RSJ Int'l Conf. on Intelligent Robots and Systems (IROS), 2007.

K. A Patwardhan, G. Sapiro, and V. Morellas, "A Graph-based Foreground Representation and its Application in Example Based People Matching in Video". IEEE Intl. Conference on Image Processing, ICIP (5) 2007: pp. 37-40.

H. Veeraraghavan, N. Bird, S. Atev, and N. Papanikolopoulos, "Classifiers for driver activity monitoring," Transportation Research Part C, vol. 15, no. 1, pp. 51-67, Feb. 2007.

E. Ribnick, S. Atev, O. Masoud, N. Papanikolopoulos, and R. Voyles. "Real-Time Detection of Camera Tampering." Proceedings of the IEEE Int'l Conf. on Advanced Video and Signal Based Surveillance (AVSS), 2006.

S. Atev, O. Masoud, and N. Papanikolopoulos, "Learning Traffic Patterns at Intersections by Spectral Clustering of Motion Trajectories", In Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS 2006), pp. 4851-4856, Oct. 2006.

N. Bird, S. Atev, N. Caramelli, R. Martin, O. Masoud, and N. Papanikolopoulos, "Real time, online detection of abandoned objects in public areas," in Proceedings of the IEEE International Conference on Robotics and Automation, pp. 3775-3780, May 2006.

S. Atev, H. Arumugam, O. Massoud, R. Janardan, and N. Papanikolopoulos, "A Vision-Based Approach to Collision Prediction at Traffic Intersections", IEEE Transactions on Intelligent Transportation Systems, vol. 6, no. 4, pp. 416-423, Dec. 2005.

H. Veeraraghavan, S Atev, N. Bird, P. Schrater, and N. Papanikolopoulos, "Driver Activity Monitoring through Supervised and Unsupervised Learning", In Proceedings of the IEEE Conference on Intelligent Transportation Systems (ITSC 2005), pp. 895-900, Sep. 2005.

S. Atev, O. Masoud, R. Janardan, and N. Papanikolopoulos, "A Collision Prediction System for Traffic Intersections", In Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS 2005), pp. 2844-2849, Aug. 2005.

N. Bird, O. Masoud, N. Pananikolopoulos, and A. Isaacs, "Detection of loitering individuals in public transportation areas," IEEE Transactions on Intelligent Transportation Systems, vol. 6, no. 2, pp. 167-177, June 2005.

S. Atev, O. Masoud, and N. Papanikolopoulos, "Practical Mixtures of Gaussians with Brightness Monitoring", In Proceedings of the IEEE Conference on Intelligent Transportation Systems (ITSC 2004), pp. 423-428, Oct. 2004.

G. Gasser, N. Bird, O. Masoud, and N. Papanikolopoulos, "Human activities monitoring and bus stops," in Proceedings of the IEEE International Conference on Robotics and Automation, vol. 1, pp. 90-95, Apr. 2004.

V. Morellas, I. Pavlidis and P. Tsiamyrtzis, "DETER: Detection of Events for Threat Evaluation and Recognition". Machine Vision and Applications, 15:29-45, 2003.

I. Pavlidis, V. Morellas, P. Tsiamyrtzis, and S. Harp, "Urban Surveillance Systems: From the Laboratory to the Commercial World". IEEE Proceedings, 89(10), pp. 1478-1497, October, 2001.

People

Faculty
Dr. Nikolaos Papanikolopoulos
Dr. Vassilios Morellas

Graduate Students
Stefan Atev
Nathaniel Bird
Anoop Cherian
David Kuo-Wei Hsu
Ajay Joshi
Robert Martin
Evan Ribnick
Ravishankar Sivalingam
Guruprasad Somasundaram

Undergraduate Students
Tyson Giraud

Alumni

Dr. Osama Masoud

Pushkar Modi

This work is supported by grants from the National Science Foundation, the Minnesota Department of Transportation, the University of Minnesota ITS Institute, and the Department of Homeland Security. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.