Monitoring Human Activity
A project of the Artifical Intelligence, Robotics and Vision Laboratory
University of Minnesota, Department of Computer Science and Engineering
Home Detection of Events Real-Time Tracking Action Recognition Learning Patterns from Video Sequences Breathing Abnormality Detection Other Projects Publications People


Attention:

We are currently looking for social and behavioral science research assistants to work on our mental health related projects. Those interested can contact Nikos Papanikolopoulos

 
The problem of using computer vision to track and understand the behavior of human beings is a very important one. It has applications in the areas of human-computer interaction, user interface design, robot learning, and surveillance, among others. At its highest level, this problem addresses recognizing human behavior and understanding intent and motive from observations alone. This is a difficult task, even for humans to perform, and misinterpretations are common. We are studying this problem in the areas of human motion recognition, surveillance, tracking, and activity detection.

In the area of surveillance, automated systems to observe pedestrian traffic areas and detect dangerous action are becoming important. Many such areas currently have surveillance cameras in place. However, all of the image understanding and risk detection is left to human security personnel. This type of observation task is not well suited to humans, as it requires careful concentration over long periods of time. Therefore, there is clear motivation to develop automated, intelligent, vision-based monitoring systems that can aid a human user in the process of risk detection and analysis.





Detection of Events


Detection of Abandoned Objects
Detection of Thrown Objects
Detection of Unusual Crowd Activity

Camera Tampering Detection
Perimeter Breach Detection
Detection of Motion in Restricted Areas

Detection of Loitering Individuals


Real-Time Tracking


Tracking People with Probabilistic Occlusion Reasoning
Tracking People Using Pan-Tilt-Zoom Cameras
Tracking People in Simultaneous Video Streams


Action Recognition


Monitoring Crowded Scenes
View-Dependent Human Motion Recognition
Online Motion Recognition

View-Independent Human Motion Recognition
Monitoring Bus Stops
General Activity Recognition


Learning Patterns From Video Sequences


Learning Static Occlusions from a Moving Figure


Breathing Abnormality Detection


Detecting Breathing Abnormalities During Radiation Treatment


Other projects


Behavioral Analysis of At-Risk Children


Publications

R. Sivalingam, D. Boley, V. Morellas, and N. Papanikolopoulos, "Tensor Sparse Coding for Region Covariances." In: K. Daniilidis, P. Maragos & N. Paragios (Eds.), ECCV 2010, LNCS Vol. 6314, pp 722-735, Springer Berlin/Heidelberg (2010).

R. Sivalingam, G. Somasundaram, V. Morellas, N. Papanikolopoulos, O. Lotfallah and Y. Park, "Dictionary Learning-based Object Detection and Counting in Traffic Scenes." Proceedings of the 4th ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC), 2010.

A. J. Joshi, F. Porikli, and N. Papanikolopoulos. "Breaking the interactive bottleneck in multi-class classification with active selection and binary feedback." IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010.

A. Cherian, J. Andersh, V. Morellas, B. Mettler, and N. Papanikolopoulos, "Motion Estimation of a Miniature Helicopter using a Single Onboard Camera." American Control Conference (ACC 2010), pp. 4456-4461, June 2010.

A. Drenner, M. Janssen, and N. Papanikolopoulos, "A Strategy for Improving Observability with Mobile Robots." Proceedings of the IEEE International Conference Robotics and Automation (ICRA 2010).

E. Ribnick, V. Morellas and N. Papanikolopoulos. "Human Motion Patterns from Single Camera Cues for Medical Applications." Proceedings of the IEEE International Conference Robotics and Automation (ICRA 2010).

A. J. Joshi, F. Porikli, and N. Papanikolopoulos. "Multi-class batch-mode active learning for image classification." Proceedings of the IEEE International Conference Robotics and Automation (ICRA 2010).

E. Ribnick and N. Papanikolopoulos. "3D Reconstruction of Periodic Motion from a Single View." International Journal of Computer Vision (IJCV), 2010.

A. J. Joshi, F. Porikli, and N. Papanikolopoulos. "Multi-class active learning with binary user feedback." Advances in Neural Information Processing Systems (NIPS), Workshop on Analysis and Design of Algorithms for Interactive Machine Learning (ADA-IML), 2009.

E. Ribnick and N. Papanikolopoulos. "View-Invariant Analysis of Periodic Motion." Proceedings of the IEEE/RSJ Int'l Conf. on Intelligent Robots and Systems (IROS 2009).

A. Cherian, J. Andersh, V. Morellas, N. Papanikolopoulos, and B. Mettler, "Autonomous Altitude Estimation of a UAV using a Single Onboard Camera." Proceedings of the IEEE/RSJ Int'l Conf. on Intelligent Robots and Systems (IROS 2009) pp. 3900-3905, Oct. 2009.

N. Bird and N. Papanikolopoulos. "Placement Quality in Structured Light Systems." Proceedings of the IEEE/RSJ Int'l Conf. on Intelligent Robots and Systems (IROS 2009).

R. Sivalingam, V. Morellas, D. Boley, and N. Papanikolopoulos, "Metric Learning for Semi-supervised Clustering of Region Covariance Descriptors." Proceedings of the 3rd ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC), 2009, pp.1-8, Aug. 30 2009-Sept. 2 2009.

E. Ribnick, S. Atev, and N. Papanikolopoulos. "Estimating 3D Positions and Velocities of Projectiles from Monocular Views." IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2009.

A. Cherian, V. Morellas, and N. Papanikolopoulos, "Accurate 3D Ground Plane Estimation from a Single Image." Proceedings of the IEEE International Conference Robotics and Automation (ICRA 2009), pp. 2243-2249, May 2009.

D. Fehr, R. Sivalingam, V. Morellas, N. Papanikolopoulos, O. Lotfallah, and Y. Park, "Counting People in Groups." Proceedings of the 6th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2009, pp.152-157, 2-4 Sept. 2009.

G. Somasundaram, V. Morellas, N. Papanikolopoulos, Lisa Austin. "Counting pedestrians and bicycles in traffic scenes." Proceedings of the 2009 IEEE Conference on Intelligent Transportation Systems Conference, St.Louis, Missouri, Oct 3-7, 2009.

E. Ribnick and N. Papanikolopoulos. "Estimating 3D Trajectories of Periodic Motions from Stationary Monocular Views." In Proceedings of the European Conference on Computer Vision (ECCV), 2008.

P. Kilambi, E. Ribnick, A. Joshi, O. Masoud, and N. Papanikolopoulos. "Estimating Pedestrian Counts in Groups." Computer Vision and Image Understanding (CVIU), 2008.

A. J. Joshi and N. Papanikolopoulos. "Learning of moving cast shadows for dynamic environments." In Proceedings of the IEEE International Conference on Robotics and Automation, 2008.

L. Fiore, G. Somasundaram, A. Drenner and N. Papanikolopoulos. "Optimal camera placement with adaptation to dynamic scenes." Proceedings of the 2008 IEEE Conference on Robotics and Automation, Pasadena, CA, May 19-23, 2008. Pages: 956-961.

A. J. Joshi and N. Papanikolopoulos. "Learning to detect moving shadows in dynamic environments." In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 30, no. 11, pages 2055-2063, 2008.

S. Atev and N. Papanikolopoulos, "Multi-View 3D Vehicle Tracking with a Constrained Filter.", In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2008), pp. 2277-2282, May 2008.

K. A Patwardhan, G. Sapiro, and V. Morellas, "Robust Foreground Detection In Video Using Pixel Layers.". IEEE Pattern Analysis and Machine Intelligence, PAMI(30), No. 4, April 2008, pp. 746-751.

L. Fiore, D. Fehr, R. Bodor, A. Drenner, G. Somasundaram, and N. Papnikolopoulos. "Multi-Camera Human Activity Monitoring." Journal of Intelligent Robotic Systems, Volume 52, Number 1, Pages 5-43, May 2008.

A. J. Joshi, S. Atev, O. Masoud, and N. Papanikolopoulos. "Moving shadow detection with low- and mid-level reasoning." In Proceedings of the IEEE International Conference on Robotics and Automation, 2007.

E. Ribnick, S. Atev, O. Masoud, N. Papanikolopoulos, and R. Voyles. "Detection of Thrown Objects in Indoor and Outdoor Scenes." IEEE/RSJ Int'l Conf. on Intelligent Robots and Systems (IROS), 2007.

K. A Patwardhan, G. Sapiro, and V. Morellas, "A Graph-based Foreground Representation and its Application in Example Based People Matching in Video.". IEEE Intl. Conference on Image Processing, ICIP (5) 2007: pp. 37-40.

H. Veeraraghavan, N. Bird, S. Atev, and N. Papanikolopoulos, "Classifiers for driver activity monitoring." Transportation Research Part C, vol. 15, no. 1, pp. 51-67, Feb. 2007.

E. Ribnick, S. Atev, O. Masoud, N. Papanikolopoulos, and R. Voyles. "Real-Time Detection of Camera Tampering." Proceedings of the IEEE Int'l Conf. on Advanced Video and Signal Based Surveillance (AVSS), 2006.

S. Atev, O. Masoud, and N. Papanikolopoulos, "Learning Traffic Patterns at Intersections by Spectral Clustering of Motion Trajectories", In Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS 2006), pp. 4851-4856, Oct. 2006.

N. Bird, S. Atev, N. Caramelli, R. Martin, O. Masoud, and N. Papanikolopoulos, "Real time, online detection of abandoned objects in public areas," in Proceedings of the IEEE International Conference on Robotics and Automation, pp. 3775-3780, May 2006.

S. Atev, H. Arumugam, O. Massoud, R. Janardan, and N. Papanikolopoulos, "A Vision-Based Approach to Collision Prediction at Traffic Intersections", IEEE Transactions on Intelligent Transportation Systems, vol. 6, no. 4, pp. 416-423, Dec. 2005.

H. Veeraraghavan, S Atev, N. Bird, P. Schrater, and N. Papanikolopoulos, "Driver Activity Monitoring through Supervised and Unsupervised Learning", In Proceedings of the IEEE Conference on Intelligent Transportation Systems (ITSC 2005), pp. 895-900, Sep. 2005.

S. Atev, O. Masoud, R. Janardan, and N. Papanikolopoulos, "A Collision Prediction System for Traffic Intersections", In Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS 2005), pp. 2844-2849, Aug. 2005.

N. Bird, O. Masoud, N. Pananikolopoulos, and A. Isaacs, "Detection of loitering individuals in public transportation areas," IEEE Transactions on Intelligent Transportation Systems, vol. 6, no. 2, pp. 167-177, June 2005.

S. Atev, O. Masoud, and N. Papanikolopoulos, "Practical Mixtures of Gaussians with Brightness Monitoring", In Proceedings of the IEEE Conference on Intelligent Transportation Systems (ITSC 2004), pp. 423-428, Oct. 2004.

G. Gasser, N. Bird, O. Masoud, and N. Papanikolopoulos, "Human activities monitoring and bus stops," in Proceedings of the IEEE International Conference on Robotics and Automation, vol. 1, pp. 90-95, Apr. 2004.

V. Morellas, I. Pavlidis and P. Tsiamyrtzis, "DETER: Detection of Events for Threat Evaluation and Recognition". Machine Vision and Applications, 15:29-45, 2003.

I. Pavlidis, V. Morellas, P. Tsiamyrtzis, and S. Harp, "Urban Surveillance Systems: From the Laboratory to the Commercial World". IEEE Proceedings, 89(10), pp. 1478-1497, October, 2001.


People



Faculty
Dr. Nikolaos Papanikolopoulos
Dr. Vassilios Morellas

Graduate Students
Stefan Atev
Nathaniel Bird
Anoop Cherian
David Kuo-Wei Hsu
Ajay Joshi
Robert Martin
Evan Ribnick
Ravishankar Sivalingam
Guruprasad Somasundaram


Undergraduate Students
Tyson Giraud

Alumni
Dr. Osama Masoud
Pushkar Modi
This work is supported by grants from the National Science Foundation, the Minnesota Department of Transportation, the University of Minnesota ITS Institute, and the Department of Homeland Security. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.