Welcome to Pengyu Hong's Homepage

Automatic Temporal-Spatial Pattern Modeling and Extraction

Automatic temporal-spatial pattern modeling is essential for unsupervised object modeling/recognition and has a wide range of applications, such as, image/video database, molecular modeling, network traffic monitoring, and so on. A temporal-spatial pattern usually consists of several primitives among which various contextual relations are defined. We adopted Attributed Relational Graph to represent samples and developed a probabilistic parametric ARG model, also called Pattern ARG, to represent temporal-spatial patterns. We proposed a EM based machine learning technique to automatically learn a Pattern ARG from unlabelled samples. The learned model characterizes both the appearance and the structure of the pattern, which is embedded in various backgrounds and recurs statistically significantly across samples. The learning procedure calculates: (a) the attributed parameters (appearance/non-spatial information) of the pattern ARG, (b) the relational parameters (spatial information) of the pattern ARG, (c) the configuration (the number of nodes and that of the relations) of the pattern ARG, and (d) the node and relation correspondences between the components of the pattern ARG and the sample ARGs. In addition, the learning procedure is able to distinguish the pattern from its backgrounds as long as the pattern recurs statistically significantly across samples.

Examples: In the following examples, the images are segmented. Each node of the ARG represents a segment of the image. The attribute of the node is the mean color (RGB) feature vector of the segment. The adjacent relations among the segments are considered. The pattern ARG models are assumed to be Contextual Gaussian Mixture models.

Example 1. Learn Patterns from Images

Sample images

Segmentations

Extracted Patterns

(a) Original video. (download)

(b) Segmentated video. (download)

(c) ARG representation of the video. (download)

(d) Learning results shown as the Pattern ARG. (download)

(e) Learning results shown as the extracted images. (download)

• Hong, P. and Huang, T. S. (2004) Spatial pattern discovery by learning a probabilistic parametric model from multiple attributed relational graphs. Journal of Discrete Applied Mathematics, vol. 130, pp. 113-135. (PDF)
• Hong, P. and Huang, T. S. Mining inexact spatial pattern. Workshop on Discrete Mathematics and Data Mining. (DM & DM) 2002.
• Hong, P., Wang, R., and Huang, T. S. Learning Patterns from Images by Combining Soft Decisions and Hard Decisions.IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina, June 13-15, 2000.