Automatic image segmentation for concealed object detection using the expectation-maximization algorithm

Dong-Su Lee; Seokwon Yeom; Jung-Young Son; Shin-Hwan Kim

doi:10.1364/OE.18.010659

1. Introduction

Recently, the researches on passive millimeter wave (PMMW) imaging techniques have increased for the security and defense applications [1–3]. The PMMW system forms interpretable imagery in the low-visibility conditions such as haze, fog, clouds, smoke, or sandstorms. The millimeter wave provides the penetrability into specific materials such as paper, plastic, clothing and fabrics [4–6]. Thus, the MMW imaging can identify concealed objects under clothing. The passive MMW imaging system largely depends on the temperature distribution of the objects with their emissivity and reflectivity [7–9]. Since the received signal is usually a weak thermal signal, the sensitivity and the noise robustness of the system are crucial to acquire high quality images. However, the passive imaging system is free of speckle and glint, and has a simple and cost-effective hardware compared to the active imagery system [10].

Image segmentation is a process to separate the foreground object from the background area [11]. The segmentation research has been extended to medical and microscopic imaging [12,13]. Various segmentation methods have been developed to detect the concealed objects on the body area in the MMW images [14–16]. One of the typical methods for the segmentation is a thresholding method [14,15]. The thresholding method is very effective when the image has the bimodal distributions and the distributions are located far enough from each other [14]. The multi-level thresholding was addressed to detect the contours of the concealed objects from Terahertz (THz) images [15]. The image segmentation using the Gaussian mixture model (GMM) has been researched on the metallic object detection [16].

In this paper, we propose the multi-level expectation maximization (EM) method to segment concealed objects from the body area. The histogram of the passive MMW image is modeled as a Gaussian mixture distribution. The EM method is an iterative solution for the maximization likelihood (ML) estimation of the GMM [17]. It has been applied to the three-dimensional color object visualization and recognition [18]. In the GMM, the probability density function (PDF) for each cluster is assumed to be Gaussian. The multi-level EM method finds two major clusters at each level. The EM algorithms are consecutively applied to one cluster, which is defined at the previous EM level. Thus, the first level EM separate the body area from background and the second level EM finds the concealed object in the body area. The performances of the conventional EM method and the proposed method are compared by the average probability of error [19–21]. In the experiments, it will be shown that the multi-level EM outperforms the conventional EM method.

The paper is organized as follows. In Section 2, we introduce the passive millimeter wave imaging system. The image segmentation using the multi-level EM algorithm is discussed in Section 3. In Section 4, we present the experimental results with the performance evaluation. The conclusions follow in Section 5.

2. A passive millimeter wave imaging system

The passive MMW imaging system is equipped with a Cassegrain dish antenna with a diameter of 0.5 m, as illustrated in Fig. 1 [5]. Two feed horn antennas are located at the focal plane of the antenna receiving the regime of 8 mm wavelength with vertical and horizontal polarization. They are apart from each other by 2 cm in the horizontal direction. The receiver channel connected to the feed horn is composed of the wave guide, the Dicke receiver, three monolithic microwave integrated circuit (MMIC) amplifiers, and a shottky diode detector. The scanning mechanism rotates both the antenna and the receiver in vertical and horizontal directions with a constant angular step. The angular resolution is around 1.1° according to the Rayleigh criterion since λ is 8 mm and D is 0.5 m. The scanning ranges are 60° and 106° in vertical and horizontal directions, respectively. The angular step size and the integrating time can be modified from 0.1° to 1° and from 20 ms to 200 ms, respectively.

Fig. 1 Passive millimeter wave imaging system.

Download Full Size | PDF

3. Image segmentation for concealed weapon detection

The overall process of the segmentation is composed of the Gaussian smoothing filtering (GSF), two-level image segmentations using the EM method, and decision processes as illustrated in Fig. 2 .

Fig. 2 Image segmentation procedure for concealed object detection.

Download Full Size | PDF

We adopt the GSF to remove the noise on the body contour for edge enhancement and the noise in the background for better visualization. The standard deviation and the window of the Gaussian smoothing filtering (GSF) are set at 0.5 and a 3 × 3 matrix, respectively. The first level EM extracts the body area from the background. The first Bayesian decision rule is followed by body area detection, which removes any segmented object inside the body area. During this process, we also perform the morphological erosion of the body area in order to estimate the accurate contour of the body area. The erosion process is also required since the body area has been dilated by the GSF. The second level EM is followed by the second Bayesian decision rule to detect the concealed objects in the body area.

At each level of the EM process, the histogram is modeled with two components of the Gaussian mixture distributions. The PDF of each pixel is assumed as follows

p (x_{t}) = \sum_{k = 1}^{N_{k}} N (x_{t} | μ_{k}, Σ_{k}) P (G_{k}), \begin{array}{c} t = 1, ..., N_{t} \end{array},

N (x_{t}; μ_{k}, Σ_{k}) = \frac{1}{{‖ 2 π Σ_{k} ‖}^{1 / 2}} e^{- \frac{1}{2} {(x_{t} - μ_{k})}^{t} Σ_{k}^{- 1} (x_{t} - μ_{k})},

where

μ_{k}

and

Σ_{k}

are the mean vector and the covariance matrix of the component k, respectively,

N_{t}

is the number of pixels,

N_{k}

is the number of clusters, which is set at 2, and the superscript t denotes matrix transpose. Therefore, solving the GMM is equivalent to estimating unknown parameters

μ_{k}

,

Σ_{k}

and

P (G_{k})

,

k = 1, ..., N_{k}

. The ML estimation of the parameters of the mixture model is as follows [17,18]

P (G_{k} | x_{t}) = \frac{N (x_{t}; μ_{k}, Σ_{k}) P (G_{k})}{\sum_{k = 1}^{N_{k}} N (x_{t}; μ_{k}, Σ_{k}) P (G_{k})},

μ_{k} = \frac{\sum_{t = 1}^{N_{t}^{}} P (G_{k} | x_{t}) x_{t}}{\sum_{t = 1}^{N_{t}} P (G_{k} | x_{t})},

Σ_{k} = \frac{\sum_{t = 1}^{N_{t}} P (G_{k} | x_{t}) (x_{t} - μ_{k}) {(x_{t} - μ_{k})}^{t}}{\sum_{t = 1}^{N_{t}} P (G_{k} | x_{t})},

P (G_{k}) = \frac{1}{N_{t}} \sum_{t = 1}^{N_{t}} P (G_{k} | x_{t}) .

Since Eqs. (3)–(6) are highly interated non-linear functions, the EM algorithm is adopted to solve the ML estimation. The EM algorithm uses a set of training data to iteratively estimate the parameters until convergence. The disadvantage of the EM algorithm can be its slow convergence, and dependent of the initial values. However, it is noted that the proposed method does not require deciding the number of the component. Figure 3 shows the block diagram of the EM algorithm; i represents the number of the iteration, ε and $i_{\max}$ are the termination criteria for the iteration, and $L_{i}$ is the likelihood $L_{i} = \sum_{t = 1}^{N_{t}} \log p_{i} (x_{t})$ . In the experiments, the size of the body is assumed to be comparable with the background and the concealed object is assumed to be smaller than the body area. Thus, we set the initial probabilities of the clusters for the first level EM as $P^{0} (G_{1}) = 0.5$ and $P^{0} (G_{2}) = 0.5$ , respectively, to extract the body area, and for the second level EM as $P^{0} (G_{1}) = 0.999$ and $P^{0} (G_{2}) = 0.001$ , respectively, to extract the concealed object. It is noted that the initial probabilities can defend on the normality of data. We have decided the initial probabilities heuristically when the best results are provided. The termination criteria is set ε at $10^{- 5}$ and $10^{- 50}$ and $i_{\max}$ at 200 and 500 for the first and the second level EM, respectively.

Fig. 3 EM algorithm.

Download Full Size | PDF

The following Bayesian decision rule [19,20] segments two areas, $G_{1}$ and $G_{2}$ .

P (G_{1} | x_{t}) {\begin{array}{l} 〉 \\ 〈 \end{array}}_{G_{2}}^{G_{1}} P (G_{2} | x_{t}) \begin{array}{c} , & t = 1, ..., N_{t} \end{array} .

In the experiment, we will show that the first EM separates the body area from the background area. The object is extracted from the body area by the second EM.

The average probability of error [19–21] is introduced for performance measure since the segmentation in the paper is equivalent with the binary hypothesis testing:

P_{e} = P (D_{1} | H_{0}) P (H_{0}) + P (D_{0} | H_{1}) P (H_{1}),

where

H_{1}

and

H_{0}

indicate the hypotheses of the foreground object and the background object, respectively, and

D_{1}

and

D_{0}

indicate the decision for the foreground object and the background object, respectively. Thus, the following equations are calculated:

P (D_{1} | H_{0}) = \frac{| F_{S} - F_{O} |}{| B_{O} |},

P (H_{0}) = \frac{| B_{O} |}{| B_{O} \cup F_{O} |},

P (D_{0} | H_{1}) = \frac{| F_{O} - F_{S} |}{| F_{O} |},

P (H_{1}) = \frac{| F_{O} |}{| B_{O} \cup F_{O} |},

where

F_{S}

,

F_{O}

, and

B_{O}

indicate the sets of pixels in the foreground object segmented, the ground truth of the foreground object, and the ground truth of the background object, respectively, and

| \cdot |

is the operation of counting the pixels in the set. With Eqs. (9)–(12), the Bayesian probability error becomes

P_{e} = \frac{| F_{S} - F_{O} | + | F_{O} - F_{S} |}{| B_{O} \cup F_{O} |} .

It is noted that, in the experiments, $F_{O}$ is the concealed object and $B_{O}$ is the body area except for the concealed object.

4. Experimental and simulation results

In this section, we present the segmentation results of the several PMMW images capturing the concealed objects. The several concealed objects are captured by the passive MMW system. Figures 4(a) and 4(b) are the images obtained by a visual camera. They represent a human subject and the object attached to the body, respectively. Figures 4(c) and 4(d) are the PMMW images with 8 mm horizontal and 8 mm vertical polarization, respectively. The bright part inside the body is the concealed axe.

Fig. 4 Visual images (a) with concealment, (b) without concealment, PMMW image of (c) 8 mm-vertical polarization with the GSF, (d) 8 mm-horizontal polarization with the GSF.

Download Full Size | PDF

Figures 5(a) and 5(b) are the fitting results by using the GMM to the histogram of the image after the first and second level EM segmentation, respectively. Figures 5(c) and 5(d) are the segmented body area and concealed object area after the 1st and 2nd segmentation processes. By fitting the Gaussian mixture distributions to the histograms, we can visualize how exactly the EM estimate the GMM models in Figs. 5(a) and 5(b).

Fig. 5 GMM fitting results of, (a) the 1st EM (b) the 2nd EM, segmented images after, (c) the 1st EM, (d) the 2nd EM.

Download Full Size | PDF

Figure 6 shows other experimental results by the multi-level EM method. The first column images are obtained by a visual camera without the concealment. The second column images are the passive MMW images. The third column images are the segmented images using the conventional EM method. The fourth column images are the segmented images using the proposed method. As show in the segmentation results, the proposed method provides the better results than the conventional EM method.

Fig. 6 Experimental results, axe, (a) without concealment, (b) PMMW image, segmented images after, (c) 1st EM, (d) 2nd EM, gun, (e) without concealment, (f) PMMW image, segmented images after, (g) 1st EM (h) 2nd EM, skin-aid, (i) without concealment, (j) PMMW image, segmented images after, (k) 1st EM, (l) 2nd EM, water-pack, (m) without concealment, (n) PMMW image, segmented images after, (o) 1st EM, (p) 2nd EM.

Download Full Size | PDF

Figure 7 shows the performance evaluation by the average probability of error. The multi-level EM has increased the performance by 36.7%, 39.4%, 68.4%, 90.0%, respectively, from the axe, gun, skin-aid, and water-pack. The average improvement is around 58.7%.

Fig. 7 Performance comparison between the conventional EM and the multi-level EM.

Download Full Size | PDF

5. Conclusions

In this paper, we have presented the image segmentation techniques for concealed weapons detected with the multi-level EM method. The passive MMW images usually have the low contrast and resolution. Also, the images are often noisy due to the low-signal level. Therefore, a special segmentation method has been developed to detect the concealed objects. This method can be extended for multi-channel images in the future, taking advantage of the polarization effects and diversity. We also leave the comparison with other segmentation approaches with non-Gaussian mixture models or GMM with normality enhancement for the future research topic.

Acknowledgments

We are grateful to Mr. Hyoung Lee and Mr. Vladmir. P. Guschin for their experimental support in passive MMW imaging. This work was supported by the Korea Science and Engineering Foundation (KOSEF) grant funded by the Korea government (MEST) (No. R01-2008-000-2118-0).

References and links

1. H.-M. Chen, S. Lee, R. M. Rao, M.-A. Slamani, and P. K. Varshney, “Imaging for concealed weapon detection: a tutorial overview of development in imaging sensors and processing,” IEEE Signal Process. Mag. 22(2), 52–61 (2005). [CrossRef]

2. R. Appleby and R. N. Anderton, “Millimeter-wave and submillimeter-wave imaging for security and surveillance,” Proc. IEEE 95(8), 1683–1690 (2007). [CrossRef]

3. T. W. Du Bosq, J. M. Lopez-Alonso, and G. D. Boreman, “Millimeter wave imaging system for land mine detection,” Appl. Opt. 45(22), 5686–5692 (2006). [CrossRef] [PubMed]

4. L. Yujiri, M. Shoucri, and P. Moffa, “Passive millimeter-wave imaging,” IEEE Microw. Mag. 4(3), 39–50 (2003). [CrossRef]

5. J. Son, V. P. Guschin, A. G. Denisov, S. Yeom, and D. Jung, “Comparisons of 3 dimensional temperature distributions of mm and IR images,” Proceedings of IEEE Conference on Microwaves, Radar and Remote Sensing Symposium, MRRS, Kiev, 132–135 (2008).

6. F. Buersgens, G. Acuna, and R. Kersting, “Millimeter wave probing of the acoustic phase for concealed object detection,” Opt. Express 15(14), 8838–8843 (2007). [CrossRef] [PubMed]

7. M. R. Fetterman, J. Grata, G. Jubic, W. L. Kiser Jr, and A. Visnansky, “Simulation, acquisition and analysis of passive millimeter-wave images in remote sensing applications,” Opt. Express 16(25), 20503–20515 (2008). [CrossRef] [PubMed]

8. K. B. Cooper, R. J. Dengler, N. Llombart, T. Bryllert, G. Chattopadhyay, I. Mehdi, and P. H. Siegel, “An approach for sub-second imaging of concealed objects using terahertz (THz) radar,” Int. J. Infrared Millim. Waves 30, 1297–1307 (2009).

9. A. Denisov, V. Gorishnyak, S. Kuzmin, V. Miklashevich, V. Obolonskv, V. Radzikhovsky, B. Shevchuk, B. Yshenko, V. Uliz'ko, and J. Son, “Some experiments concerning resolution of 32 sensors passive 8 mm wave imaging system,” Proceedings of the 20th International Symposium on Space Terahertz Technology, Charlottesville, 227–229 (2009).

10. M.-K. Jung, Y.-S. Chae, S.-K. Kim, M. Koji, and J.-K. Rhee, “Studies on the millimeter-wave passive imaging system,” J. Korean Inst. Electr. and Electro. Mater. Eng. 43, 182–188 (2006).

11. R. C. Gonzalez, Digital Image Processing 2/E (Prentice-Hall Inc., 2003).

12. T. Fabritius, S. Makita, M. Miura, R. Myllylä, and Y. Yasuno, “Automated segmentation of the macula by optical coherence tomography,” Opt. Express 17(18), 15659–15669 (2009). [CrossRef] [PubMed]

13. T. Collier, P. Shen, B. de Pradier, K.-B. Sung, R. Richards-Kortum, M. Follen, and A. Malpica, “Near real time confocal microscopy of amelanotic tissue: dynamics of aceto-whitening enable nuclear segmentation,” Opt. Express 6(2), 40–48 (2000). [CrossRef] [PubMed]

14. D.-S. Lee, S. Yeom, J. Son, and S.-H. Kim, “Image segmentation of concealed objects detected by a PMMW imaging system,” presented at the 34th International Conference on Infrared, Millimeter, and Terahertz Waves, Paradise Hotel, Busan, Korea, 21–25 (2009).

15. X. Shen, C. R. Dietlein, E. Grossman, Z. Popovic, and F. G. Meyer, “Detection and segmentation of concealed objects in terahertz images,” IEEE Trans. Image Process. 17(12), 2465–2475 (2008). [CrossRef] [PubMed]

16. C. D. Haworth, Y. Saint-Pern, D. Clark, E. Trucco, and Y. R. Petillot, “Detection and tracking of multiple metallic objects in millimeter-wave images,” Int. J. Comput. Vis. 71(2), 183–196 (2007). [CrossRef]

17. A. Dempster, N. Laird, and D. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. R. Stat. Soc., B 39, 1–38 (1977).

18. S. Yeom, B. Javidi, P. Ferraro, D. Alfieri, S. Denicola, and A. Finizio, “Three-dimensional color object visualization and recognition using multi-wavelength computational holography,” Opt. Express 15(15), 9394–9402 (2007). [CrossRef] [PubMed]

19. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification (John Wiley & Sons, 2001).

20. K. Fukunaga, Introduction to Statistical Pattern Recognition 2nd (Academic Press, Boston, 1990).

21. W. A. Yasnoff, J. K. Mui, and J. W. Bacus, “Error measures for scene segmentation,” Pattern Recognit. 9(4), 217–231 (1977). [CrossRef]

Automatic image segmentation for concealed object detection using the expectation-maximization algorithm

Abstract

1. Introduction

2. A passive millimeter wave imaging system

3. Image segmentation for concealed weapon detection

4. Experimental and simulation results

5. Conclusions

Acknowledgments

References and links

Cited By

Figures (7)

Equations (13)

Optics Express