Elsevier

Neurocomputing

Volume 288, 2 May 2018, Pages 30-42
Neurocomputing

Early fire detection using convolutional neural networks during surveillance for effective disaster management

https://doi.org/10.1016/j.neucom.2017.04.083Get rights and content

Abstract

Fire disasters are man-made disasters, which cause ecological, social, and economic damage. To minimize these losses, early detection of fire and an autonomous response are important and helpful to disaster management systems. Therefore, in this article, we propose an early fire detection framework using fine-tuned convolutional neural networks for CCTV surveillance cameras, which can detect fire in varying indoor and outdoor environments. To ensure the autonomous response, we propose an adaptive prioritization mechanism for cameras in the surveillance system. Finally, we propose a dynamic channel selection algorithm for cameras based on cognitive radio networks, ensuring reliable data dissemination. Experimental results verify the higher accuracy of our fire detection scheme compared to state-of-the-art methods and validate the applicability of our framework for effective fire disaster management.

Introduction

Disaster management, as a hybrid research area, has attracted the attention of many research communities such as business, computer science, health sciences, and environmental sciences. According to federal emergency management agency policy, there are two main categories of disaster: (1) Technological such as emergencies related to hazardous materials, terrorism, and nuclear power plants etc., and (2) Natural such as floods, earth quakes, and forest fires etc. Regardless of the nature of the disaster, certain characteristics are necessary for effective management of almost all of them. These features include prevention, advance warning, early detection, early notification to the public and concerned authorities, response mobilization, damage containment, and providing medical care as well as relief to affected citizens [1]. Disaster management has four main phases including preparedness, mitigation, response, and recovery, each of which requires different types of data, which are needed by different communities during disaster management. Such data can be processed using data analysis technologies such as information extraction, information retrieval, information filtering, data mining, and decision support [2,3]. An overview of this data flow in disaster management is shown in Fig. 1.

Fig. 1 shows that data is gathered from different sources during disaster management, which are helpful for the detection of disaster, the response of concerned authorities against the disaster and its recovery. Among the given resources, online streaming data from CCTV cameras can be helpful for early detection of different disasters such as fire [4] and flood [5], which in turn can facilitate disaster management teams in quick recovery and reducing the loss of human lives.

Fire disasters mainly occur due to human error or the failure of a system, causing economic as well as ecological damage along with endangering human lives [6]. According to [7], wildfire disasters alone in the year 2015 resulted in 494,000 victims and caused damage worth US$ 3.1 billion. Each year, an area of vegetation of 10,000 km2 is affected by fire disasters in Europe. The statistic for fire damage is about 100,000 km2 in Russia and North America. Other examples of fire disasters include (1) the disaster of Arizona (USA, June 2013) which ruined 100 houses and killed 19 firefighters, and (2) the forest fire of California (August 2013) which burned an area of 1042 km2 and damaged around 111 structures, incurring a firefighting cost of $127.35 million [8]. Considering these examples of damage, early detection of fire is of paramount interest to disaster management systems, so as to avoid such disasters. In this context, researchers have explored different approaches to fire detection including conventional fire alerting systems and visual sensors based systems. The systems belonging to the first category are based on ion or optical sensors, needing close proximity to the fire, and thus failing to provide additional information such as the fire size, location, and degree of burning. In addition to this, such systems involve heavy human intervention, such as visiting the fire location to confirm the fire in the event of any fire alarm. To cope with these limitations, many fire detection systems based on visual sensors have been presented [9], [10], [11], [12].

Visual sensors based fire detection systems are motivated by several encouraging advantages including: (1) low cost due to the existing setup of installed cameras for surveillance, (2) monitoring of larger regions, (3) comparatively fast response time due to the elimination of waiting time for heat diffusion, (4) fire confirmation without visiting the fire location, (5) flexibility for the detection of smoke and flames through adjustment of certain parameters, and (6) the availability of fire details such as size, location, and degree of burning. Due to these characteristics, they have attracted the attention of many researchers and as a result, many fire detection methods [12], [13], [14], [15], [16], [17] have been investigated based on numerous visual features, achieving good performance. But still such methods encounter several problems such as the complexity of the scenes under surveillance due to people and objects looking like fire, the irregularity of lighting (night, day, artificial, shadows, light reflections, and flickering), and the low quality of the captured images, their lower contrast, and lower transmission of signals. These problems demand urgent solutions from the concerned research communities due to their importance to disaster management systems. Further, sending all the streaming data of multiple cameras during surveillance is impractical due to network constraints. In addition to this, an alert of fire and its associated keyframes need an autonomous and reliable communication medium for transmission, to enable the disaster management team to handle it as early as possible.

To address the aforementioned problems, we propose an early fire detection framework using convolutional neural networks (CNNs) and the internet of multimedia things (IoMT) for disaster management. To this end, the major contributions of this study can be summarized as follows:

  • (1)

    Unlike traditional hand-engineered features, which are not suitable for the detection of several types of fire, we incorporate deep features of CNNs in our fire detection framework, which can detect fire at an early stage under varying conditions. For this purpose, we used Alexnet as a baseline architecture and fine-tuned it according to our problem, considering the accuracy and complexity.

  • (2)

    Due to the emergency nature of fire for disaster management, we propose an adaptive prioritization mechanism for cameras in the surveillance system, which can adaptively switch the status of camera nodes based on their importance. Furthermore, our system contains a high-resolution camera that can be activated for capturing the important scenes when fire is detected. This can be helpful for disaster management systems in confirming the fire and analyzing the disaster data in real time.

  • (3)

    We propose a dynamic channel selection algorithm for high-priority cameras based on cognitive radio networks, ensuring reliable data dissemination and an autonomous response system for disaster management.

The rest of the paper is structured as follows: Related work on fire detection and disaster management is presented in Section 2. Our proposed work is explained in Section 3. Experimental results are provided in Section 4. Finally, our work is concluded in Section 5.

Section snippets

Related work

In this section, we first critically discuss the fire detection methods reported in the current literature along with their strengths and weaknesses. Next, we briefly highlight our approach to solving the problems of some of the current methods for early fire detection. Finally, we discuss how early fire detection can be used in effective disaster management systems. Recent advance in technology have resulted in a variety of sensors for different applications such as wireless capsule sensors

The proposed framework

Early fire detection in the context of disaster management systems during surveillance of public areas, forests, and nuclear power plants can result in the saving of ecological, economic, and social damage. However, early detection is a challenging problem due to varying lighting conditions, shadows, and the movement of fire-colored objects. Thus, there is a need for an algorithm that can achieve better accuracy in the aforementioned scenarios while minimizing the number of false alarms. To

Results and discussion

This section explains in detail the experiments conducted to evaluate the performance of the proposed framework. Firstly, we provide the experimental setup along with its details. Next, we explain different experiments performed on various datasets from the literature and compare our work with the state-of-the-art methods. Finally, we present the strengths of our method against different attacks.

Conclusions

Due to recent advances, CCTV cameras are able to perform different types of processing such as object and motion detection and tracking. Considering these processing capabilities, it is possible to detect fire at its early stage during surveillance, which can be helpful to disaster management systems, avoiding huge ecological and economic losses, as well as saving a large number of human lives. With this motivation, we proposed an early fire detection method based on fine-tuned CNNs during CCTV

Acknowledgment

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (No. 2016R1A2B4011712).

Khan Muhammad (S’16) received the bachelors degree in computer science from Islamia College Peshawar, Pakistan with research in information security. He is currently pursuing M.S. leading to Ph.D. degree in digitals contents from College of Software and Convergence Technology, Sejong University, Seoul, Republic of Korea. He has been a Research Associate with the Intelligent Media Laboratory (IM Lab) since 2015. His research interests include image and video processing, wireless networks,

References (60)

  • LiuW. et al.

    A survey of deep neural network architectures and their applications

    Neurocomputing

    (2017)
  • SunM. et al.

    Learning pooling for convolutional neural network

    Neurocomputing

    (2017)
  • A.O. Bicen et al.

    Delay-sensitive and multimedia communication in cognitive radio sensor networks

    Ad Hoc Netw.

    (2012)
  • I.F. Akyildiz et al.

    A survey on wireless multimedia sensor networks

    Comput. Netw.

    (2007)
  • R. Umar et al.

    A comparative study of spectrum awareness techniques for cognitive radio oriented wireless networks

    Phys. Commun.

    (2013)
  • I. Mehmood et al.

    Saliency-directed prioritization of visual data in wireless surveillance networks

    Inf. Fusion

    (2015)
  • S. Verstockt et al.

    Video driven fire spread forecasting (f) using multi-modal LWIR and visual flame and smoke data

    Pattern Recogn. Lett.

    (2013)
  • K. Muhammad et al.

    Efficient visual attention driven framework for key frames extraction from hysteroscopy videos

    Biomed. Signal Process. Control

    (2017)
  • L. Rossi et al.

    On the use of stereovision to develop a novel instrumentation system to extract geometric fire fronts characteristics

    Fire Saf. J.

    (2011)
  • N.R. Adam et al.

    Spatial computing and social media in the context of disaster management

    IEEE Intell. Syst.

    (2012)
  • ChiR. et al.

    Real-time multi-feature based fire flame detection in video

    IET Image Process.

    (2016)
  • LoS.-W. et al.

    Cyber surveillance for flood disasters

    Sensors

    (2015)
  • ChenT.-H. et al.

    An early fire-detection method based on image processing

  • D. Guha-Sapir, F. Vos, R. Below, S. Penserre, Annual disaster statistical review 2015: the numbers and trends, 2015....
  • T. Toulouse et al.

    Benchmarking of wildland fire colour segmentation algorithms

    IET Image Process.

    (2015)
  • QiuT. et al.

    An autoadaptive edge-detection algorithm for flame and fire image processing

    IEEE Trans. Instrum. Meas.

    (2012)
  • LiuC.-B. et al.

    Vision based fire detection

  • KoB.C. et al.

    Modeling and formalization of fuzzy finite automata for detection of irregular fire flames

    IEEE Trans. Circuits Syst. Video Technol.

    (2011)
  • P. Foggia et al.

    Real-time fire detection for video-surveillance applications using a combination of experts based on color, shape, and motion

    IEEE Trans. Circuits Syst. Video Technol.

    (2015)
  • M. Mueller et al.

    Optical flow estimation for flame detection in videos

    IEEE Trans. Image Process.

    (2013)
  • Cited by (338)

    View all citing articles on Scopus

    Khan Muhammad (S’16) received the bachelors degree in computer science from Islamia College Peshawar, Pakistan with research in information security. He is currently pursuing M.S. leading to Ph.D. degree in digitals contents from College of Software and Convergence Technology, Sejong University, Seoul, Republic of Korea. He has been a Research Associate with the Intelligent Media Laboratory (IM Lab) since 2015. His research interests include image and video processing, wireless networks, information security, image and video steganography, video summarization, diagnostic hysteroscopy, wireless capsule endoscopy, deep learning, computer vision, and CCTV video analysis. He has authored over 40 papers in peer-reviewed international journals and conferences, such as IEEE Transactions on Industrial Informatics, Neurocomputing, Future Generation Computer Systems, the IEEE Access, the Journal of Medical Systems, Biomedical Signal Processing and Control, Multimedia Tools and Applications, Pervasive and Mobile Computing, SpringerPlus, the KSII Transactions on Internet and Information Systems, MITA 2015, PlatCon 2016, FIT 2016, and ICNGC 2017.

    Jamil Ahmad received his BCS degree in Computer Science from the University of Peshawar, Pakistan in 2008 with distinction. He received his Masters degree in 2014 with specialization in Image Processing from Islamia College, Peshawar, Pakistan. He is also a regular faculty member in the Department of Computer Science, Islamia College Peshawar. Currently, he is pursuing Ph.D. degree in Sejong University, Seoul, Korea. His research interests include deep learning, medical image analysis, content-based multimedia retrieval, and computer vision. He has published several journal articles in these areas in reputed journals including Journal of Real-Time Image Processing, Multimedia Tools and Applications, Journal of Visual Communication and Image Representation, PLOS One, Journal of Medical Systems, Computers and Electrical Engineering, SpringerPlus, Journal of Sensors, and KSII Transactions on Internet and Information Systems. He is also an active reviewer for IET Image Processing, Engineering Applications of Artificial Intelligence, KSII Transactions on Internet and Information Systems, Multimedia Tools and Applications, IEEE Transactions on Image Processing, and IEEE Transactions on Cybernetics. He is a student member of the IEEE.

    Sung Wook Baik received the B.S degree in computer science from Seoul National University, Seoul, Korea, in 1987, the M.S. degree in computer science from Northern Illinois University, Dekalb, in 1992, and the Ph.D. degree in information technology engineering from George Mason University, Fairfax, VA, in 1999. He worked at Datamat Systems Research Inc. as a senior scientist of the Intelligent Systems Group from 1997 to 2002. In 2002, he joined the faculty of the College of Electronics and Information Engineering, Sejong University, Seoul, Korea, where he is currently a Full Professor and Dean of Digital Contents. He is also the head of Intelligent Media Laboratory (IM Lab) at Sejong University. His research interests include computer vision, multimedia, pattern recognition, machine learning, data mining, virtual reality, and computer games. He is a professional member of the IEEE.

    View full text