Andy Rosales Elias
Chandra Krintz
ABSTRACT
We investigate the design and implementation of Where's The Bear (WTB), an end-to-end, distributed, IoT system for wildlife monitoring. WTB implements a multi-tier (cloud, edge, sensing) system that integrates recent advances in machine learning based image processing to automatically classify animals in images from remote, motion-triggered camera traps. We use non-local, resource-rich, public/private cloud systems to train the machine learning models, and "in-the-field," resource-constrained edge systems to perform classification near the IoT sensing devices (cameras).
We deploy WTB at the UCSB Sedgwick Reserve, a 6000 acre site for environmental research and use it to aggregate, manage, and analyze over 1.12M images. WTB integrates Google TensorFlow and OpenCV applications to perform automatic image classification and tagging. To avoid transferring large numbers of training images for TensorFlow over the low-bandwidth network linking Sedgwick to public clouds, we devise a technique that uses stock Google Images to construct a synthetic training set using only a small number of empty, background images from Sedgwick. Our system is able to accurately identify bears, deer, coyotes, and emtpy images and significantly reduces the time and bandwidth requirements for image transfer, as well as end-user analysis time, since WTB automatically filters the images on-site.
- M. Abadi, P. Barham, J. Chen, Zhifeng Chen, A. Davis, J. Dean, M. Devin, Sanjay Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, and X. Zheng. 2016. TensorFlow: A System for Large-Scale Machine Learning. In Symposium on Operating Systems Design and Implementation (OSDI).Google Scholar
- A. Alemi. 2016. Improving Inception and Image Classification in TensorFlow. https://research.googleblog.com/2016/08/improving-inception-and-image.html. (2016). [Online; accessed 22-Aug-2016].Google Scholar
- David Anthony, William P Bennett, Mehmet C Vuran, Matthew B Dwyer, Sebastian Elbaum, Anne Lacy, Mike Engels, and Walter Wehtje. 2012. Sensing through the continent: towards monitoring migratory birds using cellular sensor networks. In Proceedings of the 11th international conference on Information Processing in Sensor Networks. ACM, 329--340. Google ScholarDigital Library
- Aristotle Cloud Federation 2017. Aristotle Cloud Federation. (2017). https://federatedcloud.org [Online; accessed 22-Aug-2016].Google Scholar
- AWS Edge Locations 2016. AWS Edge Locations. (2016). https://aws.amazon.com/cloudfront/ [Online; accessed 22-Aug-2016].Google Scholar
- G. Bradski. 2000. OpenCV. Dr. Dobb's Journal of Software Tools (2000).Google Scholar
- Tilo Burghardt, Janko Calic, and Barry T Thomas. 2004. Tracking Animals in Wildlife Videos Using Face Detection.. In EWIMT.Google Scholar
- Cisco. 2016. Fog Data Services - Cisco. (2016). http://www.cisco.com/c/en/us/products/cloud-systems-management/fog-data-services/index.html [Online; accessed 22-Aug-2016].Google Scholar
- Cloudlet-based Mobile Computing 2016. Cloudlet-based Mobile Computing. (2016). http://elijah.cs.cmu.edu/ [Online; accessed 22-Aug-2016].Google Scholar
- Content Distribution Network 2016. Content Distribution Network. https://en.wikipedia.org/wiki/Content_delivery_network. (2016). [Online; accessed 22-Aug-2016].Google Scholar
- J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09.Google Scholar
- Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. 2014. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition.. In ICML. 647--655.Google Scholar
- ExifTool 2016. ExifTool. http://www.sno.phy.queensu.ca/~phil/exiftool/. (2016). [Online; accessed 22-Aug-2016].Google Scholar
- D. Floyer. 2016. The Vital Role of Edge Computing in the Internet of Things. (2016). http://wikibon.com/the-vital-role-of-edge-computing-in-the-internet-of-things/ [Online; accessed 22-Aug-2016].Google Scholar
- Google. 2016. Google Images API. http://images.google.com. (2016). [Online; accessed 22-Aug-2016].Google Scholar
- Google 2016. TensorFlow Image Classification for Image Classification. https://www.tensorflow.org/versions/r0.10/tutorials/image_recognition/index.html. (2016). [Online; accessed 22-Aug-2016].Google Scholar
- Google Cloud CDN 2016. Google Cloud CDN. https://cloud.google.com/cdn/docs/. (2016). [Online; accessed 22-Aug-2016].Google Scholar
- Jyh-How Huang, Ying-Yu Chen, Yu-Te Huang, Po-Yen Lin, Yi-Chao Chen, Yi-Fu Lin, Shih-Ching Yen, Polly Huang, and Ling-Jyh Chen. 2010. Rapid prototyping for wildlife and ecological monitoring. IEEE Systems Journal 4, 2 (2010), 198--209. Google ScholarCross Ref
- Jane Hunter, Charles Brooking, Wilfred Brimblecombe, Ross G Dwyer, Hamish A Campbell, Matthew E Watts, and Craig E Franklin. 2013. OzTrack--E-Infrastructure to Support the Management, Analysis and Sharing of Animal Tracking Data. In eScience (eScience), 2013 IEEE 9th International Conference on. IEEE, 140--147.Google Scholar
- Intel NUC 2016. Intel NUC. (2016). http://www.intel.com/content/www/us/en/nuc/overview.html [Online; accessed 14-Feb-2016].Google Scholar
- Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. arXiv preprint arXiv:1408.5093 (2014).Google Scholar
- Philo Juang, Hidekazu Oki, Yong Wang, Margaret Martonosi, Li Shiuan Peh, and Daniel Rubenstein. 2002. Energy-efficient computing for wildlife tracking: Design tradeoffs and early experiences with ZebraNet. ACM Sigplan Notices 37, 10 (2002), 96--107. Google ScholarDigital Library
- K-Nearest Neighbor 2017. K-Nearest Neighbor Algorithm. https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm. (2017). [Online; accessed 22-Feb-2017].Google Scholar
- C. Krintz, R. Wolski, N. Golubovic, B. Lampel, V. Kulkarni, B. Sethuramasamyraja, B. Roberts, and B. Liu. 2016. SmartFarm: Improving Agriculture Sustainability Using Modern Information Technology. In KDD Workshop on Data Science for Food, Energy, and Water.Google Scholar
- Alan Mainwaring, David Culler, Joseph Polastre, Robert Szewczyk, and John Anderson. 2002. Wireless sensor networks for habitat monitoring. In Proceedings of the 1st ACM international workshop on Wireless sensor networks and applications. ACM, 88--97. Google ScholarDigital Library
- Daniel Nurmi, Richard Wolski, Chris Grzegorczyk, Graziano Obertelli, Sunil Soman, Lamia Youseff, and Dmitrii Zagorodnov. 2009. The Eucalyptus Open-Source Cloud Computing System. In Cluster Computing and the Grid, 2009. CCGRID'09. 9th IEEE/ACM International Symposium on. IEEE, 124--131. Google ScholarDigital Library
- OpenCV. 2016. OpenCV Optical Character Recognition Using K-Nearest Neighbors. http://docs.opencv.org/3.1.0/dd/de1/classcv_1_1ml_1_1KNearest.html. (2016). [Online; accessed 22-Aug-2016].Google Scholar
- PyExifInfo 2016. PyExifInfo. https://github.com/guinslym/pyexifinfo. (2016). [Online; accessed 22-Aug-2016].Google Scholar
- K. Sato. 2016. How a Japanese cucumber farmer is using deep learning and TensorFlow. https://cloud.google.com/blog/big-data/2016/08/how-a-japanese-cucumber-farmer-is-using-deep-learning-and-tensorflow. (2016). [Online; accessed 22-Aug-2016].Google Scholar
- Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scot Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--9. Google ScholarCross Ref
- Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. 2015. Rethinking the inception architecture for computer vision. arXiv preprint arXiv:1512.00567 (2015).Google Scholar
- Eranda Tennakoon, Charith Madusanka, Kasun De Zoysa, Chamath Keppitiyagama, Venkat Iyer, Kasun Hewage, and Thiemo Voigt. 2015. Sensor-based breakage detection for electric fences. In Sensors Applications Symposium (SAS), 2015 IEEE. IEEE, 1--4. Google ScholarCross Ref
- R. Tillet, C. Onyango, and J. Marchant. 1997. Using model-based image processing to track animal movements. Computers and electronics in agriculture 17, 2 (1997), 249--261. Google ScholarCross Ref
- L. Torrey and J. Shavlik. 2009. Transfer Learning. IGI Global, E. Soria, J. Martin, R. Magdalena, M. Martinez and A. Serrano, eds. http://ftp.cs.wisc.edu/machine-learning/shavlik-group/torrey.handbook09.pdf.Google Scholar
- UCSB Sedgwick Reserve 2013. (2013). http://sedgwick.nrs.ucsb.edu [Online;accessed 14-Feb-2016].Google Scholar
- Tim Verbelen, Pieter Simoens, Filip De Turck, and Bart Dhoedt. 2012. Cloudlets: bringing the cloud to the mobile user. In ACM workshop on Mobile cloud computing and services. ACM. Google ScholarDigital Library
- Wikipedia. 2016. Deep Learning. https://en.wikipedia.org/wiki/Deep_learning.(2016). [Online; accessed 22-Aug-2016].Google Scholar
- Xiaoyuan Yu, Jiangping Wang, Roland Kays, Patrick A Jansen, Tianjiang Wang, and Thomas Huang. 2013. Automated identification of animal species in camera trap images. EURASIP Journal on Image and Video Processing 2013, 1 (2013), 1.Google ScholarCross Ref
- Matthias Zeppelzauer. 2013. Automated detection of elephants in wildlife video. EURASIP journal on image and video processing 2013, 1 (2013), 1.Google Scholar
Recommendations
Machine learning for cloud, fog, edge and serverless computing environments: comparisons, performance evaluation benchmark and future directions
The compute-intensive and latency-sensitive Internet of Things (IoT) applications need to utilise the services from various computing paradigms, but they are facing many challenges such as large value of latency, energy and network bandwidth. To analyse ...
All one needs to know about fog computing and related edge computing paradigms: A complete survey
AbstractWith the Internet of Things (IoT) becoming part of our daily life and our environment, we expect rapid growth in the number of connected devices. IoT is expected to connect billions of devices and humans to bring promising advantages ...
A Pattern for Fog Computing
VikingPLoP '16: Proceedings of the 10th Travelling Conference on Pattern Languages of ProgramsFog Computing is a new variety of the cloud computing paradigm that brings virtualized cloud services to the edge of the network to control the devices in the IoT. We present a pattern for fog computing which describes its architecture, including its ...
Comments