Skip to main content
SpringerLink
Log in

WIP - SKOD: A Framework for Situational Knowledge on Demand

  • Conference paper
  • First Online:
Book cover Heterogeneous Data Management, Polystores, and Analytics for Healthcare (DMAH 2019, Poly 2019)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 11721))

Abstract

Extracting relevant patterns from heterogeneous data streams poses significant computational and analytical challenges. Further, identifying such patterns and pushing analogous content to interested parties according to mission needs in real-time is a difficult problem. This paper presents the design of SKOD, a novel Situational Knowledge Query Engine that continuously builds a multi-modal relational knowledge base using SQL queries; SKOD pushes dynamic content to relevant users through triggers based on modeling of users’ interests. SKOD is a scalable, real-time, on-demand situational knowledge extraction and dissemination framework that processes streams of multi-modal data utilizing publish/subscribe stream engines. The initial prototype of SKOD uses deep neural networks and natural language processing techniques to extract and model relevant objects from video streams and topics, entities and events from unstructured text resources such as Twitter and news articles. Through its extensible architecture, SKOD aims to provide a high-performance, generic framework for situational knowledge on demand, supporting effective information retrieval for evolving missions.

This research is supported by Northrop Grumman Mission Systems’ University Research Program.

S. Palacios and K.M.A. Solaiman contributed equally and are considered to be co-first authors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://docs.tweepy.org/en/v3.5.0/api.html.

  2. 2.

    https://reactjs.org/.

  3. 3.

    https://github.com/cerebral/cerebral.

  4. 4.

    https://leafletjs.com/.

  5. 5.

    The Web application was developed utilizing ideas from the OATS Center at Purdue. In particular, the OADA framework https://github.com/OADA.

  6. 6.

    https://pouchdb.com/.

  7. 7.

    https://github.com/OADA/oada-cache.

  8. 8.

    https://webrtc.org/.

References

  1. Abu-El-Haija, S., et al.: YouTube-8M: a large-scale video classification benchmark. CoRR abs/1609.08675 (2016). http://arxiv.org/abs/1609.08675

  2. Adjali, O., Hina, M.D., Dourlens, S., Ramdane-Cherif, A.: Multimodal fusion, fission and virtual reality simulation for an ambient robotic intelligence. In: ANT/SEIT. Procedia Computer Science, vol. 52, pp. 218–225. Elsevier (2015)

    Google Scholar 

  3. Bienvenu, M., Bourgaux, C., Goasdoué, F.: Query-driven repairing of inconsistent DL-Lite knowledge bases. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, 9–15 July 2016, pp. 957–964 (2016). http://www.ijcai.org/Abstract/16/140

  4. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003). http://dl.acm.org/citation.cfm?id=944919.944937

    MATH  Google Scholar 

  5. Chen, Y., Wang, D.Z.: Knowledge expansion over probabilistic knowledge bases. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, SIGMOD 2014, pp. 649–660. ACM, New York (2014). https://doi.org/10.1145/2588555.2610516. http://doi.acm.org/10.1145/2588555.2610516

  6. Cugola, G., Margara, A.: Processing flows of information: from data stream to complex event processing. ACM Comput. Surv. 44(3), 15:1–15:62 (2012). https://doi.org/10.1145/2187671.2187677

    Article  Google Scholar 

  7. Dong, X., et al.: Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In: KDD, pp. 601–610. ACM (2014)

    Google Scholar 

  8. Foresti, G.L., Farinosi, M., Vernier, M.: Situational awareness in smart environments: socio-mobile and sensor data fusion for emergency response to disasters. J. Ambient Intell. Humanized Comput. 6(2), 239–257 (2015)

    Article  Google Scholar 

  9. Itria, M.L., Daidone, A., Ceccarelli, A.: A complex event processing approach for crisis-management systems. CoRR abs/1404.7551 (2014)

    Google Scholar 

  10. Kang, D., Bailis, P., Zaharia, M.: BlazeIt: fast exploratory video queries using neural networks. CoRR abs/1805.01046 (2018)

    Google Scholar 

  11. Krishna, R., et al.: Visual genome: connecting language and vision using crowdsourced dense image annotations. CoRR abs/1602.07332 (2016). http://arxiv.org/abs/1602.07332

  12. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  13. Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., McClosky, D.: The Stanford CoreNLP natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60. Association for Computational Linguistics, Baltimore, June 2014. https://doi.org/10.3115/v1/P14-5010. https://www.aclweb.org/anthology/P14-5010

  14. Meditskos, G., Vrochidis, S., Kompatsiaris, I.: Description logics and rules for multimodal situational awareness in healthcare. In: Amsaleg, L., Guðmundsson, G., Gurrin, C., Jónsson, B., Satoh, S. (eds.) MMM 2017, Part I. LNCS, vol. 10132, pp. 714–725. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-51811-4_58

    Chapter  Google Scholar 

  15. Nguyen, D.B., Abujabal, A., Tran, N.K., Theobald, M., Weikum, G.: Query-driven on-the-fly knowledge base construction. Proc. VLDB Endow. 11(1), 66–79 (2017). https://doi.org/10.14778/3151113.3151119

    Article  Google Scholar 

  16. Palacios, S., Santos, V., Barsallo, E., Bhargava, B.: MioStream: a peer-to-peer distributed live media streaming on the edge. Multimedia Tools Appl. (2019). https://doi.org/10.1007/s11042-018-6940-2

    Article  Google Scholar 

  17. Poria, S., Cambria, E., Howard, N., Huang, G.B., Hussain, A.: Fusing audio, visual and textual clues for sentiment analysis from multimodal content. Neurocomputing 174(PA), 50–59 (2016). https://doi.org/10.1016/j.neucom.2015.01.095. http://dx.doi.org/10.1016/j.neucom.2015.01.095

    Article  Google Scholar 

  18. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779–788 (2016)

    Google Scholar 

  19. Rodríguez, M.E., Goldberg, S., Wang, D.Z.: SigmaKB: multiple probabilistic knowledge base fusion. PVLDB 9(13), 1577–1580 (2016)

    Google Scholar 

  20. Wu, Q., Wang, P., Shen, C., Dick, A.R., van den Hengel, A.: Ask me anything: free-form visual question answering based on knowledge from external sources. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, 27–30 June 2016, pp. 4622–4630 (2016). https://doi.org/10.1109/CVPR.2016.500

  21. Zhu, Y., Lim, J.J., Fei-Fei, L.: Knowledge acquisition for visual question answering via iterative querying. In: CVPR, pp. 6146–6155. IEEE Computer Society (2017)

    Google Scholar 

Download references

Funding

Distribution Statement A: Approved for Public Release; Distribution is Unlimited; #19-1107; Dated 07/18/19.

Author information

Authors and Affiliations

  1. Purdue University, West Lafayette, IN, 47906, USA

    Servio Palacios, K. M. A. Solaiman, Alina Nesen & Bharat Bhargava

  2. METU, Ankara, Turkey

    Pelin Angin

  3. MIT CSAIL, Cambridge, MA, 02139, USA

    Zachary Collins, Aaron Sipser & Michael Stonebraker

  4. Northrop Grumman Corporation, Falls Church, USA

    James Macdonald

Authors
  1. Servio Palacios
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. K. M. A. Solaiman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pelin Angin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alina Nesen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bharat Bhargava
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zachary Collins
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Aaron Sipser
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Michael Stonebraker
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. James Macdonald
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Servio Palacios or K. M. A. Solaiman .

Editor information

Editors and Affiliations

  1. Massachusetts Institute of Technology, Lexington, MA, USA

    Vijay Gadepally

  2. Intel Corporation, Portland, OR, USA

    Timothy Mattson

  3. Massachusetts Institute of Technology, Cambridge, MA, USA

    Michael Stonebraker

  4. Stony Brook University, Stony Brook, NY, USA

    Fusheng Wang

  5. University of Washington, Seattle, WA, USA

    Gang Luo

  6. Google, Mountain View, CA, USA

    Yanhui Laing

  7. Lucerne University of Applied Sciences, Rotkreuz, Switzerland

    Alevtina Dubovitskaya

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Palacios, S. et al. (2019). WIP - SKOD: A Framework for Situational Knowledge on Demand. In: Gadepally, V., et al. Heterogeneous Data Management, Polystores, and Analytics for Healthcare. DMAH Poly 2019 2019. Lecture Notes in Computer Science(), vol 11721. Springer, Cham. https://doi.org/10.1007/978-3-030-33752-0_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-33752-0_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-33751-3

  • Online ISBN: 978-3-030-33752-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation