Skip to main content
SpringerLink
Log in

Analysis of Competitor Intelligence in the Era of Big Data: An Integrated System Using Text Summarization Based on Global Optimization

  • Research Paper
  • Published:
Business & Information Systems Engineering Aims and scope Submit manuscript

Abstract

Automatic text summarization can be applied to extract summaries from competitor intelligence (CI) corpora that organizations create by gathering textual data from the Internet. Such a representation of CI text is easier for managers to interpret and use for making decisions. This research investigates design of an integrated system for CI analysis which comprises clustering and automatic text summarization and evaluates quality of extractive summaries generated automatically by various text-summarization techniques based on global optimization. This research is conducted using experimentation and empirical analysis of results. A survey of practicing managers is also carried out to understand the effectiveness of automatically generated summaries from CI perspective. Firstly, it shows that global optimization-based techniques generate good quality extractive summaries for CI analysis from topical clusters created by the clustering step of the integrated system. Secondly, it shows the usefulness of the generated summaries by having them evaluated by practicing managers from CI perspective. Finally, the implication of this research from the point of view of theory and practice is discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Ackermann M, Soares C, Guidemann B (2006) Practical data mining: applications, experiences and challenges. In: SAS & PKDD workshop, Berlin

  • Alguliev RM, Aliguliyev RM, Mehdiyev CA (2011a) Sentence selection for generic document summarization using an adaptive differential evolution algorithm. Swarm Evol Comput 1(4):213–222

    Article  Google Scholar 

  • Alguliev RM, Aliguliyev RM, Hajirahimova MS, Mehdiyev CA (2011b) MCMR: maximum coverage and minimum redundant text summarization model. Expert Syst Appl 38(12):14514–14522

    Article  Google Scholar 

  • Alguliev RM, Aliguliyev RM, Isazade NR (2012) DESAMC + DocSum: differential evolution with self-adaptive mutation and crossover parameters for multi-document summarization. Knowl Based Syst 36:21–38

    Article  Google Scholar 

  • Alguliev RM, Aliguliyev RM, Isazade NR (2013) CDDS: constraint-driven document summarization models. Expert Syst Appl 40(2):458–465

    Article  Google Scholar 

  • Alguliev RM, Aliguliyev RM, Isazade NR (2014) Multiple documents summarization based on evolutionary optimization algorithm. Expert Syst Appl 40(5):1675–1689

    Article  Google Scholar 

  • Allan J, Carbonell J, Doddington G, Yamron J, Yang Y (2000) Topic detection and tracking pilot study final report. DARPA, Arlington

    Google Scholar 

  • Arnott D, Pervan G (2008) Eight key issues for the decision support systems discipline. Decis Support Syst 44(3):657–672

    Article  Google Scholar 

  • Baralis E, Cagliero L, Jabeen S, Fiori A, Shah S (2013) Multi-document summarization based on Yago ontology. Expert Syst Appl 40(17):6976–6984

    Article  Google Scholar 

  • Barzilay R, Elhadad M (1997) Using lexical chains for text summarization. In: Proceedings ISTS, pp 10–17

  • Bellegarda J, Butzberger JW, Chow Y, Coccaro NB, Naik D (1996) A novel word clustering algorithm based on latent semantic analysis. In: ICASSP, vol 1. IEEE, pp 172–175

  • Bissantz N, Hagedorn J (2009) Data mining. Bus Inf Syst Eng 1(1):118–122

    Article  Google Scholar 

  • Bitzer P, Söllner M, Leimeister JM (2015) Design principles for high-performance blended learning services delivery. Bus Inf Syst Eng 58(2):135–149

    Article  Google Scholar 

  • Bland JM, Altman DG (1995) Multiple significance tests: the Bonferroni method. BMJ 310:170

    Article  Google Scholar 

  • Blei DM (2003) Latent Dirichlet allocation. J Mach Learn Res 3:993–1022

    Google Scholar 

  • Browne GJ, Cheung C, Heinzl A, Riedl R (2017) Human information behavior. Bus Inf Syst Eng 59(1):1–2

    Article  Google Scholar 

  • Carbonell J, Goldstein J (1998) The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: Proceedings of SIGIR, pp 335–336

  • Carullo MB, Binaghi E, Gallo I (2009) An online document clustering technique for short web contents. Pattern Recognit Lett 30(10):870–876

    Article  Google Scholar 

  • Chakraborti S (2015) Multi-document text summarization for competitor intelligence: a methodology based on topic identification and artificial bee colony optimization. SAC, ACM Digital Library, Salamanca, pp 1110–1111

    Google Scholar 

  • Chakraborti S, Dey S (2014) Multi-document text summarization for competitor intelligence: a methodology. In: ISCBI-2014. IEEE Computer Society, New Delhi, pp 97–100

  • Chakraborti S, Dey S (2015) Product news summarization for competitor intelligence using topic identification and artificial bee colony optimization. ACM RACS, ACM Digital Library, Prague, pp 1–6

    Google Scholar 

  • Chakraborti S, Dey S (2016) Multi-level k-means text clustering technique for topic identification for competitor intelligence. In: Proceedings of RCIS. IEEE, Grenoble, pp 1–10

  • Codina-Filbà J, Bouayad-Agha N, Burga A, Casamayor G, Wanner L (2017) Using genre-specific features for patent summaries. Inf Process Manag 53(1):151–174

    Article  Google Scholar 

  • Cohen W, Levinthal D (1990) Absorptive capacity: a new perspective on learning and innovation. Adm Sci Q 35(1):128–152

    Article  Google Scholar 

  • Deb K (2000) An efficient constraint handling method for genetic algorithms. Comput Method Appl Mech Eng 186(2–4):311–338

    Article  Google Scholar 

  • Deerwester S (1990) Indexing by latent semantic analysis. JOASIS 41(6):391–407

    Article  Google Scholar 

  • DeJong GF (1978) Fast skimming of news stories: the FRUMP system. PhD thesis, Yale University

  • Donohue DP, Murphy PM (2016) Supporting competitive intelligence at DuPont by controlling information overload and cutting through the noise. J Inf Knowl Manag 15(1):1650004. https://doi.org/10.1142/S0219649216500040

    Article  Google Scholar 

  • Dumais S, Chen H (2000) Hierarchical classification of web content. In: SIGIR Conference on research and development in information retrieval. ACM, pp 256–263

  • Erkan G, Radev DR (2004) LexRank: graph-based lexical centrality as salience in text summarization. J Artif Intell Res 22:457–479

    Article  Google Scholar 

  • Fischer C, Winter R, Wortmann F (2010) Design theory. Bus Inf Syst Eng 2(6):387–390

    Article  Google Scholar 

  • Flath C, Nicolay D, Conte T, Dinther C, Filipova-Neumann L (2012) Cluster analysis of smart metering data. Bus Inf Syst Eng 4(1):31–39

    Article  Google Scholar 

  • Gilad B (2015) Companies collect competitive intelligence but don’t use it. Harv Bus Rev

  • Gilad B, Fuld L (2016) Only half of companies actually use the competitive intelligence they collect. Harv Bus Rev

  • Groom JR, David FR (2001) Competitive intelligence activity among small firms. SAM Adv Manag J 66(1):12–20

    Google Scholar 

  • Halko J, Martinsson P, Tropp J (2010) Finding structure with randomness: probabilistic algorithm for constructing approximate matrix decomposition. SIAM Rev 53(2):217–288

    Article  Google Scholar 

  • Heinrich P, Schwabe G (2017) Facilitating informed decision-making in financial service encounters. Bus Inf Syst Eng 60(4):317–329

    Article  Google Scholar 

  • Hevner AR (2007) A three cycle view of design science research. Scand J Inf Syst 19(2):87–92

    Google Scholar 

  • Hevner AR, March ST, Park J, Ram S (2004) Design science in information systems research. MIS Q 28(1):75–105

    Article  Google Scholar 

  • Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor

    Google Scholar 

  • Hornik K, Kober M, Feinerer I, Buchta C (2012) Spherical k-means clustering. J Stat Softw 50(10):1–22

    Article  Google Scholar 

  • Hu Y, Chen Y, Chou H (2017) Opinion mining from online hotel reviews – a text summarization approach. Inf Process Manag 53(2):436–449

    Article  Google Scholar 

  • Jain AM, Murty MN, Flynn PJ (1999) Data clustering: a review. ACM Comput Surv 31(3):264–323

    Article  Google Scholar 

  • Jishma Mohan M, Sunitha C, Ganesh A, Jaya A (2016) A study on ontology based abstractive summarization. Proc Comput Sci 87:32–37

    Article  Google Scholar 

  • Karaboga D, Akay B (2011) A modified artificial bee colony (ABC) algorithm for constrained optimization problems. Appl Soft Comput 11(3):3021–3031

    Article  Google Scholar 

  • Karaboga D, Basturk B (2007) Artificial bee colony (ABC) optimization algorithm for solving constrained optimization problems. In: Foundations of fuzzy logic and soft computing, IFSA, pp 789–798

  • Kotonya G, Sommerville I (1998) Requirements engineering processes and techniques. Wiley, Hoboken

    Google Scholar 

  • Kowalczyk M (2014) Big data and information processing in organizational decision processes. Bus Inf Syst Eng 6(5):267–278

    Article  Google Scholar 

  • Lin C (2004) ROUGE: a package for automatic evaluation of summaries. In: Proceedings of the workshop in text summarization. ACL, pp 74–81

  • Lloret E, Romá-Ferri MT, Palomar M (2013) COMPENDIUM: a text summarization system for generating abstracts of research papers. Data Knowl Eng 88:164–175

    Article  Google Scholar 

  • Lockshin LS, Spawton AL, Macintosh G (1997) Using product, brand and purchasing involvement for retail segmentation. J Retail Consum Serv 4(3):171–183

    Article  Google Scholar 

  • Luhn HP (1958) The automatic creation of literature abstracts. IBM J Res Dev 2(2):159–165

    Article  Google Scholar 

  • MacQueen JB (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of 5th Berkeley symposium on mathematical statistics and probability. University of California Press, pp 287–297

  • Mani I, Maybury M (1999) Advances in automatic text summarization. MIT Press, Cambridge

    Google Scholar 

  • Marcu D (1998) Improving summarization through rhetorical parsing tuning. In: Proceedings of the sixth workshop on very large corpora, pp 206–215

  • Mendoza M, Bonilla S, Noguera C, Cobos C, León E (2014) Extractive single-document summarization based on genetic operators and guided local search. Expert Syst Appl 41(9):4158–4169

    Article  Google Scholar 

  • Miller GA (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev 63(2):81–97

    Article  Google Scholar 

  • Moens M (2007) Summarizing court decisions. Inf Process Manag 43(6):1748–1764

    Article  Google Scholar 

  • Oberle D, Bhatti N, Brockmans S, Niemann M, Janiesch C (2009) Countering service information challenges in the internet of services. Bus Inf Syst Eng 1(5):370–390

    Article  Google Scholar 

  • Okike C, Fernandes KJ (2012) Impact of information use architecture on load and usability. Inf Process Manag 48(5):995–1016

    Article  Google Scholar 

  • O’Reilly CA (1983) The use of information in organizational decision making: a model and some propositions. Res Organ Behav 5:103–140

    Google Scholar 

  • Orenga-Roglá S, Chalmeta R (2017) Methodology for the implementation of knowledge management systems. Bus Inf Syst Eng 2:1–19

    Google Scholar 

  • Rackoff N, Wiseman C, Ulrich WA (1985) Information systems for competitive advantage: implementation of a planning process. MIS Q 9(4):285–294

    Article  Google Scholar 

  • Radev DR (2004) MEAD – a platform for multidocument multilingual text summarization. In: Proceedings of LREC, Lisbon

  • Radev DR, Jing H, Budzikowska M (2000) Centroid-based summarization of multiple documents: sentence extraction, utility-based evaluation, and user studies. In: NAACL-ANLP 2000 workshop on automatic summarization, pp 21–30

  • Radev DR, Hovy E, McKeown K (2002) Introduction to the special issue on summarization. Comput Linguist 28(4):399–408

    Article  Google Scholar 

  • Ravi K, Ravi V (2015) A survey on opinion mining and sentiment analysis: tasks, approaches and applications. Knowl Based Syst 89:14–46

    Article  Google Scholar 

  • Reeve LH, Han H, Brooks AD (2007) The use of domain-specific concepts in biomedical text summarization. Inf Process Manag 43(6):1765–1776

    Article  Google Scholar 

  • Reuters (1987) Retrieved from Reuters-21578 text categorization data set. Retrieved from Reuters-21578: http://kdd.ics.uci.edu/databases/reuters21578/reuters21578.html. Accessed 7 Jan 2017

  • Sankarasubramaniam Y, Ramanathan K, Ghosh S (2014) Text summarization using Wikipedia. Inf Process Manag 50(3):443–461

    Article  Google Scholar 

  • Simon HA (1996) The sciences of the artificial. MIT Press, Cambridge

    Google Scholar 

  • Simon B (2010) A discussion on competency management systems from a design theory perspective. Bus Inf Syst Eng 2(6):337–346

    Article  Google Scholar 

  • Storn R, Price K (1996) Differential evolution – a simple and efficient adaptive scheme for global optimization over continuous spaces. University of California, Berkeley

    Google Scholar 

  • Stroh F, Winter R, Wortmann F (2011) Method support of information requirements analysis for analytical information systems. Bus Inf Syst Eng 3(1):33–43

    Article  Google Scholar 

  • Tallon PP, Ramirez RV, Short JE (2013–2014) The information artifact in IT governance: toward a theory of information governance. J Manag Inf Syst 30(3):141–147

  • Tseng YH, Lin CJ, Lin Y (2007) Text mining techniques for patent analysis. Inf Process Manag 43(5):1216–1247

    Article  Google Scholar 

  • Wang YD, Forgionne G (2006) A decision-theoretic approach to the evaluation of information retrieval systems. Inf Process Manag 42(4):863–874

    Article  Google Scholar 

  • Wang W, Li S, Li J, Li W, Wei F (2013) Exploring hypergraph-based semi-supervised ranking for query-oriented summarization. Inf Sci 237:271–286

    Article  Google Scholar 

  • Wang N, Sun S, OuYang D (2016) Business process modeling abstraction based on semi-supervised clustering analysis. Bus Inf Syst Eng 1–18

  • Wilson TD (1981) On user studies and information needs. J Doc 37(1):3–15

    Article  Google Scholar 

  • Wright S, Pickton DW, Callow J (2002) Competitive intelligence in UK Firms: a typology. Mark Intell Plan 20(6):349–360

    Article  Google Scholar 

  • Xu M, Ong V, Duan Y, Mathews B (2011) Intelligent agent systems for executive information scanning, filtering and interpretation: perceptions and challenges. Inf Process Manag 47(2):186–201

    Article  Google Scholar 

  • Zaby C, Wilde KD (2017) Intelligent business processes in CRM. Bus Inf Syst Eng 1–16

  • Zhan J, Loh HT, Liu Y (2009) Gather customer concerns from online product reviews – a text summarization approach. Expert Syst Appl 36(2):2107–2115

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Management, S. P. Jain Institute of Management and Research (SPJIMR), Mumbai, India

    Swapnajit Chakraborti

  2. Department of Information Systems, Indian Institute of Management Indore, Indore, M.P., India

    Shubhamoy Dey

Authors
  1. Swapnajit Chakraborti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shubhamoy Dey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Swapnajit Chakraborti.

Additional information

Accepted after one revision by Natalia Kliewer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chakraborti, S., Dey, S. Analysis of Competitor Intelligence in the Era of Big Data: An Integrated System Using Text Summarization Based on Global Optimization. Bus Inf Syst Eng 61, 345–355 (2019). https://doi.org/10.1007/s12599-018-0562-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12599-018-0562-0

Keywords

Search

Navigation