Skip to main content
SpringerLink
Log in

An integer linear programming model of reviewer assignment with research interest considerations

  • S.I.: RealCaseOR
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

In the regular work process of peer review, editors have to read and understand the entire set of submissions to choose appropriate reviewers. However, due to a large number of submissions, to select reviewers manually becomes error-prone and time-consuming. In this research, a framework that considers different indispensable aspects such as topical relevance, topical authority and research interest is presented and, an integer linear programming problem is formulated with practical considerations to recommend reviewers for a group of submissions. Specifically, the topical relevance and the topical authority are utilized to recommend relevant and accredited candidates in submission-related topics, while the research interest is to exam the willingness of candidates to review a submission. To evaluate the effectiveness of the proposed approach, categories of comparative experiments were conducted on two large scholarly datasets. Experimental results demonstrate that, compared with benchmark approaches, the proposed approach is capable to capture the research interest of reviewer candidates without a significant loss in different evaluation metrics. Our work can be helpful for editors to invite matching experts in peer review and highlight the necessity to uncover valuable information from big scholarly data for expert selection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Abdi, H. (2010). Coefficient of variation. In N. J. Salkind, D. M. Dougherty, & B. Frey (Eds.), Encyclopedia of research design (pp. 169–171). Thousand Oaks, CA: SAGE Publications.

    Google Scholar 

  • Balog, K., Azzopardi, L., & Rijke, M. D. (2006). Formal models for expert finding in enterprise corpora. In SIGIR ‘06, Seattle, WA, pp. 43–50

  • Balog, K., Azzopardi, L., & Rijke, M. D. (2009). A language modeling framework for expert finding. Information Processing and Management,45(1), 1–19.

    Google Scholar 

  • Basu, C., Hirsh, H., Cohen, W. W., & Nevill-Manning, C. (1999). Recommending papers by mining the web. In IJCAI ‘99, Stockholm, Sweden, pp. 1–11.

  • Biswas, H. K., & Hasan, M. M. (2007). Using publications and domain knowledge to build research profiles: An application in automatic reviewer assignment. In Proceedings of the 2007 international conference on information and communication technology, Dhaka, Bangladesh, pp. 82–86.

  • Blei, D. M., & Lafferty, J. D. (2006) Dynamic topic models. In ICML ‘06, Pittsburgh, PA, pp. 113–120.

  • Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research,3, 993–1022.

    Google Scholar 

  • Cao, Y., Liu, J., Bao S., & Li, H. (2005) Research on expert search at enterprise track of TREC 2005. TREC.

  • Choi, T. M., Chan, H. K., & Yue, X. (2017). Recent development in big data analytics for business operations and risk management. IEEE Transactions on Cybernetics,47(1), 81–92.

    Google Scholar 

  • Cook, W. D., Golany, B., Kress, M., Penn, M., & Raviv, T. (2005). Optimal allocation of proposals to reviewers to facilitate effective ranking. Management Science,51(4), 655–661.

    Google Scholar 

  • Daud, A., Li, J., Zhou, L., & Muhammad, F. (2010). Temporal expert finding through generalized time topic modeling. Knowledge-Based Systems,23(6), 615–625.

    Google Scholar 

  • Davoodi, E., Kianmehr, K., & Afsharchi, M. (2013). A semantic social network-based expert recommender system. Applied Intelligence,39(1), 1–13.

    Google Scholar 

  • Dumais, S. T., & Nielsen, J. (1992). Automating the assignment of submitted manuscripts to reviewers. In SIGIR ‘92, Copenhagen, Denmark, pp. 233–244

  • Egghe, L. (2006). Theory and practise of the G-index. Scientometrics,69(1), 131–152.

    Google Scholar 

  • Fang, Y., Si, L., & Mathur, A. P. (2010). Discriminative models of integrating document evidence and document-candidate associations for expert search. In SIGIR ‘10, Geneva, Switzerland, pp. 683–690

  • Fang, H., & Zhai, C. (2007). Probabilistic models for expert finding. In ECIR ‘07, Rome, Italy, pp. 418–430.

  • Gollapalli, S. D., Mitra, P., & Giles, C. L. (2011). Ranking authors in digital libraries. In JCDL ‘11, Ottawa, Canada, pp. 251–254.

  • Haveliwala, T. H. (2002). Topic-sensitive PageRank: A context-sensitive ranking algorithm for Web search. In WWW ‘02, Honolulu, HI, pp. 784–796.

  • Hettich, S., & Pazzani, M. J. (2006). Mining for proposal reviewers: lessons learned at the national science foundation. In KDD ‘06, Philadelphia, PA, pp. 862–871.

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. PNAS,102(46), 16569–16572.

    Google Scholar 

  • Hu, J., Fang, Y., & Godavarthy, A. (2013). Topical authority propagation on microblogs. In CIKM ‘13, San Francisco, CA, pp. 1901–1904.

  • Jiang, Z., Liu, X., & Gao, L. (2015). Chronological citation recommendation with information-need shifting. In CIKM ‘15, Melbourne, Australia, pp. 1291–1300

  • Jin, J., Geng, Q., Zhao, Q., & Zhang, L. (2017). Integrating the trend of research interest for reviewer assignment. In WWW‘17, Perth, Australia

  • Karimzadehgan, M., & Zhai, C. (2009). Constrained multi-aspect expertise matching for committee review assignment. In CIKM ‘09, Hong Kong, China, pp. 1697–1700

  • Karimzadehgan, M., & Zhai, C. (2012). Integer linear programming for constrained multi-aspect committee review assignment. Information Processing and Management,48(4), 725–740.

    Google Scholar 

  • Karimzadehgan, M., Zhai, C., & Belford, G. (2008). Multi-aspect expertise matching for review assignment. In CIKM ‘08, Napa Valley, CA, pp. 1113–1122.

  • Kleinberg, J. M. (1999). Authoritative sources in a hyperlinked environment. Journal of the ACM,46(5), 604–632.

    Google Scholar 

  • Kou, N. M., Hou, U. L., Mamoulis, N., & Gong, Z. (2015). Weighted coverage based reviewer assignment. In SIGMOD ‘15, Melbourne, Australia, pp. 2031–2046.

  • Li, L., Wang, Y., Liu, G., Wang, M., & Wu, X. (2015). Context-aware reviewer assignment for trust enhanced peer review. PLoS ONE,10(6), 1–28.

    Google Scholar 

  • Li, X., & Watanabe, T. (2013). Automatic paper-to-reviewer assignment, based on the matching degree of the reviewers. In Proceedings of the 17th international conference on Knowledge-based and intelligent information & Engineering Systems, Kitakyushu, Japan, pp. 633–642.

  • Liu, X., Bollen, J., Nelson, M. L., & Sompel, H. V. D. (2005). Co-authorship networks in the digital library research community. Information Processing and Management,41(6), 1462–1480.

    Google Scholar 

  • Liu, X., Suel, T., & Memon, N. (2014). A robust model for paper reviewer assignment. In RecSys ‘14, Foster City, Silicon Valley, CA, pp. 25–32

  • Liu, O., Wang, J., Ma, J., & Sun, Y. (2016). An intelligent decision support approach for reviewer assignment in R&D project selection. Computers in Industry,76, 1–10.

    Google Scholar 

  • Mauro, N. D., Basile, T. M. A., & Ferilli, S. (2005). GRAPE: An expert review assignment component for scientific conference management systems. In Proceedings of the 18th international conference on industrial and engineering applications of artificial intelligence and expert systems, Bari, Italy, pp. 789–798.

  • Mimno, D., & McCallum, A. (2007). Expertise modeling for matching papers with reviewers. In KDD ‘07, San Jose, CA, pp. 500–509.

  • Petkova, D., & Croft, W. B. (2008). Hierarchical language models for expert finding in enterprise corpora. International Journal on Artificial Intelligence Tools,17(1), 5–18.

    Google Scholar 

  • Rigaux, P. (2004). An iterative rating method: application to web-based conference management. In SAC ‘04, Nicosia, Cyprus, pp. 1682–1687

  • Rosen-Zvi, M., Griffiths, T., Steyvers, M., & Smyth, P. (2004). The author-topic model for authors and documents. In UAI ‘04, Banff, Canada, pp. 487–494

  • Setaputra, R., Yue, X., & Yao, D. (2010). Impact of information systems on quick response programs. In T. Cheng & T. M. Choi (Eds.), Innovative quick response programs in logistics and supply chain management. International handbooks on information systems. Berlin: Springer.

    Google Scholar 

  • Stephen, C. H., & Erim, K. (2015). Calibration, sharpness and the weighting of experts in a linear opinion pool. Annals of Operations Research,229(1), 429–450.

    Google Scholar 

  • Sun, Y. H., Ma, J., Fan, Z.-P., & Wang, J. (2008). A group decision support approach to evaluate experts for R&D project selection. IEEE Transactions on Engineering Management,55(1), 158–170.

    Google Scholar 

  • Tang, W., Tang, J., Lei, T., Tan, C., Gao, B., & Li, T. (2012). On optimization of expertise matching with various constraints. Neurocomputing,76(1), 71–83.

    Google Scholar 

  • Tang, W., Tang, J., & Tan, C. (2010). Expertise matching via constraint-based optimization. In WI-IAT ‘10, Toronto, Canada, pp. 34–41

  • Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., & Su, Z. (2008). Arnetminer: extraction and mining of academic social networks. In KDD ‘08, Las Vegas, NV, pp. 990–998

  • Tayal, D. K., Saxena, P. C., Sharma, A., Khanna, G., & Gupta, S. (2014). New method for solving reviewer assignment problem using type-2 fuzzy sets and fuzzy functions. Applied Intelligence,40(1), 54–73.

    Google Scholar 

  • Wang, G. A., Jiao, J., Abrahams, A. S., Fan, W., & Zhang, Z. (2013). ExpertRank: A topic-aware expert finding algorithm for online knowledge communities. Decision Support Systems,54(3), 1442–1451.

    Google Scholar 

  • Wei, W. W. S. (1994). Time series analysis: Univariate and multivariate methods. Boston: Addison-Wesley.

    Google Scholar 

  • Xu, S., Shi, Q., Qiao, X., Zhu, L., Jung, H., Lee, S., & Choi, S.-P. (2014) Author-topic over time (AToT): A dynamic users’ interest model. In Mobile, ubiquitous, and intelligent computing, volume 274 of Lecture Notes in Electrical Engineering (pp. 239–245). Berlin: Springer

  • Yukawa, T., Kasahara, K., Kato, T., & Kita, T. (2001). An expert recommendation system using concept-based relevance discernment. In Proceedings of the 13th international conference on tools with artificial intelligence, 2001. IEEE.

  • Zhai, C., Velivelli, A., & Yu, B. (2004). A cross-collection mixture model for comparative text mining. In KDD ‘04, Seattle, WA, pp. 743–748.

  • Zhang, C. (2013). The H’-index, effectively improving the H-index based on the citation distribution. PLoS ONE,8(4), e59912.

    Google Scholar 

  • Zheng, H. T., Li, Q., Jiang, Y., Xia, S. T., & Zhang, L. (2013). Exploiting multiple features for learning to rank in expert finding. In ADMA ‘13, Hangzhou, China, pp. 219–230

  • Zhou, D., Orshanskiy, S. A., Zha, H., & Giles, C. L. (2007). Co-ranking authors and documents in a heterogeneous network. In ICDM ‘07, Omaha, NE, pp. 739–744.

Download references

Acknowledgements

This research was supported by NSFC (No. 71571194, 71701019), the youth fund project of ministry of education of the humanities and social sciences research (No. 16YJC870006), the fundamental research funds for the central universities (No. SKZZB2014037), Chang Jiang Scholars Program (Niu Baozhuang 2017), GDUPS (Niu Baozhuang 2017), Guangdong Soft Science Project (No. 2016A070705019), and ITF Project (No. K-ZM25).

Author information

Authors and Affiliations

  1. Department of Information Management, Beijing Normal University, Beijing, 100875, China

    Jian Jin & Qian Geng

  2. School of Business Administration, South China University of Technology, Guangzhou, 510006, China

    Baozhuang Niu

  3. Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong

    Ping Ji

Authors
  1. Jian Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Baozhuang Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ping Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qian Geng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Baozhuang Niu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jin, J., Niu, B., Ji, P. et al. An integer linear programming model of reviewer assignment with research interest considerations. Ann Oper Res 291, 409–433 (2020). https://doi.org/10.1007/s10479-018-2919-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-018-2919-7

Keywords

Search

Navigation