Skip to main content
SpringerLink
Log in

Proposal of a stochastic model to determine the bibliometric variables influencing the quality of a journal: application to the field of Dentistry

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

On the basis of the Impact Factor of Journal Citation Reports developed by ISI as a journal quality indicator, this paper puts forth an ordinal regression model to estimate the journal’s position by terciles. The set of explanatory variables includes the H-index of its Editor-in-chief, percentage of papers published in the journal that received external funding, average number of papers published yearly, and two factors concerning the scope and structure of the journal. The proposed model was applied to the field of Dentistry, Oral Surgery and Medicine, and led us to the conclusion that the above mentioned covariables alone had a significant input in the model, but not the factors. The essay performed on a sample of 30 Dentistry journals included in JCR provided a confirmatory correct classification rate (CCR) of 80%, with a predictive CCR of 75% on a sample of eight new journals not previously considered in the phase of model estimation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  • Alberts, B. (2013). Impact factor distortions. Science, 340(6134), 787.

    Article  Google Scholar 

  • Althouse, B. M., West, J. D., Bergstrom, C. T., & Bergstrom, T. (2009). Differences in impact factor across fields and over time. Journal of the American Society for Information Science and Technology, 60(1), 27–34.

    Article  Google Scholar 

  • Anscombe, F. J. (1948). The validity of comparative experiments. Journal of the Royal Statistical Society, Series A, 111(3), 181–211.

    Article  MathSciNet  Google Scholar 

  • Bollen, J., Van de Sompel, H., Hagberg, A., & Chute, R. (2009). A principal component analysis of 39 scientific impact measures. PLOS ONE, 4(6), 0006022. https://doi.org/10.1371/journal.pone.

    Article  Google Scholar 

  • Bordons, M., Fernandez, M. T., & Gomez, I. (2002). Advantages and limitations in the use of impact factor measures for the assessment of research performance in a peripheral country. Scientometrics, 53(2), 195–206.

    Article  Google Scholar 

  • Bornmann, L., Marx, W., Gasparyan, A. Y., & Kitas, G. D. (2012). Diversity, value and limitations of the journal impact factor and alternative metrics. Rheumatology International, 32(7), 1861–1867.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., Marx, W., Schier, H., & Daniel, H. D. (2011). A multilevel modeling approach to investigating the predictive validity of editorial decisions: Do the editors of a high profile journal select manuscripts that are highly cited after publication? Journal of the Royal Statistical Society, Series A, 174(4), 857–879.

    Article  MathSciNet  Google Scholar 

  • Bornmann, L., & Williams, R. (2017). Can the journal impact factor be used as a criterion for the selection of junior researchers? A large-scale empirical study based on Researcher ID data. Journal of Informetrics, 11(3), 788–799.

    Article  Google Scholar 

  • Buela, G. (2003). Evaluating quality of articles and scientific journals. Proposal of weighted impact factor and a quality index? Psicothema, 15(1), 23–35.

    Google Scholar 

  • Falagas, M. E., & Alexiou, V. G. (2008). The top-ten in journal impact factor manipulation. Archivum Immunologiae et Therapiae Experimentalis, 56(4), 223–226.

    Article  Google Scholar 

  • Falagas, M. E., Kouranos, V. D., Arencibia-Jorge, R., & Karageorgopoulos, D. E. (2008a). Comparison of SCImago journal rank indicator with journal impact factor. FASEB Journal, 22(8), 2623–2628.

    Article  Google Scholar 

  • Fassoulaki, A., Papilas, K., Paraskeva, A., & Patris, N. (2002). Impact factor bias and proposed adjustments for its determination. Acta Anaetesiologica Scandinavica, 46(7), 902–905.

    Article  Google Scholar 

  • Fassoulaki, A., Paraskeva, A., Papilas, K., & Karabinis, G. (2000). Self-citations in six anaesthesia journals and their significance in determining the impact factor. British Journal of Anaesthesia, 84(2), 266–269.

    Article  Google Scholar 

  • Garfield, E. (1955). Citation indexes for science: A new dimension in documentation through association of ideas. Science, 122(3159), 108–111.

    Article  Google Scholar 

  • Garfield, E. (1999). Journal impact factor: A brief review. Canadian Medical Association Journal, 161, 979–980.

    Google Scholar 

  • Garfield, E. (2006). The history and meaning of the journal impact factor. Journal of the American Medical Association, 295(1), 90–93.

    Article  Google Scholar 

  • Hansson, S. (1995). Impact factor as a misleading tool in evaluation of medical journals. The Lancet, 346(8979), 906–906.

    Article  Google Scholar 

  • Kay, J., Memon, M., de Sa, D., Simunovic, N., Duong, A., Karlsson, J., et al. (2017). The H-index of editorial board members correlates positively with the impact factor of Sports Medicine journals. Orthopaedic Journal of Sports Medicine, 5(3), 2325967117694024. https://doi.org/10.1177/2325967117694024.

    Article  Google Scholar 

  • Kurmis, A. P. (2003). Understanding the limitations of the journal impact factor. The Journal of Bone and Joint Surgery, 85(12), 2449–2454.

    Article  Google Scholar 

  • Leydesdorff, L. (2012). Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most-highly cited papers. Scientometrics, 92(2), 355–365.

    Article  Google Scholar 

  • Leydesdorff, L., & Bornmann, L. (2011). How fractional counting of citations affects the impact factor: Normalization in terms of differences in citation potentials among fields of Science. Journal of the American Society for Information Science and Technology, 62(2), 217–229.

    Article  Google Scholar 

  • Leydesdorff, L., & Opthof, T. (2010). Scopus’s source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations. Journal of the American Society for Information Science and Technology, 61(11), 2365–2369.

    Article  Google Scholar 

  • Lucena, C., Souza, E. M., Voinea, G. C., Pulgar, R., Valderrama, M. J., & De-Deus, G. (2017). A quality assessment of randomized controlled trial reports in Endodontics. International Endodontic Journal, 50(3), 237–250.

    Article  Google Scholar 

  • Malay, D. S. (2013). Impact factors and other measures of a journal’s influence. The Journal of Food and Ankle Surgery, 52(3), 285–287.

    Article  Google Scholar 

  • McVeigh, M. F., & Mann, S. J. (2009). The journal impact factor denominator defining citable (counted) items. Journal of the American Medical Association, 302(10), 1107–1109.

    Article  Google Scholar 

  • Moed, H. F. (2002). The impact-factors debate: The ISI’s uses and limits. Nature, 415(6873), 731–732.

    Article  Google Scholar 

  • Moed, H. F., & Vanleeuwen, T. N. (1995). Improving the accuracy of Institute for Scientific Information journal impact factors. Journal of the American Society for Information Science and Technology, 46(6), 461–467.

    Article  Google Scholar 

  • Mutz, R., & Daniel, H.-D. (2012a). The generalized propensity score methodology for estimating unbiased journal impact factors. Scientometrics, 92, 377–390.

    Article  Google Scholar 

  • Mutz, R., & Daniel, H.-D. (2012b). Skewed citation distributions and bias factors: Solutions to two core problems with the journal impact factor. Journal of Informetrics, 6(2), 169–176.

    Article  Google Scholar 

  • Roberts, R. J. (2017). An obituary for the impact factor. Nature, 546(7660), 600.

    Article  Google Scholar 

  • Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality? Journal of the Medical Library Association, 91(1), 42–46.

    Google Scholar 

  • Seglen, P. O. (1997a). Citations and journal impact factors: Questionable indicators of research quality. Allergy, 52(11), 1050–1056.

    Article  Google Scholar 

  • Seglen, P. O. (1997b). Why the impact factor of journals should not be used for evaluating research. British Medical Journal, 314(7079), 498–502.

    Article  Google Scholar 

  • Seglen, P. O. (1998). Citation rates and journal impact factors are not suitable for evaluation of research. Acta Orthopaedica Scandinavica, 69(3), 224–229.

    Article  Google Scholar 

  • Simons, K. (2008). The misused impact factor. Science, 322(5899), 165.

    Article  Google Scholar 

  • Valderrama, P., Escabias, M., Jiménez-Contreras, E. Valderrama, M. J. & Baca, P. (2017). Bibliometric variables determining the quality of a dentistry journal. In Skiadas, C. H. (Ed.), Proceedings of the 17th conference of the applied stochastic models and data analysis international society, pp. 825–831.

  • Van Noorden, R. (2016). Impact factor gets heavyweight rival. Nature, 540(7633), 325–326.

    Article  Google Scholar 

  • Vanclay, J. K. (2012). Impact factor: Outdated artefact or stepping-stone to journal certification? Scientometrics, 92(2), 211–238.

    Article  Google Scholar 

  • Varki, A. (2017). Rename the impact factor. Nature, 548(7668), 393.

    Article  Google Scholar 

  • Wagner, G., Prester, J., Roche, M., Benlian, A., & Schryen, G. (2006). Factors affecting the scientific impact of literature reviews: A scientometric study. In proceedings of the 37th international conference on information systems, Dublin 2016, Vol. 23, pp. 1659–1682.

  • Wilsdon, J., et al. (2015). The metric tide: Report of the independent review of the role of metrics in research assessment and management. HEFCE. https://doi.org/10.13140/RG.2.1.4929.1363.

  • Zitt, M., & Small, H. (2008). Modifying the journal impact factor by fractional citation weighting: The audience factor. Journal of the American Society for Information Science and Technology, 59(11), 1856–1860.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by MTM2017-88708-P of Secretaría de Estado de Investigación, Desarrollo e Innovación, Ministerio de Economía y Competitividad de España.

Author information

Authors and Affiliations

  1. Department of Statistics and Operations Research, University of Granada, Campus de Cartuja, 18071, Granada, Spain

    Pilar Valderrama, Manuel Escabias & Mariano J. Valderrama

  2. Department of Information and Communication, University of Granada, Campus de Cartuja, 18071, Granada, Spain

    Evaristo Jiménez-Contreras

  3. Department of Dentistry, University of Granada, Campus de Cartuja, 18071, Granada, Spain

    Alberto Rodríguez-Archilla

Authors
  1. Pilar Valderrama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manuel Escabias
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Evaristo Jiménez-Contreras
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alberto Rodríguez-Archilla
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mariano J. Valderrama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mariano J. Valderrama.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Additional information

Reports of my death are greatly exaggerated (Mark Twain).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Valderrama, P., Escabias, M., Jiménez-Contreras, E. et al. Proposal of a stochastic model to determine the bibliometric variables influencing the quality of a journal: application to the field of Dentistry. Scientometrics 115, 1087–1095 (2018). https://doi.org/10.1007/s11192-018-2707-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-018-2707-9

Keywords

Search

Navigation