Abstract
This paper presents an overview of the ImageCLEF 2017 evaluation campaign, an event that was organized as part of the CLEF (Conference and Labs of the Evaluation Forum) labs 2017. ImageCLEF is an ongoing initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval for providing information access to collections of images in various usage scenarios and domains. In 2017, the 15th edition of ImageCLEF, three main tasks were proposed and one pilot task: (1) a LifeLog task about searching in LifeLog data, so videos, images and other sources; (2) a caption prediction task that aims at predicting the caption of a figure from the biomedical literature based on the figure alone; (3) a tuberculosis task that aims at detecting the tuberculosis type from CT (Computed Tomography) volumes of the lung and also the drug resistance of the tuberculosis; and (4) a remote sensing pilot task that aims at predicting population density based on satellite images. The strong participation of over 150 research groups registering for the four tasks and 27 groups submitting results shows the interest in this benchmarking campaign despite the fact that all four tasks were new and had to create their own community.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
PubMed Central (PMC) is a free full-text archive of biomedical and life sciences journal literature at the U.S. National Institute of Health’s National Library of Medicine (NIH/NLM) (see http://www.ncbi.nlm.nih.gov/pmc/).
- 7.
- 8.
The dataset is available on Zenodo with the DOI 10.5281/zenodo.804602 or on demand.
- 9.
- 10.
References
Arenas, H., Baker, A., Bialczak, A., Bargiel, D., Becker, M., Gaildrat, V., Carbone, F., Heising, S., Islam, M.B., Lattes, P., Marantos, C., Menou, C., Mothe, J., Nzeh Ngong, A., Paraskevas, I.S., Penalver, M., Sciana, P., Soudris, D.: FabSpaces at population estimation (remote) task - ImageCLEF at the CLEF 2017 Labs. CLEF working notes, CEUR (2017)
Arenas, H., Islam, B., Mothe, J.: Overview of the ImageCLEF 2017 population estimation task. In: CLEF 2017 Labs Working Notes. CEUR Workshop Proceedings, CEUR-WS.org, Dublin, Ireland (2017). http://ceur-ws.org
Balikas, G., Krithara, A., Partalas, I., Paliouras, G.: BioASQ: a challenge on large-scale biomedical semantic indexing and question answering. In: Müller, H., Jimenez del Toro, O.A., Hanbury, A., Langs, G., Foncubierta Rodríguez, A. (eds.) Multimodal Retrieval in the Medical Domain. LNCS, vol. 9059, pp. 26–39. Springer, Cham (2015). doi:10.1007/978-3-319-24471-6_3
Barnston, A.G.: Correspondence among the correlation, RMSE, and heidke forecast verification measures; refinement of the heidke score. Weather Forecast. 7(4), 699–709 (1992)
Clough, P., Müller, H., Sanderson, M.: The CLEF 2004 cross-language image retrieval track. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 597–613. Springer, Heidelberg (2005). doi:10.1007/11519645_59
Dang-Nguyen, D.T., Piras, L., Riegler, M., Boato, G., Zhou, L., Gurrin, C.: Overview of ImageCLEFlifelog 2017: lifelog retrieval and summarization. In: CLEF 2017 Labs Working Notes. CEUR Workshop Proceedings, CEUR-WS.org, Dublin, Ireland (2017). http://ceur-ws.org
Dicente Cid, Y., Kalinovsky, A., Liauchuk, V., Kovalev, V., Müller, H.: Overview of ImageCLEFtuberculosis 2017 - predicting tuberculosis type and drug resistances. In: CLEF 2017 Labs Working Notes. CEUR Workshop Proceedings, CEUR-WS.org, Dublin, Ireland (2017). http://ceur-ws.org
Dogariu, M., Ionescu, B.: A textual filtering of hog-based hierarchical clustering of lifelog data, pp. 11–14. CLEF working notes, CEUR (2017)
Eickhoff, C., Schwall, I., Seco, G., de Herrera, A., Müller, H.: Overview of ImageCLEF caption 2017 - image caption prediction and concept detection for biomedical images. In: CLEF 2017 Labs Working Notes. CEUR Workshop Proceedings, CEUR-WS.org, Dublin, Ireland (2017). http://ceur-ws.org
Goeuriot, L., Kelly, L., Li, W., Palotti, J., Pecina, P., Zuccon, G., Hanbury, A., Jones, G.J.F., Müller, H.: Share/CLEF ehealth evaluation lab 2014, task 3: User-centred health information retrieval. In: Working Notes of CLEF 2014, CEUR (2014)
Hanbury, A., Müller, H., Balog, K., Brodt, T., Cormack, G.V., Eggel, I., Gollub, T., Hopfgartner, F., Kalpathy-Cramer, J., Kando, N., Krithara, A., Lin, J., Mercer, S., Potthast, M.: Evaluation-as-a-service: overview and outlook. ArXiv 1512.07454 (2015)
Seco, G., de Herrera, A., Schaer, R., Bromuri, S., Müller, H.: Overview of the ImageCLEF 2016 medical task. In: Working Notes of CLEF 2016 (Cross Language Evaluation Forum), September 2016
Islam, M.B., Becker, M., Bargiel, D., Ahmed, K.R., Duzak, P., Emana, N.G.: Sentinel-2 satellite imagery based population estimation strategies at FabSpace 2.0 Lab Darmstadt (2017)
Joly, A., Goëau, H., Glotin, H., Spampinato, C., Bonnet, P., Vellinga, W.P., Lombardo, J.C., Planqué, R., Palazzo, S., Müller, H.: Lifeclef 2017 lab overview: multimedia species identification challenges. In: Proceedings of CLEF 2017 (2017)
Kalpathy-Cramer, J., García Seco de Herrera, A., Demner-Fushman, D., Antani, S., Bedrick, S., Müller, H.: Evaluating performance of biomedical image retrieval systems: overview of the medical image retrieval task at ImageCLEF 2004–2014. Computerized Medical Imaging and Graphics 39, 55–61 (2015)
Koutsouri, K., Skepetari, I., Anastasakis, K., Lappas, S.: Population estimation using satellite imagery. In: CLEF working notes, CEUR. CEUR-WS.org, Dublin, Ireland (2017). http://ceur-ws.org
Molino, A.G.D., Mandal, B., Lin, J., Lim, J.H., Subbaraju, V., Chandrasekhar, V.: VC-I2R@ImageCLEF2017: ensemble of deep learned features for lifelog video summarization, pp. 11–14. CLEF working notes, CEUR (2017)
Müller, H., Clough, P., Deselaers, T., Caputo, B. (eds.): ImageCLEF - Experimental Evaluation in Visual Information Retrieval, The Springer International Series on Information Retrieval, vol. 32. Springer, Heidelberg (2010)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)
Pomente, A., Aleandri, D.: Convolutional expectation maximization for population estimation, pp. 11–14. CLEF working notes, CEUR (2017)
Soldaini, L., Goharian, N.: QuickUMLS: a fast, unsupervised approach for medical concept extraction. In: MedIR Workshop, SIGIR (2016)
Tsikrika, T., Herrera, A.G.S., Müller, H.: Assessing the scholarly impact of ImageCLEF. In: Forner, P., Gonzalo, J., Kekäläinen, J., Lalmas, M., Rijke, M. (eds.) CLEF 2011. LNCS, vol. 6941, pp. 95–106. Springer, Heidelberg (2011). doi:10.1007/978-3-642-23708-9_12
Tsikrika, T., Larsen, B., Müller, H., Endrullis, S., Rahm, E.: The scholarly impact of CLEF (2000–2009). In: Forner, P., Müller, H., Paredes, R., Rosso, P., Stein, B. (eds.) CLEF 2013. LNCS, vol. 8138, pp. 1–12. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40802-1_1
Villegas, M., et al.: General overview of ImageCLEF at the CLEF 2015 labs. In: Mothe, J., et al. (eds.) CLEF 2015. LNCS, vol. 9283, pp. 444–461. Springer, Cham (2015). doi:10.1007/978-3-319-24027-5_45
Villegas, M., et al.: General overview of ImageCLEF at the CLEF 2016 labs. CLEF 2016. LNCS, vol. 9822, pp. 267–285. Springer, Cham (2016). doi:10.1007/978-3-319-44564-9_25
Zhou, L., Piras, L., Rieger, M., Boato, G., Dang-Nguyen, D.T., Gurrin, C.: Organizer team at ImageCLEFlifelog 2017: baseline approaches for lifelog retrieval and summarization, pp. 11–14. CLEF working notes, CEUR (2017)
Acknowledgements
This research was supported in part by the Intramural Research Program of the National Institutes of Health (NIH), National Library of Medicine (NLM), and Lister Hill National Center for Biomedical Communications (LHNCBC). It is also partly supported European Union’s Horizon 2020 Research and Innovation programme under the Grant Agreement no693210 (FabSpace 2.0).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Ionescu, B. et al. (2017). Overview of ImageCLEF 2017: Information Extraction from Images. In: Jones, G., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2017. Lecture Notes in Computer Science(), vol 10456. Springer, Cham. https://doi.org/10.1007/978-3-319-65813-1_28
Download citation
DOI: https://doi.org/10.1007/978-3-319-65813-1_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-65812-4
Online ISBN: 978-3-319-65813-1
eBook Packages: Computer ScienceComputer Science (R0)