Frank Hopfgartner
Allan Hanbury
Martin Potthast
Krisztian Balog
Abstract
In this report, we summarize the outcome of the "Evaluation-as-a-Service" workshop that was held on the 5th and 6th March 2015 in Sierre, Switzerland. The objective of the meeting was to bring together initiatives that use cloud infrastructures, virtual machines, APIs (Application Programming Interface) and related projects that provide evaluation of information retrieval or machine learning tools as a service.
- Georgios Balikas, Anastasia Krithara, Ioannis Partalas, and Georgios Paliouras. BioASQ: A challenge on large-scale biomedical semantic indexing and questionanswering. In MRMD'15: Proceedings of the Multimodal Retrieval in the Medical Domain Workshop, 2015.Google Scholar
Digital Library
- Georgios Balikas, Ioannis Partalas, Axel-Cyrille Ngonga Ngomo, Anastasia Krithara, and Georgios Paliouras. Results of the BioASQ track of the question answering lab at CLEF 2014. In CLEF'14: Proceedings of the 5th International Conference of the CLEF Initiative, pages 1181--1193. Springer, 2014.Google Scholar
- Krisztian Balog, Liadh Kelly, and Anne Schuth. Head first: Living labs for ad-hoc search evaluation. In CIKM'14: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pages 1815--1818, 2014. Google ScholarDigital Library
- Torben Brodt and Frank Hopfgartner. Shedding Light on a Living Lab: The CLEF NEWSREEL Open Recommendation Platform. In IIiX'14: Proceedings of Information Interaction in Context Conference, pages 223--226. ACM, 08 2014. Google ScholarDigital Library
- Tim Gollub, Benno Stein, and Steven Burrows. Ousting Ivory Tower Research: Towards a Web Framework for Providing Experiments as a Service. In SIGIR'12: Proceedings of the 35th International ACM Conference on Research and Development in Information Retrieval, pages 1125--1126. ACM, 2012. Google ScholarDigital Library
- Allan Hanbury, Henning Müller, Georg Langs, Marc André Weber, Bjoern H. Menze, and Tomas Salas Fernandez. Bringing the algorithms to the data: cloud-based benchmarking for medical image analysis. In CLEF'12: Proceedings of the 3rd International Conference of the CLEF Initiative, pages 24--29. Springer Verlag, 2012. Google ScholarDigital Library
- Frank Hopfgartner, Benjamin Kille, Andreas Lommatzsch, Torben Brodt, and Tobias Heintz. Benchmarking News Recommendations in a Living Lab. In CLEF'14: Proceedings of the 5th International Conference of the CLEF Initiative, pages 250--267. Springer Verlag, 09 2014.Google ScholarCross Ref
- Benjamin Kille, Frank Hopfgartner, Torben Brodt, and Tobias Heintz. The plista dataset. In NRS'13: Proceedings of the International Workshop and Challenge on News Recommender Systems, pages 14--21. ACM, 10 2013. Google ScholarDigital Library
- Georg Langs, Henning Müller, Bjoern H. Menze, and Allan Hanbury. Visceral: Towards large data in medical imaging --- challenges and directions. In MCBR-CDS'12: Proceedings of the Third MICCAI International Workshop, pages 92--98. Springer, 2012. Google ScholarDigital Library
- Jimmy Lin and Miles Efron. Overview of the TREC-2013 Microblog Track. In TREC'13: Proceedings of the 22nd Text REtrieval Conference, Gaithersburg, Maryland, 2013.Google Scholar
- Richard McCreadie, Ian Soboroff, Jimmy Lin, Craig Macdonald, Iadh Ounis, and Dean McCullough. On building a reusable twitter corpus. In SIGIR'12: Proceedings of the 35th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1113--1114, Portland, Oregon, 2012. Google ScholarDigital Library
- Iadh Ounis, Craig Macdonald, Jimmy Lin, and Ian Soboroff. Overview of the TREC- 2011 Microblog Track. In TREC'11: Proceedings of the 20th Text REtrieval Conference, Gaithersburg, Maryland, 2011.Google Scholar
- Martin Potthast, Tim Gollub, Francisco Rangel, Paolo Rosso, Efstathios Stamatatos, and Benno Stein. Improving the Reproducibility of PAN's Shared Tasks: Plagiarism Detection, Author Identification, and Author Profiling. In CLEF'14: Proceedings of the 5th Int. Conference of the CLEF Initiative, pages 268--299. Springer Verlag, 2014.Google Scholar
- Jinfeng Rao, Jimmy Lin, and Miles Efron. Reproducible experiments on lexical and temporal feedback for tweet search. In ECIR'15: Proceedings of the 37th European Conference on Information Retrieval, pages 755--767, Vienna, Austria, 2015.Google ScholarCross Ref
Index Terms
- Report on the Evaluation-as-a-Service (EaaS) Expert Workshop
Recommendations
IWGS 2010 workshop report: The First ACM SIGSPATIAL International Workshop on Geostreaming (San Jose, California - November 2, 2010)
The ACM SIGSPATIAL International Workshop on Geostreaming (IWGS) was held for the first time in conjunction with the 18th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACM SIGSPATIAL GIS 2010). The workshop has ...
SPRINGL 2009 Workshop Report: The 2nd International Workshop on Security and Privacy in GIS and LBS (Seattle, Washington - November 3, 2009)
The second International Workshop on Security and Privacy in GIS and LBS (SPRINGL 2009) was held on 3rd November 2009 in Seattle, Washington, in conjunction with the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information ...
IWGS 2016 workshop report: The 7th ACM SIGSPATIAL International Workshop on GeoStreaming: San Francisco, CA, USA - October 31, 2016
The ACM SIGSPATIAL International Workshop on Geostreaming (IWGS) was held for the seventh time in conjunction with the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACMGIS 2016). The workshop has been a ...
Comments