Skip to main content
SpringerLink
Log in

ConvAI Dataset of Topic-Oriented Human-to-Chatbot Dialogues

  • Conference paper
  • First Online:
The NIPS '17 Competition: Building Intelligent Systems

Part of the book series: The Springer Series on Challenges in Machine Learning ((SSCML))

Abstract

This paper contains the description and the analysis of the dataset collected during the Conversational Intelligence Challenge (ConvAI) which took place in 2017. During the evaluation round we collected over 4,000 dialogues from 10 chatbots and 1,000 volunteers. Here we provide the dataset statistics and outline some possible improvements for future data collection experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The dataset is available at http://convai.io/2017/data/

  2. 2.

    http://turing.tilda.ws/

  3. 3.

    https://nips.cc/Conferences/2017

References

  • Burtsev, M., Logacheva, V., Malykh, V., Serban, I., Lowe, R., Prabhumoye, S., Black, A. W., Rudnicky, A., and Bengio, Y. (2018). The First Conversational Intelligence Challenge. NIPS 2017 Competition track Springer Proceedings.

    Google Scholar 

  • Lowe, R., Noseworthy, M., Serban, I. V., Angelard-Gontier, N., Bengio, Y., and Pineau, J. (2017). Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses. Acl, pages 1–19.

    Google Scholar 

  • Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. (2016). SQuAD: 100,000+ Questions for Machine Comprehension of Text. Emnlp, (ii):2383–2392.

    Google Scholar 

  • Serban, I. V., Lowe, R., Charlin, L., and Pineau, J. (2015). A Survey of Available Corpora for Building Data-Driven Dialogue Systems. CoRR, page 46.

    Google Scholar 

  • Serban, I. V., Sordoni, A., Lowe, R., Charlin, L., Pineau, J., Courville, A., and Bengio, Y. (2016). A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues. Proceedings of the Advances in Neural Information Processing Systems 29 (NIPS 2016), pages 1–14.

    Google Scholar 

  • Yu, Z., Xu, Z., Black, A. W., and Rudnicky, A. I. (2016). Chatbot Evaluation and Database Expansion via Crowdsourcing. WOCHAT workshop.

    Google Scholar 

Download references

Acknowledgements

The work was supported by National Technology Initiative and PAO Sberbank project ID 0000000007417F630002.

Author information

Authors and Affiliations

  1. Moscow Institute of Physics and Technology, Moscow, Russia

    Varvara Logacheva, Mikhail Burtsev, Valentin Malykh, Vadim Polulyakh & Aleksandr Seliverstov

Authors
  1. Varvara Logacheva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mikhail Burtsev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Valentin Malykh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vadim Polulyakh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Aleksandr Seliverstov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Varvara Logacheva .

Editor information

Editors and Affiliations

  1. Department Mathematics & Informatics, University of Barcelona, Barcelona, Spain

    Sergio Escalera

  2. Microsoft (United States), Redmond, WA, USA

    Markus Weimer

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Logacheva, V., Burtsev, M., Malykh, V., Polulyakh, V., Seliverstov, A. (2018). ConvAI Dataset of Topic-Oriented Human-to-Chatbot Dialogues. In: Escalera, S., Weimer, M. (eds) The NIPS '17 Competition: Building Intelligent Systems. The Springer Series on Challenges in Machine Learning. Springer, Cham. https://doi.org/10.1007/978-3-319-94042-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-94042-7_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-94041-0

  • Online ISBN: 978-3-319-94042-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation