Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Testing the limits of SMILES-based de novo molecular generation with curriculum and deep reinforcement learning

Nature Machine Intelligence volume 5, pages 386–394 (2023)Cite this article

Subjects

A preprint version of the article is available at bioRxiv.

Abstract

Deep reinforcement learning methods have been shown to be potentially powerful tools for de novo design. Recurrent-neural-network-based techniques are the most widely used methods in this space. In this work we examine the behaviour of recurrent-neural-network-based methods when there are few (or no) examples of molecules with the desired properties in the training data. We find that targeted molecular generation is usually possible, but the diversity of generated molecules is often reduced and it is not possible to control the composition of generated molecular sets. To help overcome these issues, we propose a new curriculum-learning-inspired recurrent iterative optimization procedure that enables the optimization of generated molecules for seen and unseen molecular profiles, and allows the user to control whether a molecular profile is explored or exploited. Using our method, we generate specific and diverse sets of molecules with up to 18 times more scaffolds than standard methods for the same sample size; however, our results also point to substantial limitations of one-dimensional molecular representations, as used in this space. We find that the success or failure of a given molecular optimization problem depends on the choice of simplified molecular-input line-entry system (SMILES).

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Distributions of generated libraries.
Fig. 2: TPSA distribution of molecules.
Fig. 3: Comparison of SrIOP with REINVENT with and without diversity filters.
Fig. 4: Effects of SMILES choice on substructure generation performance using ReCL and SrIOP.
Fig. 5: High-level diagram of the deep reinforcement model architecture from past works.

Similar content being viewed by others

Data availability

The trained generative model used in some of our work is already published by Patronov and colleagues39, and is available at https://github.com/m-mokaya/RIOP/blob/main/models/random.prior.new. The raw data needed to reproduce the experiments in this work are provided at https://github.com/m-mokaya/RIOP/blob/main/data. Our training data (also available in above links) are from ChEMBL: https://www.ebi.ac.uk/chembl/.

Code availability

The code used in this study is available at https://github.com/m-mokaya/RIOP. Example notebooks for each experiment are available at https://github.com/m-mokaya/RIOP/tree/main/notebooks and https://doi.org/10.5281/zenodo.7406695.

References

  1. Schneider, P. & Schneider, G. De novo design at the edge of chaos. J. Med. Chem. 59, 4077–4086 (2016).

    Article  Google Scholar 

  2. Waring, M. J. et al. An analysis of the attrition of drug candidates from four major pharmaceutical companies. Nat. Rev. Drug Discov. 14, 475–486 (2015).

    Article  Google Scholar 

  3. Hay, M., Thomas, D. W., Craighead, J. L., Economides, C. & Rosenthal, J. Clinical development success rates for investigational drugs. Nat. Biotechnol. 32, 40–51 (2014).

    Article  Google Scholar 

  4. Bunnage, M. E. Getting pharmaceutical R&D back on target. Nat. Chem. Biol. 7, 335–339 (2011).

    Article  Google Scholar 

  5. Hughes, J., Rees, S., Kalindjian, S. & Philpott, K. Principles of early drug discovery: principles of early drug discovery. Br. J. Pharmacol. 162, 1239–1249 (2011).

    Article  Google Scholar 

  6. Bohacek, R. S., McMartin, C. & Guida, W. C. The art and practice of structure-based drug design: a molecular modeling perspective. Med. Res. Rev. 16, 3–50 (1996).

    Article  Google Scholar 

  7. Kim, S. et al. PubChem substance and compound databases. Nucleic Acids Res. 44, D1202–D1213 (2016).

    Article  Google Scholar 

  8. Romano, J. D. & Tatonetti, N. P. Informatics and computational methods in natural product drug discovery: a review and perspectives. Front. Genet. 10, 368 (2019).

    Article  Google Scholar 

  9. Lin, X., Li, X. & Lin, X. A review on applications of computational methods in drug screening and design. Molecules 25, 1375 (2020).

    Article  MathSciNet  Google Scholar 

  10. Besnard, J. et al. Automated design of ligands to polypharmacological profiles. Nature 492, 215–220 (2012).

    Article  Google Scholar 

  11. Gómez-Bombarelli, R. et al. Automatic chemical design using a data-driven continuous representation of molecules. ACS Cent. Sci. 4, 268–276 (2018).

    Article  Google Scholar 

  12. Stumpfe, D. & Bajorath, J. Similarity searching. WIREs Comput. Mol. Sci. 1, 260–282 (2011).

    Article  Google Scholar 

  13. Horvath, D. A virtual screening approach applied to the search for trypanothione reductase inhibitors. J. Med. Chem. 40, 2412–2423 (1997).

    Article  Google Scholar 

  14. Surabhi, S. & Singh, B. K. Computer-aided drug design: an overview. J. Drug Deliv. Ther. 8, 504–509 (2018).

    Article  Google Scholar 

  15. Segler, M. H. S., Kogej, T., Tyrchan, C. & Waller, M. P. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS Cent. Sci. 4, 120–131 (2018).

    Article  Google Scholar 

  16. Mauser, H. & Stahl, M. Chemical fragment spaces for de novo design. J. Chem. Inf. Model. 47, 318–324 (2007).

    Article  Google Scholar 

  17. Hartenfeller, M., Proschak, E., Schüller, A. & Schneider, G. Concept of combinatorial de novo design of drug-like molecules by particle swarm optimization. Chem. Biol. Drug Des. 72, 16–26 (2008).

    Article  Google Scholar 

  18. Dey, F. & Caflisch, A. Fragment-based de novo ligand design by multiobjective evolutionary optimization. J. Chem. Inf. Model. 48, 679–690 (2008).

    Article  Google Scholar 

  19. Elton, D. C., Boukouvalas, Z., Fuge, M. D. & Chung, P. W. Deep learning for molecular design—a review of the state of the art. Mol. Syst. Des. Eng. 4, 828–849 (2019).

  20. Baldi, P. Autoencoders, unsupervised learning, and deep architectures. In Proc. ICML Workshop on Unsupervised and Transfer Learning (eds. Guyon, I. et al.) Vol. 27, 37–49 (PMLR, 2012).

  21. Jin, W., Barzilay, R. & Jaakkola, T. Junction tree variational autoencoder for molecular graph generation. Preprint at arXiv https://doi.org/10.48550/arXiv.1802.04364 (2018).

  22. Weininger, D. SMILES: a chemical language and information system: 1: introduction to methodology and encoding rules. J. Chem. Inf. Comput. Sci. 28, 31–36 (1988).

    Article  Google Scholar 

  23. Lim, J., Ryu, S., Kim, J. W. & Kim, W. Y. Molecular generative model based on conditional variational autoencoder for de novo molecular design. J. Cheminform. 10, 31 (2018).

    Article  Google Scholar 

  24. Goodfellow, I. J. et al. Generative adversarial networks. Preprint at arXiv https://doi.org/10.48550/arXiv.1406.266 (2014).

  25. Putin, E. et al. Reinforced adversarial neural computer for de novo molecular design. J. Chem. Inf. Model. 58, 1194–1204 (2018).

    Article  Google Scholar 

  26. Guimaraes, G. L., Sanchez-Lengeling, B., Outeiral, C., Farias, P. L. C. & Aspuru-Guzik, A. Objective-reinforced generative adversarial networks (ORGAN) for sequence generation models. Preprint at arXiv https://doi.org/10.48550/arXiv.1705.10843 (2017).

  27. Vaswani, A. et al. Attention Is all you need. Preprint at arXiv https://doi.org/10.48550/arXiv.1706.03762 (2017).

  28. Grechishnikova, D. Transformer neural network for protein-specific de novo drug generation as a machine translation problem. Sci. Rep. 11, 321 (2021).

    Article  Google Scholar 

  29. Bagal, V., Aggarwal, R., Vinod, P. K. & Priyakumar, U. D. MolGPT: molecular generation using a transformer-decoder model. J. Chem. Inf. Model. 62, 2064–2076 (2022).

    Article  Google Scholar 

  30. Zheng, S. et al. Deep scaffold hopping with multimodal transformer neural networks. J. Cheminform. 13, 87 (2021).

    Article  MathSciNet  Google Scholar 

  31. He, J. et al. Transformer neural network for structure constrained molecular optimization. Preprint at ChemRxiv https://doi.org/10.26434/chemrxiv.14416133.v1 (2021).

  32. Goldberg, Y. A Primer on neural network models for natural language Processing. J. Artif. Intell. Res. 57, 345–420 (2016).

  33. Kotsias, P.-C. et al. Direct steering of de novo molecular generation with descriptor conditional recurrent neural networks. Nat. Mach. Intell. 2, 254–265 (2020).

    Article  Google Scholar 

  34. Bjerrum, E. J. & Threlfall, R. Molecular generation with recurrent neural networks (RNNs). Preprint at https://arxiv.org/abs/1705.04612 (2017).

  35. Arús-Pous, J. et al. Exploring the GDB-13 chemical space using deep generative models. J. Cheminform. 11, 20 (2019).

    Article  Google Scholar 

  36. Arús-Pous, J. et al. Randomized SMILES strings improve the quality of molecular generative models. J. Cheminform. 11, 71 (2019).

    Article  Google Scholar 

  37. Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 229–256 (1992).

    Article  MATH  Google Scholar 

  38. Popova, M., Isayev, O. & Tropsha, A. Deep reinforcement learning for de novo drug design. Sci. Adv. 4, eaap7885 (2018).

    Article  Google Scholar 

  39. Guo, J. et al. Improving de novo molecular design with curriculum learning. Nat. Mach. Intell. 4, 555–563 (2022).

    Article  Google Scholar 

  40. Olivecrona, M., Blaschke, T., Engkvist, O. & Chen, H. Molecular de-novo design through deep reinforcement learning. J. Cheminform. 9, 48 (2017).

    Article  Google Scholar 

  41. Soviany, P., Ionescu, R. T., Rota, P. & Sebe, N. Curriculum learning: a survey. Preprint at arXiv https://doi.org/10.48550/arXiv.2101.10382 (2021).

  42. Krenn, M., Häse, F., Nigam, A., Friederich, P. & Aspuru-Guzik, A. Self-referencing embedded strings (SELFIES): a 100% robust molecular string representation. Mach. Learn. Sci. Technol. 1, 045024 (2020).

    Article  Google Scholar 

  43. O’Boyle, N. & Dalke, A. DeepSMILES: an adaptation of SMILES for use in machine-learning of chemical structures. Preprint at ChemRxiv https://doi.org/10.26434/chemrxiv.7097960.v1 (2018)

  44. Gaulton, A. et al. ChEMBL: a large-scale bioactivity database for drug discovery. Nucleic Acids Res. 40, D1100–D1107 (2012).

    Article  Google Scholar 

  45. Landrum, G. RDKit: Open-Source Cheminformatics (RDKit, 2006).

  46. Ertl, P. & Schuffenhauer, A. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. J. Cheminform. 1, 8 (2009).

    Article  Google Scholar 

  47. Bemis, G. W. & Murcko, M. A. The properties of known drugs. 1. Molecular frameworks. J. Med. Chem. 39, 2887–2893 (1996).

    Article  Google Scholar 

  48. Polykovskiy, D. et al. Molecular Sets (MOSES): a benchmarking platform for molecular generation models. Front. Pharmacol. 11, 565644 (2020).

  49. Elman, J. L. Learning and development in neural networks: the importance of starting small. Cognition 48, 71–99 (1993).

    Article  Google Scholar 

  50. Blaschke, T. et al. REINVENT 2.0: an AI tool for de novo drug design. J. Chem. Inf. Model. 60, 5918–5922 (2020).

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Engineering and Physical Sciences Research council (grant no. EP/S024093/1) and Exscientia. For the purpose of Open access, we have applied a CC BY public copyright licence to any Author Accepted Manuscript (AAM) version arising from this submission.

Author information

Authors and Affiliations

  1. Department of Statistics, University of Oxford, Oxford, UK

    Maranga Mokaya & Charlotte M. Deane

  2. University of California, Los Angeles, Los Angeles, CA, USA

    Fergus Imrie

  3. Exscientia, Oxford, UK

    Willem P. van Hoorn, Aleksandra Kalisz, Anthony R. Bradley & Charlotte M. Deane

Authors
  1. Maranga Mokaya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fergus Imrie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Willem P. van Hoorn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aleksandra Kalisz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Anthony R. Bradley
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Charlotte M. Deane
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M.M. developed the code. M.M., F.I., A.R.B. and C.M.D. designed the experiments. M.M. performed the experiments and analyses. M.M. wrote the manuscript and all of the other authors revised it. A.R.B. and C.M.D. supervised the work. All of the authors read and approved of the final version of the manuscript.

Corresponding author

Correspondence to Charlotte M. Deane.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Machine Intelligence thanks the anonymous reviewers for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Simple and complex target substructures.

Diagrams of (a) simple and (b) complex target structures used for Tanimoto similarity and substructure generation experiments.

Extended Data Fig. 2 Similarity distributions of generated sets in for simple and complex optimization experiments.

Generating molecules similar to (a) a simple target molecule, (b) a complex molecule without diversity filters and (c) a complex molecule with diversity filters. SrIOP (blue), ReCL (orange), training (dotted) and Prior (green) distributions for each optimization task. Both methods can produce entire datasets that match the simple target structure, however, only SrIOP is able to generate molecules similar to the complex target structure.

Extended Data Table 1 Mean property values for molecules generated from four property optimization tasks using increasing training data percentage representations
Full size table
Extended Data Table 2 Comparison of generated set composition of each agent trained during SrIOP and REINVENT for TPSA optimization
Full size table
Extended Data Table 3 Proportion of failed substructure generation attempts using different SMILES across all molecules tested using ReCL and SrIOP
Full size table

Supplementary information

Supplementary Information

Supplementary Figs. 1–14, Sections 1–7, Discussion and Tables 1–3.

Reporting Summary

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mokaya, M., Imrie, F., van Hoorn, W.P. et al. Testing the limits of SMILES-based de novo molecular generation with curriculum and deep reinforcement learning. Nat Mach Intell 5, 386–394 (2023). https://doi.org/10.1038/s42256-023-00636-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-023-00636-2

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing