Skip to main content
SpringerLink
Log in

zoNNscan: A Boundary-Entropy Index for Zone Inspection of Neural Models

  • Conference paper
  • First Online:
Monte Carlo Search (MCS 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1379))

Included in the following conference series:

  • 196 Accesses

Abstract

The training of deep neural network classifiers results in decision boundaries whose geometry is still not well understood. This is in direct relation with classification problems such as so called corner case inputs. We introduce zoNNscan, an index that is intended to inform on the boundary uncertainty (in terms of the presence of other classes) around one given input datapoint. It is based on confidence entropy, and is implemented through Monte Carlo sampling in the multidimensional ball surrounding that input. We detail the zoNNscan index, give an algorithm for approximating it, and finally illustrate its benefits on three applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    For more advanced sampling techniques in high dimensional spaces, please refer to e.g., [5].

  2. 2.

    Note that for \(X\in [0,1]^d\) and for \(r\ge 1\), \(\mathbf {B}_{\boldsymbol{\infty } }(X,r)\cap [0,1]^d = [0,1]^d\), then the space of normalized data is totally covered by the sampling process.

References

  1. Keras: Deep learning for humans. https://github.com/keras-team/keras/tree/master/examples. Accessed 25 Feb 2020

  2. Plotting high-dimensional decision boundaries. https://github.com/tmadl/highdimensional-decision-boundary-plot. Accessed 1 July 2018

  3. Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24 (2002). https://doi.org/10.1109/34.993558

  4. van den Berg, E.: Some insights into the geometry and training of neural networks. CoRR arXiv:1605.00329 (2016)

  5. Dick, J., Kuo, F.Y., Sloan, I.H.: High-dimensional integration: the quasi-monte carlo way. Acta Numer. 22, 133–288 (2013). https://doi.org/10.1017/S0962492913000044

    Article  MathSciNet  MATH  Google Scholar 

  6. Fawzi, A., Moosavi-Dezfooli, S., Frossard, P., Soatto, S.: Classification regions of deep neural networks. CoRR arXiv:1705.09552 (2017)

  7. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: ICLR (2015)

    Google Scholar 

  8. Guo, G., Zhang, H.J., Li, S.Z.: Distance-from-boundary as a metric for texture image retrieval. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 3, pp. 1629–1632 (2001). https://doi.org/10.1109/ICASSP.2001.941248

  9. Kuncheva, L., Whitaker, C., Shipp, C., Duin, R.: Limits on the majority vote accuracy in classifier fusion. Pattern Anal. Appl. 6(1), 22–31 (2003)

    Article  MathSciNet  Google Scholar 

  10. Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 6402–6413. Curran Associates, Inc. (2017). http://papers.nips.cc/paper/7219-simple-and-scalable-predictive-uncertainty-estimation-using-deep-ensembles.pdf

  11. Le Merrer, E., Pérez, P., Trédan, G.: Adversarial frontier stitching for remote neural network watermarking. Neural Comput. Appl. 32(13), 9233–9244 (2019). https://doi.org/10.1007/s00521-019-04434-z

    Article  Google Scholar 

  12. Lee, C., Landgrebe, D.A.: Feature extraction based on decision boundaries. IEEE Trans. Pattern Anal. Mach. Intell. 15(4), 388–400 (1993)

    Article  Google Scholar 

  13. Lee, C., Landgrebe, D.A.: Decision boundary feature extraction for neural networks. IEEE Trans. Neural Netw. 8(1), 75–83 (1997). https://doi.org/10.1109/72.554193

    Article  Google Scholar 

  14. Liu, Y., et al.: Trojaning attack on neural networks. In: NDSS (2018)

    Google Scholar 

  15. Mann, H.B., Whitney, D.R.: On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18(1), 50–60 (1947). https://doi.org/10.1214/aoms/1177730491

  16. Moosavi-Dezfooli, S., Fawzi, A., Fawzi, O., Frossard, P.: Universal adversarial perturbations. In: CVPR (2017)

    Google Scholar 

  17. Pei, K., Cao, Y., Yang, J., Jana, S.: Deepxplore: automated whitebox testing of deep learning systems. In: SOSP (2017)

    Google Scholar 

  18. Wang, Z., Bovik, A.C.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002). https://doi.org/10.1109/97.995823

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Technicolor, Paris, France

    Adel Jaouen

  2. Univ Rennes, Inria, CNRS, Irisa, Rennes, France

    Erwan Le Merrer

Authors
  1. Adel Jaouen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Erwan Le Merrer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erwan Le Merrer .

Editor information

Editors and Affiliations

  1. Université Paris-Dauphine, Paris, France

    Tristan Cazenave

  2. Facebook FAIR, Paris, France

    Olivier Teytaud

  3. Maastricht University, Maastricht, Limburg, The Netherlands

    Mark H. M. Winands

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jaouen, A., Le Merrer, E. (2021). zoNNscan: A Boundary-Entropy Index for Zone Inspection of Neural Models. In: Cazenave, T., Teytaud, O., Winands, M.H.M. (eds) Monte Carlo Search. MCS 2020. Communications in Computer and Information Science, vol 1379. Springer, Cham. https://doi.org/10.1007/978-3-030-89453-5_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-89453-5_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-89452-8

  • Online ISBN: 978-3-030-89453-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation