Skip to main content
SpringerLink
Log in

Basic inequalities for weighted entropies

  • Published:
Aequationes mathematicae Aims and scope Submit manuscript

Abstract

The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. In this paper, we establish a number of simple inequalities for the weighted entropies (general as well as specific), mirroring similar bounds on standard (Shannon) entropies and related quantities. The required assumptions are written in terms of various expectations of weight functions. Examples are weighted Ky Fan and weighted Hadamard inequalities involving determinants of positive-definite matrices, and weighted Cramér-Rao inequalities involving the weighted Fisher information matrix.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Belis M., Guiasu S.: A Quantitative and qualitative measure of information in cybernetic systems. IEEE Trans. Inf. Theory 14, 593–594 (1968)

    Article  Google Scholar 

  2. Clim, A.: Weighted entropy with application. In: Analele Universităţii Bucureşti, Matematică, Anul LVII, pp. 223–231 (2008)

  3. Cover T., Thomas J.: Elements of Information Theory. Wiley, New York (2006)

    MATH  Google Scholar 

  4. Cover, T.M., Thomas, J.A.: Determinant inequalities via information theory. SIAM J. Matrix Anal. Appl. 9, 384–392 (1988)

  5. Dembo A., Cover T.M., Thomas J.A.: Information theoretic inequalities. IEEE Trans. Inf. Theory 37, 1501–1518 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  6. Di Crescenzo A., Longobardi M.: Entropy based measure of uncertainty in past lifetime distributions. J. Appl. Prob. 39(3), 434–440 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  7. Dial G., Taneja I.J.: On weighted entropy of type (α,β) and its generalizations. Appl. Math. 26, 418–425 (1981)

    MathSciNet  MATH  Google Scholar 

  8. Frizelle G., Suhov Y.M.: An entropic measurement of queueing behaviour in a class of manufacturing operations. Proc. R. Soc. A 457, 1579–1601 (2001)

    Article  MATH  Google Scholar 

  9. Frizelle G., Suhov Y.M.: The measurement of complexity in production and other commercial systems. Proc. R. Soc. A 464, 2649–2668 (2008)

    Article  Google Scholar 

  10. Fuchs, A., Letta, G.: L’inégalité de Kullback. Application à la théorie de l’estimation. Séminaire de probabilités 4, pp. 108–131. Strasbourg (1970)

  11. Guiasu S.: Weighted entropy. Rep. Math. Phys. 2, 165–179 (1971)

    Article  MathSciNet  MATH  Google Scholar 

  12. Ito K.: Introduction to Probability Theory. Cambridge University Press, Cambridge (1984)

    MATH  Google Scholar 

  13. Johnson D.H., Glantz R.M.: When does interval coding occur? Neurocopmuting 59:60, 13–18 (2004)

    Article  MathSciNet  Google Scholar 

  14. Kannappan P.L., Sahoo P.K.: On the general solution of a functional equation connected to sum form information measures on open domain. Math. Sci. 9, 545–550 (1986)

    MathSciNet  MATH  Google Scholar 

  15. Kapur J.N.: Measures of Information and Their Applications, Chapter 17. Wiley, New Delhi (1994)

    MATH  Google Scholar 

  16. Fan K.: On a theorem of Weyl concerning eigenvalues of linear transformations. I. Proc. Natl. Acad. USA 35, 652–655 (1949)

    Article  MathSciNet  Google Scholar 

  17. Fan K.: On a theorem of Weyl concerning eigenvalues of linear transformations. II. Proc. Natl. Acad. USA 36, 31–35 (1950)

    Article  MathSciNet  Google Scholar 

  18. Fan K.: Maximum properies and inequalities for the eigenvalues of completely continuous operators. Proc. Natl. Acad. USA 37, 760–766 (1951)

    Article  MATH  Google Scholar 

  19. Kelbert, M., Suhov, Y.: Continuity of mutual entropy in the limiting signal-to-noise ratio regimes. In: Stochastic Analysis, pp. 281–299. Springer, Berlin (2010)

  20. Kelbert M., Suhov Y.: Information Theory and Coding by Example. Cambridge University Press, Cambridge (2013)

    Book  MATH  Google Scholar 

  21. Moslehian, M.: Ky Fan inequalities. arXiv:1108.1467 (2011)

  22. Muandet, K., Marukatat, S., Nattee, C.: Query selection via weighted entropy in graph-based semi-supervised classification. In: Advances in Machine Learning. Lecture Notes in Computer Science, vol. 5828, pp. 278–292 (2009)

  23. Parkash O., Taneja H.C.: Characterization of the quantitative-qualitative measure of inaccuracy for discrete generalized probability distributions. Commun. Stat. Theory Methods 15, 3763–3771 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  24. Sharma B.D., Mitter J., Mohan M.: On measure of ‘useful‘ information. Inf. Control 39, 323–336 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  25. Singh R.P., Bhardwaj J.D.: On parametric weighted information improvement. Inf. Sci. 59, 149–163 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  26. Sreevally A., Varma S.K.: Generating measure of cross entropy by using measure of weighted entropy. Soochow J. Math. 30(2), 237–243 (2004)

    MathSciNet  MATH  Google Scholar 

  27. Srivastava A.: Some new bounds of weighted entropy measures. Cybern. Inf. Technol. 11(3), 60–65 (2011)

    MathSciNet  Google Scholar 

  28. Suhov, Y., Yasaei Sekeh, S., Kelbert, M.: Entropy-power inequality for weighted entropy. arXiv:1502.02188

  29. Suhov, Y., Stuhl, I., Yasaei Sekeh, S.: Weighted Gaussian entropy and determinant inequalities. arXiv:1505.01753

  30. Suhov, Y., Stuhl, I., Kelbert, M.: Weight functions and log-optimal investment portfolios. arXiv:1505.01437

  31. Tuteja R.K., Chaudhary Sh., Jain P.: Weighted entropy of orders α and type β information energy. Soochow J. Math. 19(2), 129–138 (1993)

    MathSciNet  MATH  Google Scholar 

  32. Zamir R.: A proof of the Fisher information inequality via a data processing argument. IEEE Trans. Inf. Theory 44(3), 1246–1250 (1998)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. DPMMS, University of Cambridge, Cambridge, UK

    Yuri Suhov

  2. Mathematics Department, Penn State University, Philadelphia, PA, USA

    Yuri Suhov

  3. IPIT RAS, Moscow, Russian Federation

    Yuri Suhov

  4. IMS, University of São Paulo, São Paulo, SP, Brazil

    Izabella Stuhl

  5. Mathematics Department, University of Denver, Denver, CO, USA

    Izabella Stuhl

  6. University of Debrecen, Debrecen, Hungary

    Izabella Stuhl

  7. Statistics Department, Federal University of São Carlos, São Carlos, SP, Brazil

    Salimeh Yasaei Sekeh

  8. Mathematics Department, University of Swansea, Swansea, UK

    Mark Kelbert

  9. Moscow Higher School of Economics, Moscow, Russian Federation

    Mark Kelbert

Authors
  1. Yuri Suhov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Izabella Stuhl
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Salimeh Yasaei Sekeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mark Kelbert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Izabella Stuhl.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Suhov, Y., Stuhl, I., Yasaei Sekeh, S. et al. Basic inequalities for weighted entropies. Aequat. Math. 90, 817–848 (2016). https://doi.org/10.1007/s00010-015-0396-5

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00010-015-0396-5

Mathematics Subject Classification

Keywords

Search

Navigation