Skip to main content
SpringerLink
Log in

The Unbalance Penalisation Method for Metrics of Social Progress

  • Original Research
  • Published:
Social Indicators Research Aims and scope Submit manuscript

Abstract

This study contributes to the debate on the development of aggregate metrics of societal progress. Summarising societal progress into a single number poses various methodological challenges, including the choice of indicators, normalisation, weighting and aggregation. This paper addresses the issue of aggregation in the case of metrics of well-being and uses as a case study the European Union regional Social Progress Index—EU-SPI—published by the European Commission. The index is an aggregate measure of 55 social and environmental indicators observed for all the European regions grouped into 12 components. In metrics of this type, while complete substitutability among components is rarely acceptable, controlling their level of substitutability is highly desirable. To this aim, we adopt a modified version of the unbalance penalisation approach originally proposed by Casadio Tarabusi and Guarini (Soc Indic Res 112:9–45, 2013). A penalisation is applied to the regions whose performance across the index components is unbalanced, that is when they perform well on some components but worse on others. The penalisation applied by this approach depends on two parameters that, in its original formulation, are generally arbitrarily chosen. We design a data-driven approach allowing for an informed choice of the penalisation parameters. The comparison between the EU-SPI original and penalised scores shows that the penalisation effect is particularly evident for regions with a strongly unbalanced profile across the components. The proposed method allows for adjusting the level of substitutability between components when constructing an aggregate metric, an important functionality especially when measuring societal progress for policy-making.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Availability of Data and Material

Data are available at https://ec.europa.eu/regional_policy/en/information/maps/social_progress2020/.

Code Availability

Code is available upon request.

Notes

  1. https://www.method41.com/blog/2018/11/growth-for-the-sake-of-growth-is-the-ideology-of-a-cancer-cell/.

  2. These measures include components that could be valued in monetary terms and, for this reason, are here called “money-denominated”.

  3. https://ec.europa.eu/info/european-pillar-social-rights-0/european-pillar-social-rights-20-principles_en.

  4. The geographical level of the indicators included in the EU-SPI is the NUTS2, defined as the level 2 of the Nomenclature of Units for Territorial Statistics, the hierarchical system defined by Eurostat for dividing up the territory of the EU.

  5. In economics the case \(\left(1-\beta \right)<0\) is appropriate to be used in the aggregation function I when combining complementary good rather than substitutes.

  6. Arrow et al. (1961) analyse some properties of the elasticity of substitution to assess the extent to which, in economics theory, capital and labour are substitutable for each other.

  7. In case of a negative oriented index instead the penalty will be an increase and this property results in quasi-convexity: \(I\left( {\lambda x + (1 - \lambda )x^{\prime}} \right) \le max\left( {I(x),I(x^{\prime})} \right)\,for\,0 < \lambda < 1\).

  8. The EU-SPI uses the min–max normalisation.

  9. The values 0.5 and 1 were also tested for \(\alpha\). They were discarded because led to component and dimension iso-curves either too linear (\(\alpha\) = 0.5) or too sharp-cornered (\(\alpha =1)\). Results are not shown for sake of brevity.

  10. We focus on scores and not on rankings because rankings are mutually dependent.

References

  • Ács, Z., Autio, E., & Szerb, L. (2014). National systems of entrepreneurship: Measurement issues and policy implications. Research Policy, 43(3), 476–494.

    Article  Google Scholar 

  • Annoni, P., & Bolsi, P. (2020). The regional dimension of social progress in Europe: Presenting the new EU Social Progress Index. European Union Regional Policy Working Papers, WP 06/2020. Publications Office of the European Union.

  • Annoni, P., & Weziak-Bialowolska, D. (2016). A measure to target anti-poverty policies in the European Union regions. Applied Research in Quality of Life, 11, 181–207.

    Article  Google Scholar 

  • Annoni, P., Brüggemann, R., & Carlsen, L. (2015). A multidimensional view on poverty in the European Union by partial order theory. Journal of Applied Statistics, 42(3), 535–554.

    Article  Google Scholar 

  • Arrow, K. J., Chenery, H. B., Minhas, B. S., & Solow, R. M. (1961). Capital-labor substitution and economic efficiency. The Review of Economics and Statistics, 43(3), 225–250.

    Article  Google Scholar 

  • Brans, J. P., & De Smet, Y. (2016). PROMETHEE methods. In S. Greco, M. Ehrgott, & J. Figueira (Eds.), Multiple criteria decision analysis: State of the art surveys (pp. 187–219). Springer.

    Chapter  Google Scholar 

  • Brans, J. P., & Vincke, P. (1985). Note—A preference ranking organisation method. Management Science, 31(6), 647–656.

    Article  Google Scholar 

  • Bruggemann, R., Carlsen, L., & Annoni, P. (2017). Incomparable: What now, IV. Incomparabilities: A modeling challenge. In M. Fattore & R. Bruggemann (Eds.), Partial order concepts in applied sciences (pp. 35–47). Springer.

    Chapter  Google Scholar 

  • Canadian Index of Wellbeing. (2016). How are Canadians really doing? The 2016 CIW Report. Canadian Index of Wellbeing and University of Waterloo.

    Google Scholar 

  • Casadio Tarabusi, E., & Guarini, G. (2013). An unbalance adjustment method for development indicators. Social Indicators Research, 112, 9–45.

    Article  Google Scholar 

  • Casadio Tarabusi, E., & Guarini, G. (2016). Level dependence of the adjustment for unbalance and inequality for the Human Development Index. Social Indicators Research, 126, 527–553.

    Article  Google Scholar 

  • Decancq, K., & Lugo, M. A. (2013). Weights in multidimensional indices of wellbeing: An overview. Econometric Reviews, 32, 7–34.

    Article  Google Scholar 

  • Figueira, J. R., Greco, S., Roy, B., & Slowinski, R. (2013). An overview of ELECTRE methods and their recent extensions. Journal of Multi-Criteria Decision Analysis, 20(1–2), 61–85.

    Article  Google Scholar 

  • Figueira, J. R., Mousseau, V., & Roy, B. (2016). ELECTRE methods. In S. Greco, M. Ehrgott, & J. Figueira (Eds.), Multiple criteria decision analysis: State of the art surveys (pp. 155–185). Springer.

    Chapter  Google Scholar 

  • Foster, J. E., McGillivray, M., & Seth, S. (2013). Composite indices: Rank robustness, statistical association, and redundancy. Econometric Reviews, 32(1), 35–56.

    Article  Google Scholar 

  • Greco, S., Ishizaka, A., Tasiou, M., & Torrisi, G. (2019). On the methodological framework of composite indices: A review of the issues of weighting, aggregation, and robustness. Social Indicators Research, 141, 61–94.

    Article  Google Scholar 

  • Helliwell, J. F., Layard, R., Sachs, J., & De Neve, J.-E. (Eds.). (2020). World happiness report 2020. Sustainable Development Solutions Network.

    Google Scholar 

  • Hrotkó J., Rueda-Sabater E., Lang N., & Chin, V. (2019) Measure well-being to improve it: The 2019 sustainable economic development assessment. Boston Consulting Group

  • Kubiszewski, I., Costanza, R., Franco, C., Lawn, P., Talberth, J., Jackson, T., & Aylmer, C. (2013). Beyond GDP: Measuring and achieving global genuine progress. Ecological Economics, 93, 57–68.

    Article  Google Scholar 

  • Kuznets, S. (1934). National income, 1929–1932. 73rd US Congress, 2d session, Senate document no. 124.

  • Lepenies, P. (2016). The power of a single number: A political history of GDP. Columbia University Press.

    Book  Google Scholar 

  • Mazziotta, M., & Pareto, A. (2007). Un indicatore sintetico di dotazione infrastrutturale: il metodo delle penalità per coefficiente di variazione. In Lo sviluppo regionale nell’Unione Europea-Obiettivi, strategie, politiche. Atti della XXVIII Conferenza Italiana di Scienze Regionali. AISRe, Bolzano.

  • Mazziotta, M., & Pareto, A. (2018). Measuring well-being over time: The adjusted Mazziotta–Pareto index versus other non-compensatory indices. Social Indicators Research, 136(3), 967–976.

    Article  Google Scholar 

  • Munda, G. (2005). Multiple criteria decision analysis and sustainable development. In S. Greco, M. Ehrgott, & J. Figueira (Eds.), Multiple criteria decision analysis: State of the art surveys (pp. 953–986). Springer.

    Chapter  Google Scholar 

  • Munda, G. (2012). Choosing aggregation rules for composite indicators. Social Indicators Research, 109, 337–354.

    Article  Google Scholar 

  • Munda, G., & Nardo, M. (2009). Noncompensatory/nonlinear composite indicators for ranking countries: A defensible setting. Applied Economics, 41(12), 1513–1523.

    Article  Google Scholar 

  • Patil, G., & Taillie, C. (2004). Multiple indicators, partially ordered sets and linear extensions: Multi-criterion ranking and prioritization. Environmental and Ecological Statistics, 11, 199–228.

    Article  Google Scholar 

  • Pichon, E., Widuto, A., Dobreva, A., & Jensen, L. (2021). Ten composite indices for policy-making. European Parliamentary Research Service, European Parliament. PE 696.203.

  • Ravaillon, M. (2010). Troubling tradeoff in the Human Development Index. The World Bank Development Research Group, Polici Research Working Paper 5484.

  • Ruiz, N. (2011). Measuring the joint distribution of household’s income, consumption and wealth using nested Atkinson measures. OECD Working Paper, 40, 1–37.

    Google Scholar 

  • Sen, A. (1999). Development as freedom. Oxford University Press.

    Google Scholar 

  • Seth, S. (2009). Inequality, interactions, and human development. Journal of Human Development and Capabilities, 10(3), 375–396.

    Article  Google Scholar 

  • Simon, U., Bruggemann, R., Behrendt, H., Shulenberger, E., & Pudenz, S. (2006). METEOR: A step-by-step procedure to explore effects of indicator aggregation in multi criteria decision aiding—Application to Water Management in Berlin, Germany. Acta Hydrochimica Et Hydrobiologica, 34, 126–136.

    Article  Google Scholar 

  • Social Progress Imperative. (2019). The Social Progress Index 2019. https://www.socialprogress.org/

  • Stiglitz, J. E., Sen, A., & Fitoussi, J.-P. (2009). Report by the commission on the measurement of economic performance and social progress.

  • The Economist Intelligence Unit. (2019). The Global Liveability Index 2019. The Economist Intelligence Unit.

  • UN Environment Programme. (2018). Inclusive wealth report. United Nations, Kenya

  • UNDP. (2010). Human development report 2010: The real wealth of nations—pathways to human development. UNDP. http://hdr.undp.org/en/content/human-development-report-2010

  • UNDP. (2019). Human development report 2019—beyond income, beyond averages, beyond today: Inequalities in human development in the 21st century. UNDP.

    Google Scholar 

Download references

Acknowledgements

Authors are grateful to Prof. Giovanna Boccuzzo, Director of the Department of Statistics of the University of Padua (Italy), for her comments on an earlier version of the paper.

Funding

This research is the outcome of a project partially funded by the Project of Excellence titled “Statistical methods and models for complex data” of the Department of Statistical Sciences, University of Padua that provided financial support to one of the authors during her stay at the European Commission headquarters in Brussels.

Author information

Authors and Affiliations

  1. Directorate-General for Regional and Urban Policy, Policy Development and Economic Analysis Unit, European Commission, Brussels, Belgium

    Paola Annoni

  2. Department of Statistical Sciences, University of Padua, Padua, Italy

    Manuela Scioni

Authors
  1. Paola Annoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manuela Scioni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manuela Scioni.

Ethics declarations

Conflicts of Interest

The Authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Fig. 

Fig. 11
figure 11

Overall structure of the 2020 EU-SPI index (Annoni & Bolsi, 2020)

Full size image

11.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Annoni, P., Scioni, M. The Unbalance Penalisation Method for Metrics of Social Progress. Soc Indic Res 162, 1093–1115 (2022). https://doi.org/10.1007/s11205-021-02876-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11205-021-02876-4

Keywords

Search

Navigation