Skip to main content

Advertisement

SpringerLink
Log in

Optimizing performance-per-watt on GPUs in high performance computing

Temperature, frequency and voltage effects

  • Special Issue Paper
  • Published:
Computer Science - Research and Development

Abstract

The magnitude of the real-time digital signal processing challenge attached to large radio astronomical antenna arrays motivates use of high performance computing (HPC) systems. The need for high power efficiency at remote observatory sites parallels that in HPC broadly, where efficiency is a critical metric. We investigate how the performance-per-watt of graphics processing units (GPUs) is affected by temperature, core clock frequency and voltage. Our results highlight how the underlying physical processes that govern transistor operation affect power efficiency. In particular, we show experimentally that GPU power consumption increases non-linearly (quadratic) with both temperature and supply voltage, as predicted by physical transistor models. We show lowering GPU supply voltage and increasing clock frequency while maintaining a low die temperature increases the power efficiency of an NVIDIA K20 GPU by up to 37–48 % over default settings when running xGPU, a compute-bound code used in radio astronomy. We discuss how automatic temperature-aware and application-dependent voltage and frequency scaling (T-DVFS and A-DVFS) may provide a mechanism to achieve better power efficiency for a wider range of compute codes running on GPUs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://www.green500.org/lists/green201411.

  2. http://www.top500.org/lists/2014/11/.

  3. http://www.skatelescope.org.

  4. https://developer.nvidia.com/nvidia-system-management-interface.

  5. http://www.techpowerup.com/gpuz/.

  6. http://www.softpedia.com/get/System/Benchmarks/Kepler-BIOS-Tweaker.shtml.

  7. https://github.com/GPU-correlators/xGPU.

  8. http://www.nvidia.com/content/PDF/kepler/nvidia-gpu-boost-tesla-k40-06767-001-v02.pdf.

References

  1. Broekema PC, van Nieuwpoort RV, Bal HE (2012) ExaScale high performance computing in the Square Kilometer Array. In: Workshop on high-performance computing for astronomy data, p. 9. ACM Press, New York

  2. Clark MA, Plante PL, Greenhill LJ (2013) Accelerating radio astronomy cross-correlation with graphics processing units. Int. J High Perform. Comput. Appl. 27(2):178–192

    Article  Google Scholar 

  3. Collange S, Defour D, Tisserand A (2009) Power consumption of GPUs from a software perspective. Computational science—ICCS 2009. Springer, Berlin, Heidelberg, pp 914–923

  4. Ellingson SW, Taylor GB, Craig J et al (2013) The LWA1 radio telescope. IEEE Trans. Antennas Propag. 61(5):2540–2549

    Article  Google Scholar 

  5. Ge R, Vogt R, Majumder J, Alam A et al (2013) Effects of dynamic voltage and frequency scaling on a K20 GPU. In: Parallel processing (ICPP), 42nd International Conference on, pp 826–833

  6. Hong S, Kim H (2010) An integrated GPU power and performance model. ACM SIGARCH Computer Architecture News - ISCA ’10, 38(3):280–289

  7. Januszewskia R, Gillyb L, Yilmazc E, Auweterd A (2013) Cooling-making efficient choices. Tech rep, partnership for advanced computing in Europe

  8. Jiao Y, Lin H, Balaji P, Feng W (2010) Power and performance characterization of computational kernels on the GPU. In: Green computing and communications (GreenCom), IEEE/ACM Int’l Conf on, pp 221–228

  9. Kasichayanula K, Terpstra D, Luszczek P et al (2012) Power aware computing on GPUs. In: Symposium on Application Accelerators in High Performance Computing (SAAHPC), 10–11 July 2012, pp 64–73

  10. Kocz J, Greenhill LJ, Barsdell BR et al (2014) A scalable hybrid 543 FPGA/GPU FX correlator. J Astro. Instrum. 03(01):1450,002

  11. Leng J, Zu Y, Reddi VJ (2014) Energy efficiency benefits of reducing the voltage guardband on the Kepler GPU architecture. In: 10th IEEE workshop on silicon errors in logic—system effects

  12. Liao W, He L (2005) Coupled power and thermal simulation with active cooling. In: Power-aware computer systems, pp 148–163. Springer, Berlin

  13. Liao W, He L, Lepak KM (2005) Temperature and supply voltage aware performance and power modeling at microarchitecture level. IEEE Trans. Comput. Aided Des. Integr. Circ Syst 552. 24(7):1042–1053

  14. Liu Y, Dick RP, Shang L, Yang H (2007) Accurate temperature-dependent integrated circuit leakage power estimation is easy. In: Design, automation and test in Europe conf and exhibition, DATE ’07, pp 1–6

  15. Magro A, Adami KZ, Ord S (2014) Suitability of NVIDIA GPUs for SKA1-low. arXiv:1407.4698v3

  16. Mei X, Yung LS, Zhao K, Chu X (2013) A measurement study of GPU DVFS on energy conservation. In: Workshop on power-aware computing and systems. ACM Press, New York, pp 1–5

  17. Nagasaka H, Maruyama N, Nukada A et al (2010) Statistical power modeling of GPU kernels using performance counters. In: International conference on green computing (green comp), pp 115–122 IEEE

  18. Nugteren C, van den Braak GJ, Corporaal H (2014) Roofline-aware DVFS for GPUs. International workshop on Adaptive Self-tuning Computing Systems. ACM Press, New York, pp 8–10

    Chapter  Google Scholar 

  19. Perley RA, Chandler CJ, Butler BJ, Wrobel JM (2011) The expanded Very Large Array: a new telescope for new science. Astrophys. J. Lett. 739(1):L1

  20. Ren DQ, Suda R (2010) Investigation on the power efficiency of multi-core and GPU processing element in large scale SIMD computation with CUDA. In: International conference on green computing (green comp), pp 309–316. IEEE

  21. Rofouei M, Stathopoulos T, Ryffel S (2008) Energy-aware high performance computing with graphic processing units. In: HotPower’08 proceedings of the 2008 conference on power aware computing and systems

  22. Wootten A, Thompson A (2009) The Atacama Large Millimeter/submillimeter Array. In: Proceedings of the IEEE 97(8):1463–1471

  23. Zimmermann S, Meijer I, Tiwari MK, Paredes S et al (2012) Aquasar: a hot water cooled data center with direct energy reuse. Energy 43(1):237–245

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Harvard-Smithsonian Center for Astrophysics, MS 42, 60 Garden Street, Cambridge, MA, 01238, USA

    D. C. Price, B. R. Barsdell & L. J. Greenhill

  2. NVIDIA, 2701 San Tomas Expy, Santa Clara, CA, 95050, USA

    M. A. Clark & R. Babich

Authors
  1. D. C. Price
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. A. Clark
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. B. R. Barsdell
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Babich
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. L. J. Greenhill
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to D. C. Price.

Additional information

The authors acknowledge support from NSF Grants PHYS-080357, AST-1106059, and OIA-1120587. BB thanks the NVIDIA internship program for support.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Price, D.C., Clark, M.A., Barsdell, B.R. et al. Optimizing performance-per-watt on GPUs in high performance computing. Comput Sci Res Dev 31, 185–193 (2016). https://doi.org/10.1007/s00450-015-0300-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00450-015-0300-5

Keywords

Search

Navigation