Amanda Randles
Erik W. Draeger
ABSTRACT
The potential impact of blood flow simulations on the diagnosis and treatment of patients suffering from vascular disease is tremendous. Empowering models of the full arterial tree can provide insight into diseases such as arterial hypertension and enables the study of the influence of local factors on global hemodynamics. We present a new, highly scalable implementation of the lattice Boltzmann method which addresses key challenges such as multiscale coupling, limited memory capacity and bandwidth, and robust load balancing in complex geometries. We demonstrate the strong scaling of a three-dimensional, high-resolution simulation of hemodynamics in the systemic arterial tree on 1,572,864 cores of Blue Gene/Q. Faster calculation of flow in full arterial networks enables unprecedented risk stratification on a perpatient basis. In pursuit of this goal, we have introduced computational advances that significantly reduce time-to-solution for biofluidic simulations.
- J. Alastruey, A. W. Khir, K. S. Matthys, P. Segers, S. J. Sherwin, P. R. Verdonck, K. H. Parker, and J. Peiró. Pulse wave propagation in a model human arterial network: Assessment of 1-d visco-elastic simulations against in vitromeasurements. Journal of Biomechanics, 44(12):2250--2258, 2011.Google Scholar
Cross Ref
- J. Baerentzen and H. Aanaes. Signed distance computation using the angle weighted pseudonormal. IEEE Transactions on Visualization and Computer Graphics, 11(3):243--253, 2005. Google ScholarDigital Library
- M. Bernaschi, M. Bisson, T. Endo, S. Matsuoka, M. Fatica, and S. Melchionna. Petaflop biofluidics simulations on a two million-core system. In Proceedings of the 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, page 4. ACM, 2011. Google ScholarDigital Library
- J. Berry, W. Borden, D. Bravata, S. Dai, E. Ford, et al. Heart disease and stroke statistics?2012 update. Circulation, 125:e2--e220, 2012.Google Scholar
- J. Carter, M. Soe, L. Oliker, Y. Tsuda, G. Vahala, L. Vahala, and A. Macnab. Magnetohydrodynamic turbulence simulations on the earth simulator using the lattice Boltzmann method. In Proceedings of the 2005 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, SC '05. IEEE Computer Society, 2005.Google Scholar
- J. Cebral, M. Castro, J. Burgess, R. Pergolizzi, M. Sheridan, and C. Putman. Characterization of cerebral aneurysms for assessing risk of rupture by using patient-specific computational hemodynamics models. American Journal of Neuroradiology, 26(10):2550--2559, 2005.Google Scholar
- A. B. I. Collaboration et al. Ankle brachial index combined with Framingham risk score to predict cardiovascular events and mortality: a meta-analysis. JAMA: the Journal of the American Medical Association, 300(2):197, 2008.Google ScholarCross Ref
- J. S. Coogan, J. D. Humphrey, and C. A. Figueroa. Computational simulations of hemodynamic changes within thoracic, coronary, and cerebral arteries following early wall remodeling in response to distal aortic coarctation. Biomechanics and modeling in mechanobiology, 12(1):79--93, 2013.Google Scholar
- A. V. Doobay and S. S. Anand. Sensitivity and specificity of the ankle--brachial index to predict future cardiovascular outcomes a systematic review. Arteriosclerosis, Thrombosis, and Vascular Biology, 25(7):1463--1469, 2005.Google Scholar
- C. Godenschwager, F. Schornbaum, M. Bauer, H. Köstler, and U. Rüde. A framework for hybrid parallel flow simulations with a trillion cells in complex geometries. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, page 35. ACM, 2013. Google ScholarDigital Library
- L. Grinberg, D. Fedosov, and G. Karniadakis. Parallel multiscale simulations of a brain aneurysm. Journal of Computational Physics, 2012. Google ScholarDigital Library
- L. Grinberg, V. Morozov, D. Fedosov, J. Insley, M. Papka, K. Kumaran, and G. Karniadakis. A new computational paradigm in multiscale simulations: Application to brain blood flow. In Proceedings of the 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1--12. IEEE, 2011. Google ScholarDigital Library
- R. Haring, M. Ohmacht, T. W. Fox, M. K. Gschwind, D. L. Satterfield, K. Sugavanam, P. W. Coteus, P. Heidelberger, M. A. Blumrich, R. W. Wisniewski, et al. The IBM Blue Gene/Q compute chip. Micro, IEEE, 32(2):48--60, 2012. Google ScholarDigital Library
- M. Hecht and J. Harting. Implementation of on-site velocity boundary conditions for D3Q19 lattice Boltzmann simulations. Journal of Statistical Mechanics: Theory and Experiment, 2010(01):P01018, 2010.Google ScholarCross Ref
- F. Higuera, S. Succi, and R. Benzi. Lattice gas dynamics with enhanced collisions. EPL (Europhysics Letters), 9(4):345--349, 1989.Google Scholar
- T. J. Hughes and J. Lubliner. On the one-dimensional theory of blood flow in the larger vessels. Mathematical Biosciences, 18(1):161--170, 1973.Google ScholarCross Ref
- O. Malaspinas, B. Chopard, and J. Latt. General regularized boundary condition for multi-speed lattice Boltzmann models. Computers & Fluids, 49(1):29--35, 2011.Google ScholarCross Ref
- A. L. Marsden, A. J. Bernstein, V. M. Reddy, S. C. Shadden, R. L. Spilker, F. P. Chan, C. A. Taylor, and J. A. Feinstein. Evaluation of a novel y-shaped extracardiac fontan baffle using computational fluid dynamics. The Journal of Thoracic and Cardiovascular Surgery, 137(2):394--403, 2009.Google ScholarCross Ref
- G. McNamara and G. Zanetti. Use of the Boltzmann equation to simulate lattice-gas automata. Physical Review Letters, 61(20):2332--2335, 1988.Google ScholarCross Ref
- S. Melchionna, M. Bernaschi, S. Succi, E. Kaxiras, F. J. Rybicki, D. Mitsouras, A. U. Coskun, and C. L. Feldman. Hydrokinetic approach to large-scale cardiovascular blood flow. Computer Physics Communications, 181(3):462--472, 2010.Google ScholarCross Ref
- J. M. Murabito, R. B. D?Agostino, H. Silbershatz, and P. W. Wilson. Intermittent claudication a risk profile from the framingham heart study. Circulation, 96(1):44--49, 1997.Google ScholarCross Ref
- J. M. Murabito, J. C. Evans, M. G. Larson, K. Nieto, D. Levy, and P. W. Wilson. The ankle-brachial index in the elderly and risk of stroke, coronary disease, and death: the framingham study. Archives of Internal Medicine, 163(16):1939--1942, 2003.Google ScholarCross Ref
- C. M. Papamichael, J. P. Lekakis, K. S. Stamatelopoulos, T. G. Papaioannou, M. K. Alevizaki, A. T. Cimponeriu, J. E. Kanakakis, A. Papapanagiotou, A. T. Kalofoutis, and S. F. Stamatelopoulos. Ankle-brachial index as a predictor of the extent of coronary atherosclerosis and cardiovascular events in patients with coronary artery disease. The American journal of cardiology, 86(6):615--618, 2000.Google Scholar
- K. Pekkan, B. Whited, K. Kanter, S. Sharma, D. De Zelicourt, K. Sundareswaran, D. Frakes, J. Rossignac, and A. Yoganathan. Patient-specific surgical planning and hemodynamic computational fluid dynamics optimization through free-form haptic anatomy editing tool (surgem). Medical & Biological Engineering & Computing, 46(11):1139--1152, 2008.Google ScholarCross Ref
- C. S. Peskin. Numerical analysis of blood flow in the heart. Journal of Computational Physics, 25(3):220--252, 1977.Google ScholarCross Ref
- A. Peters, S. Melchionna, E. Kaxiras, J. Lätt, J. Sircar, M. Bernaschi, M. Bison, and S. Succi. Multiscale simulation of cardiovascular flows on the IBM Blue Gene/P: Full heart-circulation system at red-blood cell resolution. In Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, SC '10, 2010. Google ScholarDigital Library
- A. Peters Randles, V. Kale, J. R. Hammond, W. Gropp, and E. Kaxiras. Performance analysis of the lattice Boltzmann model beyond Navier-Stokes. In Proceedings of the 27th IEEE International Parallel and Distributed Processing Symposium, IPDPS '13, 2013. Google ScholarDigital Library
- T. Pohl, F. Deserno, N. Thurey, U. Rude, P. Lammers, G. Wellein, and T. Zeiser. Performance evaluation of parallel large-scale lattice Boltzmann applications on three supercomputing architectures. In Proceedings of the 2004 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, SC '04. IEEE Computer Society, 2004. Google ScholarDigital Library
- I. Rahimian, A. and Lashuk, S. Veerapaneni, A. Chandramowlishwaran, D. Malhotra, L. Moon, R. Sampath, A. Shringarpure, J. Vetter, R. Vuduc, et al. Petascale direct numerical simulation of blood flow on 200k cores and heterogeneous architectures. In Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1--11. IEEE Computer Society, 2010. Google ScholarDigital Library
- A. Randles, E. Draeger, and P. Bailey. Massively parallel simulations of hemodynamics in the primary large arteries of the human vasculature. In Journal of Computational Science, ICCS15, 2015. accepted.Google ScholarCross Ref
- H. E. Resnick, R. S. Lindsay, M. M. McDermott, R. B. Devereux, K. L. Jones, R. R. Fabsitz, and B. V. Howard. Relationship of high and low ankle brachial index to all-cause and cardiovascular disease mortality the strong heart study. Circulation, 109(6):733--739, 2004.Google ScholarCross Ref
- P. Reymond, F. Merenda, F. Perren, D. Rüfenacht, and N. Stergiopulos. Validation of a one-dimensional model of the systemic arterial tree. American Journal of Physiology-Heart and Circulatory Physiology, 297(1):H208--H222, 2009.Google ScholarCross Ref
- G. D. Smith, M. Shipley, and G. Rose. Intermittent claudication, heart disease risk factors, and mortality. the Whitehall study. Circulation, 82(6):1925--1931, 1990.Google ScholarCross Ref
- N. Stergiopulos, D. Young, and T. Rogge. Computer simulation of arterial flow with applications to arterial and aortic stenoses. Journal of Biomechanics, 25(12):1477--1488, 1992.Google ScholarCross Ref
- S. Succi. The Lattice Boltzmann Equation for Fluid Dynamics and Beyond. Oxford University Press, 2001.Google ScholarCross Ref
- C. Taylor, T. Hughes, and C. Zarins. Finite element modeling of blood flow in arteries. Computer Methods in Applied Mechanics and Engineering, 158(1):155--196, 1998.Google ScholarCross Ref
- I. B. G. team. The IBM Blue Gene project. IBM Journal of Research and Development, 57(1/2):0, 2013. Google ScholarDigital Library
- N. Westerhof, F. Bosman, C. J. De Vries, and A. Noordergraaf. Analog studies of the human systemic arterial tree. Journal of Biomechanics, 2(2):121--143, 1969.Google ScholarCross Ref
- S. Williams, L. Oliker, J. Carter, and J. Shalf. Extracting ultra-scale lattice Boltzmann performance via hierarchical and distributed auto-tuning. In Proceedings of the 2011 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, SC '11, pages 1--10. IEEE Computer Society, 2011. Google ScholarDigital Library
- A. J. Wood and W. R. Hiatt. Medical treatment of peripheral arterial disease and claudication. New England Journal of Medicine, 344(21):1608--1621, 2001.Google ScholarCross Ref
- N. Xiao, J. Humphrey, and C. Figueroa. Multi-scale computational model of three-dimensional hemodynamics within a deformable full-body arterial network. Journal of Computational Physics, 244:22--40, 2013. Google ScholarDigital Library
- G. Xiong and C. A. Taylor. Virtual stent grafting in personalized surgical planning for treatment of aortic aneurysms using image-based computational fluid dynamics. In Medical Image Computing and Computer-Assisted Intervention--MICCAI 2010, pages 375--382. Springer, 2010. Google ScholarDigital Library
- Q. Zou and X. He. On pressure and velocity boundary conditions for the lattice Boltzmann BGK model. Physics of Fluids, 9:1591, 1997.Google ScholarCross Ref
Index Terms
- Massively parallel models of the human circulatory system
Recommendations
1D-model of the human liver circulatory system
Highlights- A novel integrated 1D model of the entire liver circulatory system is proposed.
Graphical abstract▪
Abstract Background and objectiveBlood flow rate and pressure can be measured in vivo by invasive and non-invasive techniques in the large vessels of the hepatic vasculature, but it is not possible to do so along the entire liver ...
The development of a physiological simulation system for the human circulatory system coupling macro and micro models
This paper describes the development of a physiological simulation model which couples macro and micro models of the biological circulatory system. The macro model for the circulatory system is composed of modules for physiological functions such as ...
Mechanical ventilation and thoracic artificial lung assistance during mechanical circulatory support with PUCA pump: In silico study
Patients assisted with left ventricular assist device (LVAD) may require prolonged mechanical ventilatory assistance secondary to postoperative respiratory failure. The goal of this work is the study of the interdependent effects LVAD like pulsatile ...
Comments