Newsletter

Newsletter
 

3D Mantle Convection


As evidenced by the worldwide proliferation of conferences and workshops devoted to the use of graphics processing units (GPUs) in mainstream high-performance computing, GPUs are revolutionizing intermediate-sized computational runs. GPUs are much more cost effective and energy efficient than the Beowulf clusters data analysts have relied on for the past 15 years. With the first petascale GPU computational facility, the Tianhe, recently established in Changsa, China, we can rightfully hail the GPU movement as a shift in computational paradigm.

Many fields in science, such as molecular biology, quantum chemistry, theoretical physics, and atmospheric and ocean sciences have taken on GPUs in solving non-trivial partial differential and integral equations. Solid-earth geophysics remains one of the last bastions to have resisted the use of GPUs, especially in geodynamics. The authors aim in this work was to introduce GPUs as a viable tool in 3D mantle convection modeling. They illustrate this with a standard second-order finite-difference method and compare results from a CPU with those from a GPU using the NVIDIA Tesla C1060. The convection model they elected to compare is a constant viscosity basally heated 3-D model in Cartesian geometry.

The following assumptions were made for the convection model:

  • Boussinesq approximation
  • Fluid is incompressible
  • Basally heated
  • Viscosity of fluid constant
  • Cartesian geometry

The model equations, numerical model, and solution for solving momentum equations are illustrated in the following charts:


The authors chose MATLAB® and Jacket for the implementation of the work. With this solution the team quickly and easily implemented the model and comparison objectives. Code development took 10 weeks and provided direct access to CUBLAS and CUFFT as well as parallel for-loop execution. "Basically writing MATLAB for the GPU" was the authors' objective. The authors noted that performance, memory management, and the high cost of indexing were disadvantages of other lower level language implementations such as C, C++ and CUDA.



Hardware utilized for the work included a personal supercomputer consisting of one (1) Intel Core i7 960, three(3) NVIDIA Tesla C1060 GPUs, one (1) Nvidia Quadro GPU for visualization, 12 gigabytes of RAM, and 5 terabytes of striped storage. The authors noted the pricing of this personal supercomputer to be about $6,500US.





The following charts illustrate the comparison of the momentum equation solver on CPUs versus GPUs, a model validation and performance explanation, and visualizations of the results:



Conclusion:


The authors successfully, with Jacket and MATLAB, implemented a second order finite difference model on the GPU in single and double precision for constant viscosity mantle convection which uses a direct Fourier-based solver for the momentum equations. The project demonstrated superior speed of the discrete cosine and sine transforms (matrix-matrix products) over fast transforms (based on the FFT) for directly solving the coupled Poisson equations. Discrete solvers on GPUs proved to be significantly faster than just fast solvers. They also validated the numerical model in single and double precision on the GPU against the parallel finite-element CPU model of Zhong (2005) and presented high-resolution results from a double precision GPU simulation at Ra=10 to the 7th.

  • For the 1x1x1 FD model, speed improvements of up to 2.53X and 3.45X were achieved when run in double and single precision on the GPU, respectively.
  • For the 2x2x1 FD model, speed improvements of up to 2.88X and 4.48X were achieved.

Authors:

  • Grady B. Wright Dept. of Mathematics , Boise State University, Boise, Idaho
  • David Yuen, Minnesota Supercomputing Institute, University of Minnesota, Minnesota
  • Gregory A. Barnett, University of Colorado, Boulder, Boulder, Colorado


References:

  • [2] J.M. Cohen and M.J. Molemaker, A Fast Double Precision CFD Code using CUDA. Proceedings of Parallel CFD 2009, DESTech; extended abstract.
  • [3] J. C. Thibault and I. Senocak. CUDA implementation of a Navier-Stokes solver on multi-GPU desktop platforms for incompressible flows. 47th AIAA Aerospace Sciences Meeting, paper no: AIAA-2009-758, 2009.
  • [4] C. van Loan. Computational Frameworks for the Fast Fourier Transform. SIAM, Philadelphia, 1992.
  • [5] S. Zhong. Dynamics of thermal plumes in three-dimensional isoviscous thermal convection. Geophys. J. Int., 162, 289-300, 2005.



« Back to Case Studies