The use of mesh refinement in CFD is an efficient and widely used methodology to minimize the computational cost by solving those regions of high geometrical complexity with a finer grid. The author focuses on studying two methods, one based on multi-domain and one based on irregular meshing, to deal with mesh refinement over LBM simulations. The numerical formulation is presented in detail. Two approaches, homogeneous GPU and heterogeneous CPU+GPU, on each of the refinement methods are studied. Obviously, the use of the two architectures, CPU and GPU, to compute the same problem involves more important challenges with respect to the homogeneous counterpart. These strategies are described in detail paying particular attention to the differences among both methodologies in terms of programmability, memory management, and performance.
TopIntroduction
Advanced strategies for the efficient implementation of computationally intensive numerical methods have a strong interest in the industrial and academic community. We could define Computational Fluid Dynamics (CFD) as a set of numerical methods applied to obtain approximate solutions of problems of fluid dynamics and heat transfer (Zikanov, 2010). The CFD community has always explored new ways to take advantage of high-performance computing systems in its never-ending quest for faster and more accurate simulations. The emergence of Graphics Processing Units (GPUs) has been an important advance in this field and it has created new challenges and opportunities for increasing performance in multiple CFD solvers. Many CFD applications and software packages have already been ported and redesigned to exploit GPUs. These developments have often involved major changes because some classical solvers may turned out to be inefficient or difficult to tune (Valero-Lara, Pinelli, Favier, & Prieto-Matías, 2012) (Valero-Lara, Pinelli, & Prieto-Matías, 2014). Fortunately, other solvers are particularly well suited for GPU acceleration and are able to achieve significant performance improvements. The Lattice Boltzmann method (LBM) (Succi, 2001) is one of those examples thanks to its inherently data-parallel nature. Certainly, the computing stages of LBM are amenable to fine grain parallelization in an almost straightforward way.
This fundamental advantage of LBM has been consistently confirmed by many authors (Bernaschi, Fatica, Melchionna, Succi, & Kaxiras, 2010) (Rinaldi, Dari, Vnere, & Clausse, 2012) (Zhou, Mo, Wu, & Zhao, 2012) (Feichtinger, Habich, Kstler, Rude, & Aoki, 2015), for a large variety of problems and computing platforms. For instance, in (Pohl, Kowarchik, Wilke, Rüde, & Iglgerger, 2003) is proposed a set of possible memory access patterns to maximize the temporal locality optimizing the cache performance over multicore architectures. Also in (Rinaldi, Dari, Vnere, & Clausse, 2012) is modified the standard ordering of the LBM steps to reduce the number of memory accesses. Lattice-Boltzmann method has been ported on multiple parallel architectures, such as multicore processors (Pohl, Kowarchik, Wilke, Rüde, & Iglgerger, 2003), manycore accelerators (Bernaschi, Fatica, Melchionna, Succi, & Kaxiras, 2010) (Rinaldi, Dari, Vnere, & Clausse, 2012) (Alexandrov, Lees, Krzhizhanovskaya, Dongarra, Sloot, Crimi, Mantovani, Pivanti, Schifano, & Tripiccione, 2013) (Valero-Lara, Igual, Prieto-Matías, Pinelli, & Favier, 2015) and distributed-memory clusters (Januszewski, & Kostur, 2014). Given the growing popularity of LBM, multiple tools (Januszewski, & Kostur, 2014) have been developed recently, consolidating this method in academia and industry. In particular, in this work, we have considered LBM-HPC framework (Valero-Lara, 2016) as our reference software tool. Lattice-Boltzmann method is an efficient and fast method; however, the usage of Cartesian grids is expensive. Although scientific problems exist for which a homogeneous description of the domain is a reasonable choice, it is usually desirable to solve regions of high geometrical complexity with a