In the realm of high-performance computing (HPC), also known as supercomputing, the idea of “better, faster, stronger” is only as good as the number of tasks a computer can efficiently run at once or complete before moving on to the next step.
Jordi Wolfson-Pou, a Ph.D. student in the School of Computational Science and Engineering (CSE), is an HPC researcher who has spent the past few months traveling the globe, presenting new insights on an old solution that aims to tackle synchronization bottlenecks in supercomputers.
These new insights highlight the efficacy of using asynchronous multigrid iterative methods for solving large linear systems on exascale computers. The researchers who discovered these findings believe this approach can quicken the computing processes used in a variety of fields, such as physics and engineering.
“Solving equations in physics and engineering often requires highly accurate solutions, which means very large problems need to be solved. This is where supercomputers come in. The next generation of supercomputers will be capable of doing calculations at the exascale and will certainly be fast, but synchronization will limit their speed,” said Wolfson-Pou.
Iterative methods are old computing techniques that start with an initial guess and generate a sequence of improved approximations to the true solution. These methods have achieved remarkable results and showed measurable progress for completing calculations simultaneously. However, the nature of these methods involves one or more synchronization points within each iteration.
“Supercomputers are composed of many parallel processes doing calculations concurrently. If many processes have to synchronize, some may be idle while waiting for others to finish,” said Wolfson-Pou. “For example, this could be due to some processes having to do more calculations than others, or the underlying hardware that a process uses is slower than the hardware another uses. In asynchronous methods, the faster processes simply move on to the next step using the most available information.”
From Brazil to China, Wolfson-Pou presented the new observations he and CSE Professor Edmond Chow discovered while examining multigrid methods in an effort to understand how they can be executed asynchronously.
Their findings are detailed in the paper, Asynchronous Multigrid Methods, which was presented at the following:
- International Parallel & Distributed Computing Symposium (IPDPS), May 20-24, Rio de Janeiro, Brazil
- International Conference on Industrial and Applied Mathematics (ICIAM), July 15-19, Valencia, Spain
- International Multigrid Conference (IMG), August 11-16, Kunming, China
- AMG Summit, September 30 – October 3, Santa Fe, New Mexico
The paper’s experimental results show that asynchronous multigrid can be faster than classical multigrid in terms of reducing the time it takes to converge to the solution.
Global-res and local-res partitionings for the Multadd example presented in Section IV for each step of the computation of the corrections e0 ande1. Arrows denote moving to the next step of the computation. Sync() denotes a synchronization point, where the list of threads passed to Sync() denotes the threads that synchronize. Blue Sync() denotes a synchronization for asynchronous multigrid, and red Sync() denotes a synchronization point for synchronous multigrid. Colored points denote points used in a calculation, where t0 is assigned the purple points, t1 is assigned the yellow points, t2 is assigned the blue points, t3 is assigned the orange points, and t4 is assigned the green points. Gray points denote points not used in a calculation.