Performance Analysis of Parallel Applications for HPC

This book presents a hybrid static-dynamic approach for efficient performance analysis of parallel applications on HPC systems. Performance analysis is essential to finding performance bottlenecks and understanding the performance behaviors of parallel applications on HPC systems. However, current performance analysis techniques usually incur significant overhead. Our book introduces a series of approaches for lightweight performance analysis.

We combine static and dynamic analysis to reduce the overhead of performance analysis. Based on this hybrid static-dynamic approach, we then propose several innovative techniques for various performance analysis scenarios, including communication analysis, memory analysis, noise analysis, computation analysis, and scalability analysis. Through these specific performance analysis techniques, we convey to readers the idea of using static analysis to support dynamic analysis.

To gain the most from the book, readers should have a basic grasp of parallel computing, computer architecture, and compilation techniques.

Jidong Zhai is a Tenured Associate Professor in the Department of Computer Science and Technology of Tsinghua University. He was a Visiting Professor of Stanford University (2015-2016) and a Visiting Scholar of MSRA (Microsoft Research Asia) in 2013. He is the general secretary of ACM SIGHPC China. His current research interests include parallel computing, compilers, programming languages, and performance evaluation. He has published in prestigious conferences (such as SC, ICS, PPOPP, ASPLOS, MICRO, OSDI, ATC, and PACT) and top-tier journals (such as IEEE TC and IEEE TPDS). His research received Best Paper Award at CLUSTER'21, Best Student Paper Award at ICS’21, Best Paper Honorable Mention Award at ICDCS’20, and Best Paper Finalist at SC’14. He has served as a PC member for a number of international conferences, including SC, ICS, PPoPP, IPDPS, ICPP, Cluster, and PACT. He was a program co-chair of NPC 2018. He is currently on the editorial boards of IEEE Transactions on Computers (TC), IEEE Transactions on Parallel and Distributed Systems (TPDS), IEEE Transactions on Cloud Computing (TCC), Journal of Parallel and Distributed Computing (JPDC), and Journal of Parallel Computing (PARCO). He is the advisor of the Tsinghua Student Cluster Team. The team led by him has achieved 12 international champions in student supercomputing challenges at SC, ISC, and ASC. In 2015 and 2018, the team led by him swept all three champions. He has received a number of awards, including the CCF Outstanding doctoral dissertation Award (2010), NSFC Young Career Award (2017), IEEE TPDS Award for Editorial Excellence (2019), CCF-IEEE CS Young Computer Scientist Award (2020), and First Prize of Natural Science of CCF (2021).

Yuyang Jin is a Postdoc in the Department of Computer Science and Technology, Tsinghua University. His research interests include performance analysis and programming languages. He received a Ph.D. degree in computer science from Tsinghua University in 2022. He has published papers in international conferences and journals, including SC, PPoPP, and TPDS.

Wenguang Chen is a professor in the Department of Computer Science and Technology, Tsinghua University. His research interests are in parallel and distributed systems and programming systems. He received his Bachelor’s and Ph.D. degrees in computer science from Tsinghua University in 1995 and 2000, respectively. Before joining Tsinghua in 2003, he was the CTO of Opportunity International Inc. He was appointed as the associate head of the Department of Computer Science and Technology from 2007 to 2014. He has published over 50 papers in international conferences and journals such as Supercomputing, EuroSys, USENIX ATC, OOPSLA, and ICSE. He is a distinguished member and distinguished speaker of CCF (China Computer Foundation). He is an ACM member, vice chair of ACM China Council, and Editor-in-Chief of Communications of ACM (China Edition). He serves on the program committee of many conferences, such as PLDI, PPoPP, SC, ASPLOS, CGO, IPDPS, CCGrid, ICPP, and APSYS. He received the Distinguished Young Scholar Award of Natural Science Foundation in 2015.

Weimin Zheng is a CAE member. He is currently a Professor at the Department of Computer Science and Technology, Tsinghua University. He was the President of the China Computer Federation. Prof. Zheng has long been engaged in the research of high-performance computer architecture as well as parallel algorithms and systems. He led the establishment and application of the cluster architecture of high-performance computers in China and participated in the development of the extremely large-scale weather forecast application based on the domestic Sunway TaihuLight, which won the ACM Gordon Bell Award in 2016. Special Allowance of the State Council, the State Science and Technology Progress Award (one 1st and two 2nd prizes), the State Technological Invention Award (2nd prize), and He Liang He Li Science and Technology Progress Award, Prof. Zheng and his collaborators published more than 500 papers and more than 10 books.