HPL - A Portable Implementation of the High-Performance Linpack Benchmark for Distributed-Memory Computers

Linpack Benchmark is a software package that solves a (random) dense linear system in double precision (64 bits) arithmetic on distributed-memory computers. 同時為全球 500 大超級電腦 (TOP500 Supercomputer) 衡量系統效能的指標.
本網頁將介紹如何安裝與調校 HPL 參數.

準備一個叢集環境

使用實體機器建立叢集環境或虛擬機器建立虛擬叢集環境，使其得以運行 MPI 程式。在此我們使用 virsh/virt-manager (libvirt) 來建立虛擬叢集環境。
第一部虛擬機器為 br0 的 IP+40
第二部虛擬機器為 br0 的 IP+80 (第二部虛擬機器等安裝完 HPL 我們再建立)

安裝 HPL

由於 HPL 是用以測試分散式系統效能的軟體, 因此使用到 Massage Passing Interface MPI, 以及 Basic Linear Algebra Subprograms BLAS 或 Vector Signal Image Processing Library VSIPL. 在此我們選用 openMPI 與 ATLAS (Automatically Tuned Linear Algebra Software). 
$ ssh -X cloud@VM1_IP
$ sudo aptitude update
$ sudo aptitude install libopenmpi-dev openmpi-bin build-essential libatlas3-base
$ wget http://www.netlib.org/benchmark/hpl/hpl-2.0.tar.gz
$ tar zxvf hpl-2.0.tar.gz
$ mv hpl-2.0 HPL
$ cd HPL
$ cp setup/Make.Linux_ATHLON_CBLAS Make.LinuxKVM
$ nano Make.LinuxKVM
$ diff Make.LinuxKVM setup/Make.Linux_ATHLON_CBLAS

64c64
< ARCH         = LinuxKVM
---
> ARCH         = Linux_ATHLON_CBLAS
70c70
< TOPdir       = $(HOME)/HPL
---
> TOPdir       = $(HOME)/hpl
84c84
< MPdir        = /usr/lib/openmpi
---
> MPdir        = /usr/local/mpi
86c86
< MPlib        = $(MPdir)/lib/libmpi.so
---
> MPlib        = $(MPdir)/lib/libmpich.a
95c95
< LAdir        = /usr/lib/atlas-base/
---
> LAdir        = $(HOME)/netlib/ARCHIVES/Linux_ATHLON
97c97
< LAlib        = $(LAdir)/libcblas.so.3 $(LAdir)/libatlas.so.3
---
> LAlib        = $(LAdir)/libcblas.a $(LAdir)/libatlas.a

$ make arch=LinuxKVM
$ echo $?

0

$ make install arch=LinuxKVM
$ echo $?

0

準備叢集環境 (續)

  1. 關閉虛擬機器
  2. Clone 虛擬機器
  3. 修改第二部虛擬機器 /etc/hostname, /etc/hosts, /etc/network/interfaces
  4. 分別將兩部虛擬機器開啟

多節點設置

多節點共同運算時需透過 ssh 遠端連線完成, 因此必須完成 ssh 相互認證達到雙方免輸入密碼才能順利運作.
  1. ssh 遠端存取認證 (免密碼)
    $ ssh-keygen 

    Generating public/private rsa key pair.

    Enter file in which to save the key (/home/cloud/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 

    Your identification has been saved in /home/cloud/.ssh/id_rsa.
Your public key has been saved in /home/cloud/.ssh/id_rsa.pub.
The key fingerprint is:
0e:1d:9b:f2:74:bc:da:14:be:88:10:13:fd:c9:4a:5a cloud@demo
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|     .           |
|    . . .        |
|     . + *       |
|    o E S +      |
|     * B o o     |
|    o . o +      |
|     . . = .     |
|      . o o      |
+-----------------+
    將 public key 傳到其他節點 
    $ ssh-copy-id cloud@node1
$ ssh-copy-id cloud@node2
    同理將所有節點做過一次.
  2. 建立 node list 
    $ cd ~/HPL/bin/LinuxKVM
$ nano node.list

    VM1_IP	slots=4
VM2_IP	slots=4
  3. 修改 HPL.dat, N, P x Q 等參數後傳至各個節點 
    $ scp HPL.dat VM2_IP:`pwd`
  4. 執行 HPL 
    $ mpiexec -np 8 -hostfile node.list ./xhpl

HPL Tuning

$ cd ~/HPL/bin/LinuxKVM/
$ more HPL.dat

HPLinpack benchmark input file
Innovative Computing Laboratory, University of Tennessee
HPL.out      output file name (if any)
6            device out (6=stdout,7=stderr,file)
4            # of problems sizes (N)
29 30 34 35  Ns
4            # of NBs
1 2 3 4      NBs
0            PMAP process mapping (0=Row-,1=Column-major)
3            # of process grids (P x Q)
2 1 4        Ps
2 4 1        Qs
16.0         threshold
3            # of panel fact
0 1 2        PFACTs (0=left, 1=Crout, 2=Right)
2            # of recursive stopping criterium
2 4          NBMINs (>= 1)
1            # of panels in recursion
2            NDIVs
3            # of recursive panel fact.
0 1 2        RFACTs (0=left, 1=Crout, 2=Right)
1            # of broadcast
0            BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
1            # of lookahead depth
0            DEPTHs (>=0)
2            SWAP (0=bin-exch,1=long,2=mix)
64           swapping threshold
0            L1 in (0=transposed,1=no-transposed) form
0            U  in (0=transposed,1=no-transposed) form
1            Equilibration (0=no,1=yes)
8            memory alignment in double (> 0)


$ mpiexec -np 4 ./xhpl | tee "HPL-result.txt"

參數說明





ChiSheng Su