Running xhpl on Rapberry Pi 4

Setup

OS
```
Raspberry PI OS
```

Kernel

Linux pi04.arif.local 5.4.44-v7l+ #1320 SMP Wed Jun 3 16:13:10 BST 2020 armv7l GNU/Linux*

Bootlader:

root@pi04:~# vcgencmd bootloader_version 
May 27 2020 18:47:29
version d648db3968cd31d4948341e09cb8a925c49d2ea1 (release)
timestamp 1590601649

Everything else is stock

Download latest mpich, and compile using

tar xfz mpich-3.3.2.tar.gz
cd mpich-3.3.2
./configure --prefix=/opt/mpich/3.3.2
make -j 3
sudo make install

Download latest OpenBLAS

unzip OpenBLAS.zip
cd OpenBLAS-develop
make -j 3
sudo make install

Download latest hpl
```
tar xfz hpl-2.3.tar.gz
cd hpl-2.3
```

HPL.dat

HPLinpack benchmark input file
Innovative Computing Laboratory, University of Tennessee
HPL.out      output file name (if any) 
6            device out (6=stdout,7=stderr,file)
1            # of problems sizes (N)
19008         Ns
1            # of NBs
192           NBs
0            PMAP process mapping (0=Row-,1=Column-major)
1            # of process grids (P x Q)
1            Ps
1            Qs
16.0         threshold
1            # of panel fact
2            PFACTs (0=left, 1=Crout, 2=Right)
1            # of recursive stopping criterium
4            NBMINs (>= 1)
1            # of panels in recursion
2            NDIVs
1            # of recursive panel fact.
1            RFACTs (0=left, 1=Crout, 2=Right)
1            # of broadcast
1            BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
1            # of lookahead depth
1            DEPTHs (>=0)
2            SWAP (0=bin-exch,1=long,2=mix)
64           swapping threshold
0            L1 in (0=transposed,1=no-transposed) form
0            U  in (0=transposed,1=no-transposed) form
1            Equilibration (0=no,1=yes)
8            memory alignment in double (> 0)
##### This line (no. 32) is ignored (it serves as a separator). ######
0                               Number of additional problem sizes for PTRANS
1200 10000 30000                values of N
0                               number of additional blocking sizes for PTRANS
40 9 8 13 13 20 16 32 64        values of NB

Make.rpi4-mpich

#
#  -- High Performance Computing Linpack Benchmark (HPL)
#     HPL - 2.3 - December 2, 2018
#     Antoine P. Petitet
#     University of Tennessee, Knoxville
#     Innovative Computing Laboratory
#     (C) Copyright 2000-2008 All Rights Reserved
#
#  -- Copyright notice and Licensing terms:
#
#  Redistribution  and  use in  source and binary forms, with or without
#  modification, are  permitted provided  that the following  conditions
#  are met:
#
#  1. Redistributions  of  source  code  must retain the above copyright
#  notice, this list of conditions and the following disclaimer.
#
#  2. Redistributions in binary form must reproduce  the above copyright
#  notice, this list of conditions,  and the following disclaimer in the
#  documentation and/or other materials provided with the distribution.
#
#  3. All  advertising  materials  mentioning  features  or  use of this
#  software must display the following acknowledgement:
#  This  product  includes  software  developed  at  the  University  of
#  Tennessee, Knoxville, Innovative Computing Laboratory.
#
#  4. The name of the  University,  the name of the  Laboratory,  or the
#  names  of  its  contributors  may  not  be used to endorse or promote
#  products  derived   from   this  software  without  specific  written
#  permission.
#
#  -- Disclaimer:
#
#  THIS  SOFTWARE  IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
#  ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,  INCLUDING,  BUT NOT
#  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
#  A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE UNIVERSITY
#  OR  CONTRIBUTORS  BE  LIABLE FOR ANY  DIRECT,  INDIRECT,  INCIDENTAL,
#  SPECIAL,  EXEMPLARY,  OR  CONSEQUENTIAL DAMAGES  (INCLUDING,  BUT NOT
#  LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
#  DATA OR PROFITS; OR BUSINESS INTERRUPTION)  HOWEVER CAUSED AND ON ANY
#  THEORY OF LIABILITY, WHETHER IN CONTRACT,  STRICT LIABILITY,  OR TORT
#  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
#  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# ######################################################################
#
# ----------------------------------------------------------------------
# - shell --------------------------------------------------------------
# ----------------------------------------------------------------------
#
SHELL        = /bin/sh
#
CD           = cd
CP           = cp
LN_S         = ln -fs
MKDIR        = mkdir -p
RM           = /bin/rm -f
TOUCH        = touch
#
# ----------------------------------------------------------------------
# - Platform identifier ------------------------------------------------
# ----------------------------------------------------------------------
#
ARCH         = rpi4-mpich
#
# ----------------------------------------------------------------------
# - HPL Directory Structure / HPL library ------------------------------
# ----------------------------------------------------------------------
#
TOPdir       = $(HOME)/hpl/hpl-2.3
INCdir       = $(TOPdir)/include
BINdir       = $(TOPdir)/bin/$(ARCH)
LIBdir       = $(TOPdir)/lib/$(ARCH)
#
HPLlib       = $(LIBdir)/libhpl.a
#
# ----------------------------------------------------------------------
# - Message Passing library (MPI) --------------------------------------
# ----------------------------------------------------------------------
# MPinc tells the  C  compiler where to find the Message Passing library
# header files,  MPlib  is defined  to be the name of  the library to be
# used. The variable MPdir is only used for defining MPinc and MPlib. 
#
MPdir        = /opt/mpich/3.3.2
MPinc        = -I$(MPdir)/include
MPlib        = $(MPdir)/lib/libmpi.a
#
# ----------------------------------------------------------------------
# - Linear Algebra library (BLAS or VSIPL) -----------------------------
# ----------------------------------------------------------------------
# LAinc tells the  C  compiler where to find the Linear Algebra  library
# header files,  LAlib  is defined  to be the name of  the library to be
# used. The variable LAdir is only used for defining LAinc and LAlib.
#
LAdir        = /opt/OpenBLAS
LAinc        = $(LAdir)/include
LAlib        = $(LAdir)/lib/libopenblas.a -lpthread
#
# ----------------------------------------------------------------------
# - F77 / C interface --------------------------------------------------
# ----------------------------------------------------------------------
# You can skip this section  if and only if  you are not planning to use
# a  BLAS  library featuring a Fortran 77 interface.  Otherwise,  it  is
# necessary  to  fill out the  F2CDEFS  variable  with  the  appropriate
# options.  **One and only one**  option should be chosen in **each** of
# the 3 following categories:
#
# 1) name space (How C calls a Fortran 77 routine)
#
# -DAdd_              : all lower case and a suffixed underscore  (Suns,
#                       Intel, ...),                           [default]
# -DNoChange          : all lower case (IBM RS6000),
# -DUpCase            : all upper case (Cray),
# -DAdd__             : the FORTRAN compiler in use is f2c.
#
# 2) C and Fortran 77 integer mapping
#
# -DF77_INTEGER=int   : Fortran 77 INTEGER is a C int,         [default]
# -DF77_INTEGER=long  : Fortran 77 INTEGER is a C long,
# -DF77_INTEGER=short : Fortran 77 INTEGER is a C short.
#
# 3) Fortran 77 string handling
#
# -DStringSunStyle    : The string address is passed at the string loca-
#                       tion on the stack, and the string length is then
#                       passed as  an  F77_INTEGER  after  all  explicit
#                       stack arguments,                       [default]
# -DStringStructPtr   : The address  of  a  structure  is  passed  by  a
#                       Fortran 77  string,  and the structure is of the
#                       form: struct {char *cp; F77_INTEGER len;},
# -DStringStructVal   : A structure is passed by value for each  Fortran
#                       77 string,  and  the  structure is  of the form:  
#                       struct {char *cp; F77_INTEGER len;},
# -DStringCrayStyle   : Special option for  Cray  machines,  which  uses
#                       Cray  fcd  (fortran  character  descriptor)  for
#                       interoperation.
#
F2CDEFS      = 
#
# ----------------------------------------------------------------------
# - HPL includes / libraries / specifics -------------------------------
# ----------------------------------------------------------------------
#
HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) -I$(LAinc) $(MPinc)
HPL_LIBS     = $(HPLlib) $(LAlib) $(MPlib) -lrt -lbacktrace
#
# - Compile time options -----------------------------------------------
#
# -DHPL_COPY_L           force the copy of the panel L before bcast;
# -DHPL_CALL_CBLAS       call the cblas interface;
# -DHPL_CALL_VSIPL       call the vsip  library;
# -DHPL_DETAILED_TIMING  enable detailed timers;
#
# By default HPL will:
#    *) not copy L before broadcast,
#    *) call the BLAS Fortran 77 interface,
#    *) not display detailed timing information.
#
HPL_OPTS     = -DHPL_DETAILED_TIMING -DHPL_PROGRESS_REPORT -DHPL_CALL_CBLAS
#
# ----------------------------------------------------------------------
#
HPL_DEFS     = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)
#
# ----------------------------------------------------------------------
# - Compilers / linkers - Optimization flags ---------------------------
# ----------------------------------------------------------------------
#
CC       = gcc
CCNOOPT  = $(HPL_DEFS)
CCFLAGS  = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loops -W -Wall
#
# On some platforms,  it is necessary  to use the Fortran linker to find
# the Fortran internals used in the BLAS library.
#
LINKER       = $(CC)
LINKFLAGS    = $(CCFLAGS)
#
ARCHIVER     = ar
ARFLAGS      = r
RANLIB       = echo
#
# ----------------------------------------------------------------------

Compiling and running

We can compile using the following command

make arch=rpi4-mpich

My HPL.dat was in ~/hpl, so the fact my PWD was ~/hpl, I ran the benchmark in the following way

OMP_NUM_THREADS=4 ./hpl-2.3/bin/rpi4-mpich/xhpl

Finally, my result is here from the above environment for the 8GB board, it was using 2.82GB

================================================================================
HPLinpack 2.3  --  High-Performance Linpack benchmark  --   December 2, 2018
Written by A. Petitet and R. Clint Whaley,  Innovative Computing Laboratory, UTK
Modified by Piotr Luszczek, Innovative Computing Laboratory, UTK
Modified by Julien Langou, University of Colorado Denver
================================================================================

An explanation of the input/output parameters follows:
T/V    : Wall time / encoded variant.
N      : The order of the coefficient matrix A.
NB     : The partitioning blocking factor.
P      : The number of process rows.
Q      : The number of process columns.
Time   : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.

The following parameter values will be used:

N      :   19008
NB     :     192
PMAP   : Row-major process mapping
P      :       1
Q      :       1
PFACT  :   Right
NBMIN  :       4
NDIV   :       2
RFACT  :   Crout
BCAST  :  1ringM
DEPTH  :       1
SWAP   : Mix (threshold = 64)
L1     : transposed form
U      : transposed form
EQUIL  : yes
ALIGN  : 8 double precision words

--------------------------------------------------------------------------------

- The matrix A is randomly generated for each test.
- The following scaled residual check will be computed:
      ||Ax-b||_oo / ( eps * ( || x ||_oo * || A ||_oo + || b ||_oo ) * N )
- The relative machine precision (eps) is taken to be               1.110223e-16
- Computational tests pass if scaled residuals are less than                16.0

Column=000000192 Fraction= 1.0% Gflops=1.352e+01
Column=000000384 Fraction= 2.0% Gflops=1.353e+01
Column=000000576 Fraction= 3.0% Gflops=1.355e+01
Column=000000768 Fraction= 4.0% Gflops=1.358e+01
Column=000000960 Fraction= 5.1% Gflops=1.360e+01
Column=000001152 Fraction= 6.1% Gflops=1.359e+01
Column=000001344 Fraction= 7.1% Gflops=1.359e+01
Column=000001536 Fraction= 8.1% Gflops=1.357e+01
Column=000001728 Fraction= 9.1% Gflops=1.357e+01
Column=000001920 Fraction=10.1% Gflops=1.356e+01
Column=000002112 Fraction=11.1% Gflops=1.356e+01
Column=000002304 Fraction=12.1% Gflops=1.356e+01
Column=000002496 Fraction=13.1% Gflops=1.355e+01
Column=000002688 Fraction=14.1% Gflops=1.355e+01
Column=000002880 Fraction=15.2% Gflops=1.353e+01
Column=000003072 Fraction=16.2% Gflops=1.353e+01
Column=000003264 Fraction=17.2% Gflops=1.353e+01
Column=000003456 Fraction=18.2% Gflops=1.352e+01
Column=000003648 Fraction=19.2% Gflops=1.352e+01
Column=000003840 Fraction=20.2% Gflops=1.352e+01
Column=000004032 Fraction=21.2% Gflops=1.351e+01
Column=000004224 Fraction=22.2% Gflops=1.351e+01
Column=000004416 Fraction=23.2% Gflops=1.351e+01
Column=000004608 Fraction=24.2% Gflops=1.350e+01
Column=000004800 Fraction=25.3% Gflops=1.350e+01
Column=000004992 Fraction=26.3% Gflops=1.349e+01
Column=000005184 Fraction=27.3% Gflops=1.349e+01
Column=000005376 Fraction=28.3% Gflops=1.349e+01
Column=000005568 Fraction=29.3% Gflops=1.348e+01
Column=000005760 Fraction=30.3% Gflops=1.348e+01
Column=000005952 Fraction=31.3% Gflops=1.347e+01
Column=000006144 Fraction=32.3% Gflops=1.346e+01
Column=000006336 Fraction=33.3% Gflops=1.346e+01
Column=000006528 Fraction=34.3% Gflops=1.345e+01
Column=000006720 Fraction=35.4% Gflops=1.345e+01
Column=000006912 Fraction=36.4% Gflops=1.344e+01
Column=000007104 Fraction=37.4% Gflops=1.344e+01
Column=000007296 Fraction=38.4% Gflops=1.343e+01
Column=000007488 Fraction=39.4% Gflops=1.343e+01
Column=000007680 Fraction=40.4% Gflops=1.343e+01
Column=000007872 Fraction=41.4% Gflops=1.343e+01
Column=000008064 Fraction=42.4% Gflops=1.342e+01
Column=000008256 Fraction=43.4% Gflops=1.342e+01
Column=000008448 Fraction=44.4% Gflops=1.341e+01
Column=000008640 Fraction=45.5% Gflops=1.341e+01
Column=000008832 Fraction=46.5% Gflops=1.341e+01
Column=000009024 Fraction=47.5% Gflops=1.340e+01
Column=000009216 Fraction=48.5% Gflops=1.340e+01
Column=000009408 Fraction=49.5% Gflops=1.340e+01
Column=000009600 Fraction=50.5% Gflops=1.339e+01
Column=000009792 Fraction=51.5% Gflops=1.339e+01
Column=000009984 Fraction=52.5% Gflops=1.339e+01
Column=000010176 Fraction=53.5% Gflops=1.339e+01
Column=000010368 Fraction=54.5% Gflops=1.338e+01
Column=000010560 Fraction=55.6% Gflops=1.338e+01
Column=000010752 Fraction=56.6% Gflops=1.338e+01
Column=000010944 Fraction=57.6% Gflops=1.337e+01
Column=000011136 Fraction=58.6% Gflops=1.337e+01
Column=000011328 Fraction=59.6% Gflops=1.337e+01
Column=000011520 Fraction=60.6% Gflops=1.336e+01
Column=000011712 Fraction=61.6% Gflops=1.336e+01
Column=000011904 Fraction=62.6% Gflops=1.336e+01
Column=000012096 Fraction=63.6% Gflops=1.335e+01
Column=000012288 Fraction=64.6% Gflops=1.335e+01
Column=000012480 Fraction=65.7% Gflops=1.335e+01
Column=000012672 Fraction=66.7% Gflops=1.334e+01
Column=000012864 Fraction=67.7% Gflops=1.334e+01
Column=000013056 Fraction=68.7% Gflops=1.334e+01
Column=000013248 Fraction=69.7% Gflops=1.333e+01
Column=000013440 Fraction=70.7% Gflops=1.333e+01
Column=000013632 Fraction=71.7% Gflops=1.333e+01
Column=000013824 Fraction=72.7% Gflops=1.333e+01
Column=000014016 Fraction=73.7% Gflops=1.332e+01
Column=000014208 Fraction=74.7% Gflops=1.332e+01
Column=000014400 Fraction=75.8% Gflops=1.332e+01  
Column=000014592 Fraction=76.8% Gflops=1.331e+01
Column=000014784 Fraction=77.8% Gflops=1.331e+01
Column=000014976 Fraction=78.8% Gflops=1.331e+01
Column=000015168 Fraction=79.8% Gflops=1.331e+01
Column=000015360 Fraction=80.8% Gflops=1.330e+01
Column=000015552 Fraction=81.8% Gflops=1.330e+01
Column=000015744 Fraction=82.8% Gflops=1.330e+01
Column=000015936 Fraction=83.8% Gflops=1.330e+01
Column=000016128 Fraction=84.8% Gflops=1.330e+01
Column=000016320 Fraction=85.9% Gflops=1.330e+01
Column=000016512 Fraction=86.9% Gflops=1.329e+01
Column=000016704 Fraction=87.9% Gflops=1.329e+01
Column=000016896 Fraction=88.9% Gflops=1.329e+01
Column=000017088 Fraction=89.9% Gflops=1.329e+01
Column=000017280 Fraction=90.9% Gflops=1.329e+01
Column=000017472 Fraction=91.9% Gflops=1.329e+01
Column=000017664 Fraction=92.9% Gflops=1.329e+01
Column=000017856 Fraction=93.9% Gflops=1.328e+01
Column=000018048 Fraction=94.9% Gflops=1.328e+01
Column=000018240 Fraction=96.0% Gflops=1.328e+01
Column=000018432 Fraction=97.0% Gflops=1.328e+01
Column=000018624 Fraction=98.0% Gflops=1.328e+01
Column=000018816 Fraction=99.0% Gflops=1.328e+01
================================================================================
T/V                N    NB     P     Q               Time                 Gflops
--------------------------------------------------------------------------------
WR11C2R4       19008   192     1     1             345.24             1.3263e+01
HPL_pdgesv() start time Fri Jun  5 16:39:38 2020

HPL_pdgesv() end time   Fri Jun  5 16:45:23 2020

--VVV--VVV--VVV--VVV--VVV--VVV--VVV--VVV--VVV--VVV--VVV--VVV--VVV--VVV--VVV-
Max aggregated wall time rfact . . . :               9.47
+ Max aggregated wall time pfact . . :               3.14
+ Max aggregated wall time mxswp . . :               0.64
Max aggregated wall time update  . . :             335.26
+ Max aggregated wall time laswp . . :              10.44
Max aggregated wall time up tr sv  . :               0.49
--------------------------------------------------------------------------------
||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)=   9.59421981e-04 ...... PASSED
================================================================================

Finished      1 tests with the following results:
              1 tests completed and passed residual checks,
              0 tests completed and failed residual checks,
              0 tests skipped because of illegal input values.
--------------------------------------------------------------------------------

End of Tests.
================================================================================

The above result showed that we were able to acheive 13.26 GFlops

Note: The system was not tuned, i.e. the following things were not changed

Reduction of any services running on the pi
CPU governor
CPU overclocking