MPI-BGQ

From Parts
Jump to: navigation, search

As of the V1R2M1 driver update, the MPI directory structure is different relative to V1R2M0. This should not be a problem if people just use the wrappers. However, anyone who tries to unwind the wrappers manually will notice changes.

The differences are described below. The MPI XLC++ wrappers are used for illustration, but the same patterns apply to all wrappers. Please refer to Explanation of MPI variants to understand the different types of MPI builds available.

V1R2M0

[jhammond@miralac1 ~]$ mpixlcxx_r -show
bgxlC_r -I/bgsys/drivers/V1R2M0/ppc64 
-I/bgsys/drivers/V1R2M0/ppc64/comm/sys/include 
-I/bgsys/drivers/V1R2M0/ppc64/spi/include 
-I/bgsys/drivers/V1R2M0/ppc64/spi/include/kernel/cnk 
-L/bgsys/drivers/V1R2M0/ppc64/spi/lib 
-L/bgsys/drivers/V1R2M0/ppc64/comm/sys/lib 
-L/bgsys/drivers/V1R2M0/ppc64/spi/lib 
-L/bgsys/drivers/V1R2M0/ppc64/comm/sys/lib 
-L/bgsys/drivers/V1R2M0/ppc64/spi/lib 
-I/bgsys/drivers/V1R2M0/ppc64/comm/xl/include 
-L/bgsys/drivers/V1R2M0/ppc64/comm/xl/lib 
-lcxxmpich -lmpich -lopa -lmpl 
-lpami -lSPI -lSPI_cnk 
-lrt -lpthread -lstdc++ -lpthread

The libraries are in /bgsys/drivers/ppcfloor/comm/${build}/lib and their names are always -lcxxmpich -lmpich -lopa -lmpl -lpami and the headers are in /bgsys/drivers/ppcfloor/comm/${build}/include, where build is one of {gcc,gcc.legacy,xl,xl.legacy,xl.legacy.ndebug,xl.ndebug}. Additionally, one finds the PAMI files in /bgsys/drivers/ppcfloor/comm/{sys,sys-fast}/{lib,include}.

V1R2M1

[jhammond@vestalac1 ~]$ mpixlcxx_r -show
bgxlC_r -I/bgsys/drivers/V1R2M1/ppc64/comm/lib/xl 
-I/bgsys/drivers/V1R2M1/ppc64 
-I/bgsys/drivers/V1R2M1/ppc64/comm/sys/include 
-I/bgsys/drivers/V1R2M1/ppc64/spi/include 
-I/bgsys/drivers/V1R2M1/ppc64/spi/include/kernel/cnk 
-L/bgsys/drivers/V1R2M1/ppc64/comm/lib 
-L/bgsys/drivers/V1R2M1/ppc64/comm/lib 
-L/bgsys/drivers/V1R2M1/ppc64/comm/lib64 
-L/bgsys/drivers/V1R2M1/ppc64/comm/lib 
-L/bgsys/drivers/V1R2M1/ppc64/spi/lib 
-L/bgsys/drivers/V1R2M1/ppc64/comm/sys/lib 
-L/bgsys/drivers/V1R2M1/ppc64/spi/lib 
-L/bgsys/drivers/V1R2M1/ppc64/comm/sys/lib 
-L/bgsys/drivers/V1R2M1/ppc64/comm/lib64 
-L/bgsys/drivers/V1R2M1/ppc64/comm/lib 
-L/bgsys/drivers/V1R2M1/ppc64/spi/lib 
-I/bgsys/drivers/V1R2M1/ppc64/comm/include 
-L/bgsys/drivers/V1R2M1/ppc64/comm/lib 
-lmpichcxx-xl -lmpich-xl -lopa-xl -lmpl-xl 
-lpami-gcc -lSPI -lSPI_cnk 
-lrt -lpthread -lstdc++ -lpthread

Both the PAMI and MPI libraries are in /bgsys/drivers/ppcfloor/comm/lib and their names are -lmpichcxx-${compiler} -lmpich-${compiler} -lopa-${compiler} -lmpl-${compiler} -lpami-gcc. PAMI is always compiled with GCC.

Most of the headers are in /bgsys/drivers/ppcfloor/comm/include/ because they are compiler-independent. When they are compiler-dependent, they are in /bgsys/drivers/ppcfloor/comm/include/${compiler}. MPI Fortran modules are compiler-dependent.