Skip to content

Building and installing software on MUSICA

In order to meet the needs of the individual user or project, it will often be necessary to build additional software - software that is not already found in one of the provided repositories. There are two supported ways that this can be accomplished - both build on top of EESSI. Each option has some advantages. The preferred option is to use EasyBuild, as this allows the user to leverage EasyBuild's functionality and features. The second option is to build software manually, and is recommended only for expert users.

All software on MUSICA is built against the compatibility layer of EESSI. This means that operating system libraries are not in a standard location, so the build environment must be configured in a way that those libraries can be located both at build time and again at run time.

Building software directly on the host (i.e. completely bypassing EESSI) is not supported.

Building software with EasyBuild

EESSI-extend

In a nutshell, EESSI-extend is a module that loads EasyBuild in an environment that is configured to install software to one specific location at a time. This location may be configured by the user, by setting one of the variables mentioned below.

Using the EESSI-extend module is a short and informative introduction to building software with EasyBuild on EESSI. It is recommended that new users first read that page before proceeding.

Note that a specific version of EESSI's compatibility must be selected when building software. This will default to the most recent release available on the system if no version number is supplied. For a quick overview of EESSI-extend, see the module's help.

ml help EESSI-extend/2023.06-easybuild

Variables used to set the install location

By default, the EESSI-extend module sets the EESSI_USER_INSTALL variable. This can be configured to point to any location on the filesystem, but by default it points to $HOME/eessi. The user will have full write permissions to this location, and can build out any software that they require.

In some situations, a user might want to make software that they build available to other members of a research group. This requires that some shared storage be set up for the group and the user building the software has full write permissions to that location. As stated in the help documentation, the user must be aware of the order of precedence for the variables that are used to manage the install location.

If both EESSI_USER_INSTALL and EESSI_PROJECT_INSTALL are defined, both sets of installations are exposed, but new installations are created as user installations.

The implications of this, are that if a user wants to build software that can be shared at the group level, they must run something like the following:

ml unload EESSI-extend
unset EESSI_USER_INSTALL
export EESSI_PROJECT_INSTALL=/path/to/shared/storage
ml load EESSI-extend
Once the module has been loaded, the shell session will have access to the eb command line tool.

Searching for available build recipes

Caveat emptor (Let the buyer beware)

The quickest way to install required software is not to install it! In other words, always check to see if there is a module already available on the system in one of the provided repos. If no modules are found, the next thing to do would be to check if there are any appropriate easyconfig files available for the software we want to build. For example, to see if there are any recipes for installing the intel-compilers:

eb --search intel-compilers

The search returns a lot of results. A close examination reveals there are in fact a number of different easyconfig files available for the intel-compilers available.

If we look at the paths to those files, we can see that they are being found in a specific folder along the EESSI path.

/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen4/software/EasyBuild/$EB_VERSION/easybuild/easyconfigs

This is because the EESSI-extend module configures EasyBuild's robot-paths variable to point to that location, it is the first location that will be searched for easyconfig files any time eb --search is run.

To view the easyconfig file for a specific version run:

eb --show-ec intel-compilers-2023.2.1.eb

The next step is to tell easybuild to do a "dry-run" of the build to see what would be installed. 

eb -x -r intel-compilers-2023.2.1.eb

Finally, to install the software, omit the dry run flag (-x) and follow any further instructions that easybuild suggests. For instance, the intel compilers are proprietary software and will require passing an additional --accept-eula-for=Intel-oneAPI flag.

Note

In order to use software installed into either one of the two locations mentioned above, the EESSI-extend module must be loaded and the variable must point to the correct path where the software was installed.

Advanced

It can sometimes be a challenge to find an easyconfig file that build right away. Part of the reason for this is that some toolchains are better supported than others in EESSI. Software built with a well supported toolchain will be quicker to build with EasyBuild, because it will be easier to resolve the dependencies from already installed software. Note that passing the -r or --robot flag to EasyBuild will cause it to automatically build and install any missing dependencies. Even though this is possible, the recommended approach is to try to align the dependencies with what is already available through the existing software repositories as much as is practical. This is not only because the software will be quicker to build, but also because there will likely be better support from the community if any extra hooks or patches are required to build the software.

Manually building software on top of EESSI

The following two principles are best practices for extending/building on top of EESSI modules:

  • Use the least amount of modules possible (some are pulled as dependencies and might clash).
  • Declare precisely which module to use with the name and version module load HDF5/1.14.3-gompi-2023b instead of module load HDF5).

If one is not familiar with Easybuild or prefers building software in a more traditional way, this can be done via EESSI's buildenv module. This will also build the software using EESSI's libraries, and has the advantage that all the bundled modules are compatible with each other plus some useful settings.

Warning

Currently EESSI only provides a build environment for the
GCC + flexBLAS + OpenMPI toolchain

myname@n3001-007:~$ module load buildenv/default-foss-2023b
myname@n3001-007:~$ module list
Currently Loaded Modules:
  1) EESSI/2023.06
  2) GCCcore/13.2.0
  3) GCC/13.2.0
  4) numactl/2.0.16-GCCcore-13.2.0
  5) libxml2/2.11.5-GCCcore-13.2.0
  6) libpciaccess/0.17-GCCcore-13.2.0
  7) hwloc/2.9.2-GCCcore-13.2.0
  8) OpenSSL/1.1
  9) libevent/2.1.12-GCCcore-13.2.0
 10) UCX/1.15.0-GCCcore-13.2.0
 11) libfabric/1.19.0-GCCcore-13.2.0
 12) PMIx/4.2.6-GCCcore-13.2.0
 13) UCC/1.2.0-GCCcore-13.2.0
 14) OpenMPI/4.1.6-GCC-13.2.0
 15) OpenBLAS/0.3.24-GCC-13.2.0
 16) FlexiBLAS/3.3.1-GCC-13.2.0
 17) FFTW/3.3.10-GCC-13.2.0
 18) gompi/2023b
 19) FFTW.MPI/3.3.10-gompi-2023b
 20) ScaLAPACK/2.2.0-gompi-2023b-fb
 21) foss/2023b
 22) buildenv/default-foss-2023b

Info

All software built with that will link to the EESSI compilers and libraries.

Example: HBT-HERONS

myname@n3001-007:~$ git clone https://github.com/SWIFTSIM/HBT-HERONS.git && cd HBT-HERONS

myname@n3001-007:~$ mkdir build && cd build

myname@n3001-007:~$ module load buildenv/default-foss-2023b
myname@n3001-007:~$ module load CMake/3.27.6-GCCcore-13.2.0
myname@n3001-007:~$ module load HDF5/1.14.3-gompi-2023b

myname@n3001-007:~$ cmake ../ && make -j 128