EasyBuild setup instructions

Install new version

Due to missing bdist_wheel in the system python, first create a venv:

cd
ml uv
uv venv --seed --python 3.12
source .venv/bin/activate
uv pip install pip setuptools wheel rich

Set EB_PYTHON=$HOME/.venv/bin/python and run:

module purge
ml easybuild
eb --install-latest-eb-release

If you get an error related to the ssl certificate, prepend the command with SSL_CERT_FILE=/etc/ssl/certs/ca-bundle.crt.

Configuration

In /apps/modulefiles/standard/core/easybuild/x.y.z.lua append:

setenv("CONTAINERDIR", "/share/resources/containers/apptainer")

Create ~/.config/easybuild if it does not already exist and copy config.cfg. A blank file can be generated via

eb --confighelp > $HOME/.config/easybuild/config.cfg

To see just the uncommented lines of the config file:

$ grep ^[^#] ~/.config/easybuild/config.cfg 
[MAIN]
[basic]
repositorypath=/apps/ebscripts/easybuild/easyconfigs
robot-paths=%(repositorypath)s:%(DEFAULT_ROBOT_PATHS)s
[config]
buildpath=/tmp/uvacse
hooks=/home/uvacse/ebhook/ssz_hook.py
installpath=/apps
module-naming-scheme=SSZ_HMNS
sourcepath=/share/resources/apps/source
subdir-modules=modulefiles/standard
subdir-software=software/standard
suffix-modules-path=
[container]
[easyconfig]
[github]
[informative]
[job]
[override]
accept-eula-for=intel-compilers,imkl,impi,NVHPC,nvidia-compilers,CUDA
cuda-compute-capabilities=8.0,8.6,8.9,10.0,12.0
default-opt-level=opt
filter-deps=Autoconf,Automake,libtool,Autotools,flex,M4
fixed-installdir-naming-scheme=False
optarch=Intel:-march=skylake;GCC:-march=skylake;NVHPC:-tp=znver2
rpath=0
[package]
[regtest]
[software]
[unittest]

Hook

Copy ssz_hook.py from this repo to /home/uvacse/ebhook.

Module naming scheme

Copy ssz_hmns.py from this repo to lib/pythonX.Y/site-packages/easybuild/tools/module_naming_scheme. If the syntax has changed, start from hierarchical_mns.py:

• TOOLCHAIN = toolchains (delete this step in the next fresh installation of the entire software stack)

• class SSZ_HMNS

• det_short_module_name:

  name = ec['name']
      if name != 'R':
          name = name.lower()
      return os.path.join(name, self.det_full_version(ec))
  

• det_module_subdir: return subdir.lower()

• Install toolchains to its own subdir instead of core (delete this step in the next fresh installation of the entire software stack)

  if tc_comps is None:
      self.log.debug('ec.toolchain.TOOLCHAIN_FAMILY=%s', ec.toolchain.TOOLCHAIN_FAMILY)
      if ec['moduleclass'] == 'toolchain':
          subdir = TOOLCHAIN
      else:
          # no compiler in toolchain, system toolchain => Core module
          subdir = CORE
  

• det_modpath_extensions: add .lower() to names

  comp_name_ver = [ec['name'].lower(), self.det_full_version(ec)]
  ...
  paths.append(os.path.join(MPI, tc_comp_name.lower(), tc_comp_ver, ec['name'].lower(), fullver))
  ...
  paths.append(os.path.join(TOOLCHAIN, ec.name.lower(), ec.version))
  ...
  paths.append(os.path.join(MPI, "nvidia-compilers", fullver, ec.name.lower(), fullver))
  

• Add is_short_modname_for from mns.py. Add .lower() to name and short_modname.

Container toolchain

1. In lib/pythonX.Y/site-packages/easybuild/tools/module_naming_scheme/toolchain.py:

   if ec.toolchain.is_system_toolchain() or ec.toolchain.name == 'apptainer':

2. (Skip if copying ssz_hmns.py from this repo.) In lib/pythonX.Y/site-packages/easybuild/tools/module_naming_scheme/ssz_hmns.py:

   CONTAINER = 'container'
   CONTAINER_TOOLCHAINS = ['apptainer']
   MODULECLASS_CONTAINER = 'containersystem'
   
   # Under det_module_subdir:
           if tc_comps is None:
               ...
               elif ec.toolchain.TOOLCHAIN_FAMILY == 'CONTAINER':
                   subdir = os.path.join(CONTAINER, ec.toolchain.name, ec.toolchain.version)

3. Copy container.py from this repo to lib/pythonX.Y/site-packages/easybuild/tools/toolchain. (If syntax has changed, start from toolchains/compiler/systemcompiler.py.)

4. Copy apptainer.py from this repo to lib/pythonX.Y/site-packages/easybuild/toolchains.

5. In lib/pythonX.Y/site-packages/easybuild/tools/toolchain/toolchain.py under def prepare():

                                        vvvvvvvvvvvvvvvvvvvvvvvvvvvv
           if self.is_system_toolchain() or self.name == 'apptainer':

6. Copy apptainerimage.py from this repo to lib/pythonX.Y/site-packages/easybuild/easyblocks/a. (If syntax has changed, compare with lib/pythonX.Y/site-packages/easybuild/easyblocks/generic/binary.py.)

Custom Easyblocks

First cd lib/pythonX.Y/site-packages/easybuild/easyblocks

• In l/llvm.py around line 900 gcc_version = get_software_version('GCCcore'), delete core.
• (Optional) In generic/conda.py add or get_software_root('miniforge') for the conda_cmd = 'mamba' case.

Citation

R. Sun and K. Siller, HPC Container Management at the University of Virginia, PEARC '24: Practice and Experience in Advanced Research Computing 2024: Human Powered Computing 73, 1 (2024). doi:10.1145/3626203.3670568