Environment Modules

LRZ uses the modules approach to manage the user environment for different software versions. This is the standard method available for user environment management on all Linux-based LRZ HPC platforms. The distinct advantage of the modules approach is that the user is no longer required to explicitly specify paths for different executable versions, and try to keep the MANPATH and related environment variables coordinated. With the modules approach, users simply 'load' and 'unload' modules to control their environment.

Introduction

Modules provide an easy mechanism for managing the user's environment (PATH, MANPATH, and LD_LIBRARY_PATH, to name a few). To perform this in the current shell, the user types commands like

module load <package>

(making the settings for <package> available),

module unload <package>

(removing the settings for <package>).

On the LRZ HPC systems, setting up of the modules package is performed automatically at login. This includes

  • making the module command (as described below) available

  • loading a default set of module files: This depends on the platform used. PRACE users need to explicitly load the prace module - for details please consult the PRACE environment documentation.

Details on Setup and Restrictions

Module classes

In order to impose some structure on the large number of provided modules, a number of module classes has been defined, one of which each module falls into:

  • applications: third party application programs will be entered here

  • compilers: programs for converting language source to machine code

  • environment: modules for providing LRZ specific environment variables

  • graphics: modules for providing graphics and visualization related functionality

  • libraries: performance and functionality libraries

  • parallel: access to parallelization software, especially MPI

  • tools: development, tuning and debugging aids

There is a further, special extfiles category which will be used e.g., for applications with a rapidly changing profile or for testing.

Changing shells

While the LRZ setup of modules can cope with repeated shell calls (e.g., when opening a new xterm from your first login shell) of the same kind, changing over to a different kind of shell may be a problem: For example, executing ksh from a bash (which may especially happen when using shell scripts) will have the effect that the module command is not known within the ksh. As a consequence, environment variables, while exported to the new shell, cannot be administered there. To get around this (it won't always be actually necessary to do so), you must either

  • force execution of the new shell as a login shell (ksh -l)

  • or source /etc/profile.d/modules.sh either manually, or via your own login script pointed at by the ENV environment variable. The following lines recommend themselves for insertion into your own shell init script:

    typeset -f module > /dev/null
    if [ $? != 0 -a -r /etc/profile.d/modules.sh ] ; then
     . /etc/profile.d/modules.sh
     fi
    

    Note that this will provide you with the same set of loaded modules as given last in the calling shell.

Using modules in batch jobs

The modules package is loaded by default at login. However, the typical batch systems are not able to reliably load the modules package. Hence, if you want to use modules within batch scripts, please do not forget to issue the following command as the first line of your script:

. /etc/profile

if your are a sh/bash/ksh user, and

source /etc/csh.cshrc

in case your script is csh derived.

In all example scripts on the LRZ web server this entry is contained. Feedback and bug reports are welcome. See the Support section below for whom to ask for help in case of trouble.

Grid environments (PRACE)

The Partnership for Advanced Computing in Europe also makes use of the modules package. Details on the PRACE module setup at LRZ are available as a separate document.

Module suboptions

Here is an overview of the most important command options:

module list
This lists all the modules which are currently loaded into the users' environment.
module avail
This option lists all the modules which are available to be loaded. Notice that many of them have version numbers associated with them. Modules makes it easy to switchcompiler/application/library versions. The module name without a version number is the production default.

Additional switches are availabe for this suboption:

  • module avail -d will only list the default versions
  • module avail -c <module_class> will only list modules from the specified class of modules.
These two switches can also be used in combination.
module help modulefile
If a module looks interesting, to get more information use this command to display the `help' information contained within the given module file.
% module help gaussian
Module Specific Help for /lrz/sys/share/modules/files/applications/gaussian/03.D.01:
modulefile "gaussian/03.D.01"
Enable the usage of Gaussian QC software package.
Documentation on LRZ Web Server at
http://www.lrz.de/services/software/chemie/gaussian.
module show modulefile
Use this command to see exactly what a given modulefile will do to your environment, such as what will be added to the PATH, MANPATH, etc. environment variables.
module load modulefile
This adds one or more modulefiles to the user's current environment. It does so silently, unless there is a problem with a modulefile. For example,
% module load gaussian
will make the Gaussian package available for your use. Please also note that
  • some modules may automatically load others if a package depends on some other package to be available. Unloading dependent modules may give you trouble!
  • Certain default modules should also not be unloaded, since your environment will not function properly without the settings performed there.
module unload modulefile
This removes any listed modules from the user's current environment. The modules can be removed in any order. It is also possible to specify more than one module to be removed.
% module unload gaussian
will undo any environment variable settings performed during a preceding load of the module of the same name.
module switch modulefile_old modulefile_new
This command performs a transition from modulefile_old to modulefile_new with a single command. Note that this is not the same as an unload followed by a load, since all modules which were loaded after modulefile_old will usually also be reloaded. If this causes difficulties, you may need to revert to explicitly perform an unload/load sequence.

To get a complete list of command options you can type module help on your command prompt.

Using modules from Perl

It is possible to use the module command also from non-shell scripting languages. In the case of Perl, a LRZ specific package is available. Here is an example on how to use this:

#! /usr/bin/perl
use lib "/lrz/sys/share/modules/init";
use Environment_Modules;
module("load","fortran mkl");
module("list");
system("make myprog");
system("./dosomething");

Note the following details:

  • Use standard perl function call syntax with brackets, especially when specifying more than one argument

  • In contrast to the shell setup, only the LRZ administrative modules are loaded. Please load all other required modules yourself.

  • Within Perl, the environment variables are accessible via the %ENV hash.

  • All environment variables will be propagated to processes called e.g., via system. Hence, you can use these variables in Makefiles and scripts which are executed/called from Perl.

Additional Utilities

Some additional utilities which are provided specifically with the LRZ installation are described in this section.

  1. ismoduleloaded: if issued with an argument, will tell whether that module is loaded. If the argument is a versioned module name, the exact version will be checked, otherwise only whether any version of the given module is loaded. Output is a "YES" or "NO" to stdout as well as an error exit of 0 or 1, respectively. If the -s command line switch is provided, output to stdout is suppressed. If no module argument is given, "YES" will be written to stdout if any module is loaded, "NO" otherwise.

  2. defaultversion: if issued with a module name as argument (with or without a version number), this will return the default version of the module on STDOUT. Error 1 is returned if the module does not exist.

  3. loadedversion: if issued with a module name as argument (with or without a version number), this will return the presently loaded version of the module on STDOUT. Error 1 is returned if a module of the given name is not loaded. The version number on the argument need not be the correct or even an existing one.

  4. removemodules.[c]sh: this can be used to completely remove all module settings from your environment by sourcing the appropriate version into your running shell; all modules are unloaded, and the module alias as well as all module-related variables are unset. Beware that this will rather cripple your environment since many important settings are performed via LRZ administrative modules.

Versioned module setup on LRZ systems

For each package a module is provided for, it may happen that more than one version is available. In this case, there will be

  • a default module responsible for setting up the main production version.
    Example: the module default file for setting up the quantum chemical package "Gaussian" is gaussian

  • some module files with the same basename as the default, followed by a "/" sign and then a version number.
    Example: While the default version of Gaussian may be 03.D.01, it is possible to instead use the 03.C.02 version by loading the module gaussian/03.C.02

Note that the "/" convention for separating version information from the module name allows you to perform e.g.,

module switch fortran fortran/intel/11.1

correctly regardless which version of fortran is loaded prior to the above command.

Versioning policies

Please note that software packages as well as the module system undergo a maintenance and update process. As a consequence, the following may happen at LRZ HPC staff's discretion and - unless the user impact is considered very large - without special announcement:

  • the default version of a software package may change; the method of loading the default module will however normally stay the same.
  • the availability and the module name of non-default versions of a software package may change

We apologize for the potential impact on processing of user jobs, but given the large number of packages we make available we presently see no alternative. For a given software package, please check out its documentation page on the LRZ web server for updates and/or use the module avail <package_name> command to see the presently available releases.

User initialization files

A user can perform her/his own module initializations at login time using a shell-specific script in the $HOME/.modules subdirectory. This script must have the same name as your login shell, i.e., $HOME/.modules/$SHELL, and will typically contain

  • extension of the MODULEPATH environment variable (e.g., via the module use command) to contain some subdirectory where a user's own module files can reside

  • loading of additional LRZ or self-written modules at login as needed

Naming Conventions

When loading a module, package specific environment variables follow the conventions given in this section. In the following table, the placeholder PAC is used for the package name.

Generic NameDescription

PAC_BASE

base directory for package

PAC_LIBDIR

library directory for package, usually $PAC_BASE/lib

PAC_LIB

library setting for (static) linking, typically $PAC_BASE/lib/lib<library_name.a>

PAC_SHLIB

library setting for dynamic linking, typically -L$PAC_BASE/lib -l<library_name>

PAC_INC

include setting for compilation, usually -I$PAC_BASE/include

PAC_DOC

directory containing documentation (in other than man or info format)

PAC_WWW

URL referring to location of package-specific documentation on the LRZ WWW server.

 

Note that occasionally a module may contain subpackages e.g., SUB, in which case additional variables of the form PAC_SUB_LIB etc. may be defined.

Support and Sources of Information on Modules

If you experience problems using the module package, please notify the LRZ HPC support team.

Further information on modules can be found by reading the man pages on the LRZ HPC systems (module (1), modulefile (4)).

The sources can be found on the Sourceforge git repository. For the use of modules, a Tcl (http://www.tcl.tk/) installation is needed as a prerequisite.