Administrative aspects of the LRZ module setup

Packaging and Configuration

Packages

The LRZ version of modules is slightly modified with respect to the original, especially with respect to the initialization functionality. There a two parts to the modules package:

A RPM package to be installed on every node. Automatic setup at login is performed via

/etc/profile.d/modules.sh
/etc/profile.d/modules.csh
depending on the login shell. Test whether zsh is properly treated. Note that the RPM has the Tcl and TclX packages as dependencies; the latter would be needed if a module file wants to use system.
The configuration which is installed in the globally accessible /lrz/sys/share/modules directory (MODULESHOME). The MODULEPATH variable by default contains references to the LRZ module classes defined in the main article. A module class will in the following occasionally be referred to as <class>.
All configuration data with exception of the extfiles area will be completely overwritten by the installation procedure which is performed from the SVN-generated LRZ modules source tree.

Modules source repository at LRZ

The complete source package is kept under SVN in the LRZ HPC repository. LRZ staff can check out the source tree by issuing the command
svn checkout https://svn.lrz-muenchen.de/repos/hlr/trunk/modules

Changing to the root directory and typing make hint will give you an overview over all available targets. Available platform-dependent subdirectories are
i386_Linux (frozen) ia64_Linux powerpc_AIX (is obsolete and will soon be discarded) x86_64_Linux deisa_specific
while the platform-independent configuration stuff is placed in init. The source tree can also serve as a testing facility. To do this, please source the setup.* corresponding to your shell.
Please do not forget to perform a SVN update / SVN commit cycle if you perform any corrections/updates to the package. To get an overview of installation options, please type make in the base module source directory.

Remarks on LRZ setup

module sequence unloading. Example: mymod-xyz tries to unload fortran and ccomp, but fortran is not actually unloaded. This seems to be a bug; not used in final LRZ setup.
puts "..." must be replaced by puts stderr "..." otherwise the quoted stuff is interpreted as command. Note that this can be useful if system commands need to be issued within a modulefile; however this may re-introduces shell dependencies which need to be appropriately treated within the modulefile.
etc/rc is not automatically loaded on module initialization. At least, no modulefiles are shown as loaded. Preliminarily, load default list of modules using the etc/defaults.[c]sh scripts. This seems to be a bug.
Trying to abort "module unload" within module file does not work as one would like it to. Settings remain but module is removed from loaded list. This is semantically correct. Retreat to warnings to not unload the module.
external environment variable settings (e.g., XXX_VERSION) to influence control flow within a module prevent "module switch" from being correctly usable. Need always to do
```
        (check XXX_VERSION set consistently)
        module unload xxx
        export XXX_VERSION=(new version) 
        module load xxx
        
```
It was decided to give this up entirely and only use the postfix to differentiate software versions.
Additional shells will not perform /etc/profile.d init scripts by default since PROFILEREAD has been set to "true". Hence the files /etc/bash.bashrc.local and /etc/csh.cshrc.local must contain statements to reinitialize modules if necessary.
It is as yet unclear how zsh behaves.
Changing shells will occasionally lose the module command. The reason is that in case of sh and ksh no shell initialization scripts are performed. To propagate the module command at least when staying within one shell, the user must enter the initialization in the startup script pointed to by his ENV environment variable.
In order to automatically be able to use modules within SGE, the job startup mode in SGE needs to be changed. This is in consonance with Grid requirements of higher security, so a changeover from 'default_unix' to 'read_script_from_stdin' will be introduced, implying POSIX conforming startup, i.e. a login shell parsed from the -S qsub switch is initiated when the job starts up. Note: POSIX conforming startup was configured on the Altix, but does not seem to be effective.

Conceptual and Implementational Problems

List of default versions

With the new "/" convention the default version is provided via the modulename/.version file. This is then also indicated when performing a module avail command, and the correct default version is automatically loaded if no version information is provided.

Changes and extensions to the modulescmd.tcl file

The following changes have been applied:

The procedures prepend-path and append-path have been extended to allow to deal with database-like variables with arbitrary separators. Within a module file it is now possible to do e.g.,
```
         prepend-path VAR value " "
         
```
to expand the content of the whitespace separated variable VAR. This especially obviates the necessity of using the extra DEISA_*_INTERNAL variables. Note: This is work in progress since for consistency with the C version the --delimit switch should be added (newer modulecmd.tcl needed).
module switch reloads all modules which appear to have been loaded later than the module to be replaced. This is a nuisance for the reloading mechanism required for the deisa and lrz modules. Hence, a provision (presently a LRZ specific hack) has been added to have an exception list of modules which will always be considered as independent and hence will not be processed. This change has been accepted by the code maintainer.

Additional utilities

modulefile-already-loaded.tcl and modulefile-already-loaded-conflict.tcl: Both check whether one out of a set of module files is already loaded. The "conflict" version indicates a conflict if this is the case, the other one only sets MODULEFILE_LOADED to 1 (Tcl internal variable). Usage is e.g.,
```
set COMPONENT xyz
source "[exec dirname $env(MODULE_BIN)]/modulefile-already-loaded.tcl"
```
I've added the additional return variable MODULEFILE_LOADED_NAME containing the name of the already loaded module to make this facility more useful. Note: these should all be removed.

Reference Files

This section contains reference files which can be used as templates for writing new module files. Note that these reference files also take into account some LRZ specific settings. Also, editing Module files should be performed not in the production environment, but within the CVS sandbox. Hence the default $MODULESHOME etc. settings need to be changed by running the setup as described above.

Module file

The first example is a module file for version 1.3 of software package pac. Please edit the file <arch_specific>/files/<class>/pac/1.3 in the appropriate module class subtree to contain the following text:

#%Module

# LRZ HPC user environment
# DEISA Common Production Environment
#
# Author                : wrzlprmft@lrz.de
# Created               : Wed Jul 11 2012
#

set DIRECTORY "/lrz/sys/<class>/pac"
set VERSION   1.3_build2

# version and architecture independent part in common include file
source $env(MODULE_INCLUDE)/whatever/pac.tcl
# the following line only if DEISA enablement required.
source $env(MODULE_INCLUDE)/whatever/deisa_pac.tcl

Architecture and version independent includes

The include/<class>/pac.tcl file might then contain entries of the following form

# LRZ HPC user environment
# DEISA Common Production Environment
#
# Author                : wrzlprmft@lrz.de
# created               : Wed Jul 11 2012
#
# needs Tcl internal variables
#  DIRECTORY  base directory
#  VERSION    version subdirectory
#  WWWDoc     LRZ WWW server documentation

set WWWDoc "http://www.lrz-muenchen.de/services/software/whatever/pac"
module-whatis "Enable usage of PAC application software"

# TMPDIR needed for scratch files
prereq tempdir
# Only one version at a time
conflict pac
# Note: can also have prereq and conflict lines in module file if necessary

# need to be member of group "pac" to use
set groups [ exec groups $env(USER) ]
if { [ lsearch -exact $groups pac ] == -1 && [module-info mode] == "load" } {
  puts stderr "Your account is not in the UNIX group pac.
Please contact LRZ support to have it entered. Aborting module load."
  break
}
# Tcl internal settings of e.g. PAC_SHLIB required for DEISA setup
set          DIR         "${DIRECTORY}/${VERSION}"
setenv       PAC_BASE    "$DIR"
setenv       PAC_LIBDIR  "${DIR}/lib"
setenv       PAC_INC     "-I${DIR}/include"
set          PAC_LIB     "${DIR}/lib/libpac.a"
setenv       PAC_LIB     "$PAC_LIB"
set          PAC_SHLIB   "-L${DIR}/lib/ -lpac"
setenv       PAC_SHLIB   "$PAC_SHLIB"
setenv       PAC_DOC     "${DIR}/docs"
setenv       PAC_WWW     "$WWWDoc"
setenv       PAC_SRC     "/lrz/noarch/src/PAC"
setenv       PAC_INSTALL_DOC "/lrz/noarch/src/PAC/REAMDE_lrz"
prepend-path LD_LIBRARY_PATH "${DIR}/lib"
prepend-path PATH        "${DIR}/bin"
prepend-path MANPATH     "${DIR}/man"
prepend-path INFOPATH    "${DIR}/info"

# make application usage profile entry
# (do not use for very heavily used stuff like compilers)
if { [module-info mode] == "load" } {
  exec /usr/bin/application-control pac $VERSION $env(HOSTNAME)
}
proc ModulesHelp {} {
  global WWWDoc
  puts stderr "
Enable the usage of PAC application software.
Documentation on LRZ Web Server at
$WWWDoc."

  return 0
}


Note that some application use their own naming scheme for environment variables. This should of course take precedence to the LRZ naming scheme indicated above. The include/<class>/deisa_pac.tcl file might be of the form

#
# enable DEISA settings for pac
# requires the PAC_SHLIB internal Tcl variable
#
if { "[ lindex [ array get env DEISA_ENV ] end ]" == "yes" } {
  if { $env(DEISA_NOCOMPILER_WRAPPERS) == 1 } {
    setenv DEISA_PAC_LDFLAGS "$PAC_SHLIB"
  }
  prepend-path DEISA_LDFLAGS  "$PAC_SHLIB" " "
}


Notes:

Use is made of the prepend-path extension; this LRZ specific change will be replaced by equivalent functionality in a future release of modulecmd.tcl.
The above template files will be integrated into the scripts supporting the workflows as soon as those are ready.

Version file

Finally, a version file <arch_specific>/files/<class>/pac/.version must be created:

#%Module
# LRZ HPC user environment
# DEISA Common Production Environment
##
##  default version
##
set ModulesVersion "1.3"

Additional notes

Internet sources of original package

The source of the original modules package can be obtained from sourceforge using CVS:

   cvs -d :pserver:anonymous@cvs.sourceforge.net:/cvsroot/modules login
   (empty password)
   cvs -d :pserver:anonymous@cvs.sourceforge.net:/cvsroot/modules co modules
   (first checkout, after this updates in base directory should be OK).

All sources below subdirectory src; according to DEISA requirements, the new Tcl version should be used (see subdirectory tcl) if possible. But the module files essentially are compatible (the C version is linked with libtcl).