Development Environment for Linux-based HPC systems at LRZ

This document gives an overview of the most important development tools; a troubleshooting guide for some problems which often arise is also included

Programming Environment

Using Modules to handle environment settings

LRZ uses the modules approach to manage the user environment for different software, library or compiler versions. The distinct advantage of the modules approach is that the user is no longer required to explicitly specify paths for different executable versions, library versions and locations of other entities needed for the execution environment.

Operating environment

All nodes in the cluster run a GNU/Linux based operating environment. Depending on which segment is used, a different release of the operating system may be deployed.

SoftwareVersion
Distribution: SUSE Linux Enterprise Server SLES 11
Kernel 2.11
libc glibc 3.0
gcc release See LRZ gcc document for details
X-Window-System (Xorg-X11) 7.4
GNU all available packages
TSM Backup Client

6.4 (not available on all systems)

Batch Queuing SLURM

Development Tools and Libraries

Overview

ActivityToolsLinux versions

Source code development

Editors vi, emacs, etc.

Executable creation

Compilers

icc, icpc, ifort, gcc, gfortran, g95, pgif90, pgicc

Archiving

Library Archiver

ar

Object file inspection

Object tools

objdump

Debugging

Debuggers

gdb, idb,  Alinea DDT, Intel Inspector

Performance analysis

Profilers

Intel Amplifier, Intel Advisor

Automation

Make

make

Environment configuration

modules, embedded Tcl

module

For details see the Linux Cluster Software page for a listing of available software packages (under the headings "development tools", "mathematical libraries", and "parallelization").

Eclipse (integrated development environment)

Eclipse was designed as an integrated development environments (IDE). The Eclipse CDT installed at LRZ provides an IDE for C/C++ development, as well as keeping the (generic) support for developing Java applications and Eclipse plug ins, since Eclipse itself is programmed in Java.

Main features of the Eclipse CDT include:

  • C/C++ Editor (basic functionality, syntax highlighting, code completion etc.)
  • C/C++ Debugger (using GDB)
  • C/C++ Launcher (APIs & Default implementation, launches and external application)
  • Search Engine
  • Content Assist Provider
  • Makefile generator
  • Graphical CVS management

There are plans at LRZ to install and support additional Eclipse toolkits as these become available and/or stable enough to be recommended for general use. Among these are: Photran, which provides an IDE for Fortran90, 95, possibly 95+. It is currently in beta state, was found to be somewhat unstable on the LRZ 32-bit Linux cluster and is not available yet for IA64 platforms. It will eventually be integrated with CDT, and VTune 8.1+ with full Eclipse support, on IA64 platforms, hopefully including threading tools.

Compilers and Parallel Programming

Intel Compilers and Performance Libraries

LRZ recommends the usage of the Intel Compilers and Performance Libraries as a first choice; licensing and support agreements with Intel ensure that bug-fixes should be available within reasonably short order.

PGI Compilers

Some commercial packages still require availability of the Portland Group compiler suite; furthermore, for Opteron-based (AMD x86_64) systems this compiler may well still be the best choice. Finally, High Performance Fortran is also supported by the Fortran compiler. Hence, this package is still available on the Linux Cluster, and is also licensable in the Munich Campus area.

In order to use the PGI compiler on the interactive nodes of the Linux Cluster, please load the environment module pgi; also note that this product is not available on Itanium-based nodes. Support for the PGI compiler is limited: the LRZ HPC team will report bugs to PGI, but this is kept at low priority.

Documentation for the PGI compilers is available from the PGI web site.

MPI

Programs parallelized with MPI can of course run on the Linux cluster. Please refer to the introductory document for further details on available MPI flavours and their handling.

OpenMP

The Multi-Threading capabilities of Intel SMPs and SGI's ccNUMA can be used via the OpenMP implementations of the PGI and Intel Compilers. Examples for OpenMP programming in Fortran as well as general information about OpenMP can be found in the LRZ OpenMP introduction.

Tuning

Please also consult the SuperMUC tuning document for further details and tools for optimization. Most of these tools are also available in the Linux-Cluster.

Debuggers

Debuggers with graphical Interface (GUI)

  • DDT:             Distributed Debugging Tool: a commercial product by  Allinea Software.
  • Totalview:      A commercial product by Etnus.

The GUI driven debuggers offer a graphical user interface; simple debugging sessions can therefore be  handled without  intensive, prior study of man-pages and manual.

  • DDT and Totalview are  advanced tools for more complex  debugging, especially when it comes to debugging parallel codes (MPI, OpenMP). They allow to inspect  data structures in the different threads of a parallel program, set global breakpoints, set  breakpoints in individual threads, etc.
  • DDT is the preferred debugger at LRZ
  • Totalview can also be used in CLI  mode, whereas DDT is a pure GUI tool.
NameInterfaceProgramming
Model
LRZ moduleDocumentation 
Recommended Debuggers
 DDT GUI g77, g95, icc, ifort serial, parallel (MPI, OpenMP) module load ddt PDF ($DDT_DOC)
Totalview GUI, CLI g77, icc, ifort serial, parallel (MPI, OpenMP) module load totalview PDF ($TOTALVIEW_DOC)
HTML

Other debuggers like gdb, idb or DDD are available, but they can hardly be used on the compute nodes.

Command Line Oriented Debuggers

gdb (GCC)  and idb (Intel Compiler)are available on the Linux Clusters of LRZ

Theses debuggers are command line driven (CLI) und require some knowledge of their commands by the user. Their functionality is quite powerful,and thus supply a wealth of commands; the learning curve for a successful debugging session is therefore quite high.
The Intel debugger idb can be used in the "gdb mode", which means that a user aquainted with the gdb commands can also use these with the idb debugger.
These debuggers are mainly used for debugging serial programs.

idb supplies a graphical interface for the "casual user" in order to lower the "learning curve"; this interface  is available via the command " idb  -gui"

Threading Tools

Threading Tools allow you to perform correctness and performance checking on multi-threaded applications (running in shared memory). The parallelization method may be based on POSIX or Linux Threads, or on OpenMP. For OpenMP applications it is necessary to use the Intel compilers in combination with suitably chosen compiler switches to perform the analysis of applications:

Intel Inspector
Inspector allows you to perform correctness checking on multi-threaded applications (running in shared memory).
Intel Amplifier
Intel Amplifier XE (formerly VTune) allows you to perform performance analysis on multi-threaded applications (running in shared memory). The analyzer also collects, analyzes, and displays hardware performance data from the system-wide view down to a specific function, module, or instruction.

Troubleshooting with the Intel, PGI and GNU Compilers

The troubleshooting section is being extended and has moved to its own web page.