The Totalview Debugger

A short introduction to the usage of the Totalview Debugger and the Memoryscape memory leak detector available on high performance computing platforms at LRZ.

Introduction and Features

The Totalview debugger, in the words of it's User's Guide, is "part of a suite of software development tools for debugging, analyzing and tuning the performance of programs, including multiprocess (MPI) and/or multithreading (OpenMP) programs".

The Totalview Replay Engine is also available, which allows you to roll back the state of your program to any point in the past.

Licensing of Totalview at the LRZ

Leibniz Supercomputing Centre provides floating licenses for Totalview and Replay Engine on its high performance computing platforms.

Setting up Totalview

Preparing Your Program

In order to enable debugging, your program needs to be compiled with special switches which insert debugging information into the binary executable. Totalview is perfectly capable of debugging code which was generated without debugging information, however unless you're able to read and analyze machine code the debugging sessions might prove not to be particularly enjoyable ...

Here's a table indicating the debugging switches which are recommended for compilation and linkage before debugging an executable with Totalview:

Compiler

switches for Compilation

switches for Linkage

Remarks

ifort/mpif90 (Intel)

-g -O2 -check all -traceback

-traceback

-C activates extensive runtime error checking; for array bound checking only replace -check all by -check bounds

icc/icpc/mpicc/mpiCC (Intel)

-g -O2 [-traceback]

-traceback only useful for C programs linked into Fortran executables

Notes:

  • It is recommended to switch optimization on in addition to adding debugging symbols since otherwise the code (built at -O0) may be quite slow.
  • If you use self-built libraries it may be advisable to generate debug versions of these as well.
  • The debugger should also work with other compilers available on the LRZ HPC systems. Switch settings may require modifications for these, please check the compiler documentation.

Calling Totalview

If the environment module totalview is not loaded at login (check with module list), please issue the command

module load totalview

Then, start the GUI by entering the command

totalview &

Configuring the program run

Upon calling the GUI, two widgets will appear: The main window -

and the initial configuration window -

IA64 systems

x86_64 systems with Replay support

The four tabs in this window must be filled in with the necessary information for your run, in particular:

  • program name
  • program arguments
  • environment variables (e.g., OMP_NUM_THREADS) for threaded parallel runs.
  • stdin and stdout if applicable
  • parallel setup (e.g., MPI flavor and number of tasks to run. Important Note on MPI flavor: If any mpi.altix module is loaded, you need to select "MPT". If any mpi.intel module is loaded, you need to select "Intel MPI". The mpi.parastation module is unsupported.).

After this has been performed, you can start the run by pressing the OK button. This will start the main debugging window, the functionality of which is described in the documentation.

 

Documentation, support and further information about Totalview and Memoryscape

If you experience any problems with Totalview, please contact LRZ HPC support via the Service Desk.

Whoever likes to read manuals before actually using the software might like to refer to the Documentation page on the Totalview Web Site.

Documentation in PDF format is locally available on the login nodes of the HPC systems; the environment variable $TOTALVIEW_DOC points to the directory containing the PDFs.

LLNL's TotalView Tutorial.