ALIs

kommt noch

SGE example job scripts

 

Introductory remarks

  • It is necessary to explicitly specify the execution shell. The first #! ... line is ignored, and only left in so as to be able to run the script interactively as well.

  • Some entries are placeholders, which you must replace with correct, user-specific settings. In particular, path specifications and e-Mail addresses must be adapted.

  • For recommendations on how to do large-scale I/O please refer to the description of the file systems available on the cluster. It is recommended to keep executables within your HOME file system, in particular for parallel jobs. The example jobs reflect this, assuming that files are opened with relative path names from within the executed program.

  • In case you have to work with the environment modules package in your batch script, you also have to source the file /etc/profile.

  • Please note that the program jobperf is presently available for the Itanium-based Altix system only.

Serial jobs

x86_64 Cluster

 
#!/bin/bash
#$-M my_email_address@mydomain
#$-S /bin/bash
#$-N myjob
#$-o $HOME/mydir/myjob.$JOB_ID.out -j y
#$-l mf=3000M
#$-l march=x86_64
. /etc/profile
cd mydir
./myprog

Archiving jobs

The following job examples provide template scripts for archiving or retrieving data. These items are noteworthy:

  • While it is possible (and allowed) to perform TSM archiving on one of the interactive nodes, you may be able to increase throughput by using an SGE job instead.

  • Running a job with scratch data on the $OPT_TMP directory on altix1.lrz-muenchen.de may leave you with unaccessible data, since this area is node-local. It is recommended to use a separate SGE job to copy data to a shared file system or to the TSM archive since using your parallel production script for this purpose would waste CPU resources.

  • If using the script to archive from $OPT_TMP on Altix, you need to know on which of the systems your production job was executed. The production job scripts for parallel jobs hence need to contain an entry hostname, which provided the necessary information.

  • For archiving jobs, a special resource data_handling has been defined which allows to use a single socket on any system the job script has been assigned to.

Cluster

 
#!/bin/bash
#$-M my_email_address@my_domain
#$-S /bin/bash
#$-N tsm_job
#$-o $HOME/mydir/tsm_job.out -j y
#$-l data_handling=yes
#$-l march=x86_64

. /etc/profile
cd mydir
dsmc archive --subdir=yes "mydata.dir/*"
# dsmc retrieve --subdir=yes "./mydata.dir/*"