ALIs

kommt noch

SGE resource specifications

Introductory remarks

Batch job options and resources can be given as command line switches to qsub, or they can be embedded into a SGE job script as a comment line of the form

#$ option

for a SGE script. The example job scripts illustrate the use of these control sequences.

General options (Shell, Execution Path, Mail, Output etc.)

SGE option

Functionality

Remarks

-cwd

Start job in directory from which it was submitted.

If this is not specified, the home directory will be the starting directory of the job.

-S <shell>

Use designated shell to execute the job script

Obligatory, since the standard first line #! <shell> is ignored due to a change in the SGE configuration. The entry used for the -S switch must be consistent with the shell dialect used in the batch script.

-M <mail_addr>

User's e-mail address.

Obligatory so LRZ can contact you in case of problems. Batch requests which do not specify an e-mail address will not be processed.

-m [a|b|e]

Batch system sends e-mail when [aborting|starting|ending] job.

 

-N <req_name>

Name of batch request. Default is name of the script.

8 characters at most will be retained.

-o <filename>

write standard output to specified file.

LRZ recommends specifying the full path name. Default value is job_name.ojob_id, where job_name is the name of the job specified via the -N parameter, and job_id its identification number.

Note that under SGE it is possible to use one of the following pseudo-environment variables as part of the full name specification:

$HOME home directory
$USER user ID
$JOB_ID job ID
$JOB_NAME job name (see -N option)
$HOSTNAME name of target node
$SGE_TASK_ID job array task index

These are replaced by the run-time contents of the actual environment variables. Other environment variables are not supported via this mechanism.

-e <filename>

write standard error to specified file. Under SGE, the same pseudo-environment variables can be used as for the -o option. LRZ recommends specifying the full path name. job_name.ejob_id is used as a default if no explicit name is specified.

-j y

write standard error to the same file as standard output

Any -e specification is ignored.

-v var1[=val1][,var2[=val2],...]

exports designated environment variables into SGE job

If only the name of a variable is specified, the existing value is used in the SGE script at run time. For more than one variable, a comma-separated list must be specified. Example:
-v PATH,MYVAR=xxyy,LD_LIBRARY_PATH.
Please use with care: It is probably better to write an environment module and load that in the script.

-V

exports all presently defined environment variables into SGE job.

Beware: This setting should not be used on the LRZ HPC systems since it interferes destructively with the module based configuration environment.

Job Control, Limits and Resource Requirements

SGE option

Functionality

Remarks

-hard

All the following requirements must be fulfilled for the job to be initiated.

This is in fact the default, so you actually only need this to invalidate a previous -soft option.

-soft

after fulfillment of all -hard requirements the job is processed within the queue fulfilling as many -soft specifications as possible.

This means that at runtime not all requirements specified as -soft may be fulfilled. Hence the user's program must be able to cope with this situation.

-h

batch request is kept in User Hold (meaning it is queued but not initiated)

With qalter -hU <job_id> you can remove the user hold from your job.

-l h_rt=x:y:z

specify hard limit for job run time as x hours y minutes z seconds. All three fields must be provided.

The maximum run time value imposed by LRZ cannot be exceeded using this option.

-l mf=<memory>

explicit memory requirement (mf abbreviates mem_free)

Note that contradictory requirements (e. g., a low-memory queue and a high explicit memory requirement) will result in a non-initiating job. The "mf" setting replaces the "vf" setting, which could lead to swapping and consequent performance degradation since "vf" ("virtual_free") refers to free virtual memory, whereas "mf" refers to physical memory.

E
xamples:

-l mf=1200M specifies 1200 * 1024 * 1024 Bytes
-l mf=1200m specifies 1200 * 1000 * 1000 Bytes
-l mf=700000K specifies 700000 * 1024 Bytes
-l mf=700000k specifies 700000 * 1000 Bytes

 

-r y[es]|n[o]

Identifies the ability of a job to be rerun or not. If the value of -r is 'yes', the job will be rerun if the job was
aborted without leaving a consistent exit state. (This is typically the case if the node on which the job is running
crashes). If -r is 'no', the job will not be rerun under any circumstances.

Default is 'yes'.

-q <queue_name>

batch request targets a specific queue (mapped to a specific node)

Note that qstat -f gives you a table the first column of which gives the queue names. It is also possible to specify multiple queue names in a comma-separated list. The job then runs on the first queue becoming available. This option is deprecated for normal batch jobs; it should be only used for specific purposes (e.g., for qrsh to the Altix)

-pe <par_env> <num_cpus>

Specification of a parallel environment and the number of CPU's. The latter must be given as a fixed value.

Not supported any more at the LRZ SGE installation. Please submit a SLURM job if you need to do parallel processing.

-t n[:m[:s]]

submits an array job: n, m and s are integers with n, m and s > 0.

An array job means that in fact a number of jobs are run; each job is characterized by a value of $SGE_TASK_ID (available inside the script) which runs between n and m with a step size of s. If only n is specified, $SGE_TASK_ID takes the value n and only one task is performed.

 

Environment Variables

In addition to those environment variables specified to be exported to the job via the -v or the -V option (see above)  qsub, qsh, and qlogin add the following variables with the indicated values to the variable list (for a complete list see: man qsub)

SGE_O_HOME the home directory of the submitting client.
SGE_O_PATH the executable search path of the submitting client.
SGE_O_WORKDIR the absolute path of the current working directory of the submitting client.
SGE_STDERR_PATH
SGE_STDOUT_PATH
the pathname of the file to which the standard error/standard output stream of the job is diverted
SGE_TASK_ID The index number of the current array job task (see -t option above). This is an unique number in each
array job and can be used to reference different input data records, for example.
SGE_TASK_FIRST
SGE_TASK_LAST
SGE_TASK_STEPSIZE
The index number of the first/last/increment array job task
ENVIRONMENT The variable is set to BATCH to identify that the job is being executed under SGE control
JOB_ID
JOB_NAME
A unique identifier assigned by SGE
The jobname set by -N.
NSLOTS The number of slots (normally the same as the number of cores) for a parallel job.
RESTARTED This variable is set to 1 if a job was restarted either after a system crash or an interruption.