Generic sbatch Templates for Neumann

Windows users: please make sure to convert the script with dos2unix on the linux machine, and read the article on Linebreaks

This is page is a collection of generic templates of job scripts. The latest official template can be found on the Neumann-Cluster-Homepage.

In the variant 2, the line mpirun $debug_mpi ./mpi_app should be comment out when you want to run a mpi process. If you ran special software, such as StarCCM+ you cannot use mpirun. Replace the mpirun line with your command.

If you want to start parallel processes in your job, such as running a programm on every CPU use srun.

Scripts on this page

Variant 1

A simple example, to get familiar with the queue system and job submission syntax:

Windows users: please make sure to convert the script with dos2unix on the linux machine, and read the article on Linebreaks

justatestjob_20170822.sh
#!/bin/bash
# An example batch script to launch a dummy test script on the Neumann cluster
# From command line on Neumann, log into a node, e.g. type "ssh c002"
# then submit this job to the queue system by typing "sbatch justatestjob_20170814.sh"
# If this works correctly, you should see your job in the short queue for about half a minute
# And then it will create a tiny file called "hello" in your scratch directory
 
 
###################################################################################################
# Queue system requests
 
#SBATCH --job-name doobdoob		# job name displayed by squeue
#SBATCH --partition sw01_short		# queue in which this is going
#SBATCH --nodes 2	 		# number of nodes
#SBATCH --time 001:00:00		# time budget [HHH:MM:SS] 
#SBATCH --mem 1G			# RAM memory allocated to each node
 
#SBATCH --dependency singleton	# singleton dependency: do not start this job before any other job with the same job name has finished
 
 
 
###################################################################################################
# Now, let’s run this script
 
# Define where the working directory is, based on the username of the user submitting
WORKINGDIRECTORY="/scratch/tmp/$USER"
 
# Create the scratch folder if it does not exist, and change into it
mkdir -p $WORKINGDIRECTORY
cd $WORKINGDIRECTORY
 
# Do nothing for 30 seconds
sleep 30s
 
# Create a tiny file called "hello" in your working directory and write some text inside it
echo "hello world!" >> $WORKINGDIRECTORY/hello
 
# Write the date at the end
date +%Y-%m-%d_%H:%M:%S_%s_%Z >> $WORKINGDIRECTORY/hello

Variant 2

Windows users: please make sure to convert the script with dos2unix on the linux machine, and read the article on Linebreaks

job-default.sh
#!/bin/bash
# UPDATE: 22.09.2019 simplification by slurmProlog/epilog-script
#  please check http://www-e.uni-magdeburg.de/urzs/t100/ periodically 2016-11
#
# lines beginning with "#SBATCH" are instructions for the jobsystem (man slurm).
# lines beginning with "##SBATCH" are comments
#
#SBATCH -J job-01             # jobname displayed by squeue
#SBATCH -N 4                  # minimum number of nodes needed or minN-maxN
#  do not waste nodes (check scaling of your app), other users may need them
#SBATCH --ntasks-per-node 1   # 1 for multi-thread-codes (using 16 cores)
##SBATCH --ntasks-per-node 2  # 2 for hybrid code, 2 tasks * 8 cores/task
##                            #                     ... [* 2 threads/core]
##SBATCH --ntasks-per-node 16 # 16 for pure MPI-code or 16 single-core-apps
#SBATCH --time 01:00:00       # set 1h walltime (=maximum runtime), see sinfo
#SBATCH --mem 80000           # [MB/node], please use less than 120000 MB
# please use all cores of a node (especially small jobs fitting to one node)
# nodes will not be shared between jobs (avoiding problems) (added 2017.06)
#
# most output is for more simple debugging (better support):
# we are on a 1st node here using sbatch, but on the login node using salloc
#  mpirun will not work on the login node because it is in another subnet
#
. /cluster/apps/utils/bin/slurmProlog.sh  # output slurm settings, node healthy
#
# load modulefiles which set paths to mpirun and libs (see website)
echo "DEBUG: LOADEDMODULES=$LOADEDMODULES" # module list
#module load gcc/4.8.2                # if you need gcc or gcc-libs on nodes
#module load openblas/gcc/64/0.2.15_no_affinity # multithread basic linear algebra
#module load openmpi/gcc/64/1.10.1    # message passing interface
#module load ansys                    # Ansys-Simulations, License needed!
echo "DEBUG: LOADEDMODULES=$LOADEDMODULES" # module list
#
## please use /scratch (200TB 8GB/s), /home is for job preparation only
## do not start jobscript in /scratch but change to it to use massive disk-I/O
##   (conflicting link@master vs. mount@nodes), see website for more info
#mkdir -p /scratch/tmp/${USER}_01    # create directory if not existing
#cd /scratch/tmp/${USER}_01;echo new_pwd=$(pwd) # change to scratch-dir
#
# --- please comment out and modify the part you will need! ---
# --- for MPI-Jobs and hybrid MPI/OpenMP-Jobs only ---
# prepare debug options for small test jobs
## set debug-output for small test jobs only:
[ "$SLURM_NNODES" ] && [ $SLURM_NNODES -lt 4 ] && mpidebug="--report-bindings"
#
# prepare nodefile for software using its own MPI (ansys/fluent, starccm++)
#   self compiled openmpi-programs do not need the nodelist or hostfile
HOSTFILE=slurm-$SLURM_JOBID.hosts
scontrol show hostnames $SLURM_JOB_NODELIST > $HOSTFILE # one entry per host
#
# ## mpirun may autodetect 32 (hyper-)threads as 32 cpus and run very slow
# ##        setting it to no more than 16 will result in optimal performance
# ##        also activate core binding (some versions will auto set it)
# ## outside slurm you need $MPI_HOME/bin/mpirun or --prefix for ompi-1.10.1 
# mpirun -npernode $SLURM_NTASKS_PER_NODE --bind-to core $mpidebug ./mpi_user_app
# mpirun -machinefile $HOSTFILE ./mpi_user_app
# --fluent-- make sure ssh c001 is working without passwd with ssh-key
# fluent -g 3d   -cnf=nodelist-$SLURM_JOBID -pinfiniband -mpi=openmpi \
#   -t$SLURM_NPROCS \
#   -mpitest     # MPI-Network-Test  1.3us, 3140MB/s Reduce=43us(2*16)
#   -i inputfile # Ansys-CFD-Simulation instead of -mpitest
#   # LICENSES: 1node*16cores maximum for teaching license! 
#
# --- for multiple Single-Jobs (Job arrays) and multiple OpenMP-Jobs only ---
#
# ## for ((i=0;i<$SLURM_NPROCS;i++));do ./app1 $i;done  # serial version
# srun bash -c "./app1 \$SLURM_PROCID"                 # parallel version
#
# -------------------------- post-processing -----------
. /cluster/apps/utils/bin/slurmEpilog.sh  # cleanup
#
## Final time stamp
date +%Y-%m-%d_%H:%M:%S # date as YYYY-MM-DD_HH:MM:SS
guide/neumann/jobscript_generirc.txt · Last modified: 2020/02/14 13:26 by seengel
Back to top
CC Attribution-Share Alike 3.0 Unported
chimeric.de = chi`s home Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0