Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

...

For MPI jobs, the system provided openmpi will not bind processes to cores by default, as would be the normal default for openmpi. In addition, the system openmpi settings will treat the HT cores as processors. This may be important if you wish to run hybrid MPI/OpenMP threaded jobs.

Info

If you do not wish the HT cores to be treated as CPU processors by OpenMPI then add the following option to the mpirun command

No Format
mpirun -mca hwloc_base_use_hwthreads_as_cpus false

or set the equivalent environment variable in the job script

No Format
OMPI_MCA_hwloc_base_use_hwthreads_as_cpus=false



The binding parameters can be overridden with parameters to mpirun. Openmpi provides fine grained control of process layout. The options that are set by default should be good in most cases but can be overridden with the openmpi options for

...

If your job does not use the system openmpi, or does not use MPI, then any desired core binding will need to be set up with whatever mechanism the software uses. Otherwise, there will be no core binding. Again, that may not be a major issue. If your job does not work well with HT then run on a number of cores equal to half of the number of slots requested and the OS scheduler will minimize contention. 

new SGE utilities

While SoGE is very similar to previous versions of SGE there are some new utilities that people may find of interest. There are manual pages for each of these.

...