...
If your job does not use the system openmpi, or does not use MPI, then any desired core binding will need to be set up with whatever mechanism the software uses. Otherwise, there will be no core binding. Again, that may not be a major issue. If your job does not work well with HT then run on a number of cores equal to half of the number of slots requested and the OS scheduler will minimize contention.
new SGE utilities
While SoGE is very similar to previous versions of SGE there are some new utilities that people may find of interest. There are manual pages for each of these.
...
| Table plus |
|---|
| sortColumn | 1 |
|---|
| columnTypes | S,S,S,I,I |
|---|
|
| Queue | Node Description | Queue Manager | Slots | Total memory (GB) |
|---|
| AML | (1) mid memory | Aaron Miller | 56 | 256 |
|---|
| ANTH | (4) standard memory | Andrew Kitchen | 224 | 512 |
|---|
ARROMA | (8) standard memory | Jun Wang | 448 | 1024 |
|---|
| AS | (5) mid memory | Katharine Corum | 280 | 1280 |
|---|
| BH | (1) high memory | Bin He | 56 | 512 |
|---|
BIGREDQ | (13) mid memory | Sara Mason | 728 | 3328 |
|---|
BIOLOGY | (1) mid memory | Matthew Brockman
| 56 | 256 |
|---|
BIOSTAT | (2) standard memory | Grant Brown | 112 | 256 |
|---|
| BIO-INSTR | (3) mid memory | JJ Urich, Albert Erives | 168 | 768 |
|---|
| CBIG | (1) mid memory with P100 accelerator | Mathews Jacob | 56 | 256 |
|---|
| CBIG-HM | (1) high memory with P100 accelerator | Mathews Jacob | 56 | 512 |
|---|
| CCOM | (18) high memory
5 running jobs per user | Boyd Knosp | 1008 | 9216 |
|---|
| CCOM-GPU | (2) high memory with P100 accelerator | Boyd Knosp | 112 | 1024 |
|---|
CGRER + LMOS | (10) standard memory | Jeremie Moen | 560 | 1280 |
|---|
| CHEMISTRY | (3) mid memory | JJ Urich | 168 | 768 |
|---|
CLAS-INSTR | (2) mid memory | JJ Urich | 112 | 512 |
|---|
| CLL | (5) standard memory | Mark Wilson
Brian Miller | 280 | 640 |
|---|
| COB | (2) mid memory | Brian Heil | 112 | 512 |
|---|
| COE | (10) mid memory Note: Users are restricted to no more than three running jobs in the COE queue. | Matt McLaughlin | 560 | 2560 |
|---|
DARBROB | (1) mid memory | Benjamin Darbro | 56 | 256 |
|---|
| FERBIN | (13) standard memory | Adrian Elcock | 728 | 1664 |
|---|
MF | (6) standard memory | Michael Flatte | 336 | 768 |
|---|
| MF-HM | (2) high memory | Michael Flatte | 112 | 1024 |
|---|
FLUIDSLAB | (8) standard memory | Mark Wilson
Brian Miller | 448 | 1024 |
|---|
| AIS | (1) mid memory | Grant Brown | 56 | 256 |
|---|
GEOPHYSICS | (3) standard memory | William Barnhart | 168 | 384 |
|---|
| GV | (2) mid memory | Mark Wilson
Brian Miller | 112 | 512 |
|---|
| HJ | (10) standard memory | Hans Johnson | 560 | 1280 |
|---|
| HJ-GPU | (1) high memory with P100 accelerator | Hans Johnson | 56 | 512 |
|---|
| IFC | (10) mid memory | Mark Wilson
Brian Miller | 560 | 2560 |
|---|
| IIHG | (10) mid memory | Diana Kolbe | 560 | 256 |
|---|
INFORMATICS | (12) mid memory | Ben Rogers | 672 | 3072 |
|---|
INFORMATICS-GPU | (2) mid memory with P100 accelerator | Ben Rogers | 112 | 512 |
|---|
INFORMATICS-HM | (1) high memory | Ben Rogers | 56 | 512 |
|---|
| IVR | (4) mid memory
(1) high memory | Todd Scheetz | 280 | 1536 |
|---|
| IVR-GPU | (1) high memory with K80 accelerator | Todd Scheetz | 56 | 1536 |
|---|
| IVRVOLTA | (4) high memory with Titan V | Mike Schnieders | 224 | 2048 |
|---|
| IWA | (11) standard memory | Mark Wilson
Brian Miller | 616 | 1408 |
|---|
| JM | (3) high memory | Jake Michaelson | 168 | 512 |
|---|
| JM-GPU | (1) mid memory with P100 accelerator | Jake Michaelson | 56 | 512 |
|---|
| JP | (2) high memory | Virginia Willour | 112 | 1024 |
|---|
| JS | (10) mid memory | James Shepherd | 560 | 2560 |
|---|
| LUNG | (2) high memory with P40 accelerator | Joe Reinhardt | 112 | 1024 |
|---|
| MANSCI | (1) standard memory | Qihang Lin | 56 | 128 |
|---|
| MANSCI-GPU | (1) high memory with P100 accelerator | Qihang Lin | 56 | 512 |
|---|
| MANORG | (1) standard memory | Michele Williams/Brian Heil | 56 | 128 |
|---|
MORL | (5) mid memory | Mike Schnieders William (Daniel) Walls | 280 | 1280 |
|---|
| MORL-GPU | (105) mid memory with dual P100 acceleratoraccelerators | Mike Schnieders William (Daniel) Walls | 560280 | 25601280 |
|---|
| NEURO | (1) mid memory | Marie Gaine/Ted Abel | 56 | 256 |
|---|
| NOLA | (1) high memory | Ed Sander | 56 | 512 |
|---|
| PINC | (6) mid memory | Jason Evans | 336 | 1536 |
|---|
| REX | (4) standard memory | Mark Wilson
Brian Miller | 224 | 512 |
|---|
| REX-HM | (1) high memory | Mark Wilson
Brian Miller | 56 | 512 |
|---|
| SB | (4) standard memory | Scott Baalrud | 224 | 512 |
|---|
| STATEPI | (1) mid-memory | Linnea Polgreen | 56 | 256 |
|---|
| UDAY | (4) standard memory | Mark Wilson
Brian Miller | 224 | 512 |
|---|
| UI | (20) mid memory | | 1120 | 5120 |
|---|
UI-DEVELOP | (1) mid memory
(1) mid memory with P100 accelerator | | 112 | 512 |
|---|
| UI-GPU | (4) mid memory with P100 accelerator | | 224 | 1024 |
|---|
| UI-HM | (5) high memory | | 280 | 2560 |
|---|
| UI-MPI | (19) mid memory | | 1064 | 4864 |
|---|
| all.q | (115) standard memory
(149) mid memory
(19) mid memory with P100 accelerator
(49) high memory
(9) high memory with P100 accelerator
(2) high memory with K80 accelerator | | 19208 | 87998 |
|---|
| NEUROSURGERY | (1) high memory with K80 accelerator | Haiming Chen | 56 | 512 |
|---|
| SEMI | (1) standard memory | Craig Pryor | 56 | 128 |
|---|
| ACB | (1) mid memory | Adam Dupuy | 56 | 256 |
|---|
| FFME | (16) standard memory | Mark Wilson | 896 | 2048 |
|---|
| FFME-HM | (1) high memory | Mark Wilson | 56 | 512 |
|---|
| RP | (2) high memory | Robert Philibert | 112 | 1024 |
|---|
| LT | (2) high memory with P100 accelerator | Luke Tierney | 112 | 1024 |
|---|
| KA | (1) high memory | Kin Fai Au | 56 | 512 |
|---|
|
...
There some non-Boolean resources for GPU nodes that could be useful in a shared node scenario. Most of these are requestable but some are informational. Note that these are host based resources so are probably mostly useful when using the all.q queue for jobs. GPU jobs in investor queues will most likely want to use the Boolean resources listed in the previous table.
| Resorce | Description | Requestable |
|---|
| gpu.ncuda | number of CUDA GPUs on the host | YES |
| gpu.nopencl | number of OpenCL GPUs on the host | YES |
| gpu.ndev | total number of GPUs on the host | YES |
| gpu.cuda.N.mem_free | free memory on CUDA GPU N | YES |
| gpu.cuda.N.procs | number of processes on CUDA GPU N | NO |
| gpu.cuda.N.clock | maximum clock speed of CUDA GPU N (in MHz) | YES |
gpu.cuda.N.util | compute utilization of CUDA GPU N (in %) | NO |
gpu.cuda.procsum | total number of processes running on devices | NO |
gpu.cuda.dev_free | number of devices with no current processes | YES |
| gpu.opencl.0.clock | maximum clock speed of OpenCL GPU N (in MHz) | YES |
| gpu.opencl.0.mem | global memory of OpenCL GPU N (in MHz) | YES |
| gpu.names | semi-colon-separated list of GPU model names | YES |
For example, to request a node with at least 2G of memory available on the first GPU device:
...