...
If your job does not use the system openmpi, or does not use MPI, then any desired core binding will need to be set up with whatever mechanism the software uses. Otherwise, there will be no core binding. Again, that may not be a major issue. If your job does not work well with HT then run on a number of cores equal to half of the number of slots requested and the OS scheduler will minimize contention.
new SGE utilities
While SoGE is very similar to previous versions of SGE there are some new utilities that people may find of interest. There are manual pages for each of these.
...
| Table plus |
|---|
| sortColumn | 1 |
|---|
| allowExport | true |
|---|
| columnTypes | S,S,S,I,I |
|---|
|
| Queue | Node Description | Queue Manager | Slots | Total memory (GB) |
|---|
| AML | (1) 56-core 256GB | Aaron Miller | 56 | 256 |
|---|
| ANTH | (4) 56-core 128GB
(2) 32-core 64GB | Andrew Kitchen | 278 | 640 |
|---|
ARROMA | (8) 56-core 128GB | Jun Wang | 448 | 1024 |
|---|
| AS | (5) 56-core 256GB
(7) 32-core 64GB | Katharine Corum | 504 | 1728 |
|---|
| BH | (1) 56-core 512GB | Bin He | 56 | 512 |
|---|
BIGREDQ | (13) 56-core 256GB | Sara Mason | 728 | 3328 |
|---|
BIOLOGY | (1) 56-core 256GB | Matthew Brockman
| 56 | 256 |
|---|
BIOSTAT | (2) 56-core 128GB
(2) 32-core 256GB
(1) 24-core 512GB | Patrick Breheny
Grant Brown
Yuan Huang
Dan Sewell
Brian Smith | 200 | 1280 |
|---|
| BIO-INSTR | (3) 56-core 256GB | Brad Carson
Bin He
Jan Fassler | 168 | 768 |
|---|
| CBIG | (1) 56-core 256GB with P100 accelerator (1) 64-core 192GB, not yet equipped with accelerators | Mathews Jacob | 120 | 448 |
|---|
| CBIG-HM | (1) 56-core 512GB with P100 accelerator | Mathews Jacob | 56 | 512 |
|---|
| CCOM | (18) 56-core 512GB
5 running jobs per user | Boyd Knosp | 1008 | 9216 |
|---|
| CCOM-GPU | (2) 56-core 512GB with P100 accelerator | Boyd Knosp | 112 | 1024 |
|---|
CGRER | (10) 56-core 128GB
(6) 32-core 64GB | Jeremie Moen | 752 | 1664 |
|---|
| CHEMISTRY | (3) 56-core 256GB | Brad Carson | 168 | 768 |
|---|
CLAS-INSTR | (2) 56-core 256GB | Brad Carson | 112 | 512 |
|---|
| CLAS-INSTR-GPU | (2) 40-core 192GB with 1080Ti accelerators (One node with single, one node with two accelerators) | Brad Carson | 80 | 384 |
|---|
| CLL | (5) 56-core 128GB
(10) 32-core 64GB
(1) 32-core 256GB | Mark Wilson
Brian Miller | 632 | 1536 |
|---|
| COB | (2) 56-core 256GB
(1) 32-core 64GB | Brian Heil | 144 | 576 |
|---|
| COB-GPU | (1) 40-core 192GB with (2)Titan V accelerators | Brian Heil | 40 | 192 |
|---|
| COE | (10) 56-core 256GB
(11) 32-core 256GB
(10) 32-core 64GB Note: Users are restricted to no more than three running jobs in the COE queue. | Matt McLaughlin | 1232 | 6016 |
|---|
| COE-GPU | (2) 40-core 192GB with (4) Titan V accelerators (2) 40-core 192GB with (4) 1080Ti accelerators | Matt McLaughlin | 160 | 768 |
|---|
DARBROB | (1) 56-core 256GB | Benjamin Darbro | 56 | 256 |
|---|
| FERBIN | (13) 56-core 128GB | Adrian Elcock | 728 | 1664 |
|---|
MF | (6) 56-core 128GB
(1) 32-core 64GB | Michael Flatte | 368 | 832 |
|---|
| MF-HM | (2) 56-core 512GB | Michael Flatte | 112 | 1024 |
|---|
FLUIDSLAB | (8) 56-core 128GB | Mark Wilson
Brian Miller | 448 | 1024 |
|---|
| AIS | (1) 56-core 256GB | Grant Brown | 56 | 256 |
|---|
GEOPHYSICS | (3) 56-core 128GB | William Barnhart | 168 | 384 |
|---|
| GV | (2) 56-core 256GB | Mark Wilson
Brian Miller | 112 | 512 |
|---|
| HJ | (10) 56-core 128GB
(11) 32-core 64-GB
(4) 32-core 256-GB
(1) 24-core 512-GB | Hans Johnson | 1064 | 3520 |
|---|
| HJ-GPU | (1) 56-core 512GB with P100 accelerator | Hans Johnson | 56 | 512 |
|---|
| IFC | (10) 56-core 256GB | Mark Wilson
Brian Miller | 560 | 2560 |
|---|
| IIHG | (10) 56-core 256GB | Diana Kolbe | 560 | 256 |
|---|
INFORMATICS | (12) 56-core 256GB
(15) 32-core 256GB | Ben Rogers,
UI3 Faculty | 1152 | 6912 |
|---|
INFORMATICS-GPU | (2) 56-core 256GB with Titan V accelerators
(2) 40-core 192GB with (3) Titan V accelerators | Ben Rogers | 192 | 896 |
|---|
INFORMATICS-HM-GPU | (1) 56-core 512GB with (2) P100 accelerators | Ben Rogers | 56 | 512 |
|---|
| IVR | (4) 56-core 256GB
(1) 56-core 512GB
(4) 32-core 64GB
(1) 32-core 256GB | Todd Scheetz | 440 | 2048 |
|---|
| IVR-GPU | (1) 56-core 512GB with K80 accelerator | Todd Scheetz | 56 | 1536 |
|---|
| IVRVOLTA | (4) 56-core 512GB with Titan V | Mike Schnieders | 224 | 2048 |
|---|
| IWA | (11) 56-core 128GB | Mark Wilson
Brian Miller | 616 | 1408 |
|---|
| JM | (3) 56-core 512GB
(2) 32-core 512GB
(2) 24-core 512GB | Jake Michaelson | 280 | 3584 |
|---|
| JM-GPU | (1) 56-core 256GB with P100 accelerator | Jake Michaelson | 56 | 512 |
|---|
| JP | (2) 56-core 512GB | Virginia Willour | 112 | 1024 |
|---|
| JS | (10) 56-core 256GB | James Shepherd | 560 | 2560 |
|---|
| LUNG | (2) 56-core 512GB with P40 accelerator | Joe Reinhardt | 112 | 1024 |
|---|
| MANSCI | (1) 56-core 128GB
(1) 32-core 64GB | Qihang Lin | 88 | 192 |
|---|
| MANSCI-GPU | (1) 56-core 512GB with P100 accelerator | Qihang Lin | 56 | 512 |
|---|
| MANORG | (1) 56-core 128GB | Michele Williams/Brian Heil | 56 | 128 |
|---|
MORL | (5) 56-core 256GB | William (Daniel) Walls | 280 | 1280 |
|---|
| MS | (5) 56-core 256G with (2) P100 GPUs
(7) 40-core 96G with (4) 1080Ti GPUs
(1) 40-core 96G with (4) Titan V GPUS
(20) 32-core 64GB | Mike Schnieders | 1240 | 3328 |
|---|
| NEURO | (1) 56-core 256GB | Marie Gaine/Ted Abel | 56 | 256 |
|---|
| NOLA | (1) 56-core 512GB | Ed Sander | 56 | 512 |
|---|
| PINC | (6) 56-core 256GB | Jason Evans | 336 | 1536 |
|---|
| REX | (4) 56-core 128GB | Mark Wilson
Brian Miller | 224 | 512 |
|---|
| REX-HM | (1) 56-core 512GB | Mark Wilson
Brian Miller | 56 | 512 |
|---|
| SB | (4) 56-core 128GB
(2) 32-core 64GB | Scott Baalrud | 288 | 576 |
|---|
| STATEPI | (1) 56-core 256GB | Linnea Polgreen | 56 | 256 |
|---|
| UDAY | (4) 56-core 128GB | Mark Wilson
Brian Miller | 224 | 512 |
|---|
| UI | (20) 56-core 256GB
(66) 32-core 64GB | | 11203232 | 51209344 |
|---|
UI-DEVELOP | (1) 56-core 256GB
(1) 56-core 256GB with P100 accelerator | | 112 | 512 |
|---|
| UI-GPU | (45) 56-core 256GB with P100 accelerator
(2) 40-core 192GB with (4) 1080Ti accelerators
(4) 40-core 192Gb 192GB with (4) Titan V accelerators
(1) 40-core 192GB with (2) Titan V accelerators | | 464560 | 21762624 |
|---|
| UI-HM | (5) 56-core 512GB
(3) 24-core 512GB | | 280352 | 25604096 |
|---|
| UI-MPI | (19) 56-core 256GB | | 1064 | 4864 |
|---|
| all.q | (174) 32-core 64GB
(115) 56-core 128GB
(2) 64-core 192GB
(15449) 5632-core 256GB
(7154) 56-core 256GB with (1) P100
(8) 24-core 512GB
(2) 32-core 512GB
(42) 56-core 512GB
(10) 32-core 64GB with (1) K20 accelerator
(51) 5632-core 256GB with (21) P100 acceleratorsK20 accelerator
(2) 56-core 256GB 512GB with (1) Titan V K80 accelerator
(426) 56-core 512GB256GB with (1) P100 accelerator
(24) 56-core 512GB 256GB with (12) K80 acceleratorP100 accelerators
(98) 56-core 512GB with (1) P100 accelerator
(12) 56-core 512GB with (2) P100 accelerators
(2) 56-core 512GB with (1) P40 accelerator
(43) 56-core 512GB 256GB with (1) Titan V accelerator
(64) 4056-core 192GB 512GB with (41) Titan V acceleratorsaccelerator
(21) 40-core 192GB 96GB with (34) Titan V acceleratorsaccelerator
(16) 40-core 192GB with (24) Titan V accelerators
(42) 40-core 192GB with (43) 1080Ti Titan V accelerators
(12) 40-core 192GB with (2) 1080Ti Titan V accelerators
(17) 40-core 192GB 96GB with (14) 1080Ti acceleratoraccelerators
(14) 40-core 96GB 192GB with (1) Titan V accelerator4) 1080Ti accelerators
(71) 40-core 96GB 192GB with (42) 1080Ti accelerators
(1) 6440-core 192GB (No accelerators at this time, will be equipped with multiple)with (1) 1080Ti accelerator
| | 20192 | 9203228064 |
|
|---|
| NEUROSURGERY | (1) 56-core 512GB with K80 accelerator | Haiming Chen | 56 | 512 |
|---|
| SEMI | (1) 56-core 128GB | Craig Pryor | 56 | 128 |
|---|
| ACB | (1) 56-core 256GB | Adam Dupuy | 56 | 256 |
|---|
| FFME | (16) 56-core 128GB | Mark Wilson | 896 | 2048 |
|---|
| FFME-HM | (1) 56-core 512GB | Mark Wilson | 56 | 512 |
|---|
| RP | (2) 56-core 512GB | Robert Philibert | 112 | 1024 |
|---|
| LT | (2) 56-core 512GB with P100 accelerator
(1) 32-core 256GB with K20 accelerator
(1) 24-core 512GB | Luke Tierney | 168 | 1792 |
|---|
| KA | (1) 56-core 512GB
(1) 32-core 256GB | Kin Fai Au | 88 | 768 |
|---|
| BA | (2) 32-core 64GB | Bruce Ayati | 64 | 128 |
|---|
| DAWSON | (1) 32-core 256GB | Jeff Dawson | 32 | 256 |
|---|
| FISH | (9) 32-core 64GB | Larry Weber | 288 | 576 |
|---|
| GW | (2) 32-core 256GB | Ginny Willour | 64 | 512 |
|---|
| JES | (1) 56-core 512GB | Jacob Simmering | 56 | 512 |
|---|
| MP | (1) 32-core 256GB | Miles Pufall | 32 | 256 |
|---|
| PABLO | (6) 32-core 64GB
(1) 32-core 256GB |
| 224 | 640 |
|---|
| SH | (5) 32-core 256GB | Shizhong Han | 160 | 1280 |
|---|
| SHIP | (10) 32-core 64GB | Fred Stern | 320 | 640 |
|---|
| TB | (1) 32-core 64GB | Terry Braun | 32 | 64 |
|---|
|
The University of Iowa (UI) queue
A significant portion of the HPC cluster systems at UI were funded centrally. These nodes are put into queues named UI or prefixed with UI-.
- UI → Default queue
- UI-HM→ 56-core 512GB nodes; request only for jobs that need more memory than can be met with the standard nodes.
- UI-MPI → MPI jobs; request only for jobs that can take advantage of multiple nodes.
- UI-GPU → Contains nodes with GPU accelerators; request only if job can use a GPU accelerator.
- UI-DEVELOP → Meant for small, short running job prototypes and debugging.
...
| Centrally funded queues | Node Description | Wall clock limit | Running jobs per user |
|---|
| UI | (20) 56-core 256GB
(66) 32-core 64GB | None | 2 |
|---|
| UI-HM | (5) 56-core 512GB
(3) 24-core 512GB | None | 1 |
|---|
UI-MPI
(56 slot minimum) | (2019) 56-core 256GB | 48 hours | 1 |
|---|
| UI-GPU | (45) 56-core 256GB with P100 accelerator
(2) 40-core 192GB with (4) 1080Ti accelerators
(4) 40-core 192Gb 192GB with (4) Titan V accelerators
(1) 40-core 192GB with (2) Titan V accelerators | None | 1 |
|---|
| UI-DEVELOP | (1) 56-core 256GB
(1) 56-core 256GB with P100 accelerator | 24 hours | 1 |
|---|
...
In addition to the above, the HPC systems have some nodes that are not part of any investor queue. These are in the /wiki/spaces/hpcdocs/pages/76513448 and are used for node rentals and future purchases. The number of nodes for this purpose varies.
(66) 32-core 64GB