The Argon HPC system is the latest HPC system of the University of Iowa. It consists of 346 366 compute nodes , each of which contain 28 2.4GHz Intel Broadwell processor cores, running CentOS-7.4 Linux. There are several compute node configurations,
- standard memory → 128GB
- mid-memory → 256GB
- high-memory → 512GB
There are 16 machines with single
- 56-core 128G
- 56-core 256G
- 56-core 512G
- 40-core 96G
- 40-core 192G
There are 22 machines with Nvidia P100 accelerators, 6 2 machines with dual Nvidia P100 K80 accelerators, 2 machines with an Nvidia K80 acceleratorP40 accelerators, 2 12 machines with an Nvidia P40 accelerator1080Ti accelerators, and 6 16 machines with a Titan V acceleratoraccelerators.
| Info |
|---|
The Titan V is now considered as a supported configuration in GPU-capable compute nodes but is restricted to a single card per node. Staff have completed the qualification process for the 1080 Ti and concluded that it is not a viable solution to add to current Argon compute nodes. |
The Rpeak (theoretical Flops) is 285is 398.60 69 TFlops, not including the accelerators, with 67.25 TB of memory. In addition, there are 2 login nodes of the same Broadwell system architecture. The login nodes have 256GB of memory.
...
If your job does not use the system openmpi, or does not use MPI, then any desired core binding will need to be set up with whatever mechanism the software uses. Otherwise, there will be no core binding. Again, that may not be a major issue. If your job does not work well with HT then run on a number of cores equal to half of the number of slots requested and the OS scheduler will minimize contention.
new SGE utilities
While SoGE is very similar to previous versions of SGE there are some new utilities that people may find of interest. There are manual pages for each of these.
...
| Table plus |
|---|
| sortColumn | 1 |
|---|
| allowExport | true |
|---|
| columnTypes | S,S,S,I,I |
|---|
| columnCalculations | all.q,,,sum,sum |
|---|
| autoTotal | true |
|---|
|
| Queue | Node Description | Queue Manager | Slots | Total memory (GB) |
|---|
| AML | (1) mid memory56-core 256G | Aaron Miller | 56 | 256 |
|---|
| ANTH | (4) standard memory56-core 128G | Andrew Kitchen | 224 | 512 |
|---|
ARROMA | (8) standard memory56-core 128G | Jun Wang | 448 | 1024 |
|---|
| AS | (5) mid memory56-core 256G | Katharine Corum | 280 | 1280 |
|---|
| BH | (1) high memory56-core 512G | Bin He | 56 | 512 |
|---|
BIGREDQ | (13) mid memory56-core 256G | Sara Mason | 728 | 3328 |
|---|
BIOLOGY | (1) mid memory56-core 256G | Matthew Brockman
| 56 | 256 |
|---|
BIOSTAT | (2) standard memory56-core 128G | Grant Brown | 112 | 256 |
|---|
| BIO-INSTR | (3) mid memory56-core 256G | JJ Urich, Albert Erives | 168 | 768 |
|---|
| CBIG | (1) mid memory 56-core 256G with P100 accelerator | Mathews Jacob | 56 | 256 |
|---|
| CBIG-HM | (1) high memory 56-core 512G with P100 accelerator | Mathews Jacob | 56 | 512 |
|---|
| CCOM | (18) high memory56-core 512G
5 running jobs per user | Boyd Knosp | 1008 | 9216 |
|---|
| CCOM-GPU | (2) high memory 56-core 512G with P100 accelerator | Boyd Knosp | 112 | 1024 |
|---|
CGRER + LMOS | (10) standard memory56-core 128G | Jeremie Moen | 560 | 1280 |
|---|
| CHEMISTRY | (3) mid memory56-core 256G | JJ Urich | 168 | 768 |
|---|
CLAS-INSTR | (2) mid memory56-core 256G | JJ Urich | 112 | 512 |
|---|
| CLL | (5) standard memory56-core 128G | Mark Wilson
Brian Miller | 280 | 640 |
|---|
| COB | (2) mid memory56-core 256G | Brian Heil | 112 | 512 |
|---|
| COE | (10) mid memory56-core 256G Note: Users are restricted to no more than three running jobs in the COE queue. | Matt McLaughlin | 560 | 2560 |
|---|
DARBROB | (1) mid memory56-core 256G | Benjamin Darbro | 56 | 256 |
|---|
| FERBIN | (13) standard memory56-core 128G | Adrian Elcock | 728 | 1664 |
|---|
MF | (6) standard memory 56-core 128G | Michael Flatte | 336 | 768 |
|---|
| MF-HM | (2) high memory56-core 512G | Michael Flatte | 112 | 1024 |
|---|
FLUIDSLAB | (8) standard memory56-core 128G | Mark Wilson
Brian Miller | 448 | 1024 |
|---|
| AIS | (1) mid memory56-core 256G | Grant Brown | 56 | 256 |
|---|
GEOPHYSICS | (3) standard memory56-core 128G | William Barnhart | 168 | 384 |
|---|
| GV | (2) mid memory56-core 256G | Mark Wilson
Brian Miller | 112 | 512 |
|---|
| HJ | (10) standard memory56-core 128G | Hans Johnson | 560 | 1280 |
|---|
| HJ-GPU | (1) high memory 56-core 512G with P100 accelerator | Hans Johnson | 56 | 512 |
|---|
| IFC | (10) mid memory 56-core 256G | Mark Wilson
Brian Miller | 560 | 2560 |
|---|
| IIHG | (10) mid memory56-core 256G | Diana Kolbe | 560 | 256 |
|---|
INFORMATICS | (12) mid memory56-core 256G | Ben Rogers | 672 | 3072 |
|---|
INFORMATICS-GPU | (2) mid memory 56-core 256G with Titan V accelerators | Ben Rogers | 112 | 512 |
|---|
INFORMATICS-HM-GPU | (1) high memory 56-core 512G with (2) P100 accelerators | Ben Rogers | 56 | 512 |
|---|
| IVR | (4) mid memory56-core 256G
(1) high memory 56-core 512G | Todd Scheetz | 280 | 1536 |
|---|
| IVR-GPU | (1) high memory 56-core 512G with K80 accelerator | Todd Scheetz | 56 | 1536 |
|---|
| IVRVOLTA | (4) high memory 56-core 512G with Titan V | Mike Schnieders | 224 | 2048 |
|---|
| IWA | (11) standard memory56-core 128G | Mark Wilson
Brian Miller | 616 | 1408 |
|---|
| JM | (3) high memory56-core 512G | Jake Michaelson | 168 | 512 |
|---|
| JM-GPU | (1) mid memory 56-core 256G with P100 accelerator | Jake Michaelson | 56 | 512 |
|---|
| JP | (2) high memory56-core 512G | Virginia Willour | 112 | 1024 |
|---|
| JS | (10) mid memory56-core 256G | James Shepherd | 560 | 2560 |
|---|
| LUNG | (2) high memory 56-core 512G with P40 accelerator | Joe Reinhardt | 112 | 1024 |
|---|
| MANSCI | (1) standard memory56-core 128G | Qihang Lin | 56 | 128 |
|---|
| MANSCI-GPU | (1) high memory 56-core 512G with P100 accelerator | Qihang Lin | 56 | 512 |
|---|
| MANORG | (1) standard memory56-core 128G | Michele Williams/Brian Heil | 56 | 128 |
|---|
MORL | (5) mid memory56-core 256G | William (Daniel) Walls | 280 | 1280 |
|---|
| MS | (5) 56-core 256G with (2) P100 GPUs
(7) 40-core 96G with (4) 1080Ti GPUs
(1) 40-core 96G with (4) Titan V GPUS | Mike Schnieders | 780 | 2048 |
|---|
| NEURO | (1) mid memory56-core 256G | Marie Gaine/Ted Abel | 56 | 256 |
|---|
| NOLA | (1) high memory56-core 512G | Ed Sander | 56 | 512 |
|---|
| PINC | (6) mid memory56-core 256G | Jason Evans | 336 | 1536 |
|---|
| REX | (4) standard memory56-core 128G | Mark Wilson
Brian Miller | 224 | 512 |
|---|
| REX-HM | (1) high memory56-core 512G | Mark Wilson
Brian Miller | 56 | 512 |
|---|
| SB | (4) standard memory56-core 128G | Scott Baalrud | 224 | 512 |
|---|
| STATEPI | (1) mid56-memorycore 256G | Linnea Polgreen | 56 | 256 |
|---|
| UDAY | (4) standard memory56-core 128G | Mark Wilson
Brian Miller | 224 | 512 |
|---|
| UI | (20) mid memory56-core 256G | | 1120 | 5120 |
|---|
UI-DEVELOP | (1) mid memory56-core 256G
(1) mid memory 56-core 256G with P100 accelerator | | 112 | 512 |
|---|
| UI-GPU | (4) mid memory 56-core 256G with P100 accelerator | | 224 | 1024 |
|---|
| UI-HM | (5) high memory56-core 512G | | 280 | 2560 |
|---|
| UI-MPI | (19) mid memory56-core 256G | | 1064 | 4864 | all.q | (115) standard memory(149) mid memory
(19) mid memory with P100 accelerator
(49) high memory
(9) high memory with P100 accelerator
(2) high memory with K80 accelerator | | 19208 | 87998 |
|---|
| NEUROSURGERY | (1) high memory 56-core 512G with K80 accelerator | Haiming Chen | 56 | 512 |
|---|
| SEMI | (1) standard memory56-core 128G | Craig Pryor | 56 | 128 |
|---|
| ACB | (1) mid memory56-core 256G | Adam Dupuy | 56 | 256 |
|---|
| FFME | (16) standard memory56-core 128G | Mark Wilson | 896 | 2048 |
|---|
| FFME-HM | (1) high memory56-core 512G | Mark Wilson | 56 | 512 |
|---|
| RP | (2) high memory56-core 512G | Robert Philibert | 112 | 1024 |
|---|
| LT | (2) high memory 56-core 512G with P100 accelerator | Luke Tierney | 112 | 1024 |
|---|
| KA | (1) high memory56-core 512G | Kin Fai Au | 56 | 512 |
|---|
|
The University of Iowa (UI) queue
A significant portion of the HPC cluster systems at UI were funded centrally. These nodes are put into queues named UI or prefixed with UI-.
- UI → Default queue
- UI-HM→ High memory 56-core 512G nodes; request only for jobs that need more memory than can be met with the standard nodes.
- UI-MPI → MPI jobs; request only for jobs that can take advantage of multiple nodes.
- UI-GPU → Contains nodes with GPU accelerators; request only if job can use a GPU accelerator.
- UI-DEVELOP → Meant for small, short running job prototypes and debugging.
...
| Centrally funded queues | Node Description | Wall clock limit | Running jobs per user |
|---|
| UI | (20) mid memory56-core 256G | None | 2 |
|---|
| UI-HM | (5) high memory56-core 512G | None | 1 |
|---|
UI-MPI
(56 slot minimum) | (20) mid memory56-core 256G | 48 hours | 1 |
|---|
| UI-GPU | (4) mid memory 56-core 256G with P100 accelerator | None | 1 |
|---|
| UI-DEVELOP | (1) mid memory56-core 256G
(1) mid memory 56-core 256G with P100 accelerator | 24 hours | 1 |
|---|
...