The Argon HPC system is the latest HPC system of the University of Iowa. It consists of 366 346 compute nodes, each of which contain 28 2.4GHz Intel Broadwell processor cores, running CentOS-7.4 Linux. There are several compute node configurations,
- 56-core 128G
- 56-core 256G
- 56-core 512G
- 40-core 96G
- 40-core 192G
...
- standard memory → 128GB
- mid-memory → 256GB
- high-memory → 512GB
There are 16 machines with single Nvidia P100 accelerators, 2 6 machines with dual Nvidia K80 P100 accelerators, 2 machines with an Nvidia P40 acceleratorsK80 accelerator, 12 2 machines with 1080Ti acceleratorsan Nvidia P40 accelerator, and 16 6 machines with a Titan V acceleratorsaccelerator.
| Info |
|---|
The Titan V is now considered as a supported configuration in GPU-capable compute nodes but is restricted to a single card per node. Staff have completed the qualification process for the 1080 Ti and concluded that it is not a viable solution to add to current Argon compute nodes. |
The Rpeak (theoretical Flops) is 398is 285.69 60 TFlops, not including the accelerators, with 67.25 TB of memory. In addition, there are 2 login nodes of the Broadwell same system architecture. The login nodes have 256GB of memory.
...
If your job does not use the system openmpi, or does not use MPI, then any desired core binding will need to be set up with whatever mechanism the software uses. Otherwise, there will be no core binding. Again, that may not be a major issue. If your job does not work well with HT then run on a number of cores equal to half of the number of slots requested and the OS scheduler will minimize contention.
new SGE utilities
While SoGE is very similar to previous versions of SGE there are some new utilities that people may find of interest. There are manual pages for each of these.
...
| Table plus |
|---|
| sortColumn | 1 |
|---|
| allowExport | true |
|---|
| columnTypes | S,S,S,I,I |
|---|
| columnCalculations | all.q,,,sum,sum |
|---|
autoTotal | true |
|
| Queue | Node Description | Queue Manager | Slots | Total memory (GB) |
|---|
| AML | (1) 56-core 256Gmid memory | Aaron Miller | 56 | 256 |
|---|
| ANTH | (4) 56-core 128Gstandard memory | Andrew Kitchen | 224 | 512 |
|---|
ARROMA | (8) 56-core 128Gstandard memory | Jun Wang | 448 | 1024 |
|---|
| AS | (5) 56-core 256Gmid memory | Katharine Corum | 280 | 1280 |
|---|
| BH | (1) 56-core 512Ghigh memory | Bin He | 56 | 512 |
|---|
BIGREDQ | (13) 56-core 256Gmid memory | Sara Mason | 728 | 3328 |
|---|
BIOLOGY | (1) 56-core 256Gmid memory | Matthew Brockman
| 56 | 256 |
|---|
BIOSTAT | (2) 56-core 128Gstandard memory | Grant Brown | 112 | 256 |
|---|
| BIO-INSTR | (3) 56-core 256Gmid memory | JJ Urich, Albert Erives | 168 | 768 |
|---|
| CBIG | (1) 56-core 256G mid memory with P100 accelerator | Mathews Jacob | 56 | 256 |
|---|
| CBIG-HM | (1) 56-core 512G high memory with P100 accelerator | Mathews Jacob | 56 | 512 |
|---|
| CCOM | (18) 56-core 512Ghigh memory
5 running jobs per user | Boyd Knosp | 1008 | 9216 |
|---|
| CCOM-GPU | (2) 56-core 512G high memory with P100 accelerator | Boyd Knosp | 112 | 1024 |
|---|
CGRER + LMOS | (10) 56-core 128Gstandard memory | Jeremie Moen | 560 | 1280 |
|---|
| CHEMISTRY | (3) 56-core 256Gmid memory | JJ Urich | 168 | 768 |
|---|
CLAS-INSTR | (2) 56-core 256Gmid memory | JJ Urich | 112 | 512 |
|---|
| CLL | (5) 56-core 128Gstandard memory | Mark Wilson
Brian Miller | 280 | 640 |
|---|
| COB | (2) 56-core 256Gmid memory | Brian Heil | 112 | 512 |
|---|
| COE | (10) 56-core 256Gmid memory Note: Users are restricted to no more than three running jobs in the COE queue. | Matt McLaughlin | 560 | 2560 |
|---|
DARBROB | (1) 56-core 256Gmid memory | Benjamin Darbro | 56 | 256 |
|---|
| FERBIN | (13) 56-core 128Gstandard memory | Adrian Elcock | 728 | 1664 |
|---|
MF | (6) 56-core 128G standard memory | Michael Flatte | 336 | 768 |
|---|
| MF-HM | (2) 56-core 512Ghigh memory | Michael Flatte | 112 | 1024 |
|---|
FLUIDSLAB | (8) 56-core 128Gstandard memory | Mark Wilson
Brian Miller | 448 | 1024 |
|---|
| AIS | (1) 56-core 256Gmid memory | Grant Brown | 56 | 256 |
|---|
GEOPHYSICS | (3) 56-core 128Gstandard memory | William Barnhart | 168 | 384 |
|---|
| GV | (2) 56-core 256Gmid memory | Mark Wilson
Brian Miller | 112 | 512 |
|---|
| HJ | (10) 56-core 128Gstandard memory | Hans Johnson | 560 | 1280 |
|---|
| HJ-GPU | (1) 56-core 512G high memory with P100 accelerator | Hans Johnson | 56 | 512 |
|---|
| IFC | (10) 56-core 256G mid memory | Mark Wilson
Brian Miller | 560 | 2560 |
|---|
| IIHG | (10) 56-core 256Gmid memory | Diana Kolbe | 560 | 256 |
|---|
INFORMATICS | (12) 56-core 256Gmid memory | Ben Rogers | 672 | 3072 |
|---|
INFORMATICS-GPU | (2) 56-core 256G mid memory with Titan V accelerators | Ben Rogers | 112 | 512 |
|---|
INFORMATICS-HM-GPU | (1) 56-core 512G high memory with (2) P100 accelerators | Ben Rogers | 56 | 512 |
|---|
| IVR | (4) 56-core 256Gmid memory
(1) 56-core 512G high memory | Todd Scheetz | 280 | 1536 |
|---|
| IVR-GPU | (1) 56-core 512G high memory with K80 accelerator | Todd Scheetz | 56 | 1536 |
|---|
| IVRVOLTA | (4) 56-core 512G high memory with Titan V | Mike Schnieders | 224 | 2048 |
|---|
| IWA | (11) 56-core 128Gstandard memory | Mark Wilson
Brian Miller | 616 | 1408 |
|---|
| JM | (3) 56-core 512Ghigh memory | Jake Michaelson | 168 | 512 |
|---|
| JM-GPU | (1) 56-core 256G mid memory with P100 accelerator | Jake Michaelson | 56 | 512 |
|---|
| JP | (2) 56-core 512Ghigh memory | Virginia Willour | 112 | 1024 |
|---|
| JS | (10) 56-core 256Gmid memory | James Shepherd | 560 | 2560 |
|---|
| LUNG | (2) 56-core 512G high memory with P40 accelerator | Joe Reinhardt | 112 | 1024 |
|---|
| MANSCI | (1) 56-core 128Gstandard memory | Qihang Lin | 56 | 128 |
|---|
| MANSCI-GPU | (1) 56-core 512G high memory with P100 accelerator | Qihang Lin | 56 | 512 |
|---|
| MANORG | (1) 56-core 128Gstandard memory | Michele Williams/Brian Heil | 56 | 128 |
|---|
MORL | (5) 56-core 256Gmid memory | William (Daniel) Walls | 280 | 1280 |
|---|
| MS | (5) 56-core 256G with (2) P100 GPUs
(7) 40-core 96G with (4) 1080Ti GPUs
(1) 40-core 96G with (4) Titan V GPUS | Mike Schnieders | 780 | 2048 |
|---|
| NEURO | (1) 56-core 256Gmid memory | Marie Gaine/Ted Abel | 56 | 256 |
|---|
| NOLA | (1) 56-core 512Ghigh memory | Ed Sander | 56 | 512 |
|---|
| PINC | (6) 56-core 256Gmid memory | Jason Evans | 336 | 1536 |
|---|
| REX | (4) 56-core 128Gstandard memory | Mark Wilson
Brian Miller | 224 | 512 |
|---|
| REX-HM | (1) 56-core 512Ghigh memory | Mark Wilson
Brian Miller | 56 | 512 |
|---|
| SB | (4) 56-core 128Gstandard memory | Scott Baalrud | 224 | 512 |
|---|
| STATEPI | (1) 56mid-core 256Gmemory | Linnea Polgreen | 56 | 256 |
|---|
| UDAY | (4) 56-core 128Gstandard memory | Mark Wilson
Brian Miller | 224 | 512 |
|---|
| UI | (20) 56-core 256Gmid memory | | 1120 | 5120 |
|---|
UI-DEVELOP | (1) 56-core 256Gmid memory
(1) 56-core 256G mid memory with P100 accelerator | | 112 | 512 |
|---|
| UI-GPU | (4) 56-core 256G mid memory with P100 accelerator | | 224 | 1024 |
|---|
| UI-HM | (5) 56-core 512Ghigh memory | | 280 | 2560 |
|---|
| UI-MPI | (19) 56-core 256Gmid memory | | 1064 | 4864 |
|---|
| all.q | (115) standard memory
(149) mid memory
(19) mid memory with P100 accelerator
(49) high memory
(9) high memory with P100 accelerator
(2) high memory with K80 accelerator | | 19208 | 87998 |
|---|
| NEUROSURGERY | (1) 56-core 512G high memory with K80 accelerator | Haiming Chen | 56 | 512 |
|---|
| SEMI | (1) 56-core 128Gstandard memory | Craig Pryor | 56 | 128 |
|---|
| ACB | (1) 56-core 256Gmid memory | Adam Dupuy | 56 | 256 |
|---|
| FFME | (16) 56-core 128Gstandard memory | Mark Wilson | 896 | 2048 |
|---|
| FFME-HM | (1) 56-core 512Ghigh memory | Mark Wilson | 56 | 512 |
|---|
| RP | (2) 56-core 512Ghigh memory | Robert Philibert | 112 | 1024 |
|---|
| LT | (2) 56-core 512G high memory with P100 accelerator | Luke Tierney | 112 | 1024 |
|---|
| KA | (1) 56-core 512Ghigh memory | Kin Fai Au | 56 | 512 |
|---|
|
The University of Iowa (UI) queue
A significant portion of the HPC cluster systems at UI were funded centrally. These nodes are put into queues named UI or prefixed with UI-.
- UI → Default queue
- UI-HM→ 56-core 512G High memory nodes; request only for jobs that need more memory than can be met with the standard nodes.
- UI-MPI → MPI jobs; request only for jobs that can take advantage of multiple nodes.
- UI-GPU → Contains nodes with GPU accelerators; request only if job can use a GPU accelerator.
- UI-DEVELOP → Meant for small, short running job prototypes and debugging.
...
| Centrally funded queues | Node Description | Wall clock limit | Running jobs per user |
|---|
| UI | (20) 56-core 256Gmid memory | None | 2 |
|---|
| UI-HM | (5) 56-core 512Ghigh memory | None | 1 |
|---|
UI-MPI
(56 slot minimum) | (20) 56-core 256Gmid memory | 48 hours | 1 |
|---|
| UI-GPU | (4) 56-core 256G mid memory with P100 accelerator | None | 1 |
|---|
| UI-DEVELOP | (1) 56-core 256Gmid memory
(1) 56-core 256G mid memory with P100 accelerator | 24 hours | 1 |
|---|
...