Page Comparison

...

Note

Some tutorials suggest running R scripts with an older convention as "R CMD BATCH program.R", but this has several disadvantages compared to "Rscript program.R" in any situation, and particularly on an HPC system:

It doesn't merely interpret the script; instead, it simulates running the script in an interactive session and prints the script's output inline with the script itself. This makes the script's output more difficult to read or parse with some other program later.
It doesn't print anything to the display (stdout), so you can't use SGE or redirection (">") to capture and manage the output.
It always creates and prints to a file named like "program.Rout" in the directory where you start the script; not necessarily where your script is, or where your input or output data is, or where you prefer. HTC Best Practices describes how this can cause performance problems, but the suggested mitigations are difficult to apply.

Therefore we advise against using "R CMD BATCH".

Running MPI jobs with R, particularly R-snow, requires some special handling. If the MPI processes will all be on a single host then you can essentially start R as normal and spawn MPI slave ranks. However, if running across multiple nodes, the processes must be spawned by mpirun. This could also be done for the single node case. Since this will have R started by mpirun we need to use a wrapper that will distinguish between the master and slave processes and start up the respective R sessions accordingly. This wrapper is called RMPISNOW. The launch command would look something like the following in an SGE job script.

No Format
mpirun -np # RMPISNOW CMD BATCH --slave sample_script.R

The slot request for SGE must 1 greater than the number of workers that will be used. Unless more slots are requested for memory the mpirun command will use what is in the SGE environment so the above could be shortened to

No Format
mpirun RMPISNOW CMD BATCH --slave sample_script.R

The above will create the snow cluster and that can be referenced within the R script.

The size of the cluster can be gotten with

No Format
mpi.universe.size()

so the number of workers would be

No Format
mpi.universe.size()-1

A cluster object can be set just by referencing the cluster set up by mpirun with

No Format
cl <- getMPIcluster()

The cluster should be stopped with

No Format
stopCluster(cl)

and the R script exited with

No Format
mpi.quit()

Rscript can not be used when using the RMPISNOW wrapper, because it calls R directly, but the command could also be run as

No Format
mpirun RMPISNOW --slave <sample_script.R

Versions Compared

Old Version 4

New Version 5

Key