Hadoop Spark on XSEDE Bridges-2

Information on using the XSEDE resource Bridges-2 at PSC for University of Iowa coursework in Hadoop, Spark, and SparkR.

This resource provided by PSC allows for each student to generate a dynamic virtual Hadoop cluster of one name node and two data nodes with persistent file systems.  Each cluster is spun up upon student login, and halted upon logout.  To gain access to Bridges-2, and run hadoop, you must first sign up for an XSEDE account, then, using the single-sign-on hub, you can access Bridges-2 over an SSH connection.

• Process for creating an XSEDE account and accessing PSC (Pittsburgh Supercomputing Center) Bridges-2

• Requesting Education Allocations for use in Teaching Courses

• Commands to access and create a three node Hadoop cluster