Once you have created your XSEDE account, have set up DUO authentication, and have verified that you can connect to the XSEDE's Single Sign On (SSO) login hub, execute the following instructions to generate your personal virtual three node Hadoop cluster.
To start a Hadoop session with a persistent hdfs:
1. start an interactive job with at least 3 nodes
interact -N 3
2. load the hadoop module when the job starts
module load hadoop
IMPORTANT: do NOT run start-hadoop.sh as the module suggests.
3. instead, run this:
/opt/packages/hadoop-testing/persist-start.sh
4. Finally, load up the Hive module
source /opt/packages/hive/hive.sh
5. Run Hive
hive
For more information on Bridges see https://portal.xsede.org/psc-bridges.