Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

Once you have created your XSEDE account, have set up DUO authentication, and have verified that you can connect to the XSEDE's Single Sign On (SSO) login hub, execute the following instructions to generate your personal virtual three node Hadoop cluster.
 

To start a Hadoop session with a persistent hdfs:
 

1. start an interactive job with at least 3 nodes

interact -N 3

2. load the hadoop module when the job starts

module load hadoop

3. For a persistent file system, start hadoop with the following script:

/opt/packages/hadoop-testing/persist-start.sh

4. Finally, load up the Hive module

source /opt/packages/hive/hive.sh

5. Run Hive

hive


For more information on Bridges see https://portal.xsede.org/psc-bridges.

  • No labels