Once you have created your XSEDE account, have set up DUO authentication, and have verified that you can connect to the XSEDE's Single Sign On (SSO) login hub, and have connected to Bridges, execute the following instructions to generate your personal virtual three node Hadoop cluster. If you have not yet connected to Bridges via the Single Sign On hub, click here for instructions on how to do so.
To start a Hadoop session with a persistent hdfs:
...
3. Instead, for a persistent file system, set up your environment with the following script:
Note: The directory in which you run this command will be the home for the file-system, if you run this command again in another session, it will look for the file-system. If you are not in the same directory that you had generated your file-system in previous sessions, it will generate a new file-system. For consistency, you may want to always run this command in the root of your home directory.
/opt/packages/hadoop-testing/persist-start.sh
...
source /opt/packages/hive/hive.sh
5. Run Hive
hive
If you see java error messages, wait a minute or two, and try to run hive again.
For more information on Bridges see https://portal.xsede.org/psc-bridges.