问题:在执行yarn集群或者客户端时,报以上错误,
Exception in thread "main" java.lang.Exception: When running with master 'yarn-client' either HADOOP_CONF_DIR or YARN_CONF_DIR must be set in the environment.
问题:在执行yarn集群或者客户端时,报以上错误,
[bdata@bdata4 bin]$ ./spark-sql --master yarn-client
Exception in thread "main" java.lang.Exception: When running with master 'yarn-client' either HADOOP_CONF_DIR or YARN_CONF_DIR must be set in the environment.
解决办法
根据提示,配置HADOOP_CONF_DIR or YARN_CONF_DIR的环境变量即可
export HADOOP_HOME=/u01/hadoop-2.6.1
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
PATH=PATH:PATH: HOME/.local/bin:HOME/bin:HOME/bin: SQOOP_HOME/bin:HIVEHOME/bin:HIVEHOME/bin: HADOOP_HOME/bin