This guide will get you past the troubles of getting Hadoop installed and running on Mac OS X. I’ve tested it for Hadoop 2.2 apotheke-zag.de.x and OS X 10.9.
The meat of the process you need to follow is well documented by Michael Noll. I’m only going to add the steps here that are required over and above the Ubuntu guide for OS X. You should probably use brew unless you have a great reason not to.
in <HADOOP>/etc/hadoop/hadoop-env.sh add or edit these lines:
# The java implementation to use.
export JAVA_HOME=`/usr/libexec/java_home -v 1.6`
# Extra Java runtime options. Empty by default.
export HADOOP_OPTS=”$HADOOP_OPTS -Djava.net.preferIPv4Stack=true”
export HADOOP_OPTS=”$HADOOP_OPTS -Djava.awt.headless=true -Djava.security.krb5.realm=-Djava.security.krb5.kdc=”
YARN_OPTS=”$YARN_OPTS -Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk -Djava.awt.headless=true”
For x permissions problems over ssh
xhost +
Make sure you enable ssh using System Preferences > Sharing > Remote Login.
Starting Hadoop.
<HADOOP>/sbin/start-dfs.sh
<HADOOP>/sbin/start-yarn.sh
Running the WordCount Examples (download the files as in Michael’s guide).
bin/hdfs dfs -mkdir -p /user/hduser/gutenberg
bin/hdfs dfs -copyFromLocal -f /tmp/gutenberg/* /user/hduser/gutenberg
bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar wordcount /user/hduser/gutenberg /user/hduser/gutenberg-output
find /usr/local/hadoop -name hadoop*examples*.jar
Hello,
your post is really helpful for setup hadoop in Mac. Thank you for your post. I could able to run hadoop successfully with this.
-Hari