Blog Entries tagged hadoop
Feeds: RSS | Atom

Hadoop Streaming Error Codes

Published: 2011-01-31 08:12 UTC. Tags: hadoop

I'm using Hadoop Streaming a lot. It's exit codes has been something of a mystery, so today I decided to find out by looking at the source code.

The exit codes are listed in StreamJob.java, and are as follows:

  1. Success
  2. Job not successful, i.e. something went wrong with M/R code.
  3. Bad input path
  4. Invalid jobconf
  5. Output path already exists
  6. Error launching job. Could be any error, for example some HDFS communication error.
0 comments.

Hadoop lesson learnt: Restart datanodes after modifying dfs.balance.bandwidthPerSec

Published: 2010-09-10 13:17 UTC. Tags: hadoop

I was rebalancing one of the Hadoop clusters I run at work. It was not running very fast, so I modified the appropriate setting:

<property>
  <!-- 100Mbit/s -->
  <name>dfs.balance.bandwidthPerSec</name>
  <value>104857600</value>
</property>

I restarted the namenode and thought that would make the trick. But no, you also need to restart all your datanodes for the setting to take effect. Now I can see some action on my network graphs :-).

0 comments.