How to solve “Unable to load native-hadoop library” in Eclipse

Having just configured Eclipse for Hadoop development, I ran into the following problem:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Running jobs via the hadoop cli command worked fine; this only happened when I tried to run jobs directly from Eclipse, in local mode. After doing a little investigation, I found that the reason behind this is a java property called java.library.path that did not include the correct path.

Running from the hadoop cli command, the java.library.path property was properly set to /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop/lib/native (I am using the CDH 4.2.0 distribution of Hadoop). When the job was started from inside Eclipse, the java.library.path held it’s system default value:

native1

In order to correctly set this property you can configure Eclipse to load the Java Virtual Machine with this setting, or (and this is the better way) add the native library under the respective library from the Java Build Path. In order to do this, first right click on your project and open the Buid Path configuration screen:

native2

In this screen, find the hadoop-common library, expand the row and add the native library by pointing to the correct location:

native3

That’s it, from now on, you’ll be able to run your MapReduce jobs in local mode, and use the proper native libraries!

5 thoughts on “How to solve “Unable to load native-hadoop library” in Eclipse

  1. Thanks a lot, that really work! BTW, if you downloaded and installed hadoop directly from Apache, the native-library-location should be $HADOOP_PREFIX/lib/native/Linux-amd64-64 for 64 bit OS.

  2. m using windows 10 can anyone help me out to setup native in eclipse it shows like non modifiable
    please guide me through the proceess to install native libraries

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s