fuse hdfs 0.20.1 繼續加油

環境

• ubuntu 9.04
• kernel 2.6.24-24-generic (原本預設為 2.6.28-15 )
  • 請重新開機於grub 切換成第三個kernel，否則無法 modprobe fuse
• hadoop 0.20.1 (安裝於/opt/hadoop)
• fuse 版本
  • tarball 版為 2.7.4
  • 而/lib/modules/2.6.24-24-generic/kernel/fs/fuse/fuse.ko 此fuse模組檔為安裝 linux-image-2.6.24-24-generic.deb時就有
• automake 1.9.x 以上
  • sudo apt-get install automake 可得 1.10.2

需求

• Hadoop with compiled libhdfs.so
• Linux kernel > 2.6.9 with fuse, which is the default or Fuse 2.7.x, 2.8.x installed. See: ​http://fuse.sourceforge.net/
• modprobe fuse to load it
• fuse-dfs executable (see below)
• fuse_dfs_wrapper.sh installed in /bin or other appropriate location (see below)

方法

1. build

改 build.xml 的 1046行左右，去掉 doc相關敘述，如下：

  <target name="package" depends="compile, jar, examples, tools-jar, jar-test, ant-tasks, package-librecordio"

$ cd /opt/hadoop
$ ant compile-c++-libhdfs -Dlibhdfs=1
$ ant package
$ ant compile-contrib -Dlibhdfs=1 -Dfusedfs=1

bug fix

• check java5

改 build.xml 的 1046行左右，去掉 doc相關敘述，如下：

  <target name="package" depends="compile, jar, examples, tools-jar, jar-test, ant-tasks, package-librecordio"

• libhdfs.so does not exist: /opt/hadoop-0.20.1/build/libhdfs/libhdfs.so

mkdir /opt/hadoop-0.20.1/build/libhdfs/
ln -sf /opt/hadoop/c++/Linux-i386-32/lib/libhdfs.so /opt/hadoop-0.20.1/build/libhdfs/libhdfs.so