wiki:waue/2009/1007

Version 2 (modified by waue, 15 years ago) (diff)

--

Cloudera Desktop

  1. 安裝
    1. 加入套件庫
    2. 安裝cdh2 (hadoop0.20) & desktop
  2. 設定
    1. 修改 hdfs-site.xml
    2. 修改 mapred-site.xml
    3. 修改 hadoop-metrics.properties
    4. 重開 hadoop 叢集與 cloudera desktop
  3. 使用
  4. 參考文件

安裝

加入套件庫

新增一個檔案 /etc/apt/sources.list.d/cloudera.list，內容如下 

# $DISTRO=版本名稱，如hardy，可用 lsb_release -c 來查看
deb http://archive.cloudera.com/debian $DISTRO-testing contrib
deb-src http://archive.cloudera.com/debian $DISTRO-testing contrib

安裝cdh2 (hadoop0.20) & desktop

$ wget http://archive.cloudera.com/debian/archive.key | sudo apt-key add -
$ apt-get update 
$ sudo apt-get install hadoop-0.20 (有多少裝多少)
$ apt-get install cloudera-desktop cloudera-desktop-plugins

設定

修改 hdfs-site.xml

加入以下到 hdfs-site.xml 

<property>
  <name>dfs.namenode.plugins</name>
  <value>org.apache.hadoop.thriftfs.NamenodePlugin</value>
  <description>Comma-separated list of namenode plug-ins to be activated.
  </description>
</property>
<property>
  <name>dfs.datanode.plugins</name>
  <value>org.apache.hadoop.thriftfs.DatanodePlugin</value>
  <description>Comma-separated list of datanode plug-ins to be activated.
  </description>
</property>
<property>
  <name>dfs.thrift.address</name>
  <value>0.0.0.0:9090</value>
</property>

修改 mapred-site.xml

加入以下到 mapred-site.xml: 

<property>
  <name>jobtracker.thrift.address</name>
  <value>0.0.0.0:9290</value>
</property>
<property>
  <name>mapred.jobtracker.plugins</name>
  <value>org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin</value>
  <description>Comma-separated list of jobtracker plug-ins to be activated.
  </description>
</property>

修改 hadoop-metrics.properties

加入以下到 hadoop-metrics.properties ： 

# Exposes /metrics URL endpoint for metrics information.
dfs.class=org.apache.hadoop.metrics.spi.NoEmitMetricsContext
mapred.class=org.apache.hadoop.metrics.spi.NoEmitMetricsContext
jvm.class=org.apache.hadoop.metrics.spi.NoEmitMetricsContext
rpc.class=org.apache.hadoop.metrics.spi.NoEmitMetricsContext

重開 hadoop 叢集與 cloudera desktop

$ /etc/init.d/hadoop-0.20-namenode start
$ /etc/init.d/hadoop-0.20-datanode start
$ /etc/init.d/hadoop-0.20-jobtracker start
$ /etc/init.d/hadoop-0.20-tasktracker start
$ /etc/init.d/cloudera-desktop start

使用

用firefox 3.5 以上版本打開 ​http://localhost:8088/ 即可操作

參考文件