2013-11-30

CDH4 on hiCloud Ubuntu 12.04 VM

## 產生 zh_TW.UTF-8 語系支援
locale-gen zh_TW.UTF-8
## 安裝 Sun JDK 6
if [ ! -x /usr/bin/add-apt-repository ]; then   sudo apt-get -f -y install python-software-properties; fi
if [ ! -x /usr/bin/java ]; then
  echo "---- [1] Installing Sun Java JDK 6 ........ ----"
  sudo add-apt-repository -y 'deb http://free.nchc.org.tw/debian squeeze non-free'
  sudo apt-get update
  cat << EOF | sudo /usr/bin/debconf-set-selections
sun-java6-bin   shared/accepted-sun-dlj-v1-1    select true
sun-java6-jdk   shared/accepted-sun-dlj-v1-1    select true
sun-java6-jre   shared/accepted-sun-dlj-v1-1    select true
EOF
  sudo apt-get -y --force-yes install sun-java6-jdk sun-java6-plugin
  sudo add-apt-repository -r 'deb http://free.nchc.org.tw/debian squeeze non-free'
fi
## 下載 CDH4 設定金鑰與套件庫清單檔(/etc/apt/sources.list.d/cloudera-cdh4.list)
wget http://archive.cloudera.com/cdh4/one-click-install/precise/amd64/cdh4-repository_1.0_all.deb
dpkg -i cdh4-repository_1.0_all.deb
## 設定單機偽分散模式 Pseudo-Distributed Mode
sudo apt-get update
sudo apt-get install hadoop-0.20-conf-pseudo

## 安裝 DRBL
echo "deb http://free.nchc.org.tw/drbl-core drbl stable" > drbl.list
mv drbl.list /etc/apt/sources.list.d/
wget http://drbl.nchc.org.tw/GPG-KEY-DRBL
apt-key add GPG-KEY-DRBL
apt-get update
apt-get -y install drbl clonezilla
drblsrv -i
drblpush -i

如果你有部份或是全部的用戶端機器不想使用待會程式自動產生的電腦名稱，而是要自己指定用戶端電腦名稱，
可以先編輯 /etc/drbl/client-ip-hostname，把要指定的名稱依據 IP address 與名稱一行一行對應好。
/etc/drbl/client-ip-hostname 中沒有指定到的機器，drblpush 會給與一個自動產生的機器名稱。 

### 安裝 Ganglia 與相關套件
apt-get -f install openssh-server vim sudo ganglia-monitor ganglia-webfrontend gmetad apache2 libapache2-mod-php5 php5 lsb-release debconf-utils util-linux tar gzip bzip2 lzop pigz pbzip2 procps dialog rsync parted pciutils tcpdump bc gawk hdparm sdparm netcat file ethtool etherwake ssh syslinux mtools reiserfsprogs e2fsprogs psmisc locales wget disktype zip unzip patch iproute traceroute iputils-ping binutils expect partimage udpcast initscripts tftpd-hpa nfs-kernel-server nis curl lftp iptables libdigest-sha-perl udev memtest86+ aoetools btrfs-tools dhcp3-server dmraid dos2unix fgetty hfsprogs hfsutils isc-dhcp-server kpartx lbzip2 libdmraid1.0.0.rc16 lshw lzip lzma runit tofrodos ufsutils vblade vblade-persist bridge-utils ntfs-3g sun-java6-jdk sysrqd xfsprogs gdisk

ln -s /etc/ganglia-webfrontend/apache.conf 

HDP for Windows

• <安裝> 下載安裝檔

  wget http://public-repo-1.hortonworks.com/HDP-Win/1.3/hdp-1.3.0.0-GA.zip
  

• <文件> ​http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-Win-1.3.0/index.html
• JDBC driver for MS SQL Server
• If you are using MS SQL Server for Hive and Oozie metastores, you must install the MS SQL Server JDBC driver.
• ​http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-Win-1.3.0/bk_installing_hdp_for_windows/content/win-getting-ready-2-4-1.html

  wget http://download.microsoft.com/download/D/6/A/D6A241AC-433E-4CD2-A1CE-50177E8428F0/1033/sqljdbc_3.0.1301.101_enu.exe
  

   msiexec /i "hdp-win-1.1.msi" /lv "hdp.log" HDP_LAYOUT="C:\config\clusterproperties.txt" HDP_DIR="C:\hdp\hadoop" DESTROY_DATA="no"