= 2013-11-30 = == CDH4 on hiCloud Ubuntu 12.04 VM == {{{ #!sh ## 產生 zh_TW.UTF-8 語系支援 locale-gen zh_TW.UTF-8 ## 安裝 Sun JDK 6 if [ ! -x /usr/bin/add-apt-repository ]; then sudo apt-get -f -y install python-software-properties; fi if [ ! -x /usr/bin/java ]; then echo "---- [1] Installing Sun Java JDK 6 ........ ----" sudo add-apt-repository -y 'deb http://free.nchc.org.tw/debian squeeze non-free' sudo apt-get update cat << EOF | sudo /usr/bin/debconf-set-selections sun-java6-bin shared/accepted-sun-dlj-v1-1 select true sun-java6-jdk shared/accepted-sun-dlj-v1-1 select true sun-java6-jre shared/accepted-sun-dlj-v1-1 select true EOF sudo apt-get -y --force-yes install sun-java6-jdk sun-java6-plugin sudo add-apt-repository -r 'deb http://free.nchc.org.tw/debian squeeze non-free' fi ## 下載 CDH4 設定金鑰與套件庫清單檔(/etc/apt/sources.list.d/cloudera-cdh4.list) wget http://archive.cloudera.com/cdh4/one-click-install/precise/amd64/cdh4-repository_1.0_all.deb dpkg -i cdh4-repository_1.0_all.deb ## 設定單機偽分散模式 Pseudo-Distributed Mode sudo apt-get update sudo apt-get install hadoop-0.20-conf-pseudo }}} {{{ #!text ## 安裝 DRBL echo "deb http://free.nchc.org.tw/drbl-core drbl stable" > drbl.list mv drbl.list /etc/apt/sources.list.d/ wget http://drbl.nchc.org.tw/GPG-KEY-DRBL apt-key add GPG-KEY-DRBL apt-get update apt-get -y install drbl clonezilla drblsrv -i drblpush -i }}} {{{ #!text 如果你有部份或是全部的用戶端機器不想使用待會程式自動產生的電腦名稱,而是要自己指定用戶端電腦名稱, 可以先編輯 /etc/drbl/client-ip-hostname,把要指定的名稱依據 IP address 與名稱一行一行對應好。 /etc/drbl/client-ip-hostname 中沒有指定到的機器,drblpush 會給與一個自動產生的機器名稱。 }}} {{{ #!text ### 安裝 Ganglia 與相關套件 apt-get -f install openssh-server vim sudo ganglia-monitor ganglia-webfrontend gmetad apache2 libapache2-mod-php5 php5 lsb-release debconf-utils util-linux tar gzip bzip2 lzop pigz pbzip2 procps dialog rsync parted pciutils tcpdump bc gawk hdparm sdparm netcat file ethtool etherwake ssh syslinux mtools reiserfsprogs e2fsprogs psmisc locales wget disktype zip unzip patch iproute traceroute iputils-ping binutils expect partimage udpcast initscripts tftpd-hpa nfs-kernel-server nis curl lftp iptables libdigest-sha-perl udev memtest86+ aoetools btrfs-tools dhcp3-server dmraid dos2unix fgetty hfsprogs hfsutils isc-dhcp-server kpartx lbzip2 libdmraid1.0.0.rc16 lshw lzip lzma runit tofrodos ufsutils vblade vblade-persist bridge-utils ntfs-3g sun-java6-jdk sysrqd xfsprogs gdisk ln -s /etc/ganglia-webfrontend/apache.conf }}} == HDP for Windows == * <安裝> 下載安裝檔 {{{ wget http://public-repo-1.hortonworks.com/HDP-Win/1.3/hdp-1.3.0.0-GA.zip }}} * <文件> http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-Win-1.3.0/index.html * JDBC driver for MS SQL Server * If you are using MS SQL Server for Hive and Oozie metastores, you must install the MS SQL Server JDBC driver. * http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-Win-1.3.0/bk_installing_hdp_for_windows/content/win-getting-ready-2-4-1.html {{{ wget http://download.microsoft.com/download/D/6/A/D6A241AC-433E-4CD2-A1CE-50177E8428F0/1033/sqljdbc_3.0.1301.101_enu.exe }}} {{{ msiexec /i "hdp-win-1.1.msi" /lv "hdp.log" HDP_LAYOUT="C:\config\clusterproperties.txt" HDP_DIR="C:\hdp\hadoop" DESTROY_DATA="no" }}}