Running Kerrighed on Debian AMD64 ( kernel 2.6.20 )

[jazz]

哲源:

請協助確認 Linux Kernel 如何確定是支援 64 位元的 CPU. 建議可以比對 Fedora 64 位元的 Kernel Config 與目前 Kerrighed 2.6.20 客製化核心的差異為何.

Jazz Yao-Tsung Wang

[jazz]

from LSI 阿德大神:

64 位元的 config => http://www.lsi.idv.tw/Download/Kernel/config/linux-2.6.17.14_x86_64.conf
http://www.lsi.idv.tw/Download/Kernel/config/linux-2.6.18_x86_64.conf

最重要有幾個

x86 64 支援
highmem 支援
elf32 模擬支援
ia32  模擬支援

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_MISC=m
CONFIG_IA32_EMULATION=y
CONFIG_IA32_AOUT=y
CONFIG_COMPAT=y
CONFIG_SYSVIPC_COMPAT=y

開這幾個才可以執行 32 位元的執行檔

[jazz]

from LSI 阿德大神:

64 位元的關鍵辨識 => x86_64

A. Fedora
[lsi@temp087 ~]$ uname -a
Linux temp087.nchc.org.tw 2.6.15.7 #1 SMP Fri May 26 16:23:28 CST 2006 x86_64 x86_64 x86_64 GNU/Linux

B. uBuntu
tdw22:~> uname -a
Linux tdw22.nchc.org.tw 2.6.22-14-generic #1 SMP Sun Oct 14 21:45:15 GMT 2007 x86_64 GNU/Linux

[jazz]

from LSI 阿德大神:

換好 Kernel 之後 , 記得一定要記得執行 "cat /proc/cpuinfo" 
看看 CPU 的個數以及時脈對不對 有時候會被降頻跑 尤其是 Intel 的機器

要把 cpuspeed 或是 powernowd 服務關閉 , 
或是直接 Kernel 加入參數 => acpi=off noacpi noapic

powernowd => Intel 雙核或是四核都會被降頻 => uBuntu 64 位元才有這個 service
cpuspeed  => CentOS 才有的服務 , AMD Optron 的 CPU 會被降頻

[jazz]

from LSI 阿德大神:

最簡單的做法就是 Kernel parameter 關掉 ACIP 功能  
=> 架 server 不需要省電以及電源管理

Jazz: 同意!! 看樣子 Kerrighed 要 patch 的 config 不少喔!! 大家加油了~

[anonymous]

他官網的資訊是說目前版本可支援64bit， 但還是屬於EXPERIMENTAL性質， 但有說下一個版本會正式支援64bit， 所以應該是可以跑，只是可能會有一些狀況吧～～～

我也用一個64的系統來試試～ 因為GlusterFS目前也只能支援64bit的環境～

[jazz]

How to confirm Kernel is support 64bit CPU ?

32位元 ---> Update to 64位元 : Kernel 重編需注意的地方與部份 lib64xxx需要升級
 
茲附上今天編的kernel 2.6.20 32位元 與 lsi 之前fc 的 kernel 2.6.22 64位元的 kernel config file .
 
## 64位元 linux_kernel key point
x86 64 支援
highmem 支援
elf32 模擬支援
ia32  模擬支援
 
## lib64xxx 視需求而安裝 (以Linux krg-server 2.6.20-krg為例)
lib64asound2 - ALSA library (64 bit)
lib64asound2-dev - ALSA library development files (64 bit)
lib64bz2-1.0 - high-quality block-sorting file compressor library - 64bit runtime
lib64bz2-dev - high-quality block-sorting file compressor library - 64bit development
lib64ffi4 - Foreign Function Interface library runtime (64bit)
lib64g2c0 - Runtime library for GNU Fortran 77 applications (64bit)
lib64gcc1 - GCC support library (64bit)
lib64gfortran1 - Runtime library for GNU Fortran applications (64bit)
lib64mudflap0 - GCC mudflap shared support libraries (64bit)
lib64ncurses5 - Shared libraries for terminal handling (64-bit)
lib64ncurses5-dev - Developer's libraries for ncurses (64-bit)
lib64objc1 - Runtime library for GNU Objective-C applications (64bit)
lib64readline5 - GNU readline and history libraries, run-time libraries (64-bit)
lib64readline5-dev - GNU readline and history libraries, development files (64-bit)
lib64ssp0 - GCC stack smashing protection library (64bit)
lib64stdc++6 - The GNU Standard C++ Library v3 (64bit)
lib64stdc++6-4.1-dbg - The GNU Standard C++ Library v3 (debugging files)
lib64z1 - compression library - 64 bit runtime
lib64z1-dev - compression library - 64 bit development
libc6-dev-amd64 - GNU C Library: 64bit Development Libraries for AMD64
 
# 開啟 64 Support options for example
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.18-1.2257
# Mon Jan 14 20:30:31 2008
#
CONFIG_X86_64=y
CONFIG_64BIT=y
CONFIG_X86=y
 

# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_MISC=m
CONFIG_IA32_EMULATION=y
CONFIG_IA32_AOUT=y
CONFIG_COMPAT=y
CONFIG_SYSVIPC_COMPAT=y
 
 
PS1 : 140.110.31.201 目前是 update to  2.6.20 Patch Kerrighed 2.2.1 ( pre-version 2.2.0 without SMP options )
         ( Service有起來 , SMP開啟後四顆CPU 都正常抓到ㄌ ) 請大家幫忙測看看 , kerrighed , 文傑再麻煩你看看哪裡有問題.這部份你比較熟.
PS2 : 下面四台電腦目前會灌成 Debian amd64的 , 上面的4台還是走32位元 ( 上面則是update to 64位元的 kernel注意部份 )
PS3 : 感謝阿德學長提供許多寶貴意見
 
 
Best Regards
==========================================================
Bill Tu 涂哲源 < Rider >
National Center for High-Performance Computing
No.7, R&D Rd. VI, Hsinchu Science Park, Hsinchu 30076, Taiwan, R.O.C.
Tel : 886-3-5776085  # 246
Fax : 886-3-5638749
E-mail : rider@nchc.org.tw

[rider]

Source: /linux-2.6.20-krg/include/linux/ipc_util.h Add: #include <linux/init.h>

Source: /linux-2.6.20-krg/include/kerrighed/sched.h

/kerrighed/epm.c

Add: #include <kerrighed/syscalls.h> #include <kerrighed/signal.h> #include <kerrighed/sched.h>

Source: /linux-2.6.20-krg/kerrighed/mm.c Add: #include <asm/tlbflush.h>

PS:Some header files need to be modified after patch is done.(FOR 2.6.20-krg_x86_64)

[rider]

2.6.20-krg_x86_32 config for Kerrighed-2.2.1 (Server: 140.110.31.201 Runs)

[rider]

Using two different ways to install kerrighed-2.2.1 on linux-2.6.20-krg_x86_64 Server:140.110.31.203 1.make patch to linux-2.6.20-krg then using the 2.6.20_x86_64.config file,and

kerrighed is been built-in.Then make the kernel and make module_install install. The script of kerrighed may not be installed and some scripts are missing.

dmesg | grep i kerrighed ---> seems that it has been loaded. Kerrighed session ID : 0 Kerrighed node ID : 0 Kerrighed nb nodes : 0 Kerrighed min nodes : -1

2.Under kerrighed source document to make install the kernel is fine , but make may

get some errors.

There are still some problems for kerrighed 2.2.1 that runs on 2.6.20_x86_64 ,and maybe check the Makefile will find someting about this problem.

[rider]

Solution: Source:/usr/src/kerrighed-2.2.1/modules/rpc/rpclayer.c -> modify 386~392行

memcpy(data, desc_recv->iter->data, desc_recv->iter->size);

} else {

printk("unsufficient room for received packet (%d %lu-%lu)!\n",

desc->rpcid, desc->desc_id, desc_recv->iter->seq_id);

BUG();

}

Caution !!! Q: kerrighed: Unknown symbol alloc_page_vma Sol: CONFIG_NUMA is not set(勿開)

Q: kerrighed: Unknown symbol free_ipc_ns Sol: CONFIG_IPC_NS is not set(勿開)

[rider]

rider@krg64:~$ dmesg | grep -i kerrighed Kerrighed session ID : 0 Kerrighed node ID : 0 Kerrighed nb nodes : 0 Kerrighed min nodes : -1 Start loading Kerrighed... Init Kerrighed worker(s)...Init Kerrighed low-level framework... Init kerrighed syscall mechanism Kerrighed tools - init module Init Kerrighed low-level framework (nodeid 0) : done Init Kerrighed distributed services... Init Kerrighed distributed services: done Kerrighed... loaded!

[rider]

2.6.20-krg_x86_64 config for Kerrighed-2.2.1

[rider]

Remember to update the Intel® PRO/1000 PT Dual Port Server Adapter driver.

URL:http://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&Inst=Yes&ProductID=2248&DwnldID=9180&strOSs=39&OSFullName=Linux*&lang=eng

Q: TPIC problem Sol:Close all TPIC options in kernel and update the adapter driver.

[rider]

2.6.20-krg_x86_64 config for Kerrighed-2.2.1 (TPIC Closed)

[rider]

Intel® PRO/1000 PT Dual Port Server Adapter driver

[rider]

libreiserfs0.3-0 for amd64 (fixed for drbl deployment)

[anonymous]

Milestone 1 - 建立 Kerrighed 測試平台 deleted