Please refer to the online GPFS FAQ at http://publib.boulder.ibm.com/infocenter/clresctr/topic/com.ibm.cluster.gpfs.doc/gpfs_faqs/gpfs_faqs.html for the latest information about GPFS updates, supported kernel levels, and kernel patches before building the portability layer. Linux kernel patches for GPFS can be obtained at http://sourceforge.net/tracker/?atid=719124&group_id=130828&func=browse which is the patches link for the project "General Parallel File System (GPFS)" on Source Forge; only Linux kernel patches are kept on the Source Forge site. You can send license inquiries to gpfs@us.ibm.com or, in writing, to: GPFS Product Manager International Business Machines Company Dept. LNLA Mail Station P963 2455 South Road Poughkeepsie NY 12601-5400 United States of America To build the Linux portability interface for GPFS ------------------------------------------------- It is strongly suggested that you build the source at non root level. You will need to change the owner and group permissions accordingly in /usr/lpp/mmfs/src. When you build as a non root user it is also important that you have read access to the linux kernel header files on your machine. These headers are normally found in /usr/src/linux/include but may appear in a different place as enumerated below in step 3 section D. The simplest way to build the portability layer is to use the "Autoconfig" process to create the site.mcr file for you. 1) export SHARKCLONEROOT=/usr/lpp/mmfs/src 2) cd /usr/lpp/mmfs/src 3) make Autoconfig 4) make World 5) make InstallImages Alternatively, if you have a platform that requires customization, manual creation of the site.mcr file will be necessary as follows. 1) export SHARKCLONEROOT=/usr/lpp/mmfs/src 2) cd /usr/lpp/mmfs/src/config cp site.mcr.proto site.mcr 3) Edit and customize the site.mcr file. A) The default architecture is GPFS_ARCH_I386. Only modify the architecture choice if your platform is not Intel 32-bit based (IA32). B) Modify the Linux distribution choice according to the software distribution your machine runs. C) Modify the Linux distribution level according to the level of the software distribution your machine runs. D) Modify the LINUX_KERNEL_VERSION according to the Linux kernel level you are using with your software distribution. E) Note also that the kernel header file search path, KERNEL_HEADER_DIR, is by default /lib/modules/`uname -r`/build/include Customers who have their kernel source headers in a different directory will need to modify KERNEL_HEADER_DIR. For instance another common location for the kernel headers is /usr/src/linux/include. YOU MUST HAVE READ ACCESS TO THESE HEADERS! F) Uncomment any desired Linux kernel patches listed under LINUX_PATCH_DEFINES. The kernel will also need to be rebuilt and installed with the appropriate patches, which can be obtained from http://sourceforge.net/tracker/?atid=719124&group_id=130828&func=browse. The online GPFS FAQ has detailed information on each patch. 4) If no LINUX_PATCH_DEFINES are applied above and the kernel is not recompiled, some distributions will still require more preparation of kernel headers. A) On SuSE Enterprise Server 8 PowerPC you will need to: Install the kernel source cd /lib/modules/`uname -r`/build sh make_ppc64.sh distclean sh make_ppc64.sh clean sh make_ppc64.sh mrproper sh make_ppc64.sh cloneconfig sh make_ppc64.sh dep B) On SuSE Enterprise Server 8 AMD64 you will need to: Install the kernel source cd /lib/modules/`uname -r`/build make distclean make clean make mrproper make cloneconfig make dep C) On SuSE Enterprise Server 9 all platforms you will need to: Install the kernel source The default header file path must be changed. KERNEL_HEADER_DIR can instead be set to /lib/modules/`uname -r`/source/include 5) cd /usr/lpp/mmfs/src 6) make World All of the kernel modules and libraries reside in the "bin" subdirectory after the build. To install the Linux portability interface for GPFS into /usr/lpp/mmfs/bin -------------------------------------------------------------------------- This step installs the binaries for the portability interface. 1) su 2) make InstallImages The files mmfslinux, lxtrace, tracedev, and dumpconv will be installed in /usr/lpp/mmfs/bin. If the system is using a 2.6 kernel, there will be an additional mmfs26 binary. If this kernel configuration applies to other machines in the cluster, you may then copy these binaries to those machines, or iteratively perform the above compile and install steps on each machine. Additional Notes ---------------- Most of the source code headers are in the "ibm-kxi" and "ibm-linux" directories, while the source code is in the "gpl-linux" directory. The "config" directory provides the configuration information for the build (site.mcr) and the "misc" directory handles some items that allow us to reuse our standard development Imakefiles. Troubleshooting build problems ------------------------------ Most commonly encountered build problems are caused by an incorrect setting of KERNEL_HEADER_DIR, or an incorrect setup of the kernel source tree. GPFS requires a full set of kernel headers in order to build; a subset of kernel headers provided to build "typical" kernel modules may be insufficient and could lead to build breaks. Specifically, on SuSE Enterprise Server 9 and 10, the directory pointed to by the /lib/modules/`uname -r`/build symlink contains only a subset of kernel headers, while the directory pointed to by /lib/modules/`uname -r`/source contains a full source tree. Some files in the full source tree, most notably include/linux/version.h and include/linux/autoconf.h, are often configured dynamically by an external shell script (e.g. /etc/init.d/running-kernel on SuSE Enterprise Server), which copies a set of files into the source tree to make it match the currently booted kernel (that way one can boot different kernels but have the same source tree always configured to match it). If the kernel source tree is updated but the shell reconfiguration script hasn't run (it usually runs during the boot sequence), or the script didn't work correctly, the kernel source tree may not be configured properly. NOTE: On SuSE Enterprise Server 10 for PowerPC, /etc/init.d/running-kernel that ships with the GA level of distribution (kernel-source-2.6.16.21-0.8) contains a bug that results in the wrong set of files being copied to the kernel source tree. This bug will be fixed with SLES10 SP1. If the official fix is unavailable, the following change should also address the problem: --- running-kernel.orig 2006-10-06 14:54:36.000000000 -0500 +++ /etc/init.d/running-kernel 2006-10-06 14:59:58.000000000 -0500 @@ -53,6 +53,7 @@ arm*|sa110) arch=arm ;; s390x) arch=s390 ;; parisc64) arch=parisc ;; + ppc64) arch=powerpc ;; esac # FIXME: How to handle uml?