| 1 | Introduction |
|---|
| 2 | ------------ |
|---|
| 3 | This directory includes a set of scripts that implements HA-NFS using GPFS |
|---|
| 4 | as the file system. |
|---|
| 5 | |
|---|
| 6 | Network Setup |
|---|
| 7 | ------------- |
|---|
| 8 | Setup an IP address for the GPFS cluster, and a separate set of IP addresses |
|---|
| 9 | (one or more) for NFS serving. Use static IP address. DHCP can not be used. |
|---|
| 10 | |
|---|
| 11 | Dependencies |
|---|
| 12 | ------------ |
|---|
| 13 | (1) Linux heartbeat |
|---|
| 14 | (2) Kernel patch for lockd. (until the code is included in the kernel) |
|---|
| 15 | (3) Fixed sm-notify (until SuSe release the fixed version) |
|---|
| 16 | |
|---|
| 17 | Cluster Setup |
|---|
| 18 | ------------- |
|---|
| 19 | 1. Create a GPFS cluster and file systems. |
|---|
| 20 | |
|---|
| 21 | 2. Create a small file system for the HA-NFS shared files. This file system |
|---|
| 22 | should be configured to be mounted when GPFS starts (mmcrfs -A yes). |
|---|
| 23 | |
|---|
| 24 | 3. Add all file systems but the one for HA-NFS to /etc/exports. |
|---|
| 25 | |
|---|
| 26 | 4. Stop the GPFS cluster on all nodes with mmshutdown -a |
|---|
| 27 | or on one node use /etc/init.d/gpfs stop |
|---|
| 28 | |
|---|
| 29 | 5. Modify .sample files appropriately and copy them to /var/mmfs/etc (drop the .sample extension) |
|---|
| 30 | cp -p /usr/lpp/mmfs/samples/nfscluster/nfsdefs.sample /var/mmfs/etc/nfsdefs |
|---|
| 31 | cp -p /usr/lpp/mmfs/samples/nfscluster/nfs.nodes.sample /var/mmfs/etc/nfs.nodes |
|---|
| 32 | cp -p /usr/lpp/mmfs/samples/nfscluster/alert.sample /var/mmfs/etc/alert |
|---|
| 33 | |
|---|
| 34 | 6. Edit nfsdefs; use the fields as described in the file. |
|---|
| 35 | |
|---|
| 36 | 7. Edit nfs.nodes; each line is a mapping between GPFS IP address and NFS IP |
|---|
| 37 | address(es). One line for every node in the NFS cluster. |
|---|
| 38 | |
|---|
| 39 | 8. Edit alert is optional. This script is invoked to alert an administrator |
|---|
| 40 | that something is wrong with the NFS cluster. Edit this file to match your |
|---|
| 41 | needs. |
|---|
| 42 | |
|---|
| 43 | 9. To complete configuration of GPFS and Linux for HA-NFS on all nodes run the |
|---|
| 44 | following on one node. |
|---|
| 45 | /usr/lpp/mmfs/samples/nfscluster/install_ha-nfs.sh -a /var/mmfs/etc/nfsdefs /var/mmfs/etc/nfs.nodes |
|---|
| 46 | |
|---|
| 47 | 10. Start the GPFS cluster. |
|---|
| 48 | mmstartup -a |
|---|
| 49 | |
|---|
| 50 | |
|---|
| 51 | Notes |
|---|
| 52 | ----- |
|---|
| 53 | Configure GPFS as an HA-NFS cluster: |
|---|
| 54 | |
|---|
| 55 | The following option are already set by install_ha-nfs.sh script. |
|---|
| 56 | |
|---|
| 57 | For better read performance of big files set the following option. |
|---|
| 58 | mmchconfig nfsprefetchstrategy=1 |
|---|
| 59 | |
|---|
| 60 | To get faster recovery time: |
|---|
| 61 | Current lease time and wait time are 35 second. |
|---|
| 62 | Set leaserecoverywait to no less than 10 second. |
|---|
| 63 | mmchconfig leaserecoverywait=10 |
|---|
| 64 | |
|---|
| 65 | Set leaseDuration to no less than 20 second. |
|---|
| 66 | mmchconfig leaseDuration=20 |
|---|
| 67 | |
|---|
| 68 | user exits: |
|---|
| 69 | mmfsup, mmfsup.scr, mmfsdown, and mmfsdown.scr are used by HA-NFS |
|---|
| 70 | all old scripts in /var/mmfs/etc are saved in /var/mmfs/etc.bak |
|---|
| 71 | After HA-NFS installation the user can add a calls to its exits from |
|---|
| 72 | mmfsup.scr or mmfsdown.scr |
|---|
| 73 | The user should NOT create an mmfsup or mmfsdown scripts for now. |
|---|
| 74 | |
|---|
| 75 | bonding: |
|---|
| 76 | If bonding is used add an entry like the following to ifcfg file |
|---|
| 77 | for correct monitoring by mii-tool. |
|---|
| 78 | |
|---|
| 79 | BONDING_MODULE_OPTS='miimon=100 mode=1 use_carrier=0' |
|---|
| 80 | |
|---|
| 81 | Clean-up: |
|---|
| 82 | If GPFS or some other component terminated abnormally you can try |
|---|
| 83 | the following command to avoid reboot. (reboot is recommended) |
|---|
| 84 | /var/mmfs/etc/cleanup |
|---|
| 85 | |
|---|
| 86 | nfs.nodes example: |
|---|
| 87 | # The first column is the IP addresses used for the GPFS cluster. |
|---|
| 88 | # The second column is the IP addresses used to export NFS. |
|---|
| 89 | |
|---|
| 90 | 192.168.2.13 9.1.140.65 |
|---|
| 91 | 192.168.2.14 9.1.140.66 |
|---|
| 92 | 192.168.1.15 9.1.140.67 |
|---|
