.TH mmrestripefs 02/16/06
mmrestripefs Command
.SH "Name"
.PP
\fBmmrestripefs\fR - Rebalances or restores the replication factor of all
files in a file system.
.SH "Synopsis"
.PP
\fBmmrestripefs\fR \fIDevice\fR {\fB-m\fR | \fB-r\fR |
\fB-b\fR | \fB-p\fR} [\fB-N\fR
{\fINode\fR[,\fINode\fR...] |
\fINodeFile\fR | \fINodeClass\fR}] [\fB-P\fR \fIPoolName\fR]
.SH "Description"
.PP
Use the \fBmmrestripefs\fR command to rebalance or restore the
replication factor of all files in a file system. The
\fBmmrestripefs\fR command moves existing file system data between
different disks in the file system based on changes to the disk state made by
the
\fBmmchdisk\fR, \fBmmadddisk\fR, and \fBmmdeldisk\fR commands.
.PP
The \fBmmrestripefs\fR command attempts to restore the metadata or data
replication factor of any file in the file system.
.PP
You must specify one of the four options (\fB-b\fR, \fB-m\fR,
\fB-r\fR, or \fB-p\fR) to indicate how much file system data to
move. You can issue this command against a mounted or unmounted file
system.
.PP
If you do not use replication, the \fB-m\fR and \fB-r\fR options are
equivalent. Their behavior differs only on replicated files.
After a successful replicate (\fB-r\fR option), all suspended disks are
empty. A migrate operation, using the \fB-m\fR option, leaves data
on a suspended disk as long as at least one other replica of the data remains
on a disk that is not suspended. Restriping a file system includes
replicating it. The \fB-b\fR option performs all the operations of
the \fB-m\fR and \fB-r\fR options.
.PP
Consider the necessity of restriping and the current demands on the
system. New data that is added to the file system is correctly
striped. Restriping a large file system requires a large number of
insert and delete operations and may affect system performance. Plan to
perform this task when system demand is low.
.PP
When using SANergy, consider these points:
.RS +3
.HP 3
\(bu If the \fBmmrestripefs\fR command is issued on a file that is locked by
SANergy, the command waits until it is unlocked before proceeding.
.HP 3
\(bu I/O operations from SANergy clients must terminate before using the
\fBmmrestripefs\fR command. If not, the client applications receive
an error.
.RE
.PP
\fBDetermining how long mmrestripefs takes to complete\fR
.PP
To determine how long the \fBmmrestripefs\fR command will take to complete,
consider these points:
.RS +3
.HP 3
1. How much data is to be moved by issuing the \fBdf -k\fR command.
.HP 3
2. How many GPFS client nodes there are to do the work.
.HP 3
3. How much virtual shared disk server or Network Shared Disk (NSD)
server bandwidth is available for I/O.
.HP 3
4. If you have added new disks to a file system, after the disks have been
added determine how much free space is on each of the new disks by issuing the
\fBmmdf\fR \fIDevice\fR \fB- q\fR
command.
.RE
.PP
The restriping of a file system is done by having one thread on each node in
the cluster work on a subset of files. Consequently, the more GPFS
client nodes performing work for the restripe, the faster the
\fBmmrestripefs\fR command will complete. The nodes that should
participate in the restripe are specified on the command using the \fB-N\fR
parameter. Based on raw I/O rates, you should be able to estimate the
length of time for the restripe. However, to account for the overhead
of scanning all metadata, that value should be doubled.
.PP
Assuming that you have enough nodes to saturate the disk servers, and have
to move all of the data, the time to read and write every block of data is
roughly:
.sp
.nf
  2 * fileSystemSize / averageDiskserverDataRate
.fi
.sp
.PP
As an upper bound, due to overhead of scanning all of the metadata, this
time should be doubled. If other jobs are heavily loading the virtual
shared disk servers, this time may increase even more.
.RS +3
\fBNote:\fR
.RE
.RS +9
There is no particular reason to stop all other jobs while the
\fBmmrestripefs\fR command is running. The CPU load of the command
is minimal on each node and only the files that are being restriped at any
moment are locked to maintain data integrity.
.RE
.SH "Parameters"
.PP
.RS +3
\fB\fIDevice\fR
\fR
.RE
.RS +9
The device name of the file system to be restriped. File system
names need not be fully-qualified. \fBfs0\fR is as acceptable as
\fB/dev/fs0\fR. 
.PP
This must be the first parameter.
.RE
.PP
.RS +3
\fB-N {\fINode\fR[,\fINode\fR...] |
\fINodeFile\fR | \fINodeClass\fR}
\fR
.RE
.RS +9
Specify the nodes that participate in the restripe of the file
system. This command supports all defined node classes. The
default is \fBall\fR (all nodes in the GPFS cluster will participate in the
restripe of the file system).
.PP
For information on how to specify node names, see \fISpecifying nodes as input to GPFS commands\fR
in \fIGPFS: Administration and Programming Reference\fR.
.RE
.SH "Options"
.PP
.RS +3
\fB-b
\fR
.RE
.RS +9
Rebalances all files across all disks that are not suspended, even if they
are stopped. Although blocks are allocated on a stopped disk, they are
not written to a stopped disk, nor are reads allowed from a stopped disk,
until that disk is started and replicated data is copied onto it. The
\fBmmrestripefs\fR command rebalances and restripes the file system.
Use this option to rebalance the file system after adding, changing, or
deleting disks in a file system.
.RS +3
\fBNote:\fR
.RE
.RS +9
Rebalancing of files is an I/O intensive and time consuming operation, and is
important only for file systems with large files that are mostly
invariant. In many cases, normal file update and creation will
rebalance your file system over time, without the cost of the
rebalancing.
.RE
.RE
.PP
.RS +3
\fB-m
\fR
.RE
.RS +9
Migrates all critical data off any suspended disk in this file
system. Critical data is all data that would be lost if currently
suspended disks were removed.
.RE
.PP
.RS +3
\fB-P \fIPoolName\fR
\fR
.RE
.RS +9
Directs \fBmmrestripefs\fR to repair only files assigned to the
specified storage pool.
.RE
.PP
.RS +3
\fB-p
\fR
.RE
.RS +9
Directs \fBmmrestripefs\fR to repair the file placement within the
storage pool.
.PP
Files assigned to one storage pool, but with data in a different pool, will
have their data migrated to the correct pool. Such files are referred
to as ill-placed. Utilities, such as the \fBmmchattr\fR command, may change a file's storage
pool assignment, but not move the data. The \fBmmrestripefs\fR
command may then be invoked to migrate all of the data at once, rather than
migrating each file individually. Note that the rebalance operation,
specified by the \fB-b\fR option, also performs data placement on all
files, whereas the placement option, specified by \fB-p\fR, rebalances only
the files that it moves.
.RE
.PP
.RS +3
\fB-r
\fR
.RE
.RS +9
Migrates all data off suspended disks. It also restores all
replicated files in the file system to their designated degree of replication
when a previous disk failure or removal of a disk has made some replica data
inaccessible. Use this parameter either immediately after a disk
failure to protect replicated data against a subsequent failure, or before
taking a disk offline for maintenance to protect replicated data against
failure of another disk during the maintenance process.
.RE
.SH "Exit status"
.PP
.PP
.RS +3
\fB0
\fR
.RE
.RS +9
Successful completion.
.RE
.PP
.RS +3
\fBnonzero
\fR
.RE
.RS +9
A failure has occurred.
.RE
.SH "Security"
.PP
You must have root authority to run the \fBmmrestripefs\fR
command.
.PP
You may issue the \fBmmrestripefs\fR command from any node in the GPFS
cluster.
.PP
When using the \fBrcp\fR and \fBrsh\fR commands for remote
communication, a properly configured \fB.rhosts\fR file must exist
in the root user's home directory on each node in the GPFS
cluster. If you have designated the use of a different remote
communication program on either the
\fBmmcrcluster\fR
or the
\fBmmchcluster\fR command, you must
ensure:
.RS +3
.HP 3
1. Proper authorization is granted to all nodes in the GPFS cluster.
.HP 3
2. The nodes in the GPFS cluster can communicate without the use of a
password, and without any extraneous messages.
.RE
.SH "Examples"
.RS +3
.HP 3
1. To move all critical data from any suspended disk in file system
\fBfs0\fR, issue this command:
.sp
.nf
mmrestripefs fs0 -m
.fi
.sp
The system displays information similar to:
.sp
.nf
Scanning file system metadata, phase 1 ...
Scan completed successfully.
Scanning file system metadata, phase 2 ...
Scan completed successfully.
Scanning file system metadata, phase 3 ...
Scan completed successfully.
Scanning file system metadata, phase 4 ...
Scan completed successfully.
Scanning user file metadata ...
6 % complete on Fri Feb 10 15:45:07 2006
45 % complete on Fri Feb 10 15:48:03 2006
78 % complete on Fri Feb 10 15:49:28 2006
85 % complete on Fri Feb 10 15:49:53 2006
100 % complete on Fri Feb 10 15:53:21 2006
Scan completed successfully.
.fi
.sp
.HP 3
2. To rebalance all files in file system \fBfs1\fR across all defined,
accessible disks that are not stopped or suspended, issue this command:
.sp
.nf
mmrestripefs fs1 -b
.fi
.sp
The system displays information similar to:
.sp
.nf
GPFS: 6027-589 Scanning file system metadata, phase 1 ...\ 
  48 % complete on Wed Aug 16 16:47:53 2000
  96 % complete on Wed Aug 16 16:47:56 2000
 100 % complete on Wed Aug 16 16:47:56 2000
GPFS: 6027-552 Scan completed successfully.
GPFS: 6027-589 Scanning file system metadata, phase 2 ...\ 
GPFS: 6027-552 Scan completed successfully.
GPFS: 6027-589 Scanning file system metadata, phase 3 ...\ 
  98 % complete on Wed Aug 16 16:48:02 2000
 100 % complete on Wed Aug 16 16:48:02 2000
GPFS: 6027-552 Scan completed successfully.
GPFS: 6027-565 Scanning user file metadata ...
GPFS: 6027-552 Scan completed successfully.
.fi
.sp
.RE
.SH "See also"
.PP
.PP
mmadddisk Command
.PP
mmapplypolicy Command
.PP
mmchattr Command
.PP
mmchdisk Command
.PP
mmdeldisk Command
.PP
mmrpldisk Command
.PP
mmrestripefile Command
.SH "Location"
.PP
\fB/usr/lpp/mmfs/bin\fR
.PP