| 12 | | * '''2.2.2''' 作者先舉 '''NFS''' 來介紹： |
| 13 | | * __'''NFS:'''__ |
| 14 | | * Primary file system for clusters, and is pretty much” plug and play” on most *nix systems. |
| 15 | | * It was the first popular file system that allowed distributed systems to share data. |
| 16 | | * __'''NFSv3:'''__ |
| 17 | | * The most popular version of NFS. It was released around 1995 and |
| 18 | | * added several features including support for 64-bit file sizes and offsets (so it can handle files larger than 4GB), asynchronous write support, and TCP as a transport layer. |
| 19 | | * __'''NFSv4:'''__ |
| 20 | | * Around 2003, NFSv4 was released with some improvements. In particular, |
| 21 | | * it added some speed improvements, strong security (with compatibility for multiple security protocols), and NFS became a stateful protocol. |
| 22 | | * __'''NFS 的效能面：'''__ |
| 23 | | * The good news for NFS and NAS is that many codes don’t require lots of I/O for good performance. These codes will run very well using NFS as the storage protocol even for large runs (100+ nodes or several hundred cores). NFS provides adequate performance until the input and output files for these codes become extremely large, or if the code is run across a very large number of processors (in the thousands). |
| 24 | | * NFS still lacks the performance and scalability required by many large clusters, but that is about to change. |
| 25 | | * __'''pNFS (NFSv4.1):'''__ |
| 26 | | * adding '''''PFS capability to the NFS protocol'''''. The goal is to improve '''''performance and scalability''''' while making the changes within a standard (recall that NFS is the only true shared file system standard). |
| 27 | | * this standard is designed to be used with file based, block based, and object based storage devices with an eye towards freeing customers from vendor lock-in. |
| 28 | | * pNFS Architecture, [[Image(pNFS_arch.png)]] |
| 29 | | * pNFS Sever 連接 Client 和 Storage ，當 Client 要存取檔案時先至 pNFS Server 查詢 Metadata 找到檔案的位置，再連接到存放檔案的 Sotrage |
| | 12 | * [wiki:rock/paper/PFS_HPC/NFS '''2.2.2''' 作者先舉 NFS 來介紹：] |
| 50 | | * __''' GPFS '''__ |
| 51 | | * 早期只能使用在 AIX 系統上，後來 IBM 將其移植到 Linux，早期只能使用在 IBM 的機器上，到了2005年非IBM的機器也可使用，目前只有一家 OEM 提供GPFS (linux networx) |
| 52 | | * it’s direct attached storage (DAS) or some type of Storage Area Network (SAN) storage. In some cases, you can combine various types of storage. |
| 53 | | * GPFS -> high-speed, parallel, distributed file system. GPFS achieves high-performance by striping data across multiple disks on multiple storage devices. |
| 54 | | * 使用三種 striping 方式： |
| 55 | | 1. '''''Round Robin''''' |
| 56 | | 2. '''''Random''''' |
| 57 | | 3. '''''Balanced Random''''' |
| 58 | | * To further improve performance, GPFS uses client-side caching and deep prefetching such as '''''read-ahead and write-behind'''''. It recognizes standard '''''access patterns''''' (sequential, reverse sequential, and random) and will optimize I/O accesses for these particular patterns. Furthermore GPFS can read or write large blocks of data in a single I/O operation. |
| 59 | | * '''''Block size''''' 可選擇 16KB, 64KB, 256KB, 512KB, 1MB, and 2MB block sizes are supported with 256K being the most common |
| 60 | | * large block sizes helps improve performance when large data accesses are common |
| 61 | | * Small block sizes are used when small data accesses are common |
| 62 | | * GPFS subdivides the blocks into '''''32 sub-blocks'''''. |
| 63 | | * Block is a largest chunk of contiguous data that can be accessed. A sub-block is the smallest contiguous data that can be accessed. Sub-blocks are useful for files that are smaller than a block and are stored using the sub-blocks. This can help the performance of applications that use lots of small data files (i.e. life sciences applications). |
| 64 | | * High Availability (HA) -> GPFS uses distributed metadata so that there is no single point of failure, nor a performance bottleneck. GPFS can be configured to use logging and replication. GPFS will log (journal) the metadata of the file system. |
| 65 | | * GPFS can also be configured for fail-over both at a disk level and at a server level |
| 66 | | * GPFS 現今依舊有在使用，in the Linux world, there are GPFS clusters with over 2,400 nodes (clients). One aspect of GPFS that should be mentioned in this context is that the GPFS is priced by the node for both I/O nodes and clients. |
| 67 | | * GPFS 版本： |
| 68 | | * 3 -> only uses TCP as the transport protocol |
| 69 | | * 4 -> have native IB protocols |
| 70 | | * In addition, the I/O nodes of GPFS can act as NFS servers if NFS is required. |
| 71 | | * Feature -> multi-cluster. This allows two different GPFS file systems to be connected over a network.This is a great feature for groups in disparate locations to share data |
| 72 | | * last feature -> GPFS Open Source Portability Layer. \The portability layer allows these GPFS kernel modules to communicate with |
| 73 | | the Linux kernel. A way to create a bridge from the GPL kernel to a non-GPL set of |
| 74 | | kernel modules, it actually serves a very useful purpose. |
| | 33 | 2. [wiki:rock/paper/PFS_HPC/PFS-type2 Second type: Object based file systems] |
