1. Brief Introduction to Object Based Storage

• In a more classic file system, such as a block based file system
  • the metadata manager is contacted and a set of inodes or blocks is allocated for a file
  • Metadata manager is responsible not only for the metadata itself but also where the data is located on the storage.
  • Metadata manager is a key part of the process. It also creates the potential for a performance problem.
• Object storage takes a different approach allowing the storage devices themselves to manage where the data is stored.
  • metadata manager and storage are usually separate devices
  • metadata manager is contacted by a client about a file operation, such as a write.
  • metadata manager gets out of the way of the actual file operation and allows the client to directly contact the assigned storage devices
  • The metadata manager constantly monitors the operations
    • If there is a change in the file operation, such as another client wanting to read or write data to the file, then the metadata manager has to get involved to arbitrate the file operations
  • In general, the data operations happen without the metadata manager being in the middle

2. The Advantage to Object Based Storage

• 如同開車到百貨公司 shopping，只需把車交給停車場的服務生， shopping 完後直接跟服務生拿鑰匙和停車票 (不需 Care 車子停在哪一層、那一區)
• An object based system can optimize data layout (where your car is located), add more storage (add to the parking garage),
• Object storage has a great deal of flexibility and possibilities for improving file system performance and scalability for clusters

3. Lustre

• Open source PFS, Object based FS (scaling to ten of thousands of nodes and Petabytes of data)
• Lustre stores data as objects called containers that are very similar to files, but are not part of a directory tree
• The advantage to an object based file system is that allocation management of data is distributed over many nodes, avoiding a central bottleneck.
• Lustre has a metadata component, a data component, and a client part
  • these components to be put on different machines, or a single machine (usually only for home clusters or for testing)
  • The metadata can be distributed across machines called MetaData Servers (MDS), to ensure that the failure of one machine will not cause the file system to crash.
  • MDS support failover as well. In Lustre 1.x, you can use up to two MDS machines (one in active mode and one in standby mode) while in Lustre 2.x, the goal is to have tens or even hundreds of MDS machines.