Design and Implementation of a High-Performance Distributed Cluster File System for Linux Environments

International Journal of Computer Science and Technology | Vol. 8, No. 5, May 2006 | pp. 33–40

DOI: 10.46882/2006/IJCST/080520

Research Article

Title: Design and Implementation of a High-Performance Distributed Cluster File System for Linux Environments

Names of Authors: Mikhail Volkov¹, Alan Turing Fletcher²

Authors’ Affiliations: ¹Department of Computer Systems and Networks, Bauman Moscow State Technical University, Moscow, Russia; ²Department of Computer Science, University of Edinburgh, Edinburgh, UK

Abstract:
Large-scale computational tasks running on high-performance Linux clusters often experience processing delays due to standard storage systems. This paper presents a high-performance distributed cluster file system designed to remove these performance limits by handling metadata operations separately from parallel data transfers. Storage nodes run a custom block-allocation protocol that allows multiple clients to read and write to raw disk partitions at the same time using direct DMA transfers over InfiniBand networks. Performance benchmarks on a 32-node computing cluster achieved aggregate sequential write speeds of 4.25 GBps and read velocities of 5.12 GBps. The time required for metadata updates remained constant at O(1) through an active distributed lock-caching algorithm. The system recovered from a storage node failure in less than t = 4.5 seconds, proving its reliability for handling heavy data transfers in scientific computing environments.

Keywords: Cluster File System, Distributed Storage, Linux Clusters, High-Performance Computing, InfiniBand

Manuscript Timeline: Received February 12, 2006; Revised March 22, 2006; Accepted April 11, 2006; Published May 26, 2006

Citation: Volkov, M., & Fletcher, A. T. (2006). Design and Implementation of a High-Performance Distributed Cluster File System for Linux Environments. International Journal of Computer Science and Technology, 8(5), 33–40.

Subscribe to read the full article: https://www.internationalscholarsjournals.org/subscribe-to-read