Lustre
What is Lustre?
Lustre File System (Lustre) is an open-source parallel Distributed File System (DFS) (storage infrastructure) designed for large-scale, high-throughput computing environments such as supercomputers and scientific clusters.
- Parallel DFS for High performance computing (HPC) (HPC storage)
- Scales across many servers and clients for large shared namespaces (distributed storage)
- Supports high aggregate bandwidth for data-intensive workloads (performance storage)
- Separates metadata and object data across dedicated servers (file system architecture)
- Used in supercomputing, research, and enterprise clusters for shared POSIX file access (HPC and enterprise storage)
Show more
More About Lustre
Lustre is an open-source parallel DFS (HPC storage) designed to provide a single shared file system across many servers and clients in large compute clusters. It addresses the need for scalable, high-throughput access to shared data in environments where thousands of compute nodes process large datasets concurrently. The project focuses on high bandwidth, large capacity, and POSIX-style file system semantics over clusters used for scientific computing, simulation, analytics, and similar workloads.
The Lustre architecture (file system architecture) separates metadata and bulk data operations. Metadata operations such as file creation, directory changes, and permissions are handled by Metadata Servers (MDS) and stored on Metadata Targets (MDT). File contents are stored on Object Storage Targets (OST), which are managed by Object Storage Servers (OSS). Clients access the file system over a network, interacting with both MDS and Operations Support System (OSS) components to perform file operations. This separation allows metadata and data paths to scale and be tuned independently.
Lustre supports a POSIX-compliant file system interface (file system protocol), enabling applications to interact with it using standard file operations without modification. Files can be striped across multiple OSTs (data striping) to increase aggregate I/O bandwidth, which is relevant for workloads such as checkpointing in HPC applications, large-scale simulations, and data analysis pipelines. The system runs on Linux-based platforms and integrates with standard networking technologies such as Transmission Control Protocol (TCP) and InfiniBand where supported (cluster networking).
In enterprise and institutional environments, Lustre is used as a shared high-throughput storage backend (enterprise storage) for compute clusters in research laboratories, weather and climate modeling centers, energy and engineering organizations, and other data-intensive domains. It supports multi-petabyte deployments and is compatible with job schedulers and workload managers commonly used in HPC clusters, enabling batch and interactive jobs to share a common file system namespace. Administrators can deploy multiple file systems, manage failover configurations, and apply quota and monitoring capabilities depending on the distribution and tooling in use.
The Lustre software stack (storage software) includes client components that run on compute nodes, server components for metadata and object storage, and management utilities for configuration and maintenance. It is maintained and developed under an open-source model, with contributions from commercial vendors, research institutions, and users. In a technical directory, Lustre fits into categories such as HPC parallel file systems, distributed storage, and POSIX-compliant shared file systems for clustered computing environments.