Skip to main content

OpenStack Storlets

OpenStack Storlets is an OpenStack project that enables execution of user-defined code directly within the OpenStack Swift object storage data path for in-storage compute and data processing (data processing / storage integration).

  • In-storage compute framework for running user-defined functions on objects stored in OpenStack Swift (data processing).
  • Integration with the OpenStack Swift proxy tier to execute code close to object data (object storage).
  • Mechanisms to upload, manage, and invoke “storlets” as packaged code units associated with containers or objects (application runtime).
  • Support for offloading data transformation, filtering, and computation tasks from clients to the storage layer (data transformation).
  • Policy-based control over where and how storlets execute within the Swift cluster (governance / operations).

More About OpenStack Storlets

OpenStack Storlets is an extension to the OpenStack Swift object storage service that introduces in-storage compute capabilities, enabling user-defined code to run within the Swift data path. The project targets scenarios where data processing operations such as transformations, filtering, and analysis are executed closer to the stored data instead of on client systems or separate compute clusters. By embedding computation into the storage tier, Storlets reduces data movement between storage and applications and supports patterns where object access and processing are tightly coupled.

The core concept in the project is the “storlet,” a packaged code artifact that can be uploaded to Swift and associated with containers or individual objects. Storlets are invoked through standard Swift APIs, typically via Hypertext Transfer Protocol (HTTP) requests that include specific headers or query parameters indicating which storlet to run and with what parameters. The Swift proxy server is extended with middleware that intercepts these requests, loads the appropriate storlet, and executes it in conjunction with the object read or write operation. This places Storlets in the category of data processing frameworks integrated with object storage.

From an architectural perspective, Storlets relies on the Swift proxy layer as the primary execution point. The project introduces components that handle deployment of storlet packages, dependency management, and sandboxed execution environments. Execution is designed so that user code runs under controlled conditions, while still accessing the object data stream. This enables use cases such as server-side image resizing, format conversion, content filtering, metadata extraction, and custom access logic without requiring changes to client applications beyond invoking the storlet.

In enterprise environments, OpenStack Storlets is used when organizations deploy private or hosted OpenStack Swift clusters and require storage-integrated processing. Typical usage scenarios include preprocessing of large datasets stored as objects, implementing organization-specific logic at the storage boundary, and reducing bandwidth consumption by returning transformed or filtered data instead of raw objects. Enterprises can package storlets using languages and frameworks supported by the project and operate them as part of the storage platform.

Storlets interoperates closely with OpenStack Swift and aligns with the broader OpenStack ecosystem for authentication, multi-tenancy, and operations. It fits into enterprise taxonomies as an in-storage compute and data transformation layer for object storage, complementing compute services rather than replacing them. For organizations standardizing on OpenStack, Storlets represents a method to extend Swift beyond static object storage into programmable storage behavior that can be tailored to application and policy requirements.