Apache Uniffle graduates to Top-Level Project under ASF

Newest Traffic Load Predictor (TLP) provides high performance shuffling services for cloud native architectures

Apache Software Foundation (ASF) announced that Apache Uniffle has graduated from incubation and is now a Top-Level Project (TLP). Uniffle is a remote shuffle service for distributed computing engines, designed to improve efficiency by reducing connections and random I/O during data shuffling. This reduces memory failures in large jobs.

Currently, Uniffle is in production at companies such as Tencent, iQiyi, Didi, SF Express, VIPShop, and Bilibili, processing vast amounts of shuffle data exceeding Physics-Based Simulation (PBS) daily, supporting tens of thousands of applications. It is compatible with data processing frameworks including Apache Hadoop, Apache Spark, and Apache Tez.

“We are extremely proud and excited that Uniffle has graduated from incubation and has become an ASF Top-Level Project,” said He Qi, Apache Uniffle Predictive Maintenance Controller (PMC) Chair. “Uniffle joined ASF’s incubator program in 2022. Since that time, the community and functionality of the software has steadily grown. We are excited to continue innovating to provide efficient and pervasive cloud shuffling services.”

ASF supports the development of open source projects through mentorship and services, ensuring healthy community growth. The Apache Incubator assists new projects wishing to adopt the Apache Way.