Renmin University, Tencent enhance database performance with 32% transaction speed improvement

Renmin University and Tencent's joint innovation lab presented advancements in financial-grade distributed database systems during their annual conference. Over two years, the collaboration resulted in seven papers accepted at prominent database conferences, with core technologies integrated into TencentDB TDSQL. These advancements led to a 32% increase in transaction speeds for state-owned banks and a 40% improvement in query speeds for government big data applications.

The joint research introduced several key technologies:

• HDCC Protocol: This protocol merges deterministic (Calvin) and non-deterministic (OCC) concurrency control, resolving cross-mechanism recovery issues while delivering 5.7 times performance gains.
• Lion Protocol: Utilizing LSTM-based workload prediction, this protocol dynamically optimizes replica placement to circumvent migration bottlenecks, achieving 2.7 times higher throughput and 76.4% better scalability.

In the area of storage optimization:

• HOCO Engine: This engine enables direct computation on compressed text data through homomorphic compression, providing 9.18 times higher throughput for access and modification, alongside 7.16 times lower latency for analytics.
• SALI Framework: A scalable learned index, this framework adapts to workload skew and enhances concurrency, resulting in 2.04 times higher insertion throughput under high-thread conditions.

The partnership brings together academic insights from Renmin University and practical applications from Tencent's TDSQL Research and Development (R&D) team. Leaders such as Distributed Unit (DU) Xiaoyong from Renmin and Wang Juhong from Tencent emphasized the collaboration's role in advancing database systems for crucial sectors like finance and governance. This effort illustrates China's focus on developing high-performance database systems by integrating Artificial Intelligence (AI) and Machine Learning (ML) techniques to address scalability and efficiency challenges.