This projects presents a comprehensive study of system failures caused by partial network partitions – an atypical type of network partitioning fault. Also, we dissected the design of eight popular systems, and we found that implemented fault tolerance techniques are inadequate for modern systems. Our findings motivated us to build Nifty, a transparent communication layer that masks partial network partitions.
Slogger is a new disaster recovery system that differs from prior works in two principle ways: (i) Slogger enables DR for a linearizable distributed data store, and (ii) Slogger adopts the continuous backup approach that strives to maintain a tiny lag on the backup site relative to the primary site, thereby restricting the data loss window, due to disasters, to milliseconds. Our evaluation shows that Slogger maintains a near optimal data loss window without having any performance penalty on the primary data store.
FLAIR presents a novel protocol for accelerating read operations in leader-based consensus protocols. FLAIR leverages the capabilities of the new generation of programmable switches to serve reads from follower replicas without compromising consistency. Following FLAIR protocol, we designed FlairKV, a key-value store that implements the processing pipeline using the P4 programming language. Compared to state-of-the-art alternatives, FlairKV brings significant performance gains: up to 42% higher throughput and 35-97% lower latency for most workloads.
This project focuses on studying the impact of network partitioning failures on production-ready distributed systems. We analyzed 136 reported failures in 25 widely used distributed systems. In addition, we developed NEAT, a testing framework that is capable of injecting different types of network partitions.
NICE is a key-value storage system design that leverages new software-defined network capabilities to build cluster-based network-efficient storage system. NICE presents novel techniques to co-design network routing and multicast with storage replication, consistency, and load balancing to achieve higher efficiency, performance, and scalability.
COOL is a simple and generic structure for MPI collective operations. COOL enables highly efficient designs for all collective operations in the cloud. We explore a system design based on COOL that implements frequently used collective operations. Our design efficiently uses the intra-rack network while minimizing cross-rack communication, thus improving the application performance and scalability. We use recent software-defined networking capabilities to build optimal network paths for I/O intensive collective operations.