Netflix分享了Cassandra宽分区优化实战,用动态分区将读取延迟从秒级砍到十几毫秒,对处理海量时序数据的团队很有参考价值。
Netflix AI团队为解决Cassandra中TimeSeries Abstraction的宽分区问题,采用Time Slice重分区和动态分区两种方法。动态分区通过字节计数和Kafka检测超大分区,利用校验和验证分割,并通过Bloom过滤器将读取路由到并行子分区。优化后,平均读取延迟从秒级降至低双位数毫秒,500MB以上的分区仍可正常访问。
Netflix AI Team Cuts Wide-Partition Read Latency from Seconds to Milliseconds by Splitting Cassandra Partitions Per ID
Netflix engineers detailed how they handle wide partitions in Apache Cassandra for the TimeSeries Abstraction. Two approaches work together: Time Slice re-partitioning tunes future partitions at the table level, while dynamic partitioning detects and splits oversized partitions per TimeSeries ID on the read path. Detection runs via byte counting and Kafka, splits are checksum-validated, and Bloom filters route reads to parallel child partitions. Average read latency dropped from seconds to low double-digit milliseconds, with 500MB+ partitions staying available. The post Netflix AI Team Cuts Wide-Partition Read Latency from Seconds to Milliseconds by Splitting Cassandra Partitions Per ID appeared first on MarkTechPost .