数据库团队终于有了处理ML过滤器的数据跳过方案——用轻量元数据就能剪枝，做大数据分析或数据库内核开发的建议看看，能直接提升查询性能。
数据库厂商最近发布了可在过滤谓词中使用的AI函数，但这些函数依赖昂贵且黑盒的ML模型，带来了新的数据管理挑战。传统的数据跳过技术（如针对整数和字符串的）无法适用于这种新过滤器类型，因为没有已知机制能在读取blob存储文件时剪枝不符合条件的行组。本文首次研究ML过滤器的数据跳过技术，证明Parquet默认的min-max元数据足以实现剪枝，并借鉴了ML模型查询语言和神经网络验证两个研究方向。在ReLU架构上的初步实验表明，对于选择性低于0.1%的过滤器，平均剪枝效果达27.4%。此外，受空间连接研究启发，作者提出了一种增强元数据结构——有大小限制的2D凸包，使剪枝效果提升至38.31%，且每行组和列对仅占用最多45字节，在DuckDB中实现了相对于PyTorch的1.07倍端到端加速。
MLSkip: Data Skipping for ML Filters via Lightweight Metadata
Database vendors recently released AI functions that can be used in filter predicates. As such functions often rely on costly, black-box ML models, they unveil new data management challenges. Concretely, traditional data skipping techniques for integer and string data fail to be applicable to the new filter type. Indeed, there is no known mechanism for pruning non-qualifying row groups, e.g., when reading files from blob storage. In this work, we initiate the study of data skipping techniques for ML filters. We make the case that Parquet's default min-max metadata is enough to enable pruning. To this end, we draw connections to two lines of research: (i) the recently proposed query language for ML models and (ii) neural network verification. Our preliminary results on ReLU architectures show that on tables from TPC-H and TPC-DS, the average pruning effectiveness for filters of selectivity below 0.1% amounts to 27.4%. Finally, inspired by research on spatial joins, we propose an enhanced metadata structure: a size-bounded 2D convex hull that verification tools can make better use of, increasing the pruning effectiveness to 38.31%, while occupying at most 45 bytes per row group and column pair. We observe an end-to-end speedup of 1.07$\times$ over PyTorch in DuckDB.