做数据集成或开放数据探索的团队,终于有了一个不依赖元数据就能搜表格的工具——PIPER用LLM生成伪查询,直接基于表格内容做检索,效果比传统方法好很多,值得试试。
PIPER是一种针对表格数据集的内容驱动检索方法,专门解决数据湖、数据空间和开放数据门户中元数据不完整或质量低下的问题。它利用大语言模型(LLM)生成伪查询,并结合表格配置文件进行密集检索,从而超越传统基于元数据的方法和基于表格问答的检索方法。实验表明,PIPER在元数据匮乏的场景下表现优异,证明了LLM内容建模在表格数据集搜索中的价值。该方法适用于需要高效重用和分析表格数据的场景,如数据集成和开放数据探索。
PIPER: Content-Based Table Search via profiling and LLM-Generated Pseudoqueries
The rapid growth of tabular datasets in data lakes, data spaces, and open data portals makes effective dataset search essential for reuse and analysis. Existing search systems rely mainly on metadata, which is often incomplete or low quality, especially for tables whose meaning depends on both schema and cell values. Recent advances in Large Language Models (LLMs) enable richer, content-based representations of tables. However, prior LLM-based retrieval methods have focused on Table Question Answering, where the goal is to select a single table to answer a question, rather than retrieve and rank relevant datasets. We propose PIPER, a content-driven retrieval method for tabular datasets that uses table profiles and LLM-generated queries embedded for dense retrieval. Designed for dataset search in poor-metadata settings, PIPER outperforms both classical metadata-based baselines and strong TableQA retrieval methods, demonstrating the value of LLM-based content modeling for tabular dataset search.