给老博客补标签别硬分类——先让LLM随便编,再用向量嵌入找最接近的现有标签。Simon Willison分享的这套路,比直接要模型选1856个标签里的一个靠谱。
Simon Willison的博客有1,856个标签,直接让LLM从中挑选匹配项容易超限。Doug Turnbull提出先让模型自由生成标签,再通过向量嵌入在现有标签库中查找最接近的具体标签。示例提示词展示了如何用家具分类层级结构来引导模型生成形状合理的候选标签。这样既能利用模型对语义的理解,又能保证最终标签来自既定词表。
Don't classify. Hallucinate!
Don't classify. Hallucinate! I still have quite a bit of older content on my blog that I never got round to tagging. My blog has 1,856 tags - likely too many to feed to an LLM in one go and say "which of these tags match the following content". Doug Turnbull has a neat solution. Tell the model to output tags without any details of the existing vocabulary, then use vector embeddings against the existing corpus to find the concrete tags that are closest to the ones the model imagined might fit! His example prompt suggests including an example of the shape of your tags to help the model make a more useful guess: Your task is to create novel, never seen before, furniture, home goods, or hardware classification that best fit a search query. Product classifications might look like: Furniture / Living Room Furniture / Coffee Tables & End Tables / Coffee Tables Décor & Pillows / Decorative Pillows & Blankets / Throw Pillows Furniture / Bedroom Furniture / Dressers & Chests Kitchen & Tabletop / Kitchen Organization / Food Storage & Canisters School Furniture and Supplies / School Furniture / School Chairs & Seating / Stackable Chairs Baby & Kids / Toddler & Kids Bedroom Furniture / Kids Beds Here's the query to generate classifications for: brown coffee table Tags: search , ai , generative-ai , llms , embeddings , doug-turnbull