想快速搞懂主流Transformer架构和各家模型？这篇综述帮你理清了架构分类和应用场景，还比较了参数和能耗，适合做调研入门。
该综述将Transformer架构分为encoder-only、decoder-only、encoder-decoder、长上下文、置换基与生成对抗等变体，并涵盖2023年后指令微调、RLHF、DPO、MoE、RAG等进展。它梳理了OpenAI、Anthropic、Google、Meta、Mistral、DeepSeek等主流模型家族。在应用侧调研了医疗、金融、法律、教育等7个领域的部署。论文从参数规模与能耗的权衡、对齐方法、数据溯源及基准饱和四个维度评估模型。它还点名了值得关注的开放研究问题。
Transformer-Based Language Models Across Domain Verticals: Architectures, Applications and Critical Assessment
Transformer-based language models have become the default substrate for natural language processing and the pace of new releases has made it hard for practitioners to separate durable ideas from the noise of incremental announcements. This review works at two levels. At the level of mechanism, we organise the main transformer families into a working taxonomy, covering encoder-only, decoder-only, encoder-decoder, long-context, permutation-based, and generator-discriminator variants. We then extend the discussion to post-2023 developments that changed the picture in practice: instruction tuning, reinforcement learning from human feedback, direct preference optimisation, mixture-of-experts scaling, retrieval augmentation and the current flagship model families from OpenAI, Anthropic, Google, Meta, Mistral and DeepSeek. At the level of use, we survey deployments across healthcare, finance, legal, education, customer service, creative writing and scientific work. Based on this we link each to the specific capabilities that make a transformer the appropriate tool. The contribution of this paper is a critical assessment that is based on the survey. We compare architectures on four axes that matter to deployment decisions, we quantify the trade-off between parameter count and energy cost. We also discuss how alignment methods, data provenance and benchmark saturation change what it means to call a model "state of the art". The final section lists the research questions that we think deserve more attention.