想扎实掌握LLM原理？这50个实战项目从基础分词到高级注意力分析，用PyTorch一步步搭出来，比看论文快多了。
一本包含50个动手项目的书籍，用Python、PyTorch、Scikit-Learn等工具教你理解大型语言模型内部机制。项目覆盖分词、嵌入、输出logits、Transformer输出、注意力、MLP等核心主题。每个项目配有解决方案，如运用HellaSwag评估模型、用cosine相似度分析嵌入、实现logit lens等方法。完成全部项目可进入该领域前0.01%的水平。
How to become GOD-LEVEL with Large Language Models…
How to become GOD-LEVEL with Large Language Models.
Here are 50 hands-on projects with solutions that will teach you how Large Language Models work.
You don't need to solve all 50, but if you do, you'll be at the top 0.01% of the field.
It's all Python + Pytorch + SciKit-Learn + Pandas + Numpy + Matplotlib + Seaborn.
Here are the 50 problems from the book (link below):
Tokenization
1. Three tokenization schemes 2. Book lengths in characters, words, and tokens 3. Pandas frequency tables of token lengths 4. Token lengths in characters and bytes 5. Is tokenization compression? 6. Tokenization and compression in different languages 7. Translating between tokenizers
Embeddings
8. Distribution of cosine similarities 9. Sequential cosine similarity 10. Sequential number cosine similarity 11. Network graphs of cosine similarities 12. RSA to compare GPT-2 & BERT embeddings 13. Word similarity via distance and cosine 14. Linear semantic axes 15. Analogy vectors
Output logits
16. Softmax probability distributions 17. Probabilistic token selection 18. Token prediction accuracy 19. LLM loss function 20. Perplexity over sequences, texts, and models 21. Predict token position with linear and logistic regressions 22. Evaluating models with HellaSwag 23. Measuring language biases
Transformer outputs
24. Cosine similarities within and across layers 25. Category selectivity via cosine similarity 26. Current layer = previous layer + adjustments 27. Impact of layer-specific noise and scaling 28. Effective dimensionality of hidden layers 29. Hidden state dimensionality reduction 30. Sentiment analysis with decision trees 31. Logit lens 32. Patching hidden states in indirect object identification
Attention
33. QKV weights characteristics 34. QKV activation characteristics 35. Raw and softmax attention scores 36. Characteristics of attention adjustment magnitudes 37. Token prediction and attention KL divergences 38. Laminar profile of RSA and category selectivity 39. Token frequency, attention adjustments, QK^T 40. Downstream impacts of head silencing 41. Patching heads in IOI
MLP
42. MLP weights and activations characteristics 43. Characterizing the MLP progression 44. Grammar tuning in MLP projections 45. Minkowski distance, mutual information, and token positions 46. Statistics-based lesioning in MLP neurons 47. Supervised probing with XGBoost 48. "Can" vs. "can't" classification via logistic regression 49. Successive median-replacement of MLP activations 50. Recommender systems with MLP projections
Book link below.