Louis Aeilot's Blog
blog.aeilot.top
Rotary Position Embedding, usually called RoPE, is one of the most common positional encoding methods in modern decoder-only language models. It appears in models such as LLaMA and many of its descend…
Louis Aeilot's Blog 的其他文章
- How Close Is FlashAttention to the Limit? Understanding Attention Through Data Movement
- Beyond FLOPs: How COSMA Builds Parallel Matrix Multiplication from Communication Bounds
- The Red-Blue Pebble Game: Why Faster Processors Still Have to Move Data
- Git Needs a Trash Can
- CS336 Assignment 1: Large Language Model Training and Inference
讨论
登录后参与讨论
还没有评论，来说第一句吧。