From f55adf3308c35e0bf17c2da20eb1841e3aaa8cf0 Mon Sep 17 00:00:00 2001 From: p208p2002 Date: Fri, 3 May 2024 11:51:55 +0800 Subject: [PATCH] docs: update rope --- public/docs/rope-embedding/document.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/public/docs/rope-embedding/document.md b/public/docs/rope-embedding/document.md index 78de369..baac0dd 100644 --- a/public/docs/rope-embedding/document.md +++ b/public/docs/rope-embedding/document.md @@ -1,11 +1,11 @@ -# 旋轉位置編碼 (RoPE) 筆記與外推方法 +# 旋轉位置編碼 (RoPE) 筆記與長度拓展方法 - tags: #旋轉矩陣#RoPE#position-embedding - date: 2024/05/03 -### 理解 RoPE +## 理解 RoPE - [图解RoPE旋转位置编码及其特性](https://mp.weixin.qq.com/s/-1xVXjoM0imXMC7DKqo-Gw) - [详解基于调整RoPE旋转角度的大模型长度外推方法](https://mp.weixin.qq.com/s?__biz=MzA3MTgwODE1Ng==&mid=2247484826&idx=1&sn=8935f0bcb2e09f438cbf3ae63825d671&chksm=9f26a069a851297f568ba7cd111082e603108716928b8444a253457233f24d09d3a18447d6b9&cur_album_id=3199751010206973953&scene=189) @@ -63,7 +63,7 @@ $$ 在這邊我們還可以發現在每一個分量中，會有不同的旋轉頻率。 -## 拓展方法 +## RoPE 拓展方法 ### Position Interpolation - [Extending Context Window of Large Language Models via Positional Interpolation](https://arxiv.org/abs/2306.15595)