From f795dc439df305e48a5ebb5e9003237a8de201d6 Mon Sep 17 00:00:00 2001 From: Yanjun Qi / Jane Date: Sat, 13 Apr 2024 13:39:27 -0400 Subject: [PATCH] Update S0-L25.md --- _contents/S0-L25.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/_contents/S0-L25.md b/_contents/S0-L25.md index f3a24700..54f04080 100755 --- a/_contents/S0-L25.md +++ b/_contents/S0-L25.md @@ -1,6 +1,6 @@ --- layout: post -title: LLM Architecture +title: LLM Tooling II lecture: lectureVersion: next extraContent: @@ -19,11 +19,9 @@ In this session, our readings cover: ## Required Readings: +### Delta tuning: A comprehensive study of parameter efficient methods for pre-trained language models." +arXiv preprint arXiv:2203.06904 (2022).[4] -### Attention Mechanisms in Computer Vision: A Survey -+ Meng-Hao Guo, Tian-Xing Xu, Jiang-Jiang Liu, Zheng-Ning Liu, Peng-Tao Jiang, Tai-Jiang Mu, Song-Hai Zhang, Ralph R. Martin, Ming-Ming Cheng, Shi-Min Hu -+ https://arxiv.org/abs/2111.07624 -+ Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multi-modal tasks and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention and branch attention; a related repository this https URL is dedicated to collecting related work. We also suggest future directions for attention mechanism research. ### Recent Large Language Models Reshaping the Open-Source Arena