Code

April tricks#28

Code

Merged

dest1n1s merged 7 commits intomainOpenMOSS/Language-Model-SAEs:mainfrom AprilTricksOpenMOSS/Language-Model-SAEs:AprilTricksCopy head branch name to clipboardJul 2, 2024

+251-66

Commits

Commits on Jul 1, 2024

• feature(MEGA UPDATE): implement options in Anthropic April Update e.g. combine decoder bias norm with L1

  

  Hzfinfdu

  committed
  b8aba9e

• fix(sae): merge a standalone SAE init func with static method

  

  Hzfinfdu

  committed
  7f4091c

• fix(config): ignore mypy checking for norm init.

  

  Hzfinfdu

  committed
  b0b13b0

• fix

  

  Hzfinfdu

  committed
  25a4689

• fix

  

  Hzfinfdu

  committed
  b84b799

Commits on Jul 2, 2024

• fix(typing): fix mypy issues

  

  Hzfinfdu

  committed
  6864795

• feature(activation): add an option to prepend bos before each sentence. Recommend to be True except GPT2 SAEs

  

  Hzfinfdu

  committed
  b2d2380