Skip to content

Commit

Permalink
fix: bug
Browse files Browse the repository at this point in the history
  • Loading branch information
@duterscmy
duterscmy committed Dec 12, 2024
1 parent 220bde7 commit b44d2cc
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions cd-moe/modeling_deepseek.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -393,8 +393,8 @@ def backward(ctx, grad_output):

# hyper parameters
layer_num = 27
num_route_experts = 0
prune_layer_num = 9
num_route_experts = 6
prune_layer_num = 15

# current_dir = os.path.dirname(os.path.abspath(__file__))
current_dir = "/home/work/mt_cmy/programs/CD-MoE/cd-moe/data"
Expand Down

0 comments on commit b44d2cc

Please sign in to comment.