Add unit test for moe sigmoid router (#311) #386

Job	Run time
megatron-report-clean / clean-report	12s
flagscale-report-clean / clean-report	12s
megatron-distributed / unit-test	1m 47s
megatron-root / unit-test	1m 22s
megatron-dist_checkpointing / unit-test	16m 25s
megatron-data / unit-test	1m 23s
megatron-export / unit-test	57s
megatron-fusions / unit-test	47s
megatron-inference / unit-test	1m 27s
megatron-models / unit-test	1m 32s
megatron-pipeline_parallel / unit-test	1m 8s
megatron-tensor_parallel / unit-test	1m 48s
megatron-ssm / unit-test	47s
megatron-transformer / unit-test	1m 13s
megatron-transformer/moe / unit-test	11m 10s
flagscale-root / unit-test	2m 10s
flagscale-runner / unit-test	36s
train-mixtral / functional-test	21m 32s
train-aquila / functional-test	17m 37s
hetero_train-aquila / functional-test	35m 44s
megatron-coverage-test	0s
flagscale-coverage-test	0s
all-tests	0s
	1h 59m 49s

Provide feedback