[Megatron-LM] Merge the latest Megatron-LM repository #354

Job	Run time
megatron-report-clean / clean-report	12s
flagscale-report-clean / clean-report	12s
flagscale-runner / unit-test	35s
flagscale-root / unit-test	1m 0s
megatron-distributed / unit-test	1m 49s
megatron-root / unit-test	1m 22s
megatron-data / unit-test	1m 22s
megatron-dist_checkpointing / unit-test	15m 23s
megatron-export / unit-test	58s
megatron-fusions / unit-test	48s
megatron-inference / unit-test	1m 28s
megatron-models / unit-test	1m 32s
megatron-pipeline_parallel / unit-test	1m 9s
megatron-ssm / unit-test	48s
megatron-tensor_parallel / unit-test	1m 48s
megatron-transformer / unit-test	1m 12s
megatron-transformer/moe / unit-test	11m 16s
train-mixtral / functional-test	53s
train-aquila / functional-test	1s
hetero_train-${{ matrix.model }}	0s
megatron-coverage-test	0s
flagscale-coverage-test	0s
all-tests	0s
	43m 48s

Provide feedback