Bug: The custom model is not used at the time of training

[ID-Akash]

https://github.com/Farama-Foundation/PettingZoo/blob/849414dfcba7f423f4db02bfecc7ef061cccc567/tutorials/Ray/rllib_pistonball.py#L21C7-L21C17

The custom model is not used at the time of training the RL Agent. Instead, the default built-in model is used. This can be verified by simply changing the output dimension of Sequential layer from 512 to any random number and still the training works.

Please suggest the changes required.

[ID-Akash]

Just making the following changes would work:

config = ( PPOConfig() .environment(env=env_name, clip_actions=True) .rollouts(num_rollout_workers=4, rollout_fragment_length=128) .training( train_batch_size=512, lr=2e-5, gamma=0.99, lambda_=0.9, use_gae=True, clip_param=0.4, grad_clip=None, entropy_coeff=0.1, vf_loss_coeff=0.25, sgd_minibatch_size=64, num_sgd_iter=10, model={"custom_model": "CNNModelV2"}, ) .debugging(log_level="ERROR") .framework(framework="torch") .resources(num_gpus=int(os.environ.get("RLLIB_NUM_GPUS", "0"))) )

[jjshoots]

Would you like to make a PR for this?