UserWarning and Extremely large relative F RMSE

[ZhunyunTang]

Hi,
I encountered the following problem when fitting the mace potential.
UserWarning occurs before the fit is performed, as follows

The final result of the fit is also strange (although I used small max_num_epochs in this case). RMSE E and RMSE F are unusually small, while RELATIVE F RMSE is unusually large, as follows

my input parameter:
mace_run_train --name="MACE_model"
--train_file="train.xyz"
--config_type_weights='{"Default":1.0}'
--E0s='average'
--model="MACE"
--num_interactions=2
--num_channels=16
--max_L=2
--r_max=9.0
--batch_size=10
--max_num_epochs=100
--swa --start_swa=80
--ema
--ema_decay=0.99
--restart_latest
--default_dtype="float64"
--device=cuda
--save_cpu

Also, I've attached my log file and training structure file
MACE_model_run-123.log
train-xyz.zip

I hope you can provide me with some advice.
Best regards,
zhunyun

[gabor1]

Others will answer the user warnings, but your large fractional errors must be because your training forces are nearly zero. That's actually really suboptimal training data, you should use configurations which have nonzero forces.

[ZhunyunTang]

Thank you for your reply. It should be noted that the training set has some of the larger training forces.
The following is the result of fitting using the ace potential

Also, the same problem appeared in another of my cases (the dataset contains structures at temperatures of 100-2000K). Therefore, I suspect that it is not the near-zero forces that is causing the problem.

[gabor1]

Can you make a similar plot for MACE ? since your absolute force errors are very small, I expect it will look similar. Note that the mean relative error can be made arbitrarily large by having a single force component be arbitrarily small...

[ilyes319]

Looking at your log file, you seem to have no energies. Are you fitting just on forces? if this is the case you need to be careful and put zero weight on the energy loss. If it is not then there is probably a problem with your keys in your xyz file. Make sure to use "REF_energy" and "REF_forces" keys to name your properties in your xyz file.

[ilyes319]

I double checked and you are not using the correct keys. Make sure to rename your keys in the xyz file. The keys "forces" and "energy" are no longer safe in the newest version of ASE.

[ZhunyunTang]

Great, the fitting became normal after revising the keywords in the xyz file. as follows

By the way, can I use the MACE package to directly get comparative data for energy and force under DFT and MACE models. Because I want to get a visualization similar to the one shown below.

Also, is it possible to output the mace model for a specified step in the fitting process. For example, when my --max_num_epochs is set to 1000, I would like to get the model every 100 steps.