What's the correct way to use the MAGMOM's value when using MP API data?

[hongyi-zhao]

The testing code below is adapted from my example here:

from mp_api.client import MPRester
from pymatgen.symmetry.analyzer import SpacegroupAnalyzer
from atomate2.vasp.flows.mp import MPGGADoubleRelaxMaker
from atomate2.vasp.powerups import (
    update_user_incar_settings,
    update_user_potcar_functional,
)
from jobflow import run_locally

# Set POTCAR functional
user_potcar_functional = "PBE_64"

# Get structure from MP
material_id = "mp-126"
with MPRester() as mpr:
    # First get the task id from the materials doc
    materials = mpr.materials.search(material_ids=[material_id], fields=["task_ids"])
    task_id = materials[0].task_ids[0]
    
    # Now get the calculation that includes MAGMOM settings
    tasks = mpr.materials.tasks.search(task_ids=[task_id], 
        fields=["input.incar", "calcs_reversed"])
    
    # Get the INCAR settings from the calculation
    incar_settings = tasks[0].calcs_reversed[0].input.incar
    
    # Get the structure and convert to conventional cell
    structure = mpr.materials.get_structure_by_material_id(material_id)
    structure = SpacegroupAnalyzer(structure).get_conventional_standard_structure()

# Convert MAGMOM list to dictionary format
#if "MAGMOM" in incar_settings:
#    print("Original MAGMOM settings:", incar_settings["MAGMOM"])
#    print("Number of sites in structure:", len(structure))
#    
#    # Get unique species list
#    species_list = [site.species_string for site in structure]
#    unique_species = sorted(set(species_list))
#    print("Unique species in structure:", unique_species)
#    
#    # Set all species to the first MAGMOM value from the original calculation
#    magmom_dict = {specie: incar_settings["MAGMOM"][0] for specie in unique_species}
#    incar_settings["MAGMOM"] = magmom_dict
#    print("MAGMOM dictionary:", magmom_dict)

# Create a simple flow with just the relax
flow = MPGGADoubleRelaxMaker().make(structure)

# Update POTCAR settings
flow = update_user_potcar_functional(flow, user_potcar_functional)

# Update INCAR settings
flow = update_user_incar_settings(flow, incar_settings)

# Run the flow
responses = run_locally(flow, create_folders=True, ensure_success=True)

But the above code will trigger the following error:

Retrieving MaterialsDoc documents: 100%|████████████████████████████████| 1/1 [00:00<00:00, 12865.96it/s]
Retrieving TaskDoc documents: 100%|█████████████████████████████████████| 1/1 [00:00<00:00, 11715.93it/s]
Retrieving MaterialsDoc documents: 100%|████████████████████████████████| 1/1 [00:00<00:00, 11848.32it/s]
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/sets.py:288: BadInputSetWarning: Overriding the POTCAR functional is generally not recommended  as it significantly affects the results of calculations and compatibility with other calculations done with the same input set. Note that some POTCAR symbols specified in the configuration file may not be available in the selected functional.
  warnings.warn(
2024-11-10 08:41:06,924 INFO Started executing jobs locally
2024-11-10 08:41:06,926 INFO Starting job - MP GGA relax 1 (ecdb12aa-5e73-4d2b-97b3-d6deb29ce066)
2024-11-10 08:41:06,928 INFO MP GGA relax 1 failed with exception:
Traceback (most recent call last):
  File "/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/jobflow/managers/local.py", line 114, in _run_job
    response = job.run(store=store)
               ^^^^^^^^^^^^^^^^^^^^
  File "/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/jobflow/core/job.py", line 600, in run
    response = function(*self.function_args, **self.function_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/werner/Public/repo/github.com/materialsproject/atomate2.git/src/atomate2/vasp/jobs/base.py", line 219, in make
    write_vasp_input_set(
  File "/home/werner/Public/repo/github.com/materialsproject/atomate2.git/src/atomate2/vasp/files.py", line 190, in write_vasp_input_set
    vis = input_set_generator.get_input_set(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/sets.py", line 476, in get_input_set
    incar=self.incar,
          ^^^^^^^^^^
  File "/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/sets.py", line 572, in incar
    if uic_magmom := self.user_incar_settings.get("MAGMOM", {}).get(site.species_string):
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'list' object has no attribute 'get'

2024-11-10 08:41:06,928 INFO Finished executing jobs locally
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[6], line 56
     53 flow = update_user_incar_settings(flow, incar_settings)
     55 # Run the flow
---> 56 responses = run_locally(flow, create_folders=True, ensure_success=True)

File ~/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/jobflow/managers/local.py:181, in run_locally(flow, log, store, create_folders, root_dir, ensure_success, allow_external_references, raise_immediately)
    178 logger.info("Finished executing jobs locally")
    180 if ensure_success and not finished_successfully:
--> 181     raise RuntimeError("Flow did not finish running successfully")
    183 return dict(responses)

RuntimeError: Flow did not finish running successfully

Is there a proper way to handle MAGMOM settings when copying from MP calculations without requiring further user adjustment, say, the one done by the code snippet Convert MAGMOM list to dictionary format shown above?

See materialsproject/atomate2#1049 for the related discussion.

Regards,
Zhao

[esoteric-ephemera]

@hongyi-zhao , this is more of an atomate2 question, but the critical thing here is that the line flow = update_user_incar_settings(flow, incar_settings) is not needed. The MPGGADoubleRelaxMaker is constructed to use MP's settings by default

If you want to use the magmoms from the MP structure as the starting values of MAGMOM, then having "magmom" in the structure's site_properties is sufficient, atomate2 will use those

To be explicit, I recommend just the following:

from mp_api.client import MPRester
from atomate2.vasp.flows.mp import MPGGADoubleRelaxMaker
from atomate2.vasp.powerups import update_user_potcar_functional
from jobflow import run_locally

# Set POTCAR functional
user_potcar_functional = "PBE_64"

# Get structure from MP
material_id = "mp-126"
with MPRester() as mpr:
    structure = mpr.get_structure_by_material_id(material_id)

# If you really need the conventional structure, then uncomment this line. Otherwise, this adds unnecessary compute cost
#structure = structure.to_conventional()

flow = MPGGADoubleRelaxMaker().make(structure)

# Update POTCAR settings
flow = update_user_potcar_functional(flow, user_potcar_functional)

# Run the flow
responses = run_locally(flow, create_folders=True, ensure_success=True)

[hongyi-zhao]

But, the above code will trigger several custodian errors:

#$ pyenv shell datasci
#$ module load vasp
#$ ipython
In [1]: from mp_api.client import MPRester
   ...: from atomate2.vasp.flows.mp import MPGGADoubleRelaxMaker
   ...: from atomate2.vasp.powerups import update_user_potcar_functional
   ...: from jobflow import run_locally
   ...: 
   ...: # Set POTCAR functional
   ...: user_potcar_functional = "PBE_64"
   ...: 
   ...: # Get structure from MP
   ...: material_id = "mp-126"
   ...: with MPRester() as mpr:
   ...:     structure = mpr.get_structure_by_material_id(material_id)
   ...: 
   ...: # If you really need the conventional structure, then uncomment this line. Otherwise, this adds unnecessary compute cost
   ...: #structure = structure.to_conventional()
   ...: 
   ...: flow = MPGGADoubleRelaxMaker().make(structure)
   ...: 
   ...: # Update POTCAR settings
   ...: flow = update_user_potcar_functional(flow, user_potcar_functional)
   ...: 
   ...: # Run the flow
   ...: responses = run_locally(flow, create_folders=True, ensure_success=True)
Retrieving MaterialsDoc documents: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 13357.66it/s]
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/sets.py:288: BadInputSetWarning: Overriding the POTCAR functional is generally not recommended  as it significantly affects the results of calculations and compatibility with other calculations done with the same input set. Note that some POTCAR symbols specified in the configuration file may not be available in the selected functional.
  warnings.warn(
2024-12-20 09:20:02,603 INFO Started executing jobs locally
2024-12-20 09:20:02,607 INFO Starting job - MP GGA relax 1 (1a81cdd6-3ab5-4165-aaf5-761fc4d7e45e)
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/sets.py:757: BadInputSetWarning: Relaxation of likely metal with ISMEAR < 0 (-5). See VASP recommendations on ISMEAR for metals (https://www.vasp.at/wiki/index.php/ISMEAR).
  warnings.warn(
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:2420: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:2985: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(os.path.join(output_dir, key), mode="wt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:908: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:1648: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:2826: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:287: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:908: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/custodian/vasp/handlers.py:185: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(os.path.join(directory, self.output_filename), mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:1648: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:2826: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:287: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:312: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:4356: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as fid:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:2114: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:2365: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(self.filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:895: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="wt") as file:
ERROR:custodian.custodian:IncorrectSmearingHandler
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:908: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/custodian/vasp/handlers.py:185: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(os.path.join(directory, self.output_filename), mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:1648: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:2826: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:287: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:312: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:4356: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as fid:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:2114: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:2365: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(self.filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/core/structure.py:3175: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt", errors="replace") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:895: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="wt") as file:
ERROR:custodian.custodian:LargeSigmaHandler
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:908: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/custodian/vasp/handlers.py:185: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(os.path.join(directory, self.output_filename), mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:1648: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:2826: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:287: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:312: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:4356: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as fid:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:2114: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:2365: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(self.filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/core/structure.py:3175: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt", errors="replace") as file:
2024-12-20 09:21:04,335 INFO Finished job - MP GGA relax 1 (1a81cdd6-3ab5-4165-aaf5-761fc4d7e45e)
INFO:jobflow.core.job:Finished job - MP GGA relax 1 (1a81cdd6-3ab5-4165-aaf5-761fc4d7e45e)
2024-12-20 09:21:04,335 WARNING Response.stored_data is not supported with local manager.
WARNING:jobflow.managers.local:Response.stored_data is not supported with local manager.
2024-12-20 09:21:04,335 INFO Starting job - MP GGA relax 2 (d7b7f3fd-53a5-4dc3-9188-c2bdd1a6bfc5)
INFO:jobflow.core.job:Starting job - MP GGA relax 2 (d7b7f3fd-53a5-4dc3-9188-c2bdd1a6bfc5)
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/atomate2/common/files.py:268: UserWarning: /home/werner/test/job_2024-12-20-01-21-04-335517-47956/INCAR is not gzipped, skipping...
  file_client.gunzip(directory / file, host=host, force=force)
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/atomate2/common/files.py:268: UserWarning: /home/werner/test/job_2024-12-20-01-21-04-335517-47956/OUTCAR is not gzipped, skipping...
  file_client.gunzip(directory / file, host=host, force=force)
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/atomate2/common/files.py:268: UserWarning: /home/werner/test/job_2024-12-20-01-21-04-335517-47956/CONTCAR is not gzipped, skipping...
  file_client.gunzip(directory / file, host=host, force=force)
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/atomate2/common/files.py:268: UserWarning: /home/werner/test/job_2024-12-20-01-21-04-335517-47956/vasprun.xml is not gzipped, skipping...
  file_client.gunzip(directory / file, host=host, force=force)
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/atomate2/common/files.py:268: UserWarning: /home/werner/test/job_2024-12-20-01-21-04-335517-47956/WAVECAR is not gzipped, skipping...
  file_client.gunzip(directory / file, host=host, force=force)
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/atomate2/common/files.py:268: UserWarning: /home/werner/test/job_2024-12-20-01-21-04-335517-47956/CHGCAR is not gzipped, skipping...
  file_client.gunzip(directory / file, host=host, force=force)
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/atomate2/common/files.py:268: UserWarning: /home/werner/test/job_2024-12-20-01-21-04-335517-47956/POTCAR is not gzipped, skipping...
  file_client.gunzip(directory / file, host=host, force=force)
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/atomate2/common/files.py:268: UserWarning: /home/werner/test/job_2024-12-20-01-21-04-335517-47956/KPOINTS is not gzipped, skipping...
  file_client.gunzip(directory / file, host=host, force=force)
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/sets.py:757: BadInputSetWarning: Relaxation of likely metal with ISMEAR < 0 (-5). See VASP recommendations on ISMEAR for metals (https://www.vasp.at/wiki/index.php/ISMEAR).
  warnings.warn(
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:2420: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:2985: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(os.path.join(output_dir, key), mode="wt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:908: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/custodian/vasp/handlers.py:185: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(os.path.join(directory, self.output_filename), mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:1648: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:2826: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:287: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:312: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:4356: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as fid:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:2114: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:2365: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(self.filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:895: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="wt") as file:
ERROR:custodian.custodian:IncorrectSmearingHandler
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:908: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/custodian/vasp/handlers.py:185: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(os.path.join(directory, self.output_filename), mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:1648: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:2826: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:287: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:312: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:4356: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as fid:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:2114: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:2365: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(self.filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/core/structure.py:3175: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt", errors="replace") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:895: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="wt") as file:
ERROR:custodian.custodian:LargeSigmaHandler
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:908: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/custodian/vasp/handlers.py:185: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(os.path.join(directory, self.output_filename), mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:1648: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:2826: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/inputs.py:287: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:312: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:4356: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as fid:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:2114: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/io/vasp/outputs.py:2365: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(self.filename, mode="rt") as file:
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/pymatgen/core/structure.py:3175: EncodingWarning: We strongly encourage explicit `encoding`, and we would use UTF-8 by default as per PEP 686
  with zopen(filename, mode="rt", errors="replace") as file:
2024-12-20 09:21:56,255 INFO Finished job - MP GGA relax 2 (d7b7f3fd-53a5-4dc3-9188-c2bdd1a6bfc5)
INFO:jobflow.core.job:Finished job - MP GGA relax 2 (d7b7f3fd-53a5-4dc3-9188-c2bdd1a6bfc5)
2024-12-20 09:21:56,255 WARNING Response.stored_data is not supported with local manager.
WARNING:jobflow.managers.local:Response.stored_data is not supported with local manager.
2024-12-20 09:21:56,255 INFO Finished executing jobs locally
INFO:jobflow.managers.local:Finished executing jobs locally

While custodian provides automated error handling and parameter adjustments, initial reasonable parameter settings are still important for obtaining meaningful calculation results.

See Neraaz/HTESP#2 (comment) for more tests and comments by me.

[esoteric-ephemera]

@hongyi-zhao, your job still runs successfully because of the changes made by custodian. The final results will still be meaningful because of those changes.

You can safely set ISMEAR = 0 and SIGMA = 0.05 as general defaults instead of the MPRelaxSet. This isn't a pymatgen bug

@DanielYang59, these warnings about encoding from monty #730 are a little disruptive. Can the default be to disable this, or at least explicitly set utf-8 encoding in loadfn? We have so many places in the codebase where these warnings are getting thrown

[DanielYang59]

these warnings about encoding from materialsvirtuallab/monty#730 are a little disruptive. Can the default be to disable this,

You're absolutely correct, I should have used the PYTHONWARNDEFAULTENCODING from PEP 597 for consistency (don't display encoding warning until PYTHONWARNDEFAULTENCODING is set), sorry for the trouble.

at least explicitly set utf-8 encoding in loadfn

I just had a quick looks and it looks like loadfn already use UTF-8 in text mode and zopen would not issue an encoding warning for binary mode, can you show me what triggered this warning?

[hongyi-zhao]

@hongyi-zhao, your job still runs successfully because of the changes made by custodian. The final results will still be meaningful because of those changes.

You can safely set ISMEAR = 0 and SIGMA = 0.05 as general defaults instead of the MPRelaxSet. This isn't a pymatgen bug

For certain types of computational tasks, such as elastic tensors, the small differences in the above parameters can make a big difference in the final result, as shown here.

[esoteric-ephemera]

@DanielYang59: most instances of zopen in the codebase don't use binary mode. You can see in the warnings that it's coming from pymatgen.io - every VASP input and output object, like POSCAR's from_file use zopen(<path>,"rt"). Probably goes for every pymatgen.io module too

[esoteric-ephemera]

@hongyi-zhao that's completely fair, but that's more of a scientific methodology question and not a pymatgen bug. FWIW, the experimental uncertainty in bulk moduli, especially after applying zero point corrections for the nuclear motion to better compare with zero-temperature DFT, can be on the order of 10 GPa if not more. The <10 GPa differences in your results is still less than the experimental uncertainty

[hongyi-zhao]

@esoteric-ephemera But I have another question here: for the example of "mp-126", the following values are used in the input filed of MP's database: {'ISMEAR': 2, 'SIGMA': 0.14}, which is different from the final values automatically adjusted by custodian in my previous example, as shown below:

werner@x13dai-t:~/test$ ug -ri 'ismear|sigma' -g INCAR
job_2024-12-20-01-21-04-335517-47956/INCAR:ISMEAR = 2
job_2024-12-20-01-21-04-335517-47956/INCAR:SIGMA = 0.07321952854558407
job_2024-12-20-01-20-02-605601-93060/INCAR:ISMEAR = 2
job_2024-12-20-01-20-02-605601-93060/INCAR:SIGMA = 0.07102306201943624

However, the MP database involves a huge amount of computations, so how are detailed parameter adjustments like this achieved if they are not based on custodian's automatic adjustments?

[DanielYang59]

most instances of zopen in the codebase don't use binary mode.

@esoteric-ephemera Ah I got your point now so it's zopen instead of loadfn right? Don't worry about this then, it has been fixed by #4219, but was mistakenly reverted in #4221, would re-apply in #4222

Sorry for going astray in your thread @hongyi-zhao

[hongyi-zhao]

@DanielYang59

Sorry for going astray in your thread @hongyi-zhao

No worries, I understand. We can continue discussing the topic, or feel free to bring up any other questions you might have. 😊

[esoteric-ephemera]

@hongyi-zhao, since we add handlers to custodian over time, you might see different behavior in calculations run with a newer version of custodian. My guess is you're running a version of custodian that includes the LargeSigmaHandler, which checks to see if the electronic smearing pseudo-entropy is larger than 1 meV/atom, and reduces SIGMA if it is. That handler likely didn't exist when the MP calculation was performed