[Dev] Multiple dependencies for CI missing

[DanielYang59]

Multiple external dependencies are missing from test setup, including:

• graphviz
• critic2: https://aoterodelaroza.github.io/critic2/installation/
• zeo

Added

• Open Babel: added in Add pymatgen.io.openff module #3729
• enum_lib: Added for Ubuntu only in Add pymatgen.io.openff module #3729
• boltztrap2: Fixed
• x_trans: Added in Install optional boltztrap, vampire and openbabel in CI #3985
• OBAlign: Added in Install optional boltztrap, vampire and openbabel in CI #3985
• vampire: Added in Install optional boltztrap, vampire and openbabel in CI #3985

Not planned

• mcsqs: not planned, tested in Fix uv pip install from binary distribution (wheels) usage #4210 but the installation is too slow (~90 s)
• icet: not planned ([Dev] Multiple dependencies for CI missing #3684 (comment))
• emmet: not planned ([Dev] Multiple dependencies for CI missing #3684 (comment))
• gulp: not planned (libgfortran3 not supported by Ubuntu20 afterwards)

[janosh]

i believe missing icet was deliberate as @esoteric-ephemera added it only to requirements-optional.txt which is not used in CI.

though i could see how that is confusing which is why i've advocated for removing the requirements files in the past. @shyuep prefers to keep. i fwiw, i think the atomate2 approach is better of having a strict optional deps section in pyproject.toml

[janosh]

emmet is purposefully not installed as it comes with a host of transitive deps.

as for the others, i think it would be good to install and test their respective pymatgen integrations in CI

[esoteric-ephemera]

i believe missing icet was deliberate as @esoteric-ephemera added it only to requirements-optional.txt which is not used in CI.

That was deliberate. The same logic applies to mcsqs and a few other packages here. Tho that tends to mean that tests for these aren't actively maintained since CI never runs them (could suddenly have broken tests / interfaces if breaking changes are made in a dependency)

[DanielYang59]

Tho that tends to mean that tests for these aren't actively maintained since CI never runs them (could suddenly have broken tests / interfaces if breaking changes are made in a dependency)

But I assume it's still beneficial to keep the tests alive for robustness? (Issues are more likely to slip in if tests are skipped)

Anyway, I would try to install some of these dependencies locally and see if any broken tests show up, and I would keep you updated.

[janosh]

@DanielYang59 feel free to install some of the missing packages in CI as well to activate those tests. ideally, we don't want to be blocked by upstream packages in our development (see e.g. tblite) but if a package is actively maintained and easy to install, it makes sense to add it in CI

[DanielYang59]

A very late update, I ran tests for mcsqs locally and looks like it's not aging too badly and only one timeout exception test is failing

I haven't investigated on the design of mcsqs_caller but it seems to raise three different timeout errors. I would just replace the error message in #4209 as both points to mcsqs timeout, let me know if I'm misunderstanding anything

pymatgen/src/pymatgen/command_line/mcsqs_caller.py

Lines 136 to 169 in 31f1e1f

	try:
	for process in mcsqs_find_sqs_processes:
	process.communicate(timeout=search_time * 60)
	if instances and instances > 1:
	process = Popen(["mcsqs", "-best"])
	process.communicate()
	if os.path.isfile("bestsqs.out") and os.path.isfile("bestcorr.out"):
	return _parse_sqs_path(".")
	raise RuntimeError("mcsqs exited before timeout reached")
	except TimeoutExpired:
	for process in mcsqs_find_sqs_processes:
	process.kill()
	process.communicate()
	# Find the best sqs structures
	if instances and instances > 1:
	if not os.path.isfile("bestcorr1.out"):
	raise RuntimeError(
	"mcsqs did not generate output files, "
	"is search_time sufficient or are number of instances too high?"
	)
	process = Popen(["mcsqs", "-best"])
	process.communicate()
	if os.path.isfile("bestsqs.out") and os.path.isfile("bestcorr.out"):
	return _parse_sqs_path(".")
	os.chdir(original_directory)
	raise TimeoutError("Cluster expansion took too long.")

pymatgen/src/pymatgen/command_line/mcsqs_caller.py

Line 147 in 31f1e1f

raise RuntimeError("mcsqs exited before timeout reached")

pymatgen/src/pymatgen/command_line/mcsqs_caller.py

Lines 157 to 159 in 31f1e1f

	raise RuntimeError(
	"mcsqs did not generate output files, "
	"is search_time sufficient or are number of instances too high?"

pymatgen/src/pymatgen/command_line/mcsqs_caller.py

Line 169 in 31f1e1f

raise TimeoutError("Cluster expansion took too long.")

Update: also tested installing it in CI 3f63f4b but it's not worth it (way too slow at ~90 s)