plugins: xtriggers

[oliver-sanders]

Note: The information below is now somewhat outdated, however, the desire to make xtriggers more "pluggable" remains.

Part of #2959
Blocked by: #3497, #4106

Implement our first extension point for Xtriggers.

Problem

At present we have Xtriggers provided by two sources:

Cylc Flow
cylc.flow.xtriggers

Cylc Xtriggers
cylc.flow.xtriggers

So we have the cylc.flow.xtriggers module defined in two places which I think won't work as Python is surely just going to return the first cylc.flow.xtriggers module it finds? cylc.flow.xtriggers.__file__ can only resolve to one location.

Proposal

Use package metadata to register Xtriggers in the manner outlined in #2995

This way we can locate them using pkg_resources rather than by namespace.

At the Cylc Flow end this means:

• Providing an "endpoint" that plugins can register using.
• Registering built-in Xtriggers using this mechanism.
• Upgrading the code to use pkg_resources to locate Xtriggers rather than attempting to import them from cylc.flow.xtriggers directly.

Pull requests welcome!

[oliver-sanders]

Additional information:

At present the Xtrigger location logic is here:

cylc-flow/cylc/flow/subprocpool.py

Lines 39 to 66 in c86e3c5

	def get_func(func_name, src_dir):
	"""Find and return an xtrigger function from a module of the same name.
	Can be in <src_dir>/lib/python, CYLC_MOD_LOC, or in Python path.
	Suite source directory passed in because this is executed in an independent
	process in the command pool - and therefore doesn't know about the suite.
	"""
	if func_name in _XTRIG_FUNCS:
	return _XTRIG_FUNCS[func_name]
	# First look in <src-dir>/lib/python.
	sys.path.insert(0, os.path.join(src_dir, 'lib', 'python'))
	mod_name = func_name
	try:
	mod_by_name = __import__(mod_name, fromlist=[mod_name])
	except ImportError:
	# Then look in built-in xtriggers.
	mod_name = "%s.%s" % ("cylc.flow.xtriggers", func_name)
	try:
	mod_by_name = __import__(mod_name, fromlist=[mod_name])
	except ImportError:
	raise
	try:
	_XTRIG_FUNCS[func_name] = getattr(mod_by_name, func_name)
	except AttributeError:
	# Module func_name has no function func_name.
	raise
	return _XTRIG_FUNCS[func_name]

This is the logic which needs adjusting to use package metadata instead of the cylc.flow.xtriggers namespace.

From Riot discussions:

• We are happy with the entry points approach.
• We should use cylc.plugins.xtriggers or similar for the entry point string.
• The Cylc Flow built-in Xtriggers should be:
  • echo
  • suite_state
  • wall_clock
  • xrandom

To close this issue (the Cylc Flow side of things)

• Implement the cylc.plugins.xtriggers entry point.
• Register the built-in Xtriggers via this mechanism.

Further work to be undertaken in the cylc/cylc-xtriggers repo.

[oliver-sanders]

Closed by #5831