Lifecycle hooks for generators?

[sneakers-the-rat]

Hello everyone!

I'm doing a translation from a domain-specific schema to LinkML right now and working on some utility functions that I keep thinking should probably just be a part of the general Generator implementation.

It seems like it would be a common enough problem that people would want to customize the code generation process in idiosyncratic and hard to predict ways (and looking around the issues it seems I'm right), and so I'm wondering if we can do something similar to Sphinx's build phase hooks - eg. the user can provide a function with a specific signature and return type that is called at the different phases of generation.

So, for example, I have to modify the generated pydantic code to avoid shadowing a builtin, but that doesn't belong anywhere in the schema. To do that right now I have just made some skeletal abstraction around the most naive kind of patch that just treats the generated code as a big long string:

class Phases(StrEnum):
    post_generation = "post_generation"

@dataclass
class Patch:
    phase: Phases
    path: Path
    """Path relative to repository root"""
    match: str
    """Regex to match the thing to be replaced"""
    replacement: str
    """Regex to replace with (eg. can use capturing groups in match)"""

    instances: ClassVar[List['Patch']]
    """
    List of patches!
    """

    def __post_init__(self):
        self.instances.append(self)

and then defined some patch in particular

patch_schema_slot = Patch(
    phase=Phases.post_generation,
    path=Path('my_schema_pydantic.py'),
    match=r"^\s*(schema)(.*Field\()(.*)",
    replacement=r'    schema_\2alias="schema", \3',
)

which is then run with the simplest possible runner:

def run_patches(phase:Phases):
    patches = [p for p in Patch.instances if p.phase == phase]
    for patch in patches:
        with open(patch.path, 'r') as pfile:
            string = pfile.read()
        string = re.sub(patch.match, patch.replacement, string)
        with open(patch.path, 'w') as pfile:
            pfile.write(string)


def main():
    parser = argparse.ArgumentParser(
        description="Run patches for a given phase of code generation"
    )
    parser.add_argument("--phase",
                        choices=list(Phases.__members__.keys()),
                        type=Phases)
    args = parser.parse_args()
    run_patches(args.phase)

It seems like this could be generalized with a bit of work, the Generator class already looks like it has a decent build phase system, so it would mostly be a matter of providing some decorators/hooks/etc. to say "call this hook before/after this phase with xyz arguments and returning xyz" to be able to make this more formalized.

An incomplete list

• Post-loading yaml
• Pre-generate - before any code is generated
• Pre/post-generate-frontmatter - before/after any frontmatter/templates are processed
• Pre/post-generate {:element_type} - before generating a particular type of LinkML element
• Post-generate - After all generation is done

Wonder what y'all think? I don't have a very good picture of the overall structure of the code yet, so IDK what's possible/desirable, but figured i'd raise for discussion