Implement codemod registry

[drdavella]

Overview

Move codemods to core_codemods package; implement codemod registry

Description

• This PR moves the codemods to the core_codemods package and implements a codemod registry
• There are a few major goals with this change:
  • Separate individual codemod implementation from the codemodder package itself
  • Move in the direction of "external" validation for codemods rather than using __init_subclass__ hooks
  • Enable future third-party codemod implementation/registration
• The codemod registry is responsible for detecting and collecting all registered codemods
  • It also performs codemod validation and filtering based on ID
• Codemod docs and semgrep configurations are now more tightly coupled with the codemods themselves
• Codemods are registered with a namespace prefix that no longer needs to be hardcoded in codemodder
• I was able to remove the __init_subclass__ validation for the "base" API but some additional work will be required before we can get rid of this in the "sugar" API
  • I still think that maybe these shouldn't be class-level attributes; hopefully this moves us a bit closer to that goal
• This is a relatively big change but I also view it as just one step of incremental refactoring
  • I don't think the result is perfect yet but hopefully it moves in a good direction
  • I hope that eventually this will better enable us to clean up the codemod class hierarchy a bit

[codecov-commenter]

Codecov Report

Merging #47 (5afd594) into main (03a1d7e) will increase coverage by 0.04%.
The diff coverage is 97.51%.

@@            Coverage Diff             @@
##             main      #47      +/-   ##
==========================================
+ Coverage   95.82%   95.86%   +0.04%     
==========================================
  Files          38       39       +1     
  Lines        1413     1476      +63     
==========================================
+ Hits         1354     1415      +61     
- Misses         59       61       +2     

Files	Coverage Δ
src/codemodder/__main__.py	96.93% <100.00%> (-0.24%)	⬇️
src/codemodder/cli.py	100.00% <100.00%> (ø)
src/codemodder/codemods/api/__init__.py	95.38% <ø> (-0.14%)	⬇️
src/codemodder/codemods/base_codemod.py	98.03% <100.00%> (+2.32%)	⬆️
src/codemodder/context.py	100.00% <100.00%> (ø)
src/codemodder/semgrep.py	96.29% <100.00%> (-0.93%)	⬇️
src/core_codemods/__init__.py	100.00% <100.00%> (ø)
src/core_codemods/django_debug_flag_on.py	100.00% <ø> (ø)
.../core_codemods/django_session_cookie_secure_off.py	98.30% <ø> (ø)
src/core_codemods/harden_pyyaml.py	100.00% <ø> (ø)
... and 15 more

[drdavella]

This is a slightly awkward construct but I might suggest that in the longer time this actually outlines the API we want for the BaseCodemod class. Eventually maybe this becomes an ABC that all codemods inherit.

[clavedeluna]

Both CodemodCollection and CodemodContainer sound to me like containers of some sort, such as a list. What about making one of these inherit from list? Maybe that's a later thing, just a thought.

Also, how about renaming _CodemodContainer to CodemodMetadata or CodemodWrapper? Also I think this class in particular could be a dataclass. What clues me in to that is all the property methods, but some will have to be manual, of course.

[clavedeluna]

More thoughts, since you have the idea of this class _CodemodContainer being the "real" BaseCodemod, what do you think is the distinction between the data a BaseCodemod class should store versus this new _CodemodContainer? I still feel they should be one class but I'm listening!

[drdavella]

• I did think about making CodemodCollection a list but I'm not sure whether we really need that API. It might be worth a quick experiment though
• I do think maybe _CodemodWrapper is a better name and it would be a good dataclass
• The main difference between _CodemodContainer/Wrapper and BaseCodemod is that the container knows about the registry and associated metadata and the base codemod does not. But maybe there's a good way to consolidate those two things somehow. I think that's a future effort though.

[clavedeluna]

I'm liking this a ton., Just left some thoughts around naming to clarify concepts.

[clavedeluna]

Both CodemodCollection and CodemodContainer sound to me like containers of some sort, such as a list. What about making one of these inherit from list? Maybe that's a later thing, just a thought.

Also, how about renaming _CodemodContainer to CodemodMetadata or CodemodWrapper? Also I think this class in particular could be a dataclass. What clues me in to that is all the property methods, but some will have to be manual, of course.

[clavedeluna]

More thoughts, since you have the idea of this class _CodemodContainer being the "real" BaseCodemod, what do you think is the distinction between the data a BaseCodemod class should store versus this new _CodemodContainer? I still feel they should be one class but I'm listening!