🚧 Custom sphinx directive to document step

[stevenbal]

• specify examples for django_model_refs
• add comments above each line

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.00%. Comparing base (0c55577) to head (bf9cd75).
Report is 46 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #35      +/-   ##
==========================================
+ Coverage   96.89%   97.00%   +0.11%     
==========================================
  Files           8       23      +15     
  Lines         193      934     +741     
==========================================
+ Hits          187      906     +719     
- Misses          6       28      +22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[swrichards]

@stevenbal although it's not exactly what we discussed in terms of example YAML and such, I think we might want to look at https://autodoc-pydantic.readthedocs.io/en/stable/users/examples.html for a good 80/20 solution that's easily implementable. Let's discuss.

[stevenbal]

this PR here was mostly just to see if chatGPT could come up with a good start for displaying a YAML example :p

My thought was to use sphinx.ext.autodoc to display the docstring of the step class (for a general explanation of what the step does), this way steps in libraries can also easily be documented without repeating. And then below it we would have another directive to show a YAML example (with help texts for each line)

With the current state of this PR it looks like this (not sure if the classname can be omitted with autodoc, tried a couple of things but couldn't get it to work):

Since I have a bit of time this afternoon, I'll experiment a bit and check if autodoc-pydantic makes things slightly easier

[stevenbal]

@swrichards I played around a little bit with autodoc-pydantic, I couldn't get it to work for nested models (though I think it should be possible). We should probably discuss with Joeri what the best way to proceed is, I do think that he really wants an example .yaml, because that would make life a lot easier for devops. The help_text/examples per field might be something we could do with autodoc-pydantic then

[swrichards]

@stevenbal thanks for checking it out! Pity, I was hoping the nested models would work easily out of the box. And even more than that: I was hoping it would also deal with arbitrarily complex types (nested lists and unions). I am all for YAML examples, but I wonder based on your experiment (I tried a similar thing, also backed by an AI helper ;)) how feasible it is to generate the YAML from the spec, rather than manually providing an example in a structured way, and but having the declared examples automatically validated against the model so they don't get out of sync.

Generating the YAML for simple fields is relatively easy, but once you get unions (and nested unions, see the admin OIDC model which already has an object union in it. For one union, there's already to valid examples, and if there's more objects unions the permutations of valid example YAMLs quickly grows. I am therefore leaning towards some combination of hand-curated, but automatically validated, example YAMLs provided as part of the step, combined with more general auto-doc like for a deeper dive into specific fields.