on_page_markdown does not include snippet content

[vermiculus]

Description

MkDocs' hook for on_page_markdown does not include snippet content (reference docs). The markdown value provided to the hook contains the snippet syntax (--8<--) and not the included content.

I'm not entirely sure whether this is an issue downstream (here) or upstream (mkdocs), but I'm starting here since you're going to be more familiar with where pymdown-extensions fits into MkDocs and whether you'd expect the current implementation to be subject to the hooks defined by MkDocs.

Minimal Reproduction

1. Have a README that includes another file
2. Include an on_page_markdown hook that would alter the markdown content of the included file
3. Run mkdocs build

# docs/README.md
the following content is included

--8<-- "docs/included.md"

# docs/included.md
hello, world

# mkdocs.yml
...
hooks:
  - my_hooks.py
...

# my_hooks.py
def on_page_markdown(markdown, page, config, files):
    return markdown.replace("hello", "goodbye")

Version(s) & System Info

• Operating System: macOS 13.2.1
• Python Version: 3.11.3
• Package Version: MkDocs 1.4.2, pymdown-extensions 9.9.2

[facelessuser]

I'm not exactly sure what the complaint is here. Per the MkDocs documentation it states:

The page_markdown event is called after the page's markdown is loaded from file and can be used to alter the Markdown source text. The meta-data has been stripped off and is available as page.meta at this point.

So, it loads the Markdown source, which contains --8<-- and allows you to alter it. What is happening is that the the on_page_markdown event in MkDocs let's you alter the Markdown source before the Python Markdown parser processes the content, and the Snippets extension is a Python Markdown extension, not a MkDocs plugin. Hopefully that makes sense.

[facelessuser]

Keep in mind that Snippets is a preprocessor, yes, but a preprocessor extension in Python Markdown. Python Markdown has no knowledge of what MkDocs is doing. I have no idea what you are trying to do either in regards to MkDocs and Snippets as that context has not been provided here.

But yes, if you absolutely needed the Snippets content in on_page_markdown, you'd almost have to implement your own approach, or process the content in post, but depending on what you are doing, post may be too late.

[facelessuser]

To answer your bullet points more directly:

1. The hook runs before Python Markdown does its work. Save MkDocs injecting its own extension in the Python Markdown pipeline ensuring it is the last preprocessor extension to be run, it will have no way of doing what you want. I don't know if this is anything they'd want to do or not.
2. Incorrect, it is an event run directly before Markdown is processed and gives you the content that will be passed to Python Markdown. That is its intention and exactly what it is doing, it just may not be what you were specifically hoping for.

[vermiculus]

Yes, I'm aware of the relationship between python-markdown and mkdocs (in that mkdocs is a consumer of python-markdown and python-markdown (and its extensions) have no knowledge of mkdocs).

(1) and (2) were either/or options – it was assumed that only one could be true :-)

I'll raise your potential approach for (1) to mkdocs. It seems this would be the only realistic way to get at the markdown source after pre-processing – since I assume python-markdown doesn't have a 'run pre-processors only' runmode. But I'm hardly the expert here :-)

Thanks for all your help! I'm not sure how to close discussions (they've been added to github since I was active here), but consider this resolved as far as pymdown-extensions is concerned.

[facelessuser]

It's probably still not going to work as you hope. One thing that should be noted is that fenced code blocks are run as preprocessor. This is due to some awkward limitations of how the Python Markdown parser works. That is an example of something that is a preprocessor that may not expect to be a preprocessor, and may be surprising in on_page_markdown.

[vermiculus]

Likely not, but it will make my actual use case (implementing basic shortlinks – a la this example from git-commit(1)) possible for included files.

[facelessuser]

Moving to discussions