9.10a2 General Purpose Blocks Feedback

[facelessuser]

Alpha 2

Documentation: https://facelessuser.github.io/pymdown-extensions/extensions/blocks/

The purpose of this discussion is to allow feedback for the latest iteration of General Purpose Blocks 9.10.a2. It should be noted that any and all feedback is welcome, but there is a focus on feedback specifically related to syntax. In particular, we have two main points we are hoping to get feedback on, but again, any feedback on syntax is welcome.

Hopefully, Alpha 2 will be the last related alpha to finish nailing down syntax. It is hoped that we can begin the API discussion in the next alpha.

Changes in Alpha 2:

• NEW: General blocks no longer use YAML fences for per-block options, but instead use a special token to denote the
  line is part of the config.
• NEW: Add temporary alpha/beta option yaml_indent to control whether per-block YAML configs use indentation or a
  leading special character: / for /// syntax and : for ::: syntax (colon_syntax must be true to use :::
  syntax).
• NEW: Ensure that / character can be escaped when registering the blocks extension.
• FIX: Fix some block nesting issues.

Feedback Needed

Currently, we have two general options that are only available during the alpha (and maybe beta) stage. By the final release, we'd like to remove these options with a permanent solution.

1. Which is preferred, syntax with ::: or ///? Currently, the option colon_syntax allows selecting between the current default syntax of /// and the colon syntax (:::). Which is generally preferred by the community?

2. The new general purpose Blocks extension allows YAML configs in block headers to set per block options. A change in Alpha 2 now requires YAML configs to be either prefixed with / or : (if colon_syntax is enabled) or 4 spaces if yaml_indent option is enabled. Which is generally preferred by the community, using indentation or a special character like /?

   Using a special character:

   /// name
   / option: value
   
   Content
   ///
   

   Using indented syntax:

   /// name
       option: value
   
   Content
   ///
   

[Andre601]

Why was the yaml block replaced with that / (or :) and indent solution.
Feels a bit having such a syntax.

And at least for docs that use / (i.e. as part of an rest api doc) is escaping the / annoying, so the colon syntax is at least less breaking here (I assume).

So, I for my part would prefer the colon syntax, especially since it feels more natural for me after working with Docosaurus a bit and using the CommonMark solution they have (I know python markdown doesn't follow CommonMark, but who says you couldn't encourage such a format?)

[facelessuser]

Why was the yaml block replaced with that / (or :) and indent solution.

Because in editors and things like Github, when viewing, turns the header into a documentation header:

/// name
---

Becomes:

/// name

Trying to find a syntax that will generally be less problematic. This also isn't too different from what RST does or the Markdown implementation that inspired this. With that said, indentation is an experiment as editors or other parsers will either treat it as a code block or ignore the content. I think it might be a fairly decent approach, especially if prefixes of : and / are not desired.

So, I for my part would prefer the colon syntax, especially since it feels more natural for me after working with Docosaurus a bit and using the CommonMark solution they have (I know python markdown doesn't follow CommonMark, but who says you couldn't encourage such a format?)

I'm less concerned about what other parsers are doing, but I do realize that does play into some people's preferences. There is no standard per se, even in CommonMark for :. Only a few parsers adopt : for Admonitions and/or more general purpose blocks. With that said, how they implement their blocks with : can be different.

With that said, I will note that : is preferred by you.

[waylan]

I am strongly opposed to any prefix other than whitespace for the YAML block as I detailed previously. In short, editors have tools/features for editing indented blocks, but not blocks prefixed by random characters. I do not want to be fighting my editor on this. For this reason alone, I would much prefer delimitators over prefixes despite the objections raised to them.

I also strongly prefer / over : because the colon just feels too busy to me. Subjective, I know. But I'm just sharing my opinion as you asked.

[facelessuser]

I am strongly opposed to any prefix other than whitespace for the YAML block as I detailed previously

Yep, noted. I can definitely see its appeal. In my editor, it shows up as code, which seems fine as the content only contains YAML. The important part is that the content of the block is not indented, and an indented config doesn't force deeply nested indentation as the content is not indented.

I also strongly prefer / over : because the colon just feels too busy to me. Subjective, I know. But I'm just sharing my opinion as you asked.

My personal leaning is probably / specifically to keep people from confusing it with other implementations, but if we have an overwhelming majority feeling the opposite, I'm probably okay with : as I really only have a weak preference.

[Andre601]

This would probs be a bit weird, but what about a JSON-like aproach for the meta stuff?

Like for example:

::: info | Some title
{
"key": "value",
"another_key": "Another value"
}

Text goes here
:::

...And people could minify/condense it if needed:

::: info | Some title
{"key":"value","another_key":"Another value"}

Text goes here
:::

Tho, this could (and probably will) conflict with the meta plugin most likely, so another character set would be needed then ([]?)

Just a random idea.

[facelessuser]

If you want to break the config up with comments? You can do that though:

/// block
    # comment
    key: value
    # More comments
    key2: value

Content
///

[facelessuser]

I guess you could just use a single # 🤷🏻

[2bndy5]

No worries, I just found this snippet confusing:

/// block

    key: value

The indented block is code
///

Incidentally we posted at the exact same time 😄 .

[facelessuser]

If I could look ahead and see all the blocks and consume them, it would be easier, but I can't.

[facelessuser]

Yep, I think it is interesting to give insight into why some decisions are made. I'd probably be more flexible with certain parts of the syntax if Python Markdown didn't make it hard on me 🙃 .

[facelessuser]

I've been a bit busy, but I'm hoping to start getting an alpha 3 ready soon. The focus will be on the API.

I'm thinking, that I'm probably going to end up using the /// syntax. I'm not seeing an overwhelming preference either way. I'm also not getting an overwhelming about of feedback either 🙃, but I am grateful for the feedback I have received. I do understand the arguments for :::, but there is no standardization across parsers, not even CommonMark, and the few who have adopted such syntax either limit them to Admonitions or implement more generic blocks in a way that we do not fully emulate. By avoiding the use :::, if there ever is a future where something is standardized, we still leave that open, assuming that is even something we wish to target.

As for per-block configs in the header, I'll probably go with an indentation approach for now.

My general worry is that I'm not going to get enough feedback before I release this into the wild, so I'm starting to think that even after we have an official release with this out, I'll probably keep it marked as experimental for a while. I feel like I'm not really going to get the true feedback I need related to the API until it is formally shipped in an official release. But hopefully, I'll be able to get some during the alpha/beta period.

[facelessuser]

I don't plan on making it an ambiguous pattern, it is going to be one or the other so that other plugins can predict what character it is using in case another plugin wants to use those characters. I am not going to reserve infinite characters. If I allow the plugin to allow any character, then the plugin becomes unpredictable to other plugin creators. It gives users the false impression they can use whatever they want, but a user could choose --- and it will break lists.

Unfortunately, right now I have a very, very small sample of people giving feedback. With like maybe one person that feels strongly about one character, one that feels strongly about another, and like one that is somewhere in the middle.

The reality is that I cannot make everyone happy. Someone will be disappointed no matter what I choose. It may be a win for you if I pick your favorite, but I lose no matter what I pick as I will get a complaint from someone regardless.

I was hoping to get more feedback than I have received. If I had a strong amount of people requesting one way or the other, I would be happy to concede. At most I have a single person that feels strongly one way and another that feels strongly another way. That's not enough, so I have to lean on my preference in that case.

[facelessuser]

Just to expound further, the best argument I've received for either side is that people like X more than Y. The fact that some parsers have used ::: for Admonitions is or ::: for general purpose blocks is pretty weak as it is not a standardized format and is implemented differently in every parser that you see them used. Additionally, we follow none of those formats except that we happen to use :::, everything else is a bit different.

If the best argument is "I like this one more than that one", I will need a clear, obvious preference from the community. My personal preference, which I'm willing to give up if the majority of the community feels differently, is ///.

[2bndy5]

That's not enough, so I have to lean on my preference in that case.

Sounds like you'll have to make an executive decision.

Over the years, I've become flexible in the face of "similar but different" syntaxes. In the end, some will feel dissappointed.

The fact that some parsers have used ::: for Admonitions is or ::: for general purpose blocks is pretty weak as it is not a standardized format and is implemented differently in every parser

I actually agree with this. Migrating to a new syntax is a one-time ordeal. If users are finding themselves trying different MD syntaxes, then they may have not fully understood the appeal(s) that each parser offers. After the decision is made to use this newer syntax (which I feel is inevitable), the /// shouldn't be a detriment, just a simple requirement.

[facelessuser]

Yep, if I was implementing a well-known standardized ::: block, I would definitely adopt the same syntax, but we aren't, so I'll just have to make a choice.

[waylan]

Setting aside my personal preferences, I think the plan outlined here is reasonable given the situation. Go ahead and make an official release (labeled experimental) and then see what happens. Changes can always be made in the future is there is an overwhelming demand for it.

As an aside, that is what happened when we implemented fenced code blocks. Michelf (of PHP Markdown) and I used ~~~. A few years later GitHub introduced ```, which became overwhelmingly popular. Today almost no-one remembers the original syntax . Relevant to this discussion, we did consider ``` but rejected it for reasons I don't exactly recall (I believe there were arguments for and against each). The point is that an initial release of a new syntax does not lock in that syntax. But is can perhaps inspire something which later becomes wildly popular. Sometimes it is best to go forward with what you have and be willing to adapt as things play out.

[kno10]

For me, pandoc compatibility is very useful/important. Hence, I'd prefer the triple colons syntax, and would appreciate if also the other features are somewhat supported, i.e., https://pandoc.org/MANUAL.html#divs-and-spans
But of course I am fine if I can choose options to be compatible.

[kno10]

Its about interoperability for me, and not having to worry too much about x different syntaxes in different tools.
Right now, I am preparing contents for a web site that is going to be either Jekyll or Pelican. Pelican I think defaults to python-markdown, but can be switched to Pandoc. Here I still have some choice. "Vanilla" is likely more robust.
But I also want to be able to generate PDFs from these contents (lecture notes), and I am considering to use decker in the future for my slides (which is based on reveal.js), which is built onto of Pandoc.
I probably won't use all functionality, so if there is a large enough common subset that is fine for me. It just seems like a good idea to me to have things not completely disagree on syntax to be able to switch tools more easily. Also, e.g., some editor might be using one tool or another for rendering a preview.

[2bndy5]

It just seems like a good idea to me to have things not completely disagree on syntax to be able to switch tools more easily.

1. There's a fundamental motive to have the syntax distinguishable from other parsing tools' expected syntax.
2. Interoperability isn't a goal here; making that a goal would defeat the founding idea to have this new syntax.

[facelessuser]

Its about interoperability for me, and not having to worry too much about x different syntaxes in different tools.

I understand people's concerns about interoperability. But whose interoperability wins? You like Pandoc, but you aren't the only one.
CommonMark was created to solve this issue, but even CommonMark parsers have their own special plugins you can use for non-common stuff, like blocks with :::, and their implementations vary.

Python Markdown is an old school parser, and it does carry some tech debt backage with it. Its never going to be CommonMark at this point. Fenced blocks in Python Markdown has never been particularly straight forward, so there is a reason it hasn't formally been done before now.

As I said, I have very specific goals and motivations for providing this work, it's what gives me the incentive to spend time on this. I could surely implement a simple extension that does nothing but what Pandoc does, but if I'm doing this at all, I'm doing it once, so it needs to be a generic solution, and it needs to be powerful enough that I'm not wanting to implement yet another solution.
That means it needs to be extensible and people can manipulate it to do just about anything they want it to do (under the limitations of Python Markdown).

[kno10]

Ok, then I will stick to Pandoc, bye.

[facelessuser]

👋🏻

[facelessuser]

Been working on the next alpha. I think I have most of the API documented now.

Will be adding a replacement for "definitions" extension as well. Simply converts blocks that are not ol and ul to terms (dt) and then converts list items (li) of root level ol and ul tags to descriptions (dd).

I haven't settled on the name. Currently using def, but I guess something like define might be better?

/// def
word1

word2

- A definition

- Another definition
///

<dl>
<dt>word1</dt>
<dt>word2</dt>
<dd>
<p>A definition</p>
</dd>
<dd>
<p>Another definition</p>
</dd>
</dl>

I thought about allowing the word to be placed on the declaration line:

/// def | word

- A definition

///

But sense you can define multiple words, it seemed this might be awkward.

/// def | word1
word2

- A definition

- Another definition
///

I guess if people asked for it, we could make placing the first word in the declaration optional for one word cases, but for now, I'm just going to leave it out.

[facelessuser]

A pre-release for Alpha 3 has been made. Discussion for Alpha 3 is here: #1940.

While the /// vs ::: discussion is still open and ongoing, Alpha 3 is opening up discussion regarding the API. Feel free to check out the documentation, or even play with the API yourself. We hope to identify maybe some missing pieces (if any), areas that maybe could be simplified, or even areas that are unnecessary. It's an open discussion, so feel free to share your thoughts. We are getting very close to something we can release to a wider audience.

Closes this discussion in favor of the linked Alpha 3 topic.