Include unrendered pages in sections, add render property to templates

[clarfonthey]

• Have you checked to ensure there aren't other open Pull Requests for the same update/change?
• Are you doing the PR on the next branch?
• Have you created/updated the relevant documentation page(s)?

On my blog, I make use of pages which aren't rendered but included in a section to make templating a little easier. What feels like the best way to ensure this is done correctly on the upstream zola is to add these pages to sections, but explicitly allow accessing the render field from templates to filter them out (or, in my case, not show permalinks, since they would not be rendered).

This seems reasonable considering how drafts also can be detected in templates as well, when drafts are enabled.

I can't imagine this affecting many blogs, but I can go for an alternative approach otherwise.

Note that this only affects directly sections, and does not affect taxonomies. Taxonomies explicitly exclude unrendered pages, and will not even let unrendered pages add new terms.

[Keats]

That would be a big breaking change I think, and things like the sitemap would start showing them afaik? Maybe taxonomies as well. Do they need to be mixed with the other pages?

[clarfonthey]

Note that they'd only be included in taxonomies if they were explicitly added to them, and at least imho, it's kind of weird to add a page you explicitly don't want to render to a taxonomy instead of just using the extra data. Although I'll test to see if they're added to taxonomies anyway, since I believe it just affects sections.

At least on my site which is using the change, they still aren't included on the sitemap.

I'll look into adding a few tests that check how unrendered pages are added to various parts of the site; these would be nice anyway considering how afaik this feature remains largely untested in the code.

[clarfonthey]

Current progress: updated the site test to add an explicit check that an unrendered page is not in the sitemap, and also added an extra test that they can be accessed via templates. Still need to test that they don't show up in taxonomies, though.

[clarfonthey]

So, at least for now, I'm excluding taxonomies entirely. There may be a benefit to also including unrendered pages in taxonomies, but at least for now, they are explicitly excluded and there are tests for this.

[Keats]

I'm still a bit on the fence: the pages will be included in the basic sections but not if they are paginated (obviously).

To expand a bit on:

On my blog, I make use of pages which aren't rendered but included in a section to make templating a little easier. What feels like the best way to ensure this is done correctly on the upstream zola is to add these pages to sections, but explicitly allow accessing the render field from templates to filter them out (or, in my case, not show permalinks, since they would not be rendered).

Can you detail exactly what you want to achieve? I'm wondering if there's another way we can handle that

[clarfonthey]

So, the current way I'm using it is on my CV, where I can separate different items into pages without actually rendering them separately. The logic ends up working out that rendering it as a section works very well, but I don't need or want to figure out how to render the individual pages.

Right now there isn't really a great way to make use of this otherwise. I get a lot of the nice aspects of pages, like dates, sorting, slugification for anchors, and automatic markdown parsing that I'd otherwise have to do by hand. My first pass at doing this without unrendered pages involved manually parsing markdown files, and I also had to modify the code anyway to parse front matter, and it was way more work than just leveraging the page system.

Totally open to finding an alternative solution, this one just felt like the simplest.

Another thing to keep in mind is that right now, Zola isn't clever enough to determine load_data dependencies for templates and thus doing that method also will require manual rebuilds. I have this problem with a few TOML assets I use where I end up having to no-op edit pages to get the results to show up.

[dubrowgn]

I wonder if the concept of a "for organization purposes only" section would be helpful. Basically, a more targeted version of this, where you have to opt in at a section level.

I ran into the same issue trying to keep news feed items as separate posts. It would be convenient for a lot of reasons, but it doesn't work out. Zola wants to render them as individual pages (which I don't want), but setting render = false makes them disappear from the section. Your use case sounds similar to this, where the child pages exist purely as an organizational tool.

Alternatively, you could emulate these use cases by asking for "unrendered" children when you loop over the section contents. You would still have to explicitly ask for them, avoiding breaking existing sites. E.g., something like {%- for page in paginator.all_pages %}.

[clarfonthey]

I mean, ultimately, that's a small change on this one: just, templates explicitly filter out the unrendered pages and then offer the "real" list separately. I don't like the idea because it does complicate things, but, it is technically an option.

[en9inerd]

I get a lot of the nice aspects of pages, like dates, sorting, slugification for anchors, and automatic markdown parsing that I'd otherwise have to do by hand.

I understand that using pages gives you some useful features like dates, sorting, and automatic markdown parsing. However, it feels like there might be a more streamlined approach, depending on your use case. For example, I’ve been able to achieve similar results without the overhead of managing full pages by using shortcodes along with load_data. In my case, I implemented a CV where data can be stored as JSON/YAML/TOML ( https://github.com/en9inerd/en9inerd.github.io/tree/master/data/cv ) and fetched with load_data.
You can check out my shortcode implementation here: https://github.com/en9inerd/zola-hacker/blob/master/templates/shortcodes/cv.md?plain=1
In my opinion, this way offers more flexibility, but I could be wrong.

I'm still a bit on the fence: the pages will be included in the basic sections but not if they are paginated (obviously).

I ran into the same issue trying to keep news feed items as separate posts. It would be convenient for a lot of reasons, but it doesn't work out. Zola wants to render them as individual pages (which I don't want), but setting render = false makes them disappear from the section.

I’m facing the same problem on my end when I don’t want to render pages under specific section but still need to access their content and paginate them in index.html or home.html template. It would be great to have the ability to retrieve unrendered pages from paginator.

Additionally, having an option in the front matter of the section file to mark all pages within that section as unrendered (instead of setting render = false individually for each page) would be a useful feature.

[Keats]

I think the concept of "virtual"/"organisational" sections where all pages are effectively render = false makes sense. I believe this would solve @clarfonthey problem as well. Although I do like the file approach and would probably prefer that if i had to choose since it's more flexible.

I’m facing the same problem on my end when I don’t want to render pages under specific section but still need to access their content and paginate them in index.html or home.html template

What do you mean by paginate them? Include them in the paginator? What does it link to if the page is not rendered? Or is the data just the markdown content of the page with no

[en9inerd]

Include them in the paginator?

Yes

What does it link to if the page is not rendered? Or is the data just the markdown content of the page with no

By page content, I mean front matter fields + markdown content, fields that depend on the path/link to the rendered page (HTML file) probably should be set to something like "null”. In Hugo, as I recall, it's possible to avoid rendering the page to disk while still assigning a permalink, but I don’t think a permalink/link would be useful in this case. Things like specified taxonomies should likely be ignored in this case, since they rely on paths to rendered pages too.

I think the concept of "virtual"/"organisational" sections where all pages are effectively render = false makes sense.

It reminds me of collections in Jekyll. In the context of Zola, I’d say it’s like sections; there, you have the option to disable rendering for an entire collection, allowing you to iterate over the items in the collection, access their front matter fields and markdown content, and paginate them with a plugin. Overall, in my opinion "virtual sections" would be a good solution in most cases.