Add PropertyString, cleanHtml helper and escapeHtmlExt helper. #4004

EreMaijala · 2024-10-11T10:20:21Z

Related to #3998, this is a draft for PropertyString that can carry additional information along with a plain text string. It now has explicit support for HTML content, and there's also a cleanHtml helper to handle sanitization. escapeHtmlExt is a replacement for escapeHtml allowing the HTML version to be returned if desired.

@maccabeelevine What do you think? Obviously this needs some template changes where HTML is desirable, but should otherwise be fairly easy to use.

TODO:

Add unit tests

demiankatz

Thanks, @EreMaijala, looks like a great start! A few minor thoughts/suggestions...

themes/bootstrap5/templates/RecordDriver/DefaultRecord/data-summary.phtml

demiankatz · 2024-10-11T13:58:46Z

module/VuFind/src/VuFind/View/Helper/Root/EscapeHtmlExt.php

+ * @license  http://opensource.org/licenses/gpl-2.0.php GNU General Public License
+ * @link     https://vufind.org/wiki/development Wiki
+ */
+class EscapeHtmlExt extends \Laminas\View\Helper\Escaper\AbstractHelper


I don't love this name, but I also can't think of a better one... :-)

Me neither. I wanted to extend EscapeHtml, but the silly @Final thing prevents that. And while we could substitute our own EscapeHtml, the Laminas class is used in so many places, that it's not quite straightforward. We could of course change all the references to an interface, but that would require more wide-ranging changes. If you think that'd be a better way forward, I'd be happy to work on that. What would support that is the fact that EscapeHtmlAttr needs a substitute as well (to be able to control the IE-compatibility of the escaping process), so doing both at the same time would probably make sense.

I don't have really strong feelings on this -- no solution feels obviously like the best. Maybe @maccabeelevine will have some thoughts to share... ;-)

And while we could substitute our own EscapeHtml, the Laminas class is used in so many places, that it's not quite straightforward.

This was certainly my original idea with #3998, to make the template changes as simple as possible, and it's safer here than in my PoC implementation since it would do nothing unless you had a PropertyStringInterface value and passed allowHtml. But I'm sure there are complications I'm not seeing.

A month later, a fresh thought on the name: would escapeOrCleanHtml be more descriptive than escapeHtmlExt, since that's what this is doing -- escaping the HTML unless told to clean it instead?

If Ere agrees, I like that naming!

I like it a lot better than escapeHtmlExt!

module/VuFind/src/VuFind/View/Helper/Root/EscapeHtmlExt.php

module/VuFind/src/VuFind/RecordDriver/DefaultRecord.php

maccabeelevine

I love the idea here, and it's so much better than the original #3998 PoC. I added some specific questions, but I guess my primary question (and maybe the hardest to answer) is how we deal with generic templates that may or may not have HTML in certain fields for some backends.

So, would it be appropriate in the default record driver's core.html to change

<h1<?=$this->schemaOrg()->getAttributes(['property' => 'name'])?>><?=$this->escapeHtml($this->driver->getShortTitle() . ' ' . $this->driver->getSubtitle() . ' ' . $this->driver->getTitleSection())?></h1>

to

<h1<?=$this->schemaOrg()->getAttributes(['property' => 'name'])?>><?=$this->escapeHtmlExt($this->driver->getShortTitle(), allowHtml: true)?> <?=$this->escapeHtml($this->driver->getSubtitle() . ' ' . $this->driver->getTitleSection())?></h1>

Performance-wise, this should be ok, since for any other record driver the title would not be a PropertyString and so the allowHtml would be ignored. But is it ok from a template complexity standpoint?

module/VuFind/src/VuFind/Content/Summaries/Demo.php

module/VuFind/src/VuFind/View/Helper/Root/CleanHtmlFactory.php

maccabeelevine · 2024-10-14T13:49:35Z

module/VuFind/src/VuFind/View/Helper/Root/EscapeHtmlExt.php

+ * @license  http://opensource.org/licenses/gpl-2.0.php GNU General Public License
+ * @link     https://vufind.org/wiki/development Wiki
+ */
+class EscapeHtmlExt extends \Laminas\View\Helper\Escaper\AbstractHelper


And while we could substitute our own EscapeHtml, the Laminas class is used in so many places, that it's not quite straightforward.

This was certainly my original idea with #3998, to make the template changes as simple as possible, and it's safer here than in my PoC implementation since it would do nothing unless you had a PropertyStringInterface value and passed allowHtml. But I'm sure there are complications I'm not seeing.

demiankatz · 2024-10-14T15:33:54Z

Regarding @maccabeelevine's question about changing the default driver's core.html to support HTML, I don't think I would have a problem with that in the interest of greater flexibility....

EreMaijala · 2024-10-15T06:16:56Z

@demiankatz Do we want to support HTML for all fields or for longer textual fields like summary? Different choices will lead to different complications e.g. with search term highlighting.

Co-authored-by: Demian Katz <[email protected]>

demiankatz · 2024-10-15T10:46:35Z

@demiankatz Do we want to support HTML for all fields or for longer textual fields like summary? Different choices will lead to different complications e.g. with search term highlighting.

If we wanted to start small, I would think the bare minimum would be longer textual fields and titles. But I'm not opposed to rolling out the support more widely if it's easier to do it all at once than piecemeal. I tend to favor the most pragmatic strategy, whatever that turns out to be. :-)

EreMaijala · 2024-10-25T13:17:01Z

Here are a couple of thought:

Could add a default "allow HTML" toggle and/or setting to the escaper so that the parameter doesn't need to be repeated. Downside: easy to accidentally leave enabled outside of the intended scope.
Could add configurable defaults per field type ('title', 'alt-title', 'summary' etc.). This would allow specifying the fields with HTML in configuration.
truncate helper is going to have trouble with HTML. Perhaps we can make it smarter, but it won't be easy.
titles will be a problem with any tags that you can't have inside a heading like <h1>. Perhaps we need the field type from 2. to also indicate which tags are allowed.
escapeHtmlExt needs to deviate a bit further from the invoke params available with Laminas' helper. It seems that it'd be better to decouple.

demiankatz · 2024-10-25T13:27:13Z

1. Could add a default "allow HTML" toggle and/or setting to the escaper so that the parameter doesn't need to be repeated. Downside: easy to accidentally leave enabled outside of the intended scope.

I think the downsides probably outweigh the benefits here, since I can't think of any scenario where this doesn't potentially lead to unexpected side effects or confusion. I think it's better to be explicit.

2. Could add configurable defaults per field type ('title', 'alt-title', 'summary' etc.). This would allow specifying the fields with HTML in configuration.

...and this would be a good way to be explicit in a number of contexts.

3. truncate helper is going to have trouble with HTML. Perhaps we can make it smarter, but it won't be easy.

Possible lazy solution: truncate based on a tag-stripped version. If the stripped version is under the limit, display the HTML as-is. If the stripped version is too long, show the truncated, stripped version, and the user will have to "see more" to get the rich version. Not ideal, but maybe a quick place to start.

4. titles will be a problem with any tags that you can't have inside a heading like `<h1>`. Perhaps we need the field type from 2. to also indicate which tags are allowed.

Maybe we need the ability to define multiple named tag allow-lists. Then we could have a "default" list and a "heading" list and the configurable settings could refer to an allow-list, or "none". This would empower users to create their own more granular lists as needed, but we could use these two obvious ones as a starting point.

5. escapeHtmlExt needs to deviate a bit further from the invoke params available with Laminas' helper. It seems that it'd be better to decouple.

Given the way Laminas seems to be gradually cutting off the ability to extend anything, I agree that building tools that meet our needs and wrap around Laminas' public interfaces is better than trying to build upon those interfaces directly.

demiankatz · 2024-11-05T14:33:59Z

This PR was accidentally closed by the deletion of the dev-11.0 branch; I have restored and reopened it. Sorry for the inconvenience!

demiankatz

I took another look at this and had just a few more minor thoughts and questions...

demiankatz · 2024-11-11T19:32:48Z

module/VuFind/src/VuFind/View/Helper/Root/EscapeHtmlExt.php

+ * @license  http://opensource.org/licenses/gpl-2.0.php GNU General Public License
+ * @link     https://vufind.org/wiki/development Wiki
+ */
+class EscapeHtmlExt extends \Laminas\View\Helper\Escaper\AbstractHelper


A month later, a fresh thought on the name: would escapeOrCleanHtml be more descriptive than escapeHtmlExt, since that's what this is doing -- escaping the HTML unless told to clean it instead?

demiankatz · 2024-11-11T19:36:35Z

themes/bootstrap3/templates/RecordDriver/DefaultRecord/data-summary.phtml

+        '<br>',
+        array_map(
+            function ($summary) {
+                $htmlContent = str_starts_with($summary, '<') && str_ends_with($summary, '>');


I wonder if there's a way to eliminate this hacky check for angle brackets. Should we do the check in the record driver and wrap the string in a PropertyString? Should the new helper have a switch to enable this check, or even make this behavior part of the standard "allowHtml"? I'm not totally sure, but it feels like we can do better here, especially since as written, there's no safety on the summary in the special case.

demiankatz · 2024-11-11T19:38:17Z

module/VuFind/src/VuFind/View/Helper/Root/CleanHtmlFactory.php

+        // which ensures that the libxml2 options (namely keepBlanks) are set up
+        // properly, and whitespace nodes are preserved. This should not be an
+        // issue from libxml2 version 2.9.5, but during testing the issue was
+        // still intermittently present. Regardless of that, CentOS 7.x have an


Is this note about CentOS 7 still relevant now that it's so thoroughly EOL'ed?

Add PropertyString, cleanHtml helper and escapeHtmlExt helper.

3b17e2a

EreMaijala requested review from demiankatz and maccabeelevine October 11, 2024 10:20

EreMaijala marked this pull request as draft October 11, 2024 10:20

Add EscapeHtmlExt.

be71629

demiankatz requested changes Oct 11, 2024

View reviewed changes

Fixes.

d5c0ca5

maccabeelevine reviewed Oct 14, 2024

View reviewed changes

EreMaijala and others added 5 commits October 15, 2024 11:07

Add a static factory method.

c1ff4e9

Don't try to be too clever.

da9165f

Fix comment.

61ef47e

Fix indentation.

9bcfe47

Co-authored-by: Demian Katz <[email protected]>

Fix indentation more.

1461a00

demiankatz deleted the branch vufind-org:dev November 1, 2024 18:01

demiankatz closed this Nov 1, 2024

demiankatz reopened this Nov 5, 2024

demiankatz changed the base branch from dev-11.0 to dev November 5, 2024 14:33

demiankatz added this to the 11.0 milestone Nov 5, 2024

maccabeelevine mentioned this pull request Nov 6, 2024

Add backend: ProQuest Federated Search Gateway #3991

Open

3 tasks

Merge remote-tracking branch 'origin/dev' into dev-11.0-propertystring

a1210ce

demiankatz requested changes Nov 11, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add PropertyString, cleanHtml helper and escapeHtmlExt helper. #4004

Add PropertyString, cleanHtml helper and escapeHtmlExt helper. #4004

EreMaijala commented Oct 11, 2024

demiankatz left a comment

demiankatz Oct 11, 2024

EreMaijala Oct 11, 2024

demiankatz Oct 11, 2024

maccabeelevine Oct 14, 2024

demiankatz Nov 11, 2024

maccabeelevine Nov 12, 2024

EreMaijala Nov 12, 2024

maccabeelevine left a comment •

edited

Loading

maccabeelevine Oct 14, 2024

demiankatz commented Oct 14, 2024

EreMaijala commented Oct 15, 2024

demiankatz commented Oct 15, 2024

EreMaijala commented Oct 25, 2024

demiankatz commented Oct 25, 2024

demiankatz commented Nov 5, 2024

demiankatz left a comment

demiankatz Nov 11, 2024

demiankatz Nov 11, 2024

demiankatz Nov 11, 2024

Add PropertyString, cleanHtml helper and escapeHtmlExt helper. #4004

Are you sure you want to change the base?

Add PropertyString, cleanHtml helper and escapeHtmlExt helper. #4004

Conversation

EreMaijala commented Oct 11, 2024

demiankatz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maccabeelevine left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

demiankatz commented Oct 14, 2024

EreMaijala commented Oct 15, 2024

demiankatz commented Oct 15, 2024

EreMaijala commented Oct 25, 2024

demiankatz commented Oct 25, 2024

demiankatz commented Nov 5, 2024

demiankatz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maccabeelevine left a comment •

edited

Loading