[Compatibility Tool] Improve experience for user based on AMP compatibility

[postphotos]

As a WordPress site, my Compatibility Tool should give visual context to the validation data. A user should be able to understand this data and understand how to action what's being displayed.

• AC1: Assess and outline all output from the Compatability Tool.
• AC2: Create a new experience flow for this that more elegantly allows a user to understand errors and issues: Closed in favor subticket Generate userflow based on Validation Workflow efforts #1359
• AC3: Write steps for the user (i.e. documentation for the product site, wiki, etc.) to be able to resolve any issues, based on final implementation of design.

While there's a decent reporting mechanism currently built for this plugin, it'd be great if, beyond output, if something was more contextual to non-technical users.
https://files.slack.com/files-pri/T02UB976M-F9N2ADXQF/image.png

[postphotos]

The main issue description has been updated with a formal Story and Acceptance Criteria.

[kienstra]

Question About Design

Hi @westonruter,
It sounds like we're going to ask @jwold to work on the design of the 'Compatibility Tool.' Should I update him on the current state of the 'Compatibility Tool,' maybe with a screencast or a detailed comment here?

[westonruter]

@kienstra yes, that sounds good. I think he's seen the screenshots already to a degree. But having a screencast would be helpful.

[kienstra]

Request For Design Work, Update On Current State

Hi @jwold,
Are you available to work on the design of the 'Compatibility Tool' (AC2 on this issue)? This screencast shows the current state, using the dev site.

Also, this 'Steps To Test' comment describes more 'Recheck' user flows.

[postphotos]

Hi @kienstra - thank you for this. I just watched your screencast.

A request and a few questions regarding this component (cc: @jwold):

1. Can you (as a gist perhaps, or in a throwaway repo?) share on that AMP-invalid plugin?

2. Thinking about desired functionality for MVP here, the Validation Status page is indeed confusing.

I think a notice that gives content context to this listing would be helpful, e.g. This page is used to show you current AMP errors on your site. To resolve these errors, a) remove tags from selected posts, b) deactivate these plugins or c) simply allow the AMP plugin to strip this content. Just a thought to a partial solution. Your thoughts here?

3. Count (in table view): What does this mean? Count across the site? Number of posts where this happens? How many times that error occurs on the site?

4. The Validation Status listing gives us context by error. Is there a contextual way to select/filter/view by plugin? Or post? Thinking about workflows to using this component, it'd be nice to have several ways to address AMP errors from the user perspective... Maybe these are three separate listings or an interface that @jwold builds that allows this.

5. On the single Validation Status page, it reads "Validation Status" // "Validation Errors" - it should probably surface a contextual error - tell me, "What's wrong here? How can I fix it?" Give me at least the title or the name of the spec that's not working as expected.

6. On this single view, I have to scroll down the page (about half a long page) to see the invalid_element error.
   a) Is there a way to make this more elegant but allow the error to be surfaced faster? (i.e. show me the offending code or output so I can quickly debug.)
   b) Since we've got the name of the error available to us, Is there a simple way to link the error to direct documentation (i.e. to the AMP project?) so that a user can see how to fix this or read more on what AMP is supposed to do with this tag or feature?

Thank you again, both, for your help here, and I look forward to seeing this work even better!

[westonruter]

Can you (as a gist perhaps, or in a throwaway repo?) share on that AMP-invalid plugin?

@postphotos Here's the plugin: https://gist.github.com/westonruter/e660643f9b8f0e7b5d8a272824525eb2

[kienstra]

Thanks, Good Suggestions

Hi @postphotos,
Thanks for your ideas here.

I think a notice that gives content context to this listing would be helpful...

Yes, it'd be great to have a message at the top like yours. Copying your suggestion:

This page is used to show you current AMP errors on your site. To resolve these errors, a) remove tags from selected posts, b) deactivate these plugins or c) simply allow the AMP plugin to strip this content.

It'd also help to direct them to the sources field. If they see a plugin in it, they could consider deactivating it.

3. Count (in table view): What does this mean?

This is the number of URLs that have that error. For example, if you see 2 as the count, and click the post, you'll see 2 URLs at the bottom of its post.php page (screenshot).

4. The Validation Status listing gives us context by error. Is there a contextual way to select/filter/view by plugin? Or post?

No, the current /wp-admin/edit.php page is now the only way to view the errors. But you raise a good point.

6. On this single view, I have to scroll down the page (about half a long page) to see the invalid_element error.
   a) Is there a way to make this more elegant but allow the error to be surfaced faster? (i.e. show me the offending code or output so I can quickly debug.)

It looks like the code of the invalid_element is showing on the dev site (screenshot). But maybe there's a case where it doesn't display.

b) Since we've got the name of the error available to us, Is there a simple way to link the error to direct documentation (i.e. to the AMP project?) so that a user can see how to fix this or read more on what AMP is supposed to do with this tag or feature?

Good suggestion. For invalid_element or invalid_attribute errors that involve an AMP component, we could probably link to the documentation for it. The URL for components seems to always be:

https://www.ampproject.org/docs/reference/components/<example-component-name>

This might not always be helpful, like if it removes an onclick from a component. That's not allowed in any component.

Otherwise, when there isn't a message field in an invalid_element, we could probably link to HTML Tags. And HTML Attributes for invalid_attribute.

[postphotos]

Hi @kienstra, thank you for your thoughts here and I know they'll help better inform next steps. I'm recording some notes on the progress of this current ticket:

• @jwold, Maxim and I met briefly on Friday to discuss this and they were excited to help organize this visually.
• As a result of our backlog grooming session yesterday, this ticket is likely going to lead to several new development tickets.

[westonruter]

For a more holistic look at how the validation errors can be integrated into the experience, see #1087

[MackenzieHartung]

From Maxim: I met with Leo and Joshua to discuss what the compatibility tool should achieve, did some research of similar tools and mocked up what I think would be a big improvement based on the existing tool. I then shared to more people, including Ryan and Thierry and implemented feedback. I am open to more feedback based on technical requirements and would love to push this further if anyone has ideas!

https://www.figma.com/file/EZ6bu3BQbjjSwJYyigqG19PD/Compability-Tool

[postphotos]

Hi @westonruter - I've added AC3 here, broken out subticket #1273 to discuss that issue in its own ticket. Thanks for all your work here in improving the interfaces of the AMP plugin, it's gone a long way! 🥇

[kienstra]

Question about AC3

Hi @postphotos,

AC3: Write steps for the user to be able to resolve any issues and integrate to design.

For AC3, were you thinking of changing the design of the validator, to help users resolve issues?

Here's a screencast I happened to make yesterday on the validator workflow.

[postphotos]

Sharing the in-progress work from today's workshop with @jwold here.

AC1

These are the current proposals of UI:

• Initial prototype shared in Add ability to acknowledge and suppress/ignore specific validation errors #1003
• Current interface from v1.0-Beta1
• Maxim's interface shared above (Figma doc)
• @ThierryA 's internal sketches of toggles and faceted view. Also shared at Google I/O as a prototype sketch in his presentation.

We generally have lots of information around what an error is, what caused it, and where it appears.

---

AC2

Questions in mind:

1. How does a user view each error by context?
2. How does a user view each error by type?
3. How does a user view each new error?
4. How does a user work to understand each error?

This is our new user flow as proposed so far.

Current user flow:

1) Settings Page

• I enable AMP - if in Paired mode and errors are found, then it is suggested I go review the Validator.
• What I do here affects the index page.

2) Index page

• I need to be aware of what I can do based on the settings currently enabled.
• I need to be able to view a given subset of errors based on a faceted/filtered/searchable view. (By plugin, by post type, what's inside the theme itself)(JS, CSS, AMPHTML)
• I need to be able to action on the things that I see. (Quick edit - Remove or ignore.)
• Possible toggles in listings (with active and new states) that provide shorter context to what the errors are.

The "Bulk Actions" menu items (and key actions on a per error basis) are to:

• Suppress/Enable the plugin to remove invalid content/css/js
• Ignore/Prevent the plugin in doing so
• Mark as acknowledged, then assume the default

Note: Wording around working with AMP errors is totally still up for debate.

3) Single view

• I need to understand the errors I see and be able to respond to them. (Remove or ignore.)

[jwold]

Additional thoughts from Leo:

• Index listing view - We still have questions about how faceted search would work in the index. Tab, filter, toggles, dropdown, etc. May even need multiple filters based on content types. So how do we architecturally make this kind of stuff make sense in interface?
• Single context - What we want to validate is what is required. How much info should we give to user? Is there too much info there? What’s the easiest way to tell a user what’s going on without giving them too much information? How do we suppress content to just the most critical information regarding errors.

[jwold]

Just had a great workshop with Leo where we worked through the things that need to be shown in the UI for the error pages. First, we are thinking it'd make sense to split up the way you can view errors into a few different pages. You should be able to see errors by the type of error they are, and then you should be able to see errors by the page they're shown on.

In addition, when you are on either of those pages you should see a notice at the top that gives you a sense of why you're seeing certain errors, and whether you have the ability to action on them. This would be conditional based on the settings you've chosen for AMP as a whole on your site.

Following is the rough sketch:

[westonruter]

First, we are thinking it'd make sense to split up the way you can view errors into a few different pages. You should be able to see errors by the type of error they are, and then you should be able to see errors by the page they're shown on.

This is how it works currently, actually. There is the URL-centric view to see what errors are on a given page, and then there is the error-centric view to see which URLs have a given error.

There is the the Invalid URLs screen:

And the Validation Errors screen:

The thing that is not present is filtering by type. There is filtering by status (new, accepted, rejected), but not type (invalid_element, invalid_attribute, etc). Having facets for types will require changes to the data model. Each validation error is a taxonomy term, so if we want to query taxonomy terms that have a common type, this would at first seem like a taxonomy for a taxonomy. On the other hand, what is supported today is searching across validation errors. Since the validation error type (code) is part of the JSON that is in the term description, you can do a search for a given error code or even for a given plugin name and see results that match anywhere in the term description. It won't guarantee that you're only going to get errors of a given type, but it's something (if very rough).

[jwold]

Wanting to share an update for where things stand right now based on the work Leo and I have done so far: https://v.usetapes.com/N5r5al2XXN

[jwold]

Working on the single pages with Leo and Jill today:

cc @postphotos

[kienstra]

Moving To 'Ready For Merging'

Hi @postphotos,
If it's alright, I'm moving this to 'Ready For Merging,' as the PRs for the Compatibility Tool issues have been merged (#1361, #1362, etc.).