Recent change regressed touchscreen functionality in history-chart-timeline

[vangorra]

Checklist

• I have updated to the latest available Home Assistant version.
• I have cleared the cache of my browser.
• I have tried a different browser to see if it is related to my browser.

Describe the issue you are experiencing

Recent change #18036 broke touchscreen experience for history graph.

When using a touch screen device. Clicking on a block in history-chart-timeline flashes a tooltip with details and immediately opens more-info. Prior to this change, no action would occur when clicking a block in the timeline.

Describe the behavior you expected

The new functionality should be turned off until the feature properly accounts for the regression.

On touchscreen (no mouse):

• No hover exists, do nothing.
• Click a block opens the tooltip with details. (as it always has)
• Long-press opens more-info.

On desktop (with mouse):

• Hover over a block opens the tooltip with details.
• Mouse press opens more-info.

Steps to reproduce the issue

1. Create a history-graph card.
2. Add entities with non-numeric states.
3. On a touch screen device, click one of the blocks.

What version of Home Assistant Core has the issue?

latest

What was the last working version of Home Assistant Core?

2023.8.x

In which browser are you experiencing the issue with?

Chrome, Firefox

Which operating system are you using to run this browser?

Linux, Android

State of relevant entities

No response

Problem-relevant frontend configuration

No response

Javascript errors shown in your browser console/inspector

No response

Additional information

No response

[rubenpeters91]

This is true for any history graph, also for numeric entities and also occurs when using the Android or iOS app.

Right now it's impossible to check historic values with the tooltip on tablet/phone, since it will instantly open the more-info dialog.
My partner and I really want to be able to inspect the shiny new long-term statistics in the history graph on our phone, but currently reading the tooltip is only possible on desktop/laptop

[k3a]

I don't see the problem here?

On desktop, tooltips work on mouse hover as before. Clicking opens More info dialog (new feature).

On android, tap and release opens More Info dialog. Yes, this changes the old behavior.
But you can always tap & hold the finger there and move it to show the tooltip on individual data points.
Why is this a problem? Am I missing something?

ha-more-info-ok.mp4

[rubenpeters91]

I don't see the problem here?

On desktop, tooltips work on mouse hover as before. Clicking opens More info dialog (new feature).

On android, tap and release opens More Info dialog. Yes, this changes the old behavior. But you can always tap & hold the finger there and move it to show the tooltip on individual data points. Why is this a problem? Am I missing something?
ha-more-info-ok.mp4

Good question, I think it's not a great user experience. Normal users (like my partner) expect a tooltip to appear when when pressing the graph on a mobile device. The fact that the tooltip flashes shortly before going to the more info dialog feels like a bug.

Furthermore long pressing on a mobile device feels weird and doesn't always work as expected. On my device it always starts to select text in the tooltip when long pressing, which obfuscates part of the tooltip. I also can't hold and move over the graph, since my finger covers the tooltip.

If we need to have functionality like on pc, the other way around would make more sense: short press shows tooltip and long press shows more info.

[afflux]

My user experience is this:

1. I see an interesting sensor entity where I want to see historical data, so I click on it
2. "more info" dialog pops up and shows some historical data
3. I want to specify the time range, so I click "show more" to arrive at the History panel
4. I select my interesting time range
5. Now I click somewhere in the graph, because I want to read the tooltip to see the values
6. Dialog from step 2 pops up?!?

Touch-and-hold is not an option, as described by @rubenpeters91 - my finger covers the tooltip.

The "more info" dialog provides zero additional info in this context. I propose to either:

1. set clickForMoreInfo=false on the History panel (and maybe make it configurable for the card?)
2. disable the clickForMoreInfo behavior on touch inputs

[vangorra]

I was going to create a video to demonstrate but @afflux described the issue much better.

On mobile, interaction with any data is broken.
History page, broken. Energy dashboard, broken. Additionally, on mobile, the finger covers 20% if the screen and the area you are trying to "hover"is entirely occluded. It's simply not possible to 1:1 transfer desktop interactions to mobile.

This feature should be disabled by default to avoid breaking the rest of the app. This would still allow it to be turned on for a case by case basis for cards (which suggests to have been the intention).

When a solution that factors in all platforms is provided, only then, enable by default.

[k3a]

Wait, is it all about History section (page)? The Click for More Info could be made disabled in History section easily, I think More Info is less useful there as @afflux suggests. Maybe we could close this and one more issue by just disabling it for the History section.

Thanks @rubenpeters91 for showing this bug with the text selection on the tooltip, I never witnessed this myself. EDIT: It is easily reproducible in History section but it can be easily fixed by .chartTooltip CSS in the chart-base. I tested it locally and I will probably send it upstream. The text of the chart tooltip should never be selectable.

@vangorra how exactly is the Energy dashboard "broken" for you? "the finger covers 20% if the screen" that sounds like you need a bigger device! "area you are trying to "hover"is entirely occluded" - again: try to repeat precisely: 1. you tap where you want as before, but keep the finger at the screen for 1 second. 2. release your finger. 3. Tooltip is visible - celebrate! You actually don't HAVE TO move your finger to avoid Click for More Info!

[karwosts]

I believe when we get the onClick event that opens more-info we can differentiate mouse vs touch by looking at the e.native.pointerType, I think it should be mouse for desktop and touch for touchscreens.

Maybe when we're on desktop, the click should open more-info, and when we have touchscreen, the touch can show the tooltip, and then if we do a second touch on the tooltip that can open more-info?

[i-jk]

@k3a I think I can see a way to fix this and restore expected behaviour while still allowing access to more info directly from the charts...

Core issue:
The issue with jumping to 'more info' from the chart lines/data is it fundamentally counter-intuitive because it is not behaviour that we see anywhere else. The usual UX experienced across the web on sites/apps with interactive charts (e.g. finance, sports etc) is touching/clicking the data lines does not navigate, it selects the point closest to the touch. This is the previous, preferred behaviour. Due to the recent changes (and small screens/fat fingers 😄), there often isn't space to release the long press into whitespace to properly choose a data point for the overlay, which means all these unexpected jumps to 'more info' that people are marking as bugs...

Proposed fix:
Rather than wiring this event to the chart lines/data itself, for all the reasons noted above, it would be better to use the already-existing pattern of long-press on the name of the entity. On these charts that means we should only perform this on the entity titles in the header (or side of the bar chart). They already hide/show the data when toggled, so making them also have a long press for more info would be in alignment with many other behaviours in the HA front end. Then the more-info jumps from the data can be removed and several issues disappear with it.

Just a thought, hope this helps -- and thank you for your contributions to HA.

[k3a]

@karwosts e.native.pointerType can indeed be used to detect touch/mouse and alter the behavior on mobile. I am not sure tapping on the tooltip would be intuitive enough but it looks to me like the best option so far, especially considering tooltips "stay there" on mobile. Tooltip clicking event would need to be dispatched from the ha-chart-base. What I like on this option is that it would work for both line charts and timeline. I will try it out locally and see how it goes.

@i-jk Thanks for giving an additional suggestion but it is not as easy as you probably think. The contextmenu event on the legend would work but timeline history chart on mobile doesn't have a HTML legend and onClick event of the chartjs is dispatched only when clicking inside the timeline chart area. I am currently not sure if it would be possible to listen on contextmenu event on the canvas and get the category (this is how its called in timeline) under the finger. Probably yes, but it sounds like it would require some deeper integration.

[karwosts]

As of #19606, graph touches on mobile no longer activate the more-info window, that's now a mouse feature only.