Read aloud stops at certain html content

[cadi3s]

Hi, reading aloud text works great, for both plain text and html chunks, except in one case.
When I launch with a chunk with mimeType: 'text/html' and contents a shown below:

• read aloud proceeds until the last paragraph, and there it stops
• if I click on text in that last paragraph (starting with 'Romeo and Juliet has been adapted') I see a script error

Removing the <p> tags from the content avoids the problem, but I would rather not do that.
Can anyone confirm that this is a bug in the SDK ?

Sample html chunk

<p style="margin-bottom: 10px;">
	<span style="font-size: 12px;">Romeo and Juliet is a tragedy written by William Shakespeare early in his career about two young star-crossed lovers whose deaths ultimately reconcile their feuding families. It was among Shakespeare's most popular plays during his lifetime and along with Hamlet, is one of his most frequently performed plays. Today, the title characters are regarded as archetypal young lovers.</span>
</p>
<p style="margin-bottom: 10px;">
	<span style="font-size: 12px;">Romeo and Juliet belongs to a tradition of tragic romances stretching back to antiquity. The plot is based on an Italian tale translated into verse as The Tragical History of Romeus and Juliet by Arthur Brooke in 1562 and retold in prose in Palace of Pleasure by William Painter in 1567. Shakespeare borrowed heavily from both but expanded the plot by developing a number of supporting characters, particularly Mercutio and Paris. Believed to have been written between 1591 and 1595, the play was first published in a quarto version in 1597. The text of the first quarto version was of poor quality, however, and later editions corrected the text to conform more closely with Shakespeare's original.</span>
</p>
<p style="margin-bottom: 10px;">
	<span style="font-size: 12px;">Shakespeare's use of his poetic dramatic structure (especially effects such as switching between comedy and tragedy to heighten tension, his expansion of minor characters, and his use of sub-plots to embellish the story) has been praised as an early sign of his dramatic skill. The play ascribes different poetic forms to different characters, sometimes changing the form as the character develops. Romeo, for example, grows more adept at the sonnet over the course of the play.</span>
</p>
<p style="margin-bottom: 10px;">
	<span style="font-size: 12px;">Romeo and Juliet has been adapted numerous times for stage, film, musical, and opera venues. During the English Restoration, it was revived and heavily revised by William Davenant. David Garrick's 18th-century version also modified several scenes, removing material then considered indecent, and Georg Benda's Romeo und Julie omitted much of the action and added a happy ending. Performances in the 19th century, including Charlotte Cushman's, restored the original text and focused on greater realism. John Gielgud's 1935 version kept very close to Shakespeare's text and used Elizabethan costumes and staging to enhance the drama. In the 20th and into the 21st century, the play has been adapted in versions as diverse as George Cukor's 1936 film Romeo and Juliet, Franco Zeffirelli's 1968 version Romeo and Juliet, and Baz Luhrmann's 1996 MTV-inspired Romeo + Juliet.</span>
</p>

Script error

react-dom.production.min.js:101 Uncaught TypeError: Cannot read properties of undefined (reading 'height')
    at t.scrollSelectedTokenIntoView (virtualized-content-pane.tsx:1856:58)
    at t.setSelectedExpressionForTts (virtualized-content-pane.tsx:1639:18)
    at t.setCurrentlyReadToken (virtualized-content-pane.tsx:1762:18)
    at t.selectToken (virtualized-content-pane.tsx:1719:14)
    at n.onClick (virtualized-content-pane.tsx:730:19)
    at Object.We (react-dom.production.min.js:52:317)
    at Je (react-dom.production.min.js:52:471)
    at react-dom.production.min.js:53:35
    at _o (react-dom.production.min.js:100:68)
    at Do (react-dom.production.min.js:101:380)

Tested with https://ircdname.azureedge.net/immersivereadersdk/immersive-reader-sdk.1.3.0.js

[cadi3s]

I guess the bug occurs in the Azure Immersive Reader itself, not this Immersive Reader SDK.
So feel free to close this issue.
Would appreciate if it could be forwarded to the Azure Immersive Reader team though. Will ask around on SO.

[cristobal-buenrostro]

Hi @cadi3s , I tried to repro this issue but was not possible, can you please provide repro steps more in detail?

I tried both text/plain and text/html, and also starting in the phrase you mentioned, and everything is read out loud.

[cadi3s]

Hi @cristobal-buenrostro my code is pretty standard, I think. The weird behavior seems in this particular content (although there is nothing out of the ordinary in the content as well that I can see) or maybe simply the length of the content.

I include immersive-reader-sdk.1.3.0.js
After getting configuration parameters, options and a token (all standard stuff), I call
ImmersiveReader.launchAsync(token, subdomain, data, options)

The data contains a title (just a string) and two chunks:

The string in the first chunk is rather long, html and contains a couple of paragraphs:

<p style="margin-bottom: 10px;"><span style="font-size: 12px;">Romeo and Juliet is a tragedy written by William Shakespeare early in his career about two young star-crossed lovers whose deaths ultimately reconcile their feuding families. It was among Shakespeare's most popular plays during his lifetime and along with Hamlet, is one of his most frequently performed plays. Today, the title characters are regarded as archetypal young lovers.</span></p><p style="margin-bottom: 10px;"><span style="font-size: 12px;">Romeo and Juliet belongs to a tradition of tragic romances stretching back to antiquity. The plot is based on an Italian tale translated into verse as The Tragical History of Romeus and Juliet by Arthur Brooke in 1562 and retold in prose in Palace of Pleasure by William Painter in 1567. Shakespeare borrowed heavily from both but expanded the plot by developing a number of supporting characters, particularly Mercutio and Paris. Believed to have been written between 1591 and 1595, the play was first published in a quarto version in 1597. The text of the first quarto version was of poor quality, however, and later editions corrected the text to conform more closely with Shakespeare's original.</span></p><p style="margin-bottom: 10px;"><span style="font-size: 12px;">Shakespeare's use of his poetic dramatic structure (especially effects such as switching between comedy and tragedy to heighten tension, his expansion of minor characters, and his use of sub-plots to embellish the story) has been praised as an early sign of his dramatic skill. The play ascribes different poetic forms to different characters, sometimes changing the form as the character develops. Romeo, for example, grows more adept at the sonnet over the course of the play.</span></p><p style="margin-bottom: 10px;"><span style="font-size: 12px;">Romeo and Juliet has been adapted numerous times for stage, film, musical, and opera venues. During the English Restoration, it was revived and heavily revised by William Davenant. David Garrick's 18th-century version also modified several scenes, removing material then considered indecent, and Georg Benda's Romeo und Julie omitted much of the action and added a happy ending. Performances in the 19th century, including Charlotte Cushman's, restored the original text and focused on greater realism. John Gielgud's 1935 version kept very close to Shakespeare's text and used Elizabethan costumes and staging to enhance the drama. In the 20th and into the 21st century, the play has been adapted in versions as diverse as George Cukor's 1936 film Romeo and Juliet, Franco Zeffirelli's 1968 version Romeo and Juliet, and Baz Luhrmann's 1996 MTV-inspired Romeo + Juliet.</span></p>

The reader launches fine, and starts to read aloud. The first few paragraphs are read aloud ok.
Then, it stops reading aloud after the phrase "over the course of the play." In my page, it occurs 'almost' always. Sometimes if I scroll down to that phrase, and start reading aloud at that sentence, the problem does not occur. If I start reading at the top of the page, it always happen.

When it hangs, there is no script error. I can still select words in the paragraphs above and including that phrase "over the course of the play."
But if I click on a word below that paragraph (in a section that was never read aloud), the word is not highlighted, and I see a script error:

[cristobal-buenrostro]

can you please try disabling all your browser extensions? what's your current browser?
I tried following your repro steps and it keeps working fine for me in Edge.

[cadi3s]

I saw it in Chrome, Edge and Firefox. I will try to set up a simple test page that is publicly accessible.

[cristobal-buenrostro]

@cadi3s please let us know if you could publish a test page to repro the issue

[cadi3s]

@cristobal-buenrostro I created a test page at https://demo.cumlaudelearning.com/test-ir.html
Please open it, click on the launch button, then click on the Play button.
Let it read the entire text (it is a bit long, but I could not create a test page with a shorter text).

When I do that, reading stops on its own after ".. over the course of the play".
If you then click on a word below that, e.g. on 'Juliet' in the line below, you see the script error.
I reproduced it in Chrome, Edge and Firefox.

Please let me know if you can or cannot reproduce the issue. At some point I will have to remove that test page.

[cristobal-buenrostro]

@cadi3s I tried in chrome, edge and firefox and all the context was read correctly.

[cadi3s]

@cristobal-buenrostro did you let it run to the indicated line? I can consistently reproduce the bug if I let Immersive Reader read the text, uninterrupted. Also, do not resize the window or click anywhere, just press Play and wait. It usually takes around 1:30 minutes to get to ".. over the course of the play".
At that point, reading stops, and the grey overlay with white word highlighting disappears.
If you click on the word 'Juliet' below (again without resizing the browser), you should get that script error.
We could try to set up a Teams Meeting where I can show what we see.

[austingayler]

I'm seeing this same issue. I'm sending quite a lot of text (HTML content) to the IR service, but in the docs it says the max size is 50MB, so I'm definitely fine on that front. It will read until a certain line, then start throwing that r.line error.

[austingayler]

in.mov

[austingayler]

Seems to be a problem with the React virtualized renderer lib the IR is using.

Screen.Recording.2024-11-19.at.11.34.24.mov

Resizing the viewport 1px fixes the issue.

@cadi3s @cristobal-buenrostro

[cadi3s]

@austingayler indeed my link above still has the same issue and script error, but only for some viewport sizes.
I can consistently reproduce the issue if I maximize my browser, with a viewport of 1920 x 919. On a smaller viewport, I sometimes get the issue, sometimes it runs fine.

@cristobal-buenrostro maybe you want to have another go at this, see if you can reproduce it with my link or Austins info?

[austingayler]

@cadi3s this works:

  launchAsync(token, ACCESSIBILITY_SUBDOMAIN, content, options);

  const resize = () => {
    const iframe = document.querySelector('iframe');
    if (!iframe) return;

    iframe.style.height = '99vh'; // 99.9 is too much and won't cause a re-render
    setTimeout(() => {
      iframe.style.height = '100vh';
    }, 5);
  };

  window.addEventListener('message', e => {
    if (e.data === 'ImmersiveReader-LaunchSuccessful') {
      setTimeout(resize, 500);
    }
  });

This is luckily randomly exposed in this:
https://github.com/microsoft/immersive-reader-sdk/blob/dev/js/src/launchAsync.ts

Hacky but it works.

[cadi3s]

@austingayler thanks I will try that out. If it works, it work :-)
Still hoping that the SDK people can find and fix the script error from the stack trace and other information above..

[cristobal-buenrostro]

thanks for all the feedback, I was finally able to repro the issue. Added to our internal backlog.