When I use a HEVC 256 stream, VLC can play normally, but HLS cannot play normally

[XiaoxiRao]

What version of Hls.js are you using?

1.6.0

What browser (including version) are you using?

Chrome version 131.0.6778.86 (arm64)

What OS (including version) are you using?

mac

Test stream

https://schoolfighting.oss-cn-hangzhou.aliyuncs.com/test/teshh265/output.m3u8

Configuration

{
    liveDurationInfinity: true,
    levelLoadingTimeOut: 15000,
    fragLoadingTimeOut: 25000,
    enableWorker: true,
    nudgeMaxRetry: 1000,
  }

Additional player setup steps

No response

Checklist

• The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• The issue occurs in the stable client (latest release) on https://hlsjs.video-dev.org/demo and not just on my page
• The issue occurs in the latest client (main branch) on https://hlsjs-dev.video-dev.org/demo and not just on my page
• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

1. Open the webpage
2. Click to play

Expected behaviour

Play normally

What actually happened?

Interface black screen

Console output

{
    "type": "mediaError",
    "details": "fragParsingError",
    "fatal": false,
    "error": {},
    "frag": {
        "_byteRange": null,
        "_url": "https://schoolfighting.oss-cn-hangzhou.aliyuncs.com/test/teshh265/6719b8152cdfb78408001bb7(1).ts",
        "_stats": {
            "aborted": false,
            "loaded": 0,
            "retry": 0,
            "total": 229360,
            "chunkCount": 1,
            "bwEstimate": 431649.13465134724,
            "loading": {
                "start": 28576.299999952316,
                "first": 28577.600000023842,
                "end": 28578.299999952316
            },
            "parsing": {
                "start": 28578.399999976158,
                "end": 28589.899999976158
            },
            "buffering": {
                "start": 0,
                "first": 0,
                "end": 28590.199999928474
            }
        },
        "_streams": {
            "audio": null,
            "video": null,
            "audiovideo": null
        },
        "base": {
            "url": "https://schoolfighting.oss-cn-hangzhou.aliyuncs.com/test/teshh265/output.m3u8"
        },
        "relurl": "6719b8152cdfb78408001bb7(1).ts",
        "_decryptdata": null,
        "_programDateTime": null,
        "_ref": null,
        "rawProgramDateTime": null,
        "tagList": [
            [
                "INF",
                "10"
            ]
        ],
        "duration": 10,
        "sn": 2,
        "type": "main",
        "loader": null,
        "keyLoader": null,
        "level": 0,
        "cc": 0,
        "start": 20,
        "playlistOffset": 20,
        "bitrateTest": false,
        "title": null,
        "initSegment": null,
        "gap": true,
        "urlId": 0
    },
    "reason": "Found no media in msn 2 of level \"https://schoolfighting.oss-cn-hangzhou.aliyuncs.com/test/teshh265/output.m3u8\"",
    "errorAction": {
        "action": 2,
        "flags": 1,
        "retryConfig": {
            "maxNumRetry": 6,
            "retryDelayMs": 1000,
            "maxRetryDelayMs": 8000
        },
        "retryCount": 2,
        "resolved": true
    }
}

Chrome media internals output

No response

[XiaoxiRao]

I tested this issue on the master branch.

[robwalch]

HLS.js does not support everything that VLC does. Please provide a better reason for triaging the included media.

The included error details suggest that the segments do not include media in a format supported by HLS.js. How was it encoded and packaged?

[XiaoxiRao]

HLS.js does not support everything that VLC does. Please provide a better reason for triaging the included media.

The included error details suggest that the segments do not include media in a format supported by HLS.js. How was it encoded and packaged?

Thank you for your response. I believe the primary reason the stream fails to play is due to data loss in the video stream. However, I am not entirely sure about the root cause of the data loss. My role is simply to work with the given stream and ensure it plays correctly.

I did notice that when I commented out the line this.resetTransmuxer(); in the updateLevelTiming function within the base-stream-controller.ts class, the stream was able to play normally in hls.js. This leads me to believe that hls.js has the capability to handle this type of stream. However, its compatibility with certain erroneous streams could still be improved.

[robwalch]

HEVC in M2TS is a community contributed feature made up of these PRs (CC @devoldemar):

• Feature/mpeg ts hevc #5847
• Fix extraction of NALu overflowing PES packet #6268
• Fix PPS array growth in HEVC TS parser #6724

Provide more information about the asset encoding and packaging, including any identifiable out-of-spec features.

Try triaging the issue further in HLS.js. Why does HLS.js find no media in "6719b8152cdfb78408001bb7(1).ts" (set "enableWorker": false to debug through tsdemuxer)? It looks to me like this is an issue in the HevcVideoParser - The SPS and PPS are discarded when the VPS is not encountered first (skips everything in block requiringtrack.params to be an object):

hls.js/src/demux/video/hevc-video-parser.ts

Lines 121 to 130 in 1dbc3ce

	// VPS
	case 32:
	push = true;
	if (!track.vps) {
	const config = this.readVPS(unit.data);
	track.params = { ...config };
	this.initVPS = unit.data;
	}
	track.vps = [unit.data];
	break;

should probably be:

        // VPS
        case 32:
          push = true;
          track.params ||= {};
          Object.assign(track.params, this.readVPS(unit.data));
          this.initVPS = unit.data;
          track.vps = [unit.data];
          break;

Then

hls.js/src/demux/video/hevc-video-parser.ts

Lines 132 to 146 in 1dbc3ce

	// SPS
	case 33:
	push = true;
	spsfound = true;
	if (typeof track.params === 'object') {
	if (
	track.vps !== undefined &&
	track.vps[0] !== this.initVPS &&
	track.sps !== undefined &&
	!this.matchSPS(track.sps[0], unit.data)
	) {
	this.initVPS = track.vps[0];
	track.sps = track.pps = undefined;
	}
	if (!track.sps) {

should always update track params (adding them if necessary)

        // SPS
        case 33:
          push = true;
          spsfound = true;
          if (
            track.vps !== undefined &&
            track.vps[0] !== this.initVPS &&
            track.sps !== undefined &&
            !this.matchSPS(track.sps[0], unit.data)
          ) {
            this.initVPS = track.vps[0];
            track.sps = track.pps = undefined;
          }
          if (!track.sps) {
            track.params ||= {};

And finally

hls.js/src/demux/video/hevc-video-parser.ts

Lines 171 to 175 in 1dbc3ce

	// PPS
	case 34:
	push = true;
	if (typeof track.params === 'object') {
	if (!track.pps) {

should also always update track params (adding them if necessary)

        // PPS
        case 34:
          push = true;
          if (!track.pps) {
            track.params ||= {};