Try to re-factor the integration-boot.js file (and related code)

[Snuffleupagus]

For consistency/maintainability reasons, it would probably be helpful if the new integration-boot.js (and related code) would look more like the existing code used with the unit/font-tests. To that end, I'd suggest the following (not an exhaustive list):

• Move, and re-name, the new file to test/integration/jasmine-boot.js and possibly adding a new test/integration/integration_test.html file as well as part of these changes.
• Re-factor the integration-boot.js, and any related code in test/test.js, file to follow the same general pattern as used with e.g. https://github.com/mozilla/pdf.js/blob/master/test/unit/jasmine-boot.js and https://github.com/mozilla/pdf.js/blob/master/test/unit/unit_test.html respectively https://github.com/mozilla/pdf.js/blob/master/test/font/jasmine-boot.js and https://github.com/mozilla/pdf.js/blob/master/test/font/font_test.html
• Try to use the same result-reporting mechanism as used with e.g. the unit/font-tests.
• Actually include the browser name in the output, since that's often helpful; note that the current output doesn't include this information:

  pdf.js/test/integration-boot.js

  Lines 35 to 40 in 00b4f86

  	if (result.failedExpectations.length > 0) {
  	++results.failures;
  	console.log(`TEST-UNEXPECTED-FAIL | ${result.description}`);
  	} else {
  	console.log(`TEST-PASSED | ${result.description}`);
  	}

  whereas the unit/font-tests does:

  pdf.js/test/test.js

  Lines 797 to 798 in 00b4f86

  	var message =
  	data.status + " | " + data.description + " | in " + session.name;

[aashrafh]

Can I work on it?

[timvandermeij]

Sure, feel free to work on this.

[simran212530]

Hey,

Can I work on this?

[Snuffleupagus]

This issue is unlikely to be a good beginner bug, if you're not already at least somewhat familiar with the PDF.js code-base and its test-suites.

[timvandermeij]

I have looked into this, but unfortunately this is more difficult than I had hoped. The main issue is that the unit/font tests run in the browser itself whereas the integration tests run in Node.js and only control the browsers under test. This means that Jasmine also only runs once in Node.js instead of twice in each browser. This makes it very difficult to use the same general structure or reporting mechanism because those rely on being run inside the browser under test.

One way in which I can imagine this working is if we run Puppeteer inside the individual browsers. This should be possible according to https://pptr.dev/guides/running-puppeteer-in-the-browser, but in quickly trying this I haven't gotten it to work yet. Even if this works though we'll have to find a solution for tests that currently rely on Node.js packages (first-party like os or third-party like pngjs) because those aren't available in the browser context by default. Another complication are tests that spawn new browser themselves; we have one or two of those and I don't really see how that should work if Node.js is not the orchestrator here.

I therefore think the path of making the browsers orchestrate the tests here isn't going to fly. What might work instead is to let Node.js remain the orchestrator, but find some way to parametrize the tests for Jasmine so that it can report on the individual browsers inside of each test running the logic per browser inside the test. This would basically mean moving the pages.map calls outside of the test somehow, but given that the pages are built up dynamically in beforeAll blocks this also doesn't really seem trivial.

I think mainly having the browser name in the test output would be most helpful for e.g. debugging intermittents, but I don't yet see how we can easily achieve that. If there are ideas about this I'd love to hear about it. I did make PR #19254 for the first point mentioned here to at least somewhat improve consistency.