Agenda 2024-12-17

[jgraham]

• Score updates?
• Continuation of investigation next year
  • Scheduling

[jcscottiii]

Notes

• Updates to scores
  • Browser updates:
    • Apple - No results now
    • Google - Still creating issues for failures. James S. is investigating the failures before sending bugs out.
    • Mozilla - No updates. Will look into the intermittent issue with runs
  • Triaging differences
    • Spreadsheet for one-off runs
    • Colab notebook on finding differences between Desktop and Mobile
  • Conclusion
    • Firefox results are within 3% of desktop, Chrome is lower suggesting infra issues somewhere
    • On Safari Viewport size presents a problem for reftests, can also affect testharness tests
    • Shouldn't be a problem if we have mobile-specific focus areas where we pick tests that can work with the mobile viewport sizes we actually have
• Next year
  • Seems to be a broad desire to continue the focus area
  • A question is about focus: how much work will be infra vs developing new testing APIs for viewport features. e.g. what happens when the URL bar appears in browsers where it's not fixed.
  • One proposal also touched on in-app browser etc. which are more problematic to test in an automatic way since the behaviour might depend on the container app.
  • Should we continue this meeting next year?
    • Yes. Is the time good? Maybe the best option we have.
    • Attach this document to this invite and the infra invite.