Add Playwright

[davorinrusevljan]

Description

Adding Playwright tests

When running from dev container, VSCode should have Playwright extension installed,
and tests can be run directly from it, under testing icon on the far left.

Currently in dev container tests are run in
headless mode, so do not try Show browser option in VS Code, unless you wish
to restart machine anyways.

Currently there are 3 Playwright configurations available, they differ in which server fastagency
server is used, and where are corresponding playwrigth tests located:

• llm-sans (no llms in play just mesop ui testing)
• llm (end to end testing with llms in play)
• gunicorn (run fastagency with gunicorn)

When switching playwright configuration in VS Code, execute Run global teardown and Run global setup,
since they do not seem to be executed by configuration switch.

Directory e2e/llm-sans contain tests that do not involve llms, and typically run faster,
but concentrate on testing Mesop functionality only.

Directory e2e/llm contain tests (actually currently only one) that do involve llms, and test
integration of Mesop with Providers running Autogen

To run non llm tests select playwright.llm-sans.config.ts in VS Code,
or in shell run:

npx playwright test -c "playwright.llm-sans.config.ts"

To run tests that require llms select playwright.llm.config.ts in VS Code,
or in shell run:

npx playwright test -c "playwright.llm.config.ts"

Note that llms tests take longer to execute, and sometimes they do timeout.

To run non llm tests with gunicorn select playwright.gunicorn.config.ts in VS Code,
or in shell run:

npx playwright test -c "playwright.gunicorn.config.ts"

note that gunicorn for some reason takes some time to spin-up, and this can also cause
tests to run slower, and sometimes even to error due to time out.

Github test workflow integration

• Currently llm-sans set of test is run inside test workflow.
• Unless for Python 3.9 which is not supported
• Code coverage data is not collected

Administrative info

• Documentation (typos, code examples, or any documentation updates)
• Bug fix (a non-breaking change that resolves an issue)
• [ x] New feature (a non-breaking change that adds functionality)
• Breaking change (a fix or feature that would disrupt existing functionality)
• This change requires a documentation update

Checklist

• I have conducted a self-review of my own code
• I have made the necessary changes to the documentation
• My changes do not generate any new warnings
• I have added tests to validate the effectiveness of my fix or the functionality of my new feature
• I have included code examples to illustrate the modifications

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

see 2 files with indirect coverage changes