Callback POST to URL

[gnarf]

Adds a --callback-url and --callback-header CLI param - causes each test run completion to send an http post

TODOs:

• Add documentation for CLI / environment params
• Add tests for callback post
• (Maybe? Better tests than these snapshot ones?)

[jugglinmike]

This looks directionally good to me. Beyond the documentation you have in mind, I'd also appreciate some tests.

Although time has proven the entire "agent"/"host" abstraction to be superfluous, I wonder if it's worth honoring here. That is to say: did you consider reporting from the host process? I think it might help with consistency (where the host's responsibilities are limited to collecting data), and it ought to also reduce the amount of plumbing (since we won't have to forward the new parameters across the boundary).

[jugglinmike]

Let's store this as _capabilities and expose an async getCapabilities method. That'll increase parity with the WebDriver class and also make the call site a little cleaner.

[gnarf]

The WebDriver getCapabilities() ends up giving back an object that then has get(keyName) method, would you want me to kind of mirror that same API, or just return the whole object at that point?

  async getCapabilities() {
    const capabilities = await this.webDriver.getCapabilities();
    const browserName = capabilities.get('browserName');
    const browserVersion = capabilities.get('browserVersion');
    await this.atDriver.ready;
    const { atName, atVersion, platformName } = this.atDriver.capabilities;
    return { atName, atVersion, browserName, browserVersion, platformName };
  }

note the browserName and browserVersion = capabilities.get() calls

The current at-driver we are using (PAC's) does not support getting the capabilities back at any time other than the response to the session.new, so we still need to store it here on the connection for now.

[jugglinmike]

Since this is intended to hold a secret, it seems possible that the value will include the colon character. However, if we split like this, we'll end up truncating the secret in those cases.

[gnarf]

I wasn't 100% sure we wanted to use the header named Secret: - so put in a single split here, two ways to solve it:

1. Hard code the header name (not a terrible option, just want to know for sure what the aria-at-app is going to be looking for)
2. Change the split to only match the first /:\s+/ in the header.

[gnarf]

Although time has proven the entire "agent"/"host" abstraction to be superfluous, I wonder if it's worth honoring here. That is to say: did you consider reporting from the host process? I think it might help with consistency (where the host's responsibilities are limited to collecting data), and it ought to also reduce the amount of plumbing (since we won't have to forward the new parameters across the boundary).

I had not considered posting from the host side of the puzzle, but the advantages you mention here are making me think it is probably the better option overall anyway! Will convert it over.

[gnarf]

Added some snapshot testing of the callback functionality in 39d5ac6 - doing much more than this may require new test harness surface for things (all tests are currently just snapshot wrappers around exec)

[gnarf]

Bringing this one out of "DRAFT" status - would be happy to land as is, or go a little deeper with testing if @jugglinmike thinks it'd be worth the effort.

[jugglinmike]

This seems like a job for yargs' .env method.

[gnarf]

28e292c

[jugglinmike]

Let's make this value distinct so that the tests demonstrate we're not mishandling the expectation value, e.g. mocked output for assertion "${assertion.expectation}"

[gnarf]

28e292c

[gnarf]

Tested with the env('ARIA_AT') change in https://github.com/bocoup/aria-at-gh-actions-helper/actions/runs/6498859092/job/17651049573

Report from the "requestbin" callback url:

{
    "method": "POST",
    "path": "/",
    "query": {},
    "client_ip": "13.91.223.73",
    "url": "https://eorkylzwxzzltt.m.pipedream.net/",
    "headers": {
        "host": "eorkylzwxzzltt.m.pipedream.net",
        "content-length": "196",
        "accept": "*/*",
        "accept-encoding": "gzip, deflate, br",
        "content-type": "application/json",
        "secret": "42",
        "user-agent": "node-fetch"
    },
    "body": {
        "testId": 1,
        "capabilities": {
            "atName": "NVDA",
            "atVersion": "2023.2",
            "browserName": "chrome",
            "browserVersion": "117.0.5938.132",
            "platformName": "windows"
        },
        "responses": [
            " alert\n Hello",
            " alert\n Hello"
        ]
    }
}

[jugglinmike]

Nice work!