Skip to content

Commit

Permalink
[test-credential] Credential relay for browser tests (Azure#29616)
Browse files Browse the repository at this point in the history
### Packages impacted by this PR

- `@azure-tools/dev-tool`
- `@azure-tools/test-credential`

### Describe the problem that is addressed by this PR

We want to move to user auth and `DefaultAzureCredential` to
authenticate our tests, but `DefaultAzureCredential` does not work in
the browser, blocking this transition.

This PR proposes a solution -- create a short-lived, local-only, server
which can create tokens in Node using DefaultAzureCredential. In the
browser, createTestCredential provides a credential implementation which
calls this server whenever it needs a token, and the server relays the
request to DefaultAzureCredential.

Here's what's changed:

- `test-credential` now returns a credential which requests tokens from
a local web endpoint when running browser tests against a live service.
- `dev-tool` contains an implementation of this web endpoint. The
dev-tool scripts used to run browser tests now start this server
automatically when running browser tests. A separate command to start
the server is also provided for pipelines.
- Update live test pipelines to start the server when running browser
tests.

### Provide a list of related PRs _(if any)_

- Draft version with loads of commits and `/azp run` spam: Azure#29581 
- Harsha's earlier PR to enable DefaultAzureCredential:
Azure#29577
  • Loading branch information
timovv authored May 9, 2024
1 parent 685c5c5 commit c66cad2
Show file tree
Hide file tree
Showing 14 changed files with 787 additions and 408 deletions.
777 changes: 391 additions & 386 deletions common/config/rush/pnpm-lock.yaml

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions common/tools/dev-tool/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
"dependencies": {
"@_ts/min": "npm:typescript@~4.2.4",
"@_ts/max": "npm:typescript@latest",
"@azure/identity": "^4.2.0",
"@rollup/plugin-commonjs": "^25.0.7",
"@rollup/plugin-json": "^6.0.1",
"@rollup/plugin-multi-entry": "^6.0.1",
Expand All @@ -55,6 +56,7 @@
"decompress": "^4.2.1",
"dotenv": "^16.0.0",
"env-paths": "^2.2.1",
"express": "^4.19.2",
"fs-extra": "^11.2.0",
"minimist": "^1.2.8",
"prettier": "^3.2.5",
Expand All @@ -73,6 +75,8 @@
"@microsoft/api-extractor": "^7.42.3",
"@types/archiver": "~6.0.2",
"@types/decompress": "^4.2.7",
"@types/express": "^4.17.21",
"@types/express-serve-static-core": "4.19.0",
"@types/fs-extra": "^11.0.4",
"@types/minimist": "^1.2.5",
"@types/node": "^18.0.0",
Expand Down
1 change: 1 addition & 0 deletions common/tools/dev-tool/src/commands/run/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export default subCommand(commandInfo, {
"extract-api": () => import("./extract-api"),
bundle: () => import("./bundle"),
"build-test": () => import("./build-test"),
"start-browser-relay": () => import("./startBrowserRelay"),

// "vendored" is a special command that passes through execution to dev-tool's own commands
vendored: () => import("./vendored"),
Expand Down
32 changes: 32 additions & 0 deletions common/tools/dev-tool/src/commands/run/startBrowserRelay.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license

import { leafCommand, makeCommandInfo } from "../../framework/command";

import { startRelayServer } from "../../util/browserRelayServer";

export const commandInfo = makeCommandInfo(
"start-browser-relay",
"Start the browser credential relay, used for authenticating browser tests.",
{
listenHost: {
kind: "string",
default: "localhost",
description: "Host to listen on",
},
port: {
kind: "string",
default: "4895",
description: "Port to listen on",
},
},
);

export default leafCommand(commandInfo, async (options) => {
startRelayServer({
listenHost: options.listenHost,
port: Number(options.port),
});

return true;
});
25 changes: 21 additions & 4 deletions common/tools/dev-tool/src/commands/run/testBrowser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,36 @@

import { leafCommand, makeCommandInfo } from "../../framework/command";
import { isModuleProject } from "../../util/resolveProject";
import { shouldStartRelay, startRelayServer } from "../../util/browserRelayServer";
import { runTestsWithProxyTool } from "../../util/testUtils";

export const commandInfo = makeCommandInfo(
"test:browser",
"runs the browser tests using karma with the default and the provided options; starts the proxy-tool in record and playback modes",
{
"relay-server": {
description: "Start the relay server for browser credentials",
kind: "boolean",
default: true,
},
},
);

export default leafCommand(commandInfo, async (options) => {
const karmaArgs = options["--"]?.length
? options["--"].join(" ")
: `${(await isModuleProject()) ? "karma.conf.cjs" : ""} --single-run`;
return runTestsWithProxyTool({
command: `karma start ${karmaArgs}`,
name: "browser-tests",
});

const stopRelay =
options["relay-server"] && (await shouldStartRelay()) ? startRelayServer() : undefined;

try {
const result = await runTestsWithProxyTool({
command: `karma start ${karmaArgs}`,
name: "browser-tests",
});
return result;
} finally {
stopRelay?.();
}
});
29 changes: 23 additions & 6 deletions common/tools/dev-tool/src/commands/run/testVitest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import concurrently from "concurrently";
import { leafCommand, makeCommandInfo } from "../../framework/command";
import { runTestsWithProxyTool } from "../../util/testUtils";
import { createPrinter } from "../../util/printer";
import { shouldStartRelay, startRelayServer } from "../../util/browserRelayServer";

const log = createPrinter("test:vitest");

Expand All @@ -24,6 +25,13 @@ export const commandInfo = makeCommandInfo(
default: false,
description: "whether to use browser to run tests",
},
"relay-server": {
shortName: "rs",
description:
"Start the relay server for browser credentials. Only takes effect if using browser to test.",
kind: "boolean",
default: true,
},
},
);

Expand Down Expand Up @@ -64,11 +72,20 @@ export default leafCommand(commandInfo, async (options) => {
name: "vitest",
};

if (options["test-proxy"]) {
return runTestsWithProxyTool(command);
}
const stopRelayServer =
options.browser && options["relay-server"] && (await shouldStartRelay())
? startRelayServer()
: undefined;

try {
if (options["test-proxy"]) {
return await runTestsWithProxyTool(command);
}

log.info("Running vitest without test-proxy");
await concurrently([command]).result;
return true;
log.info("Running vitest without test-proxy");
await concurrently([command]).result;
return true;
} finally {
stopRelayServer?.();
}
});
128 changes: 128 additions & 0 deletions common/tools/dev-tool/src/util/browserRelayServer.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

import express from "express";
import type { Express } from "express-serve-static-core";
import { DefaultAzureCredential, type TokenCredential } from "@azure/identity";
import { randomUUID } from "node:crypto";
import { createPrinter } from "./printer";

const printer = createPrinter("browser-relay");

export interface TestCredentialServerOptions {
/**
* Port to listen on. Defaults to 4895.
*/
port?: number;

/**
* Host to listen on. Defaults to `localhost`. Caution: do not expose this server to the network.
*/
listenHost?: string;
}

function isValidScopes(scopes: unknown): scopes is string | string[] {
return (
typeof scopes === "string" ||
(Array.isArray(scopes) && scopes.every((s) => typeof s === "string"))
);
}

function buildServer(app: Express) {
const credentials: Record<string, TokenCredential> = {};

app.use(express.json());
app.use((_req, res, next) => {
res.set("Access-Control-Allow-Methods", "GET, PUT");
res.set("Access-Control-Allow-Origin", "*");
next();
});

app.get("/health", (_req, res) => {
res.status(204).send();
});

// Endpoint for creating a new credential
app.put("/credential", (req, res) => {
const id = randomUUID();
try {
const cred = new DefaultAzureCredential(req.body);
credentials[id] = cred;
res.status(201).send({ id });
} catch (error: unknown) {
res.status(400).send({ error });
return;
}
});

// Endpoint for getting a token using a pre-created credential
app.get("/credential/:id/token", async (req, res) => {
const credential = credentials[req.params.id];
if (!credential) {
res.status(404).send({ error: "Credential not found, create a credential first" });
return;
}

const scopes = req.query["scopes"];

if (!isValidScopes(scopes)) {
res.status(400).send({ error: "Scopes must be provided" });
return;
}

const options = JSON.parse(req.query["options"]?.toString() ?? "{}");

try {
const token = await credential.getToken(scopes, options);
res.status(200).send(token);
} catch (error: unknown) {
res.status(400).send({ error });
}
});
}

async function isRelayAlive(options: TestCredentialServerOptions = {}): Promise<boolean> {
try {
const res = await fetch(
`http://${options.listenHost ?? "localhost"}:${options.port ?? 4895}/health`,
);

if (res.ok) {
printer("Browser relay is already alive");
return true;
} else {
throw new Error(`Browser relay responded with an error: ${await res.text()}`);
}
} catch (e) {
printer("Browser relay is not yet alive");
return false;
}
}

export async function shouldStartRelay(
options: TestCredentialServerOptions = {},
): Promise<boolean> {
const testMode = (process.env.TEST_MODE ?? "playback").toLowerCase();
if (testMode !== "record" && testMode !== "live") {
printer("Not in record or live mode; not starting relay");
return false;
}

return !(await isRelayAlive(options));
}

/**
* Create and start the relay server used by test credential to provide credentials to the browser tests.
* @param options Options for the relay server.
* @returns A callback which, when called, will stop the server.
*/
export function startRelayServer(options: TestCredentialServerOptions = {}): () => void {
const app = express();
buildServer(app);

const { listenHost = "localhost", port = 4895 } = options;

printer(`Starting browser relay on http://${listenHost}:${port}/`);
const server = app.listen(port, listenHost);
return () => server.close();
}
19 changes: 19 additions & 0 deletions eng/pipelines/templates/jobs/live.tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,25 @@ jobs:
workingDirectory: $(Build.SourcesDirectory)/eng/tools/eng-package-utils
displayName: "Get package path"
- pwsh: |
Start-Process "node" -ArgumentList "launch.js run start-browser-relay" -NoNewWindow -WorkingDirectory "$(Build.SourcesDirectory)/common/tools/dev-tool"
for ($i = 0; $i -lt 10; $i++) {
try {
Invoke-WebRequest -Uri "http://localhost:4895/health" | Out-Null
exit 0
} catch {
Write-Warning "Failed to successfully connect to browser credential relay. Retrying..."
Start-Sleep 6
}
}
Write-Error "Could not connect to browser credential relay."
exit 1
displayName: "Start browser credential relay"
condition: and(succeeded(), eq(variables['TestType'], 'browser'))
env:
TEST_MODE: "live"
${{ insert }}: ${{ parameters.EnvVars }}
- template: ../steps/use-node-test-version.yml

- script: |
Expand Down
6 changes: 5 additions & 1 deletion rush.json
Original file line number Diff line number Diff line change
Expand Up @@ -573,7 +573,11 @@
{
"packageName": "@azure/dev-tool",
"projectFolder": "common/tools/dev-tool",
"versionPolicyName": "utility"
"versionPolicyName": "utility",
// Add Identity to decoupledLocalDependencies so that dev-tool uses the package from npm, avoiding a cyclic dependency.
"decoupledLocalDependencies": [
"@azure/identity"
]
},
{
"packageName": "@azure/eventgrid",
Expand Down
6 changes: 5 additions & 1 deletion sdk/test-utils/test-credential/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@

Updates the `createTestCredential` method to consume `DefaultAzureCredential` instead of `ClientSecretCredential` in order to offer autonomy to the devs and to move away from client secrets in environment varaibles.

- `NoOpCredential` is offered for playback and `DefaultAzureCredential` in record/live modes.
- `NoOpCredential` is offered for playback.
- In record and live modes:
- `DefaultAzureCredential` is offered in Node.
- In the browser, a custom credential is provided that fetches tokens from a locally running Node server. The server is provided in the dev-tool package, and must be running while the browser
tests are running for the credential to work. The server uses `DefaultAzureCredential` on the host machine to generate tokens.
- [`User Auth` and `Auth via development tools`](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/identity/identity#authenticate-users) are preferred in record mode to record the tests.

## 2.0.0 (2024-04-09)
Expand Down
4 changes: 3 additions & 1 deletion sdk/test-utils/test-credential/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,9 @@
"dependencies": {
"@azure/core-auth": "^1.3.2",
"@azure-tools/test-recorder": "^4.0.0",
"@azure/identity": "^4.0.1"
"@azure/identity": "^4.0.1",
"@azure/core-util": "^1.9.1",
"tslib": "^2.6.2"
},
"devDependencies": {
"@azure/dev-tool": "^1.0.0",
Expand Down
10 changes: 9 additions & 1 deletion sdk/test-utils/test-credential/review/test-credential.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,15 @@ import { DefaultAzureCredentialResourceIdOptions } from '@azure/identity';
import { TokenCredential } from '@azure/core-auth';

// @public
export function createTestCredential(tokenCredentialOptions?: DefaultAzureCredentialClientIdOptions | DefaultAzureCredentialResourceIdOptions | DefaultAzureCredentialOptions): TokenCredential;
export function createTestCredential(tokenCredentialOptions?: CreateTestCredentialOptions): TokenCredential;

// @public
export type CreateTestCredentialOptions = DefaultAzureCredentialCombinedOptions & {
browserRelayServerUrl?: string;
};

// @public
export type DefaultAzureCredentialCombinedOptions = DefaultAzureCredentialClientIdOptions | DefaultAzureCredentialResourceIdOptions | DefaultAzureCredentialOptions;

// @public
export class NoOpCredential implements TokenCredential {
Expand Down
Loading

0 comments on commit c66cad2

Please sign in to comment.