Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase with actions/runner main. #3

Merged
merged 19 commits into from
Nov 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
2c03d74
Handle runner not found (#3536)
ericsciple Nov 5, 2024
6ef5803
Publish job telemetry to run-service. (#3545)
TingluoHuang Nov 8, 2024
d37a7ae
Fetch repo-level runner groups from API in v2 flow (#3546)
lucavallin Nov 12, 2024
d16fb2c
Allow runner to check service connection in background. (#3542)
TingluoHuang Nov 12, 2024
694ae12
Expose ENV for cache service v2. (#3548)
TingluoHuang Nov 12, 2024
1e1f784
Update runner docker image. (#3511)
TingluoHuang Nov 12, 2024
59f2be2
Bump Azure.Storage.Blobs from 12.19.1 to 12.23.0 in /src (#3549)
dependabot[bot] Nov 12, 2024
074d9c0
fix dotnet-upgrade.yml to print right version (#3550)
TingluoHuang Nov 12, 2024
6c94f78
Upgrade dotnet sdk to v8.0.404 (#3552)
github-actions[bot] Nov 12, 2024
27580ef
Configure dependabot to check github-actions updates (#3333)
Goooler Nov 13, 2024
4700649
Bump actions/checkout from 3 to 4 (#3556)
dependabot[bot] Nov 13, 2024
bc18743
Bump version to 2.320.1 to test new arm builds
satmandu Oct 23, 2024
2399521
Add note about changed version.
satmandu Oct 23, 2024
02a2ea0
Adjust workflow to build artifacts.
satmandu Oct 23, 2024
2ee726d
Update to .Net 9.0 to fix ARM32 Y2038 breakage.
satmandu Oct 23, 2024
25f8032
Bump versions of packages to avoid NU1903 NU1902 warnings.
satmandu Oct 23, 2024
e0961a9
Update to .Net 9.0 Final.
satmandu Nov 13, 2024
d9896cb
More updates to use .Net 9.0 and the latest Ubuntu LTS (Ubuntu 24.04/…
satmandu Nov 13, 2024
14d933e
Merge branch 'main' into arm_build_test
satmandu Nov 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "Actions Runner Devcontainer",
"image": "mcr.microsoft.com/devcontainers/base:focal",
"image": "mcr.microsoft.com/devcontainers/base:noble",
"features": {
"ghcr.io/devcontainers/features/docker-in-docker:1": {},
"ghcr.io/devcontainers/features/dotnet": {
Expand Down
5 changes: 5 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ updates:
schedule:
interval: "daily"
target-branch: "main"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
target-branch: "main"
- package-ecosystem: "nuget"
directory: "/src"
schedule:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ jobs:

runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

# Build runner layout
- name: Build & Layout Release
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/dotnet-upgrade.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
DOTNET_CURRENT_MAJOR_MINOR_VERSION: ${{ steps.fetch_current_version.outputs.DOTNET_CURRENT_MAJOR_MINOR_VERSION }}
steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Get current major minor version
id: fetch_current_version
shell: bash
Expand Down Expand Up @@ -51,7 +51,7 @@ jobs:
run: echo "::error links::feature/dotnet-sdk-upgrade${{ steps.fetch_latest_version.outputs.DOTNET_LATEST_MAJOR_MINOR_PATCH_VERSION }} https://github.com/actions/runner/tree/feature/dotnet-sdk-upgrade${{ steps.fetch_latest_version.outputs.DOTNET_LATEST_MAJOR_MINOR_PATCH_VERSION }}::Branch feature/dotnetsdk-upgrade/${{ steps.fetch_latest_version.outputs.DOTNET_LATEST_MAJOR_MINOR_PATCH_VERSION }} already exists. Please take a look and delete that branch if you wish to recreate"
- name: Create a warning annotation if no need to update
if: ${{ steps.fetch_latest_version.outputs.SHOULD_UPDATE == 0 && steps.fetch_latest_version.outputs.BRANCH_EXISTS == 0 }}
run: echo "::warning ::Latest DotNet SDK patch is ${{ steps.fetch_latest_version.outputs.DOTNET_LATEST_MAJOR_MINOR_PATCH_VERSION }}, and we are on ${{ steps.fetch_latest_version.outputs.DOTNET_CURRENT_MAJOR_MINOR_PATCH_VERSION }}. No need to update"
run: echo "::warning ::Latest DotNet SDK patch is ${{ steps.fetch_latest_version.outputs.DOTNET_LATEST_MAJOR_MINOR_PATCH_VERSION }}, and we are on ${{ steps.fetch_current_version.outputs.DOTNET_CURRENT_MAJOR_MINOR_PATCH_VERSION }}. No need to update"
- name: Update patch version
if: ${{ steps.fetch_latest_version.outputs.SHOULD_UPDATE == 1 && steps.fetch_latest_version.outputs.BRANCH_EXISTS == 0 }}
shell: bash
Expand Down Expand Up @@ -89,7 +89,7 @@ jobs:
if: ${{ needs.dotnet-update.outputs.SHOULD_UPDATE == 1 && needs.dotnet-update.outputs.BRANCH_EXISTS == 0 }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
ref: feature/dotnetsdk-upgrade/${{ needs.dotnet-update.outputs.DOTNET_LATEST_MAJOR_MINOR_PATCH_VERSION }}
- name: Create Pull Request
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Compute image version
id: image
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
if: startsWith(github.ref, 'refs/heads/releases/') || github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

# Make sure ./releaseVersion match ./src/runnerversion
# Query GitHub release ensure version is not used
Expand Down Expand Up @@ -87,7 +87,7 @@ jobs:

runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

# Build runner layout
- name: Build & Layout Release
Expand Down Expand Up @@ -130,7 +130,7 @@ jobs:
runs-on: ubuntu-latest
steps:

- uses: actions/checkout@v3
- uses: actions/checkout@v4

# Download runner package tar.gz/zip produced by 'build' job
- name: Download Artifact (win-x64)
Expand Down Expand Up @@ -296,7 +296,7 @@ jobs:
IMAGE_NAME: ${{ github.repository_owner }}/actions-runner
steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Compute image version
id: image
Expand Down
2 changes: 1 addition & 1 deletion docs/checks/sslcert.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ To let the runner trusts your CA certificate, you will need to:
- macOS: ![trust ca cert](./../res/macOStrustCA.gif)
- Linux: Refer to the distribution documentation
1. RedHat: https://www.redhat.com/sysadmin/ca-certificates-cli
2. Ubuntu: http://manpages.ubuntu.com/manpages/focal/man8/update-ca-certificates.8.html
2. Ubuntu: https://manpages.ubuntu.com/manpages/noble/man8/update-ca-certificates.8.html
3. Google search: "trust ca certificate on [linux distribution]"
4. If all approaches failed, set environment variable `SSL_CERT_FILE` to the CA bundle `.pem` file we get.
> To verify cert gets installed properly on Linux, you can try use `curl -v https://sitewithsslissue.com` and `pwsh -Command \"Invoke-WebRequest -Uri https://sitewithsslissue.com\"`
Expand Down
11 changes: 6 additions & 5 deletions images/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Source: https://github.com/dotnet/dotnet-docker
FROM mcr.microsoft.com/dotnet/runtime-deps:6.0-jammy as build
FROM mcr.microsoft.com/dotnet/runtime-deps:9.0-noble as build

ARG TARGETOS
ARG TARGETARCH
ARG RUNNER_VERSION
ARG RUNNER_CONTAINER_HOOKS_VERSION=0.6.1
ARG DOCKER_VERSION=27.1.1
ARG BUILDX_VERSION=0.16.2
ARG DOCKER_VERSION=27.3.1
ARG BUILDX_VERSION=0.18.0

RUN apt update -y && apt install curl unzip -y

Expand All @@ -32,7 +32,7 @@ RUN export RUNNER_ARCH=${TARGETARCH} \
"https://github.com/docker/buildx/releases/download/v${BUILDX_VERSION}/buildx-v${BUILDX_VERSION}.linux-${TARGETARCH}" \
&& chmod +x /usr/local/lib/docker/cli-plugins/docker-buildx

FROM mcr.microsoft.com/dotnet/runtime-deps:6.0-jammy
FROM mcr.microsoft.com/dotnet/runtime-deps:9.0-noble

ENV DEBIAN_FRONTEND=noninteractive
ENV RUNNER_MANUALLY_TRAP_SIG=1
Expand All @@ -47,7 +47,8 @@ RUN apt update -y \
# Configure git-core/ppa based on guidance here: https://git-scm.com/download/linux
RUN add-apt-repository ppa:git-core/ppa \
&& apt update -y \
&& apt install -y --no-install-recommends git
&& apt install -y git \
&& rm -rf /var/lib/apt/lists/*

RUN adduser --disabled-password --gecos "" --uid 1001 runner \
&& groupadd docker --gid 123 \
Expand Down
2 changes: 1 addition & 1 deletion releaseNote.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## What's Changed

- Bump version to active builds for arm with .Net 8.0
- Bump version to active builds for arm with .Net 9.0
- Adding Snapshot additional mapping tokens https://github.com/actions/runner/pull/3468
- Create launch httpclient using the right handler and setting https://github.com/actions/runner/pull/3476
- Fix missing default user-agent for jitconfig runner https://github.com/actions/runner/pull/3473
Expand Down
3 changes: 2 additions & 1 deletion src/Runner.Common/BrokerServer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
using GitHub.DistributedTask.WebApi;
using GitHub.Runner.Sdk;
using GitHub.Services.Common;
using GitHub.Services.WebApi;
using Sdk.RSWebApi.Contracts;
using Sdk.WebApi.WebApi.RawClient;

Expand Down Expand Up @@ -92,7 +93,7 @@ public Task ForceRefreshConnection(VssCredentials credentials)

public bool ShouldRetryException(Exception ex)
{
if (ex is AccessDeniedException ade)
if (ex is AccessDeniedException || ex is RunnerNotFoundException)
{
return false;
}
Expand Down
4 changes: 3 additions & 1 deletion src/Runner.Common/RunServer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ Task CompleteJobAsync(
IList<StepResult> stepResults,
IList<Annotation> jobAnnotations,
string environmentUrl,
IList<Telemetry> telemetry,
CancellationToken token);

Task<RenewJobResponse> RenewJobAsync(Guid planId, Guid jobId, CancellationToken token);
Expand Down Expand Up @@ -76,11 +77,12 @@ public Task CompleteJobAsync(
IList<StepResult> stepResults,
IList<Annotation> jobAnnotations,
string environmentUrl,
IList<Telemetry> telemetry,
CancellationToken cancellationToken)
{
CheckConnection();
return RetryRequest(
async () => await _runServiceHttpClient.CompleteJobAsync(requestUri, planId, jobId, result, outputs, stepResults, jobAnnotations, environmentUrl, cancellationToken), cancellationToken);
async () => await _runServiceHttpClient.CompleteJobAsync(requestUri, planId, jobId, result, outputs, stepResults, jobAnnotations, environmentUrl, telemetry, cancellationToken), cancellationToken);
}

public Task<RenewJobResponse> RenewJobAsync(Guid planId, Guid jobId, CancellationToken cancellationToken)
Expand Down
72 changes: 50 additions & 22 deletions src/Runner.Common/RunnerDotcomServer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,11 @@ public async Task<List<TaskAgent>> GetRunnerByNameAsync(string githubUrl, string
var githubApiUrl = "";
var gitHubUrlBuilder = new UriBuilder(githubUrl);
var path = gitHubUrlBuilder.Path.Split('/', '\\', StringSplitOptions.RemoveEmptyEntries);
if (path.Length == 1)
var isOrgRunner = path.Length == 1;
var isRepoOrEnterpriseRunner = path.Length == 2;
var isRepoRunner = isRepoOrEnterpriseRunner && !string.Equals(path[0], "enterprises", StringComparison.OrdinalIgnoreCase);

if (isOrgRunner)
{
// org runner
if (UrlUtil.IsHostedServer(gitHubUrlBuilder))
Expand All @@ -58,21 +62,31 @@ public async Task<List<TaskAgent>> GetRunnerByNameAsync(string githubUrl, string
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://{gitHubUrlBuilder.Host}/api/v3/orgs/{path[0]}/actions/runners?name={Uri.EscapeDataString(agentName)}";
}
}
else if (path.Length == 2)
else if (isRepoOrEnterpriseRunner)
{
// repo or enterprise runner.
if (!string.Equals(path[0], "enterprises", StringComparison.OrdinalIgnoreCase))
// Repository runner
if (isRepoRunner)
{
return null;
}

if (UrlUtil.IsHostedServer(gitHubUrlBuilder))
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/{path[0]}/{path[1]}/actions/runners?name={Uri.EscapeDataString(agentName)}";
if (UrlUtil.IsHostedServer(gitHubUrlBuilder))
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/repos/{path[0]}/{path[1]}/actions/runners?name={Uri.EscapeDataString(agentName)}";
}
else
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://{gitHubUrlBuilder.Host}/api/v3/repos/{path[0]}/{path[1]}/actions/runners?name={Uri.EscapeDataString(agentName)}";
}
}
else
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://{gitHubUrlBuilder.Host}/api/v3/{path[0]}/{path[1]}/actions/runners?name={Uri.EscapeDataString(agentName)}";
// Enterprise runner
if (UrlUtil.IsHostedServer(gitHubUrlBuilder))
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/{path[0]}/{path[1]}/actions/runners?name={Uri.EscapeDataString(agentName)}";
}
else
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://{gitHubUrlBuilder.Host}/api/v3/{path[0]}/{path[1]}/actions/runners?name={Uri.EscapeDataString(agentName)}";
}
}
}
else
Expand All @@ -90,7 +104,11 @@ public async Task<List<TaskAgentPool>> GetRunnerGroupsAsync(string githubUrl, st
var githubApiUrl = "";
var gitHubUrlBuilder = new UriBuilder(githubUrl);
var path = gitHubUrlBuilder.Path.Split('/', '\\', StringSplitOptions.RemoveEmptyEntries);
if (path.Length == 1)
var isOrgRunner = path.Length == 1;
var isRepoOrEnterpriseRunner = path.Length == 2;
var isRepoRunner = isRepoOrEnterpriseRunner && !string.Equals(path[0], "enterprises", StringComparison.OrdinalIgnoreCase);

if (isOrgRunner)
{
// org runner
if (UrlUtil.IsHostedServer(gitHubUrlBuilder))
Expand All @@ -102,21 +120,31 @@ public async Task<List<TaskAgentPool>> GetRunnerGroupsAsync(string githubUrl, st
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://{gitHubUrlBuilder.Host}/api/v3/orgs/{path[0]}/actions/runner-groups";
}
}
else if (path.Length == 2)
else if (isRepoOrEnterpriseRunner)
{
// repo or enterprise runner.
if (!string.Equals(path[0], "enterprises", StringComparison.OrdinalIgnoreCase))
// Repository Runner
if (isRepoRunner)
{
return null;
}

if (UrlUtil.IsHostedServer(gitHubUrlBuilder))
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/{path[0]}/{path[1]}/actions/runner-groups";
if (UrlUtil.IsHostedServer(gitHubUrlBuilder))
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/repos/{path[0]}/{path[1]}/actions/runner-groups";
}
else
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://{gitHubUrlBuilder.Host}/api/v3/repos/{path[0]}/{path[1]}/actions/runner-groups";
}
}
else
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://{gitHubUrlBuilder.Host}/api/v3/{path[0]}/{path[1]}/actions/runner-groups";
// Enterprise Runner
if (UrlUtil.IsHostedServer(gitHubUrlBuilder))
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://api.{gitHubUrlBuilder.Host}/{path[0]}/{path[1]}/actions/runner-groups";
}
else
{
githubApiUrl = $"{gitHubUrlBuilder.Scheme}://{gitHubUrlBuilder.Host}/api/v3/{path[0]}/{path[1]}/actions/runner-groups";
}
}
}
else
Expand Down
8 changes: 7 additions & 1 deletion src/Runner.Listener/BrokerMessageListener.cs
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@
using GitHub.Runner.Common;
using GitHub.Runner.Listener.Configuration;
using GitHub.Runner.Sdk;
using GitHub.Services.Common;
using GitHub.Runner.Common.Util;
using GitHub.Services.Common;
using GitHub.Services.OAuth;
using GitHub.Services.WebApi;

namespace GitHub.Runner.Listener
{
Expand Down Expand Up @@ -241,6 +242,10 @@ public async Task<TaskAgentMessage> GetNextMessageAsync(CancellationToken token)
{
throw;
}
catch (RunnerNotFoundException)
{
throw;
}
catch (Exception ex)
{
Trace.Error("Catch exception during get next message.");
Expand Down Expand Up @@ -324,6 +329,7 @@ private bool IsGetNextMessageExceptionRetriable(Exception ex)
ex is TaskAgentPoolNotFoundException ||
ex is TaskAgentSessionExpiredException ||
ex is AccessDeniedException ||
ex is RunnerNotFoundException ||
ex is VssUnauthorizedException)
{
Trace.Info($"Non-retriable exception: {ex.Message}");
Expand Down
2 changes: 1 addition & 1 deletion src/Runner.Listener/JobDispatcher.cs
Original file line number Diff line number Diff line change
Expand Up @@ -1198,7 +1198,7 @@ private async Task ForceFailJob(IRunServer runServer, Pipelines.AgentJobRequestM
jobAnnotations.Add(annotation.Value);
}

await runServer.CompleteJobAsync(message.Plan.PlanId, message.JobId, TaskResult.Failed, outputs: null, stepResults: null, jobAnnotations: jobAnnotations, environmentUrl: null, CancellationToken.None);
await runServer.CompleteJobAsync(message.Plan.PlanId, message.JobId, TaskResult.Failed, outputs: null, stepResults: null, jobAnnotations: jobAnnotations, environmentUrl: null, telemetry: null, CancellationToken.None);
}
catch (Exception ex)
{
Expand Down
5 changes: 5 additions & 0 deletions src/Runner.Listener/MessageListener.cs
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,10 @@ public async Task<TaskAgentMessage> GetNextMessageAsync(CancellationToken token)
{
throw;
}
catch (RunnerNotFoundException)
{
throw;
}
catch (Exception ex)
{
Trace.Error("Catch exception during get next message.");
Expand Down Expand Up @@ -457,6 +461,7 @@ private bool IsGetNextMessageExceptionRetriable(Exception ex)
ex is TaskAgentPoolNotFoundException ||
ex is TaskAgentSessionExpiredException ||
ex is AccessDeniedException ||
ex is RunnerNotFoundException ||
ex is VssUnauthorizedException)
{
Trace.Info($"Non-retriable exception: {ex.Message}");
Expand Down
7 changes: 7 additions & 0 deletions src/Runner.Listener/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
using System.Runtime.InteropServices;
using System.Threading.Tasks;
using GitHub.DistributedTask.WebApi;
using GitHub.Services.WebApi;

namespace GitHub.Runner.Listener
{
Expand Down Expand Up @@ -144,6 +145,12 @@ private async static Task<int> MainAsync(IHostContext context, string[] args)
trace.Error(e);
return Constants.Runner.ReturnCode.TerminatedError;
}
catch (RunnerNotFoundException e)
{
terminal.WriteError($"An error occurred: {e.Message}");
trace.Error(e);
return Constants.Runner.ReturnCode.TerminatedError;
}
catch (Exception e)
{
terminal.WriteError($"An error occurred: {e.Message}");
Expand Down
Loading