Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed Android build + improved CI/CI with xharness + added AllRuntimes package #216

Merged
merged 79 commits into from
Oct 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
80ba789
WIP
sandrohanea Oct 10, 2024
35dc117
a
sandrohanea Oct 10, 2024
c6e7351
T
sandrohanea Oct 10, 2024
d12ab7c
a
sandrohanea Oct 10, 2024
56dc3b1
USed @${{ github.head_ref }}
sandrohanea Oct 10, 2024
fde75fd
Try again
sandrohanea Oct 10, 2024
0531fbe
one more
sandrohanea Oct 10, 2024
de72c0f
a
sandrohanea Oct 10, 2024
94fff04
renamed yaml -> yml
sandrohanea Oct 10, 2024
962ebb8
Permissions fix
sandrohanea Oct 10, 2024
ecbb820
v3 and v4 issue
sandrohanea Oct 10, 2024
0159c28
added
sandrohanea Oct 10, 2024
d4b8862
Fixed linux-openvino-native-build.yml
sandrohanea Oct 10, 2024
67cc3cc
a
sandrohanea Oct 10, 2024
324e4e2
Try to fix
sandrohanea Oct 10, 2024
efdd699
Try again
sandrohanea Oct 10, 2024
8fd0570
Again
sandrohanea Oct 10, 2024
3f52c54
Otra vez
sandrohanea Oct 10, 2024
1385471
Removed strace
sandrohanea Oct 10, 2024
5658ff5
Tried to change the libraryName temp
sandrohanea Oct 10, 2024
b5c93e2
Again
sandrohanea Oct 10, 2024
6960870
aaa
sandrohanea Oct 10, 2024
1f0e67e
ffs
sandrohanea Oct 10, 2024
bf95c44
a
sandrohanea Oct 10, 2024
bdb1eb2
aaa
sandrohanea Oct 10, 2024
2574676
Again
sandrohanea Oct 10, 2024
2ef453b
Added UniversalLibraryLoader (for >net6 targets)
sandrohanea Oct 11, 2024
a8998a5
Removed patchelf, added android, macos, ios, tvos, maccatalyst tests
sandrohanea Oct 11, 2024
2f428e1
Try again
sandrohanea Oct 11, 2024
3c33dd2
Removed RuntimeIdentifier
sandrohanea Oct 11, 2024
5a6cc93
Removed usemaui, we want classical android, ios, not maui
sandrohanea Oct 11, 2024
ada001f
silenced XA0101
sandrohanea Oct 11, 2024
b864a92
aaa
sandrohanea Oct 11, 2024
0447d2a
Added Android tests
sandrohanea Oct 12, 2024
c47ba1a
Added conditional USE_WHISPER_MAUI
sandrohanea Oct 12, 2024
dd08391
Android fix
sandrohanea Oct 12, 2024
f85ae20
Changed dotnet-android =>dotnet-maui on macos
sandrohanea Oct 12, 2024
2b87a6d
Fixed xharness version
sandrohanea Oct 12, 2024
7be362d
--version
sandrohanea Oct 12, 2024
92eec90
Added tvos
sandrohanea Oct 12, 2024
1c70545
path / \
sandrohanea Oct 12, 2024
0942bec
Tried to set arch to arm64-v8 and fix runtimes for ios
sandrohanea Oct 12, 2024
0c59532
This is annoying
sandrohanea Oct 12, 2024
33897ac
Not starting
sandrohanea Oct 12, 2024
22b262e
One more try
sandrohanea Oct 12, 2024
b7cbe1e
Removed linux-vulkan
sandrohanea Oct 12, 2024
ba5487a
Commented maui build
sandrohanea Oct 12, 2024
92b382f
Added linux openvino
sandrohanea Oct 12, 2024
5a75fee
Fixed linux cuda build
sandrohanea Oct 12, 2024
f6f9e2e
Fixed builds
sandrohanea Oct 12, 2024
d864620
a
sandrohanea Oct 13, 2024
6de0575
aa
sandrohanea Oct 13, 2024
cad5580
aaaa
sandrohanea Oct 13, 2024
7e58a22
Changed publish to build and added linux noavx
sandrohanea Oct 13, 2024
516f658
Tried to use tabs everywhere
sandrohanea Oct 13, 2024
9d9c0c2
Inlined NDK
sandrohanea Oct 13, 2024
52762b9
Removed old endif
sandrohanea Oct 13, 2024
6b89d94
Added WindowsNoAvx
sandrohanea Oct 13, 2024
0c8858f
I hate whitespaces in makefiles
sandrohanea Oct 13, 2024
6dcf191
D
sandrohanea Oct 13, 2024
9ddc971
Moved runtiems to owndirectory
sandrohanea Oct 13, 2024
ad2fe02
a
sandrohanea Oct 13, 2024
30a10b1
ffs
sandrohanea Oct 13, 2024
8016354
tab
sandrohanea Oct 13, 2024
f99f31d
ffs
sandrohanea Oct 13, 2024
6ab4315
Lets see
sandrohanea Oct 13, 2024
adad18f
Again
sandrohanea Oct 13, 2024
8fd4246
fixed ios-simulator no libggml
sandrohanea Oct 13, 2024
d2e82c2
Ofc I broke the makefile
sandrohanea Oct 13, 2024
eeab981
One more try
sandrohanea Oct 13, 2024
ea3e2c7
Testing
sandrohanea Oct 13, 2024
d8be859
Updated readme and xharness args
sandrohanea Oct 13, 2024
3c867a8
a
sandrohanea Oct 13, 2024
176bcd5
Trying to find libomp
sandrohanea Oct 13, 2024
1d752ed
fff
sandrohanea Oct 13, 2024
a721cc7
a
sandrohanea Oct 13, 2024
d483268
Disabled openmp for android
sandrohanea Oct 13, 2024
5b04702
Oh yeah
sandrohanea Oct 13, 2024
16014ff
Fixed wasm for both net8 and net6
sandrohanea Oct 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 0 additions & 33 deletions .github/workflows/android-native-build.yaml

This file was deleted.

30 changes: 30 additions & 0 deletions .github/workflows/android-native-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Android Native build

on:
workflow_dispatch:
workflow_call:

jobs:
native-build:
runs-on: ubuntu-20.04

steps:
- uses: actions/checkout@v4
with:
submodules: true
ref: ${{ github.head_ref }}

- name: Install Android NDK
uses: nttld/[email protected]
with:
ndk-version: r25c

- name: Run android build
run: make android

- name: Upload Android Build Artifacts
uses: actions/upload-artifact@v4
with:
name: android-build
path: "runtimes/Whisper.net.Run*/android-*/*"
retention-days: 7
95 changes: 95 additions & 0 deletions .github/workflows/build-all.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# This workflow will build a .NET project
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-net

name: CI (Native + dotnet)

on:
push:
branches: ["main"]
pull_request:
branches: ["main"]
workflow_dispatch:
inputs:
BuildGpuLibs:
description: 'Enable GPU libraries build and publish'
required: true
default: 'false'
env:
BuildGpuLibs: false
jobs:
android:
uses: ./.github/workflows/android-native-build.yml

macos:
uses: ./.github/workflows/macos-native-build.yml

windows:
uses: ./.github/workflows/windows-native-build.yml

windows-no-avx:
uses: ./.github/workflows/windows-noavx-native-build.yml

wasm:
uses: ./.github/workflows/wasm-native-build.yml

linux:
uses: ./.github/workflows/linux-native-build.yml

linux-no-avx:
uses: ./.github/workflows/linux-noavx-native-build.yml

windows-cuda:
if: ${{ inputs.BuildGpuLibs == 'true' }}
uses: ./.github/workflows/windows-cuda-native-build.yml

windows-vulkan:
if: ${{ inputs.BuildGpuLibs == 'true' }}
uses: ./.github/workflows/windows-vulkan-native-build.yml

windows-openvino:
if: ${{ inputs.BuildGpuLibs == 'true' }}
uses: ./.github/workflows/windows-openvino-native-build.yml

linux-cuda:
if: ${{ inputs.BuildGpuLibs == 'true' }}
uses: ./.github/workflows/linux-cuda-native-build.yml

linux-openvino:
if: ${{ inputs.BuildGpuLibs == 'true' }}
uses: ./.github/workflows/linux-openvino-native-build.yml

macos-coreml:
if: ${{ inputs.BuildGpuLibs == 'true' }}
uses: ./.github/workflows/macos-coreml-native-build.yml

dotnet-build:
needs:
- android
- macos
- windows
- wasm
- linux
uses: ./.github/workflows/dotnet.yml

dotnet-maui-build-and-test:
needs:
- android
- macos
- windows
- wasm
- linux
uses: ./.github/workflows/dotnet-maui.yml

pack-all:
if: ${{ inputs.BuildGpuLibs == 'true' }}
needs:
- dotnet-build
- windows-cuda
- windows-vulkan
- windows-openvino
- linux-cuda
- linux-openvino
- macos-coreml
- linux-no-avx
- windows-no-avx
uses: ./.github/workflows/pack-all.yml
94 changes: 94 additions & 0 deletions .github/workflows/dotnet-maui.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# This workflow will build a .NET project
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-net

name: Dotnet Maui Build and Test

on:
workflow_call:

jobs:
dotnet-maui-build:
runs-on: macos-14

env:
USE_WHISPER_MAUI: "TRUE"

steps:
- uses: actions/checkout@v4

- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: |
8.0.x

- name: Download Artifacts
id: download-artifact
uses: actions/download-artifact@v4
with:
merge-multiple: true
path: runtimes

- name: Install dotnet workloads
run: |
dotnet workload install maui --ignore-failed-sources
dotnet workload install tvos --ignore-failed-sources

- name: Restore dependencies
run: dotnet restore ./Whisper.net.Maui.Tests.sln

- name: Publish android build
run: dotnet publish ./tests/Whisper.net.Maui.Tests/Whisper.net.Maui.Tests.csproj -c Debug -f net8.0-android -p:EmbedAssembliesIntoApk=true

- name: Upload apk and ios app
uses: actions/upload-artifact@v4
with:
name: maui-build-artifacts
path: |
./tests/Whisper.net.Maui.Tests/bin/Debug/net8.0-android/com.companyname.whisper.net.maui.tests-Signed.apk

dotnet-maui-test-android:
runs-on: ubuntu-latest
needs: dotnet-maui-build

env:
USE_WHISPER_MAUI: "TRUE"

steps:
- uses: actions/checkout@v4

- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: |
8.0.x

- name: Download Artifacts
id: download-artifact
uses: actions/download-artifact@v4
with:
name: maui-build-artifacts
path: maui-build-artifacts

- name: Enable KVM group perms
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm

- name: Install xharness
run: dotnet tool install Microsoft.DotNet.XHarness.CLI --global --add-source https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json --version "10.0.0-*"

- name: run android tests
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: 29
script: xharness android test --app=./maui-build-artifacts/com.companyname.whisper.net.maui.tests-Signed.apk -p com.companyname.whisper.net.maui.tests -i com.companyname.whisper.net.maui.tests.AndroidMauiTestInstrumentation -o=./test-results/android

- name: Upload Maui Android xharness test results
if: ${{ always() }}
uses: actions/upload-artifact@v4
with:
name: xharness-android-test-results
path: |
./test-results/*
129 changes: 91 additions & 38 deletions .github/workflows/dotnet.yml
Original file line number Diff line number Diff line change
@@ -1,45 +1,98 @@
# This workflow will build a .NET project
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-net

name: CI
name: Dotnet Build and Test

on:
push:
branches: [ "main" ]
#pull_request:
# branches: [ "main" ]

workflow_call:

jobs:
build:

runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest]

dotnet-macos:
runs-on: macos-14

steps:
- uses: actions/checkout@v3
- name: Setup .NET
uses: actions/setup-dotnet@v3
with:
dotnet-version: 7.0.x
- name: Install Java
if: runner.os == 'Windows'
uses: actions/setup-java@v2
with:
distribution: 'microsoft'
java-version: '11'
- name: Install .NET Workloads
if: runner.os == 'Windows'
run: |
dotnet workload install android --ignore-failed-sources
dotnet workload install ios --ignore-failed-sources
dotnet workload install tvos --ignore-failed-sources
dotnet workload install maccatalyst --ignore-failed-sources
- name: Restore dependencies
run: dotnet restore ./Whisper.net.sln
- name: Build
run: dotnet build ./Whisper.net.sln --no-restore -warnaserror
- name: Test
run: |
dotnet test ./Whisper.net.sln --no-build
- uses: actions/checkout@v4

- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: |
8.0.x
6.0.x

- name: Download Artifacts
id: download-artifact
uses: actions/download-artifact@v4
with:
merge-multiple: true
path: runtimes

- name: Restore dependencies
run: dotnet restore ./Whisper.net.sln

- name: Build
run: dotnet build ./Whisper.net.sln --no-restore -warnaserror

- name: Test
run: |
dotnet test ./Whisper.net.sln --no-build

dotnet-windows:
runs-on: windows-latest

steps:
- uses: actions/checkout@v4

- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: |
8.0.x
6.0.x

- name: Download Artifacts
id: download-artifact
uses: actions/download-artifact@v4
with:
merge-multiple: true
path: runtimes

- name: Restore dependencies
run: dotnet restore ./Whisper.net.sln

- name: Build
run: dotnet build ./Whisper.net.sln --no-restore -warnaserror

- name: Test
run: |
dotnet test ./Whisper.net.sln --no-build

dotnet-linux:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: |
8.0.x
6.0.x

- name: Download Artifacts
id: download-artifact
uses: actions/download-artifact@v4
with:
merge-multiple: true
path: runtimes

- name: Restore dependencies
run: dotnet restore ./Whisper.net.sln

- name: Build
run: dotnet build ./Whisper.net.sln --no-restore -warnaserror

- name: Test
run: |
dotnet test ./Whisper.net.sln --no-build
Loading