Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cistuff #40

Merged
merged 4 commits into from
May 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .JuliaFormatter.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
style = "sciml"
82 changes: 66 additions & 16 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
@@ -1,35 +1,81 @@
name: CI
on:
- push
pull_request:
branches:
- main
- dev
paths-ignore:
- "docs/**"
push:
branches:
- main
paths-ignore:
- "docs/**"
jobs:
formatter:
runs-on: ${{ matrix.os }}
strategy:
matrix:
julia-version: [1]
julia-arch: [x86]
os: [ubuntu-latest]
steps:
- uses: julia-actions/setup-julia@latest
with:
version: ${{ matrix.julia-version }}

- uses: actions/checkout@v4
- name: Install JuliaFormatter and format
# This will use the latest version by default but you can set the version like so:
#
# julia -e 'using Pkg; Pkg.add(PackageSpec(name="JuliaFormatter", version="0.13.0"))'
run: |
julia -e 'using Pkg; Pkg.add(PackageSpec(name="JuliaFormatter", version="1.0.50"))'
julia -e 'using JuliaFormatter; format(".", verbose=true)'
- name: Format check
run: |
julia -e '
out = Cmd(`git diff`) |> read |> String
if out == ""
exit(0)
else
@error "Some files have not been formatted !!!"
write(stdout, out)
exit(1)
end'
test:
name: Julia ${{ matrix.version }} - ${{ matrix.os }} - ${{ matrix.arch }} - ${{ github.event_name }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
version:
- "1.10"
- 'nightly'
- "1.9" # Minimal version of Julia for PerfChecker.jl
- "1" # automatically expands to the latest stable 1.x release of Julia
- nightly
os:
- ubuntu-latest
- macOS-latest
- windows-latest
arch:
- x64
- x86
exclude:
include:
# test macOS and Windows with latest Julia only
- os: macOS-latest
arch: x86
arch: x64
version: 1
- os: windows-latest
arch: x64
version: 1
- os: windows-latest
arch: x86
version: 1
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- uses: julia-actions/setup-julia@v1
with:
version: ${{ matrix.version }}
arch: ${{ matrix.arch }}
- uses: actions/cache@v1
- uses: actions/cache@v4
env:
cache-name: cache-artifacts
with:
Expand All @@ -42,25 +88,29 @@ jobs:
- uses: julia-actions/julia-buildpkg@v1
- uses: julia-actions/julia-runtest@v1
- uses: julia-actions/julia-processcoverage@v1
- uses: codecov/codecov-action@v1
- uses: codecov/codecov-action@v4
with:
file: lcov.info
docs:
name: Documentation
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- uses: julia-actions/setup-julia@v1
with:
version: '1'
- uses: julia-actions/julia-buildpkg@v1
- uses: julia-actions/julia-docdeploy@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }}
- run: |
julia --project=docs -e '
using Pkg
Pkg.develop(PackageSpec(path=pwd()))
Pkg.instantiate()'
- run: |
julia --project=docs -e '
using Documenter: DocMeta, doctest
using PerfChecker
DocMeta.setdocmeta!(PerfChecker, :DocTestSetup, :(using PerfChecker); recursive=true)
doctest(PerfChecker)'
- run: julia --project=docs docs/make.jl
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }}
37 changes: 32 additions & 5 deletions .github/workflows/CompatHelper.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,43 @@
name: CompatHelper
on:
schedule:
- cron: 0 0 * * *
workflow_dispatch:
permissions:
contents: write
pull-requests: write
jobs:
CompatHelper:
runs-on: ubuntu-latest
steps:
- name: Pkg.add("CompatHelper")
run: julia -e 'using Pkg; Pkg.add("CompatHelper")'
- name: CompatHelper.main()
- name: Check if Julia is already available in the PATH
id: julia_in_path
run: which julia
continue-on-error: true
- name: Install Julia, but only if it is not already available in the PATH
uses: julia-actions/setup-julia@v1
with:
version: "1"
arch: ${{ runner.arch }}
if: steps.julia_in_path.outcome != 'success'
- name: "Add the General registry via Git"
run: |
import Pkg
ENV["JULIA_PKG_SERVER"] = ""
Pkg.Registry.add("General")
shell: julia --color=yes {0}
- name: "Install CompatHelper"
run: |
import Pkg
name = "CompatHelper"
uuid = "aa819f21-2bde-4658-8897-bab36330d9b7"
version = "3"
Pkg.add(; name, uuid, version)
shell: julia --color=yes {0}
- name: "Run CompatHelper"
run: |
import CompatHelper
CompatHelper.main()
shell: julia --color=yes {0}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COMPATHELPER_PRIV: ${{ secrets.DOCUMENTER_KEY }}
run: julia -e 'using CompatHelper; CompatHelper.main()'
13 changes: 13 additions & 0 deletions .github/workflows/SpellCheck.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: Spell Check

on: [pull_request]

jobs:
typos-check:
name: Spell Check with Typos
runs-on: ubuntu-latest
steps:
- name: Checkout Actions Repository
uses: actions/checkout@v4
- name: Check spelling
uses: crate-ci/[email protected]
16 changes: 16 additions & 0 deletions .github/workflows/TagBot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,22 @@ on:
types:
- created
workflow_dispatch:
inputs:
lookback:
default: "3"
permissions:
actions: read
checks: read
contents: write
deployments: read
issues: read
discussions: read
packages: read
pages: read
pull-requests: read
repository-projects: read
security-events: read
statuses: read
jobs:
TagBot:
if: github.event_name == 'workflow_dispatch' || github.actor == 'JuliaTagBot'
Expand Down
9 changes: 5 additions & 4 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ authors = ["Azzaare <[email protected]>"]
version = "0.2.0"

[deps]
BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
Chairmarks = "0ca39b1e-fe0b-4e98-acfc-b1656634c4de"
CoverageTools = "c36e975a-824b-4404-a568-ef97ca766997"
CpuId = "adafc99b-e345-5852-983c-f28acb93d879"
Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b"
Expand All @@ -28,10 +26,13 @@ Chairmarks = "1.2"
CoverageTools = "1"
CpuId = "0.3.1"
TypedTables = "1"
julia = "1"
julia = "1.9"

[extras]
Aqua = "4c88cf16-eb10-579e-8560-4a9242c79595"
BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
PatternFolds = "c18a7f1d-76ad-4ce4-950d-5419b888513b"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["Test"]
test = ["Aqua", "BenchmarkTools", "PatternFolds", "Test"]
8 changes: 3 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,10 @@
[![Dev](https://img.shields.io/badge/docs-dev-blue.svg)](https://JuliaConstraints.github.io/PerfChecker.jl/dev)
[![Build Status](https://github.com/JuliaConstraints/PerfChecker.jl/workflows/CI/badge.svg)](https://github.com/JuliaConstraints/PerfChecker.jl/actions)
[![codecov](https://codecov.io/gh/JuliaConstraints/PerfChecker.jl/branch/main/graph/badge.svg?token=YVJhN4dpBp)](https://codecov.io/gh/JuliaConstraints/PerfChecker.jl)
[![Code Style: Blue](https://img.shields.io/badge/code%20style-blue-4495d1.svg)](https://github.com/invenia/BlueStyle)
[![SciML Code Style](https://img.shields.io/static/v1?label=code%20style&message=SciML&color=9558b2&labelColor=389826)](https://github.com/SciML/SciMLStyle)
[![Chat: Mattermost](https://img.shields.io/badge/chat-mattermost-blueviolet.svg)](https://nohost.iijlab.net/chat/signup_user_complete/?id=nnuc1g14gtrqtnas6thu193xmr)
[![Website: JuliaConstraints](https://img.shields.io/badge/website-JuliaConstraints-informational.svg)](https://juliaconstraints.github.io/)



`PerfChecker` is a set of performance checking tools for Julia packages. The ultimate aim is to create an environment where the tool can run similarly to a test environment. By doing so, it would be possible to test the performance of a package `P` in separate Julia instances. This would allow for each version of `P`:

- The use of the latest compatible versions of Julia and other dependencies of P
Expand Down Expand Up @@ -123,7 +121,7 @@ using PerfChecker
using Test

# Target(s)
using CompositionalNetworks # lastest release: 0.3.1
using CompositionalNetworks # latest release: 0.3.1

# Direct dependencies of this script
using ConstraintDomains
Expand Down Expand Up @@ -166,7 +164,7 @@ using PerfChecker
using BenchmarkTools

# Target(s)
using CompositionalNetworks # lastest release: 0.3.1
using CompositionalNetworks # latest release: 0.3.1

# Direct dependencies of this script
using ConstraintDomains
Expand Down
28 changes: 14 additions & 14 deletions docs/make.jl
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
using PerfChecker
using Documenter

DocMeta.setdocmeta!(PerfChecker, :DocTestSetup, :(using PerfChecker); recursive=true)
DocMeta.setdocmeta!(PerfChecker, :DocTestSetup, :(using PerfChecker); recursive = true)

makedocs(;
modules=[PerfChecker],
authors="Azzaare <[email protected]>",
repo="https://github.com/JuliaConstraints/PerfChecker.jl/blob/{commit}{path}#{line}",
sitename="PerfChecker.jl",
format=Documenter.HTML(;
prettyurls=get(ENV, "CI", "false") == "true",
canonical="https://JuliaConstraints.github.io/PerfChecker.jl",
assets=String[],
modules = [PerfChecker],
authors = "Azzaare <[email protected]>",
repo = "https://github.com/JuliaConstraints/PerfChecker.jl/blob/{commit}{path}#{line}",
sitename = "PerfChecker.jl",
format = Documenter.HTML(;
prettyurls = get(ENV, "CI", "false") == "true",
canonical = "https://JuliaConstraints.github.io/PerfChecker.jl",
assets = String[]
),
pages=[
"Home" => "index.md",
],
pages = [
"Home" => "index.md"
]
)

deploydocs(;
repo="github.com/JuliaConstraints/PerfChecker.jl",
devbranch="main",
repo = "github.com/JuliaConstraints/PerfChecker.jl",
devbranch = "main"
)
13 changes: 8 additions & 5 deletions ext/BenchmarkToolsExt/benchmark.jl
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,12 @@ function PerfChecker.default_options(::Val{:benchmark})
)
end

PerfChecker.check(d::Dict, block::Expr, ::Val{:benchmark}) = quote
d = $d
using BenchmarkTools
return @benchmark $block samples=d[:samples] seconds=d[:seconds] evals=d[:evals] overhead=d[:overhead] gctrial=d[:gctrial] gcsample=d[:gcsample] time_tolerance=d[:time_tolerance] memory_tolerance=d[:memory_tolerance]
function PerfChecker.check(d::Dict, block::Expr, ::Val{:benchmark})
quote
d = $d
using BenchmarkTools
return @benchmark $block samples=d[:samples] seconds=d[:seconds] evals=d[:evals] overhead=d[:overhead] gctrial=d[:gctrial] gcsample=d[:gcsample] time_tolerance=d[:time_tolerance] memory_tolerance=d[:memory_tolerance]
end
end

PerfChecker.prep(::Dict, block::Expr, ::Val{:benchmark}) = quote
Expand All @@ -29,5 +31,6 @@ PerfChecker.post(d::Dict, ::Val{:benchmark}) = d[:check_result]
function PerfChecker.to_table(bench::BenchmarkTools.Trial)
ti = bench.times
l = length(ti)
return Table(times=ti, gctimes=bench.gctimes, memory=fill(bench.memory, l), allocs=fill(bench.allocs, l))
return Table(times = ti, gctimes = bench.gctimes,
memory = fill(bench.memory, l), allocs = fill(bench.allocs, l))
end
1 change: 0 additions & 1 deletion ext/ChairmarksExt/ChairmarksExt.jl
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,3 @@ include("b.jl")
#include("be.jl")
#include("chairmark.jl")
end

2 changes: 1 addition & 1 deletion ext/ChairmarksExt/b.jl
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,5 @@ function PerfChecker.to_table(chair::Chairmarks.Benchmark)
gctimes = [chair.samples[i].gc_fraction for i in 1:l]
bytes = [chair.samples[i].bytes for i in 1:l]
allocs = [chair.samples[i].allocs for i in 1:l]
return Table(times=times, gctimes=gctimes, bytes=bytes, allocs=allocs)
return Table(times = times, gctimes = gctimes, bytes = bytes, allocs = allocs)
end
12 changes: 6 additions & 6 deletions perf/GLM/allocs.jl
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@ using PerfChecker

result = @check :alloc Dict(:targets => ["GLM"], :path => @__DIR__) begin
using GLM, Random, StatsModels
end begin
end begin
n = 2_500_000
rng = Random.MersenneTwister(1234321)
tbl = (
x1=randn(rng, n),
x2=Random.randexp(rng, n),
ss=rand(rng, string.(50:99), n),
y=zeros(n),
x1 = randn(rng, n),
x2 = Random.randexp(rng, n),
ss = rand(rng, string.(50:99), n),
y = zeros(n)
)
f = @formula(y ~ 1 + x1 + x2 + ss)
f = @formula(y~1 + x1 + x2 + ss)
f = apply_schema(f, schema(f, tbl))
resp, pred = modelcols(f, tbl)
B = randn(rng, size(pred, 2))
Expand Down
14 changes: 7 additions & 7 deletions perf/GLM/bench.jl
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@ using PerfChecker, BenchmarkTools

t = @check :benchmark Dict(:path => @__DIR__, :evals => 1, :samples => 100, :seconds => 100) begin
using GLM, Random, StatsModels
end begin
end begin
n = 2_500_000
rng = Random.MersenneTwister(1234321)
tbl = (
x1 = randn(rng, n),
x2 = Random.randexp(rng, n),
ss = rand(rng, string.(50:99), n),
y = zeros(n),
)
f = @formula(y ~ 1 + x1 + x2 + ss)
x1 = randn(rng, n),
x2 = Random.randexp(rng, n),
ss = rand(rng, string.(50:99), n),
y = zeros(n)
)
f = @formula(y~1 + x1 + x2 + ss)
f = apply_schema(f, schema(f, tbl))
resp, pred = modelcols(f, tbl)
B = randn(rng, size(pred, 2))
Expand Down
Loading
Loading