Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rust: merge current status of experimental extractor into main #17435

Merged
merged 101 commits into from
Sep 11, 2024
Merged
Show file tree
Hide file tree
Changes from 89 commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
0e3c867
Toy rust program listing definitions of a cargo project
Aug 16, 2024
9277100
Rust: add some configuration and logging
Aug 27, 2024
2a2b79e
Rust: skeleton trap file emission code
Aug 27, 2024
f40901f
Rust: archiving + skeleton def translator
Aug 28, 2024
590a146
Rust: some basic extraction of function names (with locations!)
Aug 29, 2024
04753b2
Rust: tweak `trap_key`
Aug 30, 2024
3da6cee
Merge branch 'main' into rust-experiment
Aug 30, 2024
4f0fe1c
Rust: bazel packaging
Aug 30, 2024
7e1290a
Rust: reuse shared rust trap library
Aug 30, 2024
885e89a
Rust: first running tests
Aug 30, 2024
b23e482
Merge branch 'main' into rust-experiment
Sep 5, 2024
52ef1e3
Add 'version' field to generated Cargo.toml file
aibaars Sep 2, 2024
be16954
Rust: add autobuild.sh
aibaars Sep 2, 2024
60b349f
Rust: remove 'take(1)' from crates iterator
aibaars Sep 2, 2024
d8eb5d4
Rust: run cargo fmt
aibaars Sep 5, 2024
d93330b
Rust: make printAst work
aibaars Sep 5, 2024
6fd469c
Rust: use autodiscovery for sysroot
aibaars Sep 5, 2024
0516505
Rust: update dependencies
aibaars Sep 5, 2024
76c35be
Rust: add Expr, Pat, Stmt to schema
aibaars Sep 5, 2024
46d6bbb
Rust: update generated code
aibaars Sep 5, 2024
679d81a
Rust: extract dummy Function body
aibaars Sep 5, 2024
8ecba44
Rust: add `experimental` bazel target
Sep 6, 2024
ce2fdb3
Address comments
aibaars Sep 6, 2024
122e5a7
Rust: remove accidental IDE files
Sep 6, 2024
e14b626
Add 'BlockBase' class
aibaars Sep 6, 2024
a33c770
Add Label class
aibaars Sep 6, 2024
8c5cc2e
Rust: generate test code from schema docstrings
Sep 6, 2024
6111a74
Rust: remove accidental schema annotation
Sep 6, 2024
18c423e
Rust: fix formatting of code snippet in docstring
Sep 6, 2024
ac9e977
Rust: remove accidental test source file
Sep 6, 2024
928f3f1
Rust: remove `<>` from function wrapper
Sep 6, 2024
bc6da11
Fix type of Closure::arg_types
aibaars Sep 6, 2024
7511336
Add InlineAsm::expr
aibaars Sep 6, 2024
69b05e8
Add fields for OffsetOf
aibaars Sep 6, 2024
e827701
Add subclasses for Array
aibaars Sep 6, 2024
406d03e
Rename IfLet::else_branch to IfLet::else_
aibaars Sep 6, 2024
1a9b232
Add SlicePat::{slice, suffix}
aibaars Sep 6, 2024
c8f7358
Rust: codegen: fix generated code
aibaars Sep 7, 2024
3cd8aaf
Rust: simplify rust doc test annotation
Sep 9, 2024
7637393
Rust: resolve clippy and rustc warnings
Sep 9, 2024
ef34d24
Merge pull request #17390 from github/aibaars/rust-experiment
aibaars Sep 9, 2024
d8f37cf
Rust: add codegen pre commit hook
Sep 9, 2024
4512e42
Rust: add CI
Sep 9, 2024
ee2a8b0
Rust: fix formatting
Sep 9, 2024
c979a94
Rust: fix CI codegen check
Sep 9, 2024
10a23c0
Merge branch 'rust-experiment' into redsun82/rust-doctest-gen
Sep 9, 2024
9d08d24
Merge pull request #17396 from github/redsun82/rust-doctest-gen
redsun82 Sep 9, 2024
56e1278
Rust/Codegen: fix generated list files
Sep 9, 2024
573b8b9
Merge branch 'rust-experiment' into redsun82/rust-ci
Sep 9, 2024
b0ca793
Rust: update generated list file
Sep 9, 2024
d6b8dd1
Rust: add dummy integration test
Sep 9, 2024
45c9eb1
Merge pull request #17411 from github/redsun82/rust-ci-internal
redsun82 Sep 9, 2024
7c43839
Filter out trailing empty strings
aibaars Sep 5, 2024
da8750e
Rust: extract each project only once
aibaars Sep 5, 2024
6e6942b
Rust: initial extractor for expressions and patterns
aibaars Sep 7, 2024
6acc107
Prevent a panic on generated labels
aibaars Sep 7, 2024
b404764
Rust: schema.py add '| child' markers
aibaars Sep 9, 2024
85e51bf
Rust: PrintAstNode: tolerate elements without toString
aibaars Sep 9, 2024
86215b4
Rename IfLet to IfStmt
aibaars Sep 9, 2024
6f36ea9
Merge branch 'main' into rust-experiment
Sep 9, 2024
61592a3
Rust: fix locations
aibaars Sep 9, 2024
20ca880
Revert "Rust: PrintAstNode: tolerate elements without toString"
aibaars Sep 9, 2024
42b1112
Rust: add default Element::toString
aibaars Sep 9, 2024
37afad2
Merge pull request #17410 from github/redsun82/rust-ci
redsun82 Sep 10, 2024
38c25f9
Rust: add linting pre-commit hook
Sep 10, 2024
437b671
Merge branch 'rust-experiment' into aibaars/rust-experiment
Sep 10, 2024
ef06b55
Rust: accept test changes
Sep 10, 2024
46bfefc
Address comments
aibaars Sep 10, 2024
b2451c6
Improve schema
aibaars Sep 10, 2024
2ae7257
Don't use _ to ignore things so rustc will show warnings for all the …
aibaars Sep 10, 2024
a5d1d9e
Make implementation more complete
aibaars Sep 10, 2024
26b9de3
Merge branch 'main' into rust-experiment
Sep 10, 2024
1e830da
Add ra_ap_paths dependency
aibaars Sep 10, 2024
8213bbb
Rust: Add a README.md
geoffw0 Sep 10, 2024
43e54fb
Use custom target folder
aibaars Sep 10, 2024
f37b4ae
Only extract function bodies for local crates,
aibaars Sep 10, 2024
bffdf2b
Replace println with log statements
aibaars Sep 10, 2024
ac9afea
Rust: Add experimental warning and Bazelisk link.
geoffw0 Sep 10, 2024
d9a3711
Update rust/README.md
geoffw0 Sep 10, 2024
9c8f614
Merge pull request #17408 from github/aibaars/rust-experiment
redsun82 Sep 10, 2024
d6d52a9
Fix definition of trap compression option
aibaars Sep 10, 2024
00bc407
Merge pull request #17427 from github/aibaars/rust-experiment
aibaars Sep 10, 2024
d04f020
Rust: skip change note check for now
Sep 10, 2024
4f90f5f
Rust: accept test changes
Sep 10, 2024
6b0d7bc
Rust: Remove repo warning because we're merging into main real soon n…
geoffw0 Sep 10, 2024
7c95bb2
Rust: Update example path.
geoffw0 Sep 10, 2024
f8c9d96
Bazel: remove non-working fake `tree-sitter-extractor` workaround
Sep 11, 2024
91d5171
Add base setup for control flow graph construction
paldepind Sep 9, 2024
6d972be
Rust: Add a few control flow tree classes
paldepind Sep 10, 2024
f464759
Merge pull request #17434 from github/redsun82/rust-experiment
redsun82 Sep 11, 2024
304e68d
Merge pull request #17426 from geoffw0/readme
redsun82 Sep 11, 2024
2a7533c
Rust: add to `labeler.yml
Sep 11, 2024
809d040
Make more classes private and final
paldepind Sep 11, 2024
f624a1b
Ruby: fix extractor pack script after workspace root change
Sep 11, 2024
aa6726a
Bazel: bump up `rules_rust` version
Sep 11, 2024
d8db0e4
Ruby: update new cargo target directory in make and actions
Sep 11, 2024
8bf3c42
Merge branch 'main' into rust-experiment
Sep 11, 2024
857edb7
Rust: Fix control flow tree for function and block expression
paldepind Sep 11, 2024
e4766b6
Rust: make QL test crates workspaces
Sep 11, 2024
4398421
Merge branch 'main' into rust-experiment
Sep 11, 2024
fea6017
Merge pull request #17415 from paldepind/rust-control-flow-graph
paldepind Sep 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
5 changes: 3 additions & 2 deletions .github/workflows/check-change-note.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,12 @@ on:
- "shared/**/*.qll"
- "!**/experimental/**"
- "!ql/**"
- "!rust/**"
- ".github/workflows/check-change-note.yml"

jobs:
check-change-note:
env:
env:
REPO: ${{ github.repository }}
PULL_REQUEST_NUMBER: ${{ github.event.number }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -33,7 +34,7 @@ jobs:
!contains(github.event.pull_request.labels.*.name, 'no-change-note-required')
run: |
change_note_files=$(gh api "repos/$REPO/pulls/$PULL_REQUEST_NUMBER/files" --paginate --jq '.[].filename | select(test("/change-notes/.*[.]md$"))')

if [ -z "$change_note_files" ]; then
echo "No change note found. Either add one, or add the 'no-change-note-required' label."
exit 1
Expand Down
51 changes: 51 additions & 0 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: "Rust"

on:
pull_request:
paths:
- "rust/**"
- "misc/bazel/**"
- "misc/codegen/**"
- "shared/**"
- "MODULE.bazel"
- .github/workflows/rust.yml
- .github/actions/**
- codeql-workspace.yml
- "!**/*.md"
- "!**/*.qhelp"
branches:
- rust-experiment
- main
- rc/*
- codeql-cli-*

permissions:
contents: read

jobs:
rust-check:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Format
working-directory: rust/extractor
shell: bash
run: |
cargo fmt --check
- name: Compilation
working-directory: rust/extractor
shell: bash
run: cargo check
- name: Clippy
working-directory: rust/extractor
shell: bash
run: |
cargo clippy --fix
git diff --exit-code
- name: Code generation
shell: bash
run: |
bazel run //rust/codegen
git add .
git diff --exit-code HEAD
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,9 @@ node_modules/

# bazel-built in-tree extractor packs
/*/extractor-pack

# Jetbrains IDE files
.idea

# cargo build directory
/target
22 changes: 18 additions & 4 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ repos:
rev: v3.2.0
hooks:
- id: trailing-whitespace
exclude: /test/.*$(?<!\.ql)(?<!\.qll)(?<!\.qlref)|.*\.patch
exclude: /test/.*$(?<!\.qlref)|.*\.patch$|.*\.qll?$
- id: end-of-file-fixer
exclude: /test/.*$(?<!\.ql)(?<!\.qll)(?<!\.qlref)|.*\.patch
exclude: /test/.*$(?<!\.qlref)|.*\.patch$|.*\.qll?$
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was .ql deleted from this on purpose?

Copy link
Contributor Author

@redsun82 redsun82 Sep 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

before, ql files were included, now they are excluded always (it's in the qll? pattern): I found a corner case where this check was not agreeing with our ql formatter, who was inserting two newlines at the end. So we can leave our formatter deal with ql and qll files, and no other linter to interfere with that.


- repo: https://github.com/pre-commit/mirrors-clang-format
rev: v17.0.6
Expand Down Expand Up @@ -45,7 +45,7 @@ repos:

- id: sync-files
name: Fix files required to be identical
files: \.(qll?|qhelp|swift)$|^config/identical-files\.json$
files: \.(qll?|qhelp|swift|toml)$|^config/identical-files\.json$
language: system
entry: python3 config/sync-files.py --latest
pass_filenames: false
Expand All @@ -58,7 +58,7 @@ repos:

- id: swift-codegen
name: Run Swift checked in code generation
files: ^swift/(schema.py$|codegen/|.*/generated/|ql/lib/(swift\.dbscheme$|codeql/swift/elements)|ql/\.generated.list)
files: ^misc/codegen/|^swift/(schema.py$|codegen/|.*/generated/|ql/lib/(swift\.dbscheme$|codeql/swift/elements)|ql/\.generated.list)
language: system
entry: bazel run //swift/codegen -- --quiet
pass_filenames: false
Expand All @@ -69,3 +69,17 @@ repos:
language: system
entry: bazel test //misc/codegen/test
pass_filenames: false

- id: rust-codegen
name: Run Rust checked in code generation
files: ^misc/codegen/|^rust/(schema.py$|codegen/|.*/generated/|ql/lib/(rust\.dbscheme$|codeql/rust/elements)|\.generated.list)
language: system
entry: bazel run //rust/codegen -- --quiet
pass_filenames: false

- id: rust-lint
name: Run fmt and clippy on Rust code
files: ^rust/extractor/(.*rs|Cargo.toml)$
language: system
entry: python3 rust/lint.py
pass_filenames: false
Loading
Loading