Skip to content

Commit

Permalink
feat: update js engine and grammars (#869)
Browse files Browse the repository at this point in the history
  • Loading branch information
@slevithan
slevithan authored Dec 19, 2024
1 parent c489859 commit 5a26f77
Show file tree
Hide file tree
Showing 8 changed files with 2,613 additions and 521 deletions.
60 changes: 30 additions & 30 deletions docs/references/engine-js-compat.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Compatibility reference of all built-in grammars with the [JavaScript RegExp engine](/guide/regex-engines#javascript-regexp-engine-experimental).

> Generated on Monday, December 16, 2024
> Generated on Thursday, December 19, 2024
>
> Version `1.24.2`
>
Expand All @@ -13,8 +13,8 @@ Compatibility reference of all built-in grammars with the [JavaScript RegExp eng
| | Count |
| :-------------- | --------------------------: |
| Total Languages | 218 |
| Supported | [199](#supported-languages) |
| Mismatched | [13](#mismatched-languages) |
| Supported | [206](#supported-languages) |
| Mismatched | [6](#mismatched-languages) |
| Unsupported | [6](#unsupported-languages) |

## Supported Languages
Expand All @@ -39,6 +39,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| ballerina | ✅ OK | 230 | - | |
| bash | ✅ OK | 148 | - | |
| bat | ✅ OK | 58 | - | |
| beancount | ✅ OK | 39 | - | |
| berry | ✅ OK | 18 | - | |
| bibtex | ✅ OK | 19 | - | |
| bicep | ✅ OK | 27 | - | |
Expand Down Expand Up @@ -74,7 +75,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| elixir | ✅ OK | 708 | - | |
| elm | ✅ OK | 244 | - | |
| emacs-lisp | ✅ OK | 153 | - | 22 |
| erb | ✅ OK | 1312 | - | |
| erb | ✅ OK | 1786 | - | |
| erlang | ✅ OK | 147 | - | |
| fennel | ✅ OK | 31 | - | |
| fish | ✅ OK | 25 | - | |
Expand All @@ -83,8 +84,8 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| fortran-free-form | ✅ OK | 328 | - | |
| fsharp | ✅ OK | 239 | - | |
| fsl | ✅ OK | 30 | - | |
| gdresource | ✅ OK | 159 | - | |
| gdscript | ✅ OK | 95 | - | |
| gdresource | ✅ OK | 158 | - | |
| gdscript | ✅ OK | 94 | - | |
| gdshader | ✅ OK | 39 | - | |
| genie | ✅ OK | 20 | - | |
| gherkin | ✅ OK | 19 | - | |
Expand All @@ -98,6 +99,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| groovy | ✅ OK | 133 | - | |
| haml | ✅ OK | 562 | - | |
| handlebars | ✅ OK | 699 | - | |
| haskell | ✅ OK | 157 | - | |
| haxe | ✅ OK | 174 | - | |
| hcl | ✅ OK | 67 | - | |
| hjson | ✅ OK | 57 | - | |
Expand All @@ -120,6 +122,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| jssm | ✅ OK | 30 | - | |
| jsx | ✅ OK | 376 | - | |
| julia | ✅ OK | 1168 | - | |
| kotlin | ✅ OK | 58 | - | |
| latex | ✅ OK | 283 | - | |
| lean | ✅ OK | 32 | - | |
| less | ✅ OK | 280 | - | |
Expand All @@ -131,6 +134,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| make | ✅ OK | 51 | - | |
| marko | ✅ OK | 926 | - | |
| matlab | ✅ OK | 88 | - | |
| mdc | ✅ OK | 783 | - | |
| mdx | ✅ OK | 197 | - | |
| mermaid | ✅ OK | 134 | - | |
| mipsasm | ✅ OK | 17 | - | |
Expand All @@ -146,26 +150,29 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| ocaml | ✅ OK | 178 | - | |
| pascal | ✅ OK | 23 | - | |
| perl | ✅ OK | 941 | - | |
| php | ✅ OK | 1131 | - | |
| plsql | ✅ OK | 45 | - | |
| postcss | ✅ OK | 49 | - | |
| powerquery | ✅ OK | 30 | - | |
| powershell | ✅ OK | 88 | - | |
| prisma | ✅ OK | 26 | - | |
| prisma | ✅ OK | 28 | - | |
| prolog | ✅ OK | 26 | - | |
| proto | ✅ OK | 33 | - | |
| pug | ✅ OK | 686 | - | |
| puppet | ✅ OK | 61 | - | |
| python | ✅ OK | 218 | - | |
| qml | ✅ OK | 408 | - | |
| qmldir | ✅ OK | 7 | - | |
| qss | ✅ OK | 31 | - | |
| r | ✅ OK | 71 | - | |
| racket | ✅ OK | 69 | - | 8 |
| racket | ✅ OK | 69 | - | |
| raku | ✅ OK | 52 | - | |
| reg | ✅ OK | 9 | - | |
| regexp | ✅ OK | 34 | - | |
| rel | ✅ OK | 17 | - | |
| riscv | ✅ OK | 36 | - | |
| ruby | ✅ OK | 1307 | - | |
| rst | ✅ OK | 2031 | - | |
| ruby | ✅ OK | 1781 | - | |
| rust | ✅ OK | 89 | - | |
| sas | ✅ OK | 101 | - | |
| sass | ✅ OK | 69 | - | |
Expand Down Expand Up @@ -199,7 +206,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| tsv | ✅ OK | 1 | - | |
| tsx | ✅ OK | 376 | - | |
| turtle | ✅ OK | 15 | - | |
| twig | ✅ OK | 1984 | - | |
| twig | ✅ OK | 2426 | - | |
| typescript | ✅ OK | 363 | - | |
| typespec | ✅ OK | 80 | - | |
| typst | ✅ OK | 78 | - | |
Expand Down Expand Up @@ -235,31 +242,24 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s

Languages that do not throw with the JavaScript RegExp engine, but will produce different results than the WASM engine. Please use with caution.

| Language | Highlight Match | Patterns Parsable | Patterns Failed | Diff |
| --------- | :----------------------------------------------------------------------------- | ----------------: | --------------: | ---: |
| apex | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=apex) | 187 | - | 236 |
| beancount | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=beancount) | 39 | - | 171 |
| cpp | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=cpp) | 512 | - | 8 |
| haskell | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=haskell) | 157 | - | 39 |
| kotlin | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=kotlin) | 58 | - | 1953 |
| kusto | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=kusto) | 60 | - | 40 |
| markdown | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=markdown) | 118 | - | 915 |
| mdc | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=mdc) | 783 | - | 702 |
| nginx | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=nginx) | 378 | - | 4 |
| php | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=php) | 1131 | - | 605 |
| po | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=po) | 23 | - | 423 |
| pug | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=pug) | 686 | - | 164 |
| rst | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=rst) | 1857 | - | 62 |
| Language | Highlight Match | Patterns Parsable | Patterns Failed | Diff |
| -------- | :---------------------------------------------------------------------------- | ----------------: | --------------: | ---: |
| apex | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=apex) | 187 | - | 33 |
| cpp | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=cpp) | 512 | - | 8 |
| kusto | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=kusto) | 60 | - | 40 |
| markdown | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=markdown) | 118 | - | 100 |
| nginx | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=nginx) | 378 | - | 122 |
| po | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=po) | 23 | - | 114 |

## Unsupported Languages

Languages that throw with the JavaScript RegExp engine, either because they contain syntax we can't polyfill yet or because the grammar contains an invalid Oniguruma regex (that would also fail when using the WASM engine, but silently). You can try these languages with the `forgiving` option to skip errors, but usually it's best to use the Oniguruma engine instead.
Languages that throw with the JavaScript RegExp engine, either because they contain syntax we can't polyfill yet or because the grammar contains an invalid Oniguruma regex (that would also fail when using the WASM engine, but silently). You can try these languages with the `forgiving` option to skip errors.

| Language | Highlight Match | Patterns Parsable | Patterns Failed | Diff |
| ---------- | :-------------- | ----------------: | --------------: | ---: |
| ada | ✅ OK | 201 | 1 | |
| hack | ❌ Error | 947 | 1 | 114 |
| purescript | ❌ Error | 72 | 1 | 42 |
| csharp | ❌ Error | 306 | 3 | 204 |
| swift | ❌ Error | 326 | 3 | 40 |
| razor | ❌ Error | 952 | 5 | 22 |
| purescript | ❌ Error | 72 | 1 | |
| csharp | ❌ Error | 306 | 3 | 148 |
| swift | ❌ Error | 326 | 3 | 8 |
| razor | ❌ Error | 952 | 5 | 14 |
8 changes: 5 additions & 3 deletions packages/engine-javascript/src/index.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,13 +57,15 @@ export function defaultJavaScriptRegexConstructor(pattern: string, options?: Oni
global: true,
hasIndices: true,
rules: {
// Needed since TextMate grammars merge backrefs across patterns
allowOrphanBackrefs: true,
// Dropping unhandled `\G` anchors without erroring allows more grammars to pass but also
// allows some false positive and negative matches. It's mostly because this is true that
// the JS engine is marked as experimental
// Removing `\G` anchors in cases when they're not supported for emulation allows
// supporting more grammars, but also allows some mismatches
allowUnhandledGAnchors: true,
// Improves search performance for generated regexes
asciiWordBoundaries: true,
// Follow `vscode-oniguruma` which enables this Oniguruma option by default
captureGroup: true,
},
...options,
},
Expand Down
Loading

0 comments on commit 5a26f77

Please sign in to comment.