Default languages/grammars

[Daeraxa]

I originally posted this on Discord but I realised this is clearly the correct place to actually put this.

I had a dive through some of the more popular editors that are closest to Atom/Pulsar in terms of functionality/intended use so I could see what grammars new users will probably be expecting to work "out of the box" rather than having to go and install a package just to get basic syntax highlighting. Unfortunately Pulsar does come out a bit lacking at the moment.

Just to point out this is grammars we are talking about at this point, both textmate & tree-sitter rather than any discussion on language servers which are its own thing.

I mostly base the comparisons here on VSCode, Brackets, Sublime and Lapce as the most "modern", in-development text-editors-come-IDEs that have an emphasis on good "out of the box" usability as well as extensibility.
The older and distro "default" editors like Geany, Xed, Kate, Vim are taken into account but often include a huge number of older and mostly obsolete languages which would be better served by an optional package in Pulsar.
Editors like Helix, Neovim really cater to a slightly different user base so have also not been my main focus.
Zed is an interesting point of comparison as it in very early development and has support for the most obvious languages but is also a nice case to see what the community are currently asking for to get an idea of what people want out of a modern editor.

The full table can be found in this gist.

So, to me, the most obvious inclusions for Pulsar (that are currently not default) would be:

• Swift
• Dart
• Kotlin
• Groovy
• Lua
• Julia
• Haskell
• Dockerfile (+ Compose)

To extend the list to some "nice to haves"

• HandleBars
• Haxe
• Scala
• OCaml
• R
• Matlab
• Visual basic
• TeX (+ LaTeX, BibTeX)

I think there is merit in picking up some community packages if possible and adopting them or simply reusing parts of them for a core language package. Tree-sitter grammars should be prioritised over any legacy textmate grammars for obvious reasons.

In terms of implementation there are a couple of routes we could go:

• Don't include any more default grammars (obviously not what I'm advocating here)
• Have functionality that can recognise a given file type and recommend installation of a language package for download (but don't include it in core) including any extensions like "IDE" packages.
• Add the grammars into core and prompt/recommend to install extensions like "IDE" packages (which is how VSCode often does it).

See also:

• PR about tree-sitter bump that may break existing packages.
• Discussion about tree-sitter conversion to N-API (Discord)

[kaosine]

Have functionality that can recognise a given file type and recommend installation of a language package for download (but don't include it in core) including any extensions like "IDE" packages.

Well, we're partially here, with this part here of customizing the config.cson file like this:

How do I make Atom recognize a file with extension X as language Y?

Atom includes a feature called "custom file types" which you can use by adding some entries into your config.cson that look like this:

core:
  customFileTypes:
    'source.ruby': [
      'Cheffile'
      'this-is-also-ruby'
    ]
    'source.cpp': [
      'h'
    ]

The key (for example source.ruby in the above snippet) is the language's scope name. The value is an array of file extensions, without the period, to match to that scope name.

Which we could implement these ourselves if we really wanted and extend upon it since it's already there. (Note: this section is in the atom-archive section of the FAQ if you want to use it in your section we were talking about the other day)

[confused-Techie]

So to continue off the conversation @mauricioszabo originally started on another discussion.

I'm fully on board with having more default themes. I think the biggest concern we would have by adding more is size, and startup time.

But as I've understood so far, adding a WASM based tree-sitter grammar is literally just a few small files. So much so I remember him talking about how he wasn't even sure he wanted to require a few package for how small they were.

So if that's still the case, and they have no impact on startup time, I think the only thing stopping us from including every single language we want to would be maintainance cost.

Since essentially we should hope that each language has a 'champion' of that language. Essentially someone familiar with the language enough to field any language specific errors. But beyond that, I'd say lets go for it!

As for me personally the biggest one I want to see become bundled in the editor is powershell