[exploration] An API for syntax highlighting

[savetheclocktower]

Summary

We have a need for an API that accepts some text and a language… and returns syntax-highlighted text styled with the user's currently active syntax theme.

Relatedly, we have a need for a service that renders Markdown and uses the above API to transform each of the code blocks in the Markdown input.

These things technically exist now, but they're hard to use and present performance challenges. I'd like to imagine alternative approaches, but first I want to make sure I understand all our use cases, so this ticket is about collecting those use cases. If you have a need for this API, please explain your use case in a comment.

What benefits does this feature provide?

In general, we should be using the user's syntax-highlighting theme whenever we show them code, even if it's not meant to be edited. We do this in styleguide and in markdown-preview. Various packages — among them atom-typescript and atom-ide-datatip — attempt to do similar things in other contexts.

If there's a “blessed” way to do this in the editor, then those packages won't have to invent their own “clever” (but sub-optimal) ways of doing the same thing.

Any alternatives?

The way we do this now is to create a read-only TextEditor and insert its custom element where we need it to be. This isn't great because there's a cost involved in spinning up a new TextEditor (as you might imagine), so we want to avoid situations where someone triggers the creation of an editor just to highlight something once. We want to reuse TextEditors if we can, even if we have to switch the language mode they're using on the fly.

There's also a cost involved in moving an editor around because there are automatic callbacks that run when the atom-text-editor element is detached and attached. This is a performance bottleneck; and though the bottleneck could probably be widened, it's also worth wondering if we can improve performance by not trying to put the TextEditor into the content in the first place.

[savetheclocktower]

Right now, I think the best approach would be one of the approaches used by markdown-preview:

1. Make it so that all styles that applied within an atom-text-editor also applied to, say, a pre.atom-text-editor
2. Use an invisible TextEditor instance (or a pool of editors for better parallelization, maybe) whenever we want to highlight something
3. Grab its raw contents after the highlighting and transplant each line into the pre

This is one gigantic hack, but it's theoretically so much faster than trying to create a TextEditor whenever highlighted code needs to exist somewhere. If the code is read-only, we don't need the overhead of a full editor anyway, so we should do whatever is fastest.

The most painful part of this might be the first step… actually, I excerpted the wrong block last time. This one doesn't seem so bad!

---

#984 implements syntax highlighting in markdown-preview in a way that involves putting TextEditors on the page, but being very careful about reusing them whenever possible so that new ones aren't created and injected unnecessarily. This works great for markdown-preview, but is impractical for most use cases because of the bookkeeping challenges it puts on the consumer. For simple cases, we want something that a package can use without having to care about how it got created.