Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[core] New UI API #763

Merged
merged 31 commits into from
Nov 9, 2023
Merged

[core] New UI API #763

merged 31 commits into from
Nov 9, 2023

Conversation

confused-Techie
Copy link
Member

As previously discussed on Discord, there seems to be a good need for the creation of a UI API, that at least at first, can make it's focus on rendering Markdown content.


Within Pulsar there are many packages that have to render Markdown content, each of which containing disparate versions of the same dependencies over and over. Where when updating one of these, many packages have to be modified, and there have to be many checks for changes in behavior.

Additionally, over in pulsar-edit/package-frontend there have been many changes made to how we render Markdown content from package readmes, that for a while I've wanted to bring over to settings-view, such as converting links to the Atom docs to use WebArchive, or converting links to other packages published to Pulsar into links using the new Pulsar domain, etc. But these changes have never been easily possibly because settings-view was using marked, meanwhile package-frontend uses markdown-it.

But with all of this said, onto this proposed solution:

This PR creates a new core API endpoint of atom.ui that exposes several functions that can aid in packages parsing Markdown content. With the intention that we can remove all markdown handling from settings-view, notifications, markdown-preview, deprecation-cop, autocomplete-plus, etc. And we instead move all of these packages to utilize this new endpoint.

This change will then of course be available to community package's as well for these improvements.

The benefit of a change like this can potentially be far reaching, while concretely gives us real world benefits today:

  • With only single versions of the dependencies needed, this can reduce our overall install size
  • With only single versions of the dependencies needed, we also have to worry less about keeping things up to date, and are easier to update when needed
  • Like mentioned we can bring in the improvements to community package readmes everywhere, not just the Pulsar Package Website
  • In the future, we could introduce settings or greater control over how markdown is parsed, and it will apply to every package that uses this API. Such as enabling emojis, adding copy buttons on code blocks, disabling syntax highlighting, modifying links, or images.

Copy link

@AbdullahWins AbdullahWins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Job!

@DeeDeeG
Copy link
Member

DeeDeeG commented Oct 17, 2023

This sounds cool.

I'm not super familiar with the usage of markdown throughout the app, but this does sounds like it consolidates dependencies into one place, and makes for one source of truth for markdown, internally to the app and to whichever community packages that would opt-in to using it.

I don't know if a different name for it would make sense, like markdownUI? But I don't see us adding a ton of new APIs all the time, so it may not matter regarding parking "the good, short, general-sounding names".

Also it might be "illuminating" for reviewers to implement a usage of this API, maybe on a separate demonstration branch, but I don't think I would totally get it intuitively even if that were done. Not asking for that, but suggesting it might help people understand it if there are indeed reviewers forthcoming who are familiar with this stuff, which isn't a given. (Don't want to impose on your real time for a hypothetical reviewing-oriented benefit, though!)

@confused-Techie
Copy link
Member Author

@DeeDeeG Thanks for taking a look here. But exactly, seems like a really cool and beneficial concept.

But don't worry about want for an implementation example, the reason this is still in a draft is I'm hoping to migrate all core packages over to this new API prior to making the PR ready to review. As that'll ensure we can test it properly, as well as will give an implementation example, or even several.

As for naming, I'd be totally fine changing it, or even be totally fine appending new UI type APIs to this same endpoint. To where it does more than just markdown handling, but that remains to be seen as even an option or idea, so if you'd rather change it, by all means we can, but also like you said, not the highest priority.

…ify base64 links, make test results match universal GitHub links
Most browsers already strip self closing tags within void elements. DOMPurify itself also strips this character.
1. `master` isn't always true. We can use `HEAD` now and GitHub will automatically redirect to the main branch.
2. We already have the logic inside our markdown to correct links to use `blob` or `raw` depending on if directing to a link or image, and use `HEAD` like mentioned above
@confused-Techie confused-Techie marked this pull request as ready for review October 27, 2023 23:29
@confused-Techie
Copy link
Member Author

confused-Techie commented Oct 27, 2023

Alright it's taken some time, but this one is finally ready for a review!

In this PR I have successfully implemented the Markdown features of the new UI API, which gives all packages access to a Markdown Parser, that's extremely configurable supporting as many features as possible, and with all core packages migrated over.

Features Supported

  • Uses domPurify to sanitize all output
  • Supports markdown-it-emoji plugin
  • Supports markdown-it-github-headings plugin
  • Supports markdown-it-task-checkbox plugin
  • Can resolve paths to files or assets locally on the filesystem
  • Can resolve paths to files or assets remotely on GitHub
  • Converts local paths to the fully qualified paths (Supporting root paths or local paths in the browser)
  • If a link points to an image on GitHub the path will automatically point to GitHub's raw resource to ensure it's displayed properly
  • If a link points to GitHub that's being modified in any way, we use HEAD instead of main or master to ensure it works no matter the default branch of a resource
  • Converts any links pointing to https://atom.com/packages to https://web.pulsar-edit.dev/packages
  • Converts any links pointing to https://atom.com to that same exact URL on Web Archive
  • All above link support works for normal links and for image paths
  • Supports Syntax Highlighting via Pulsar's existing Syntax Highlighting System
  • Supports a strict mode which disables many features:
    • 1heading disabled
    • Code Block content is stripped
    • Fence Block content is stripped
    • Images are removed
    • Breaklines are the only inline HTML element supported
    • Headings are stripped
    • Bullet List content is stripped
    • Ordered List content is stripped
    • All HTML Blocks are stripped
  • YAML Front Matter is supported

Packages Migrated

  • autocomplete-plus
  • deprecation-cop
  • markdown-preview (Although within this package, the usage of this new API or the original method is configurable, so markdown-preview supports both internal parsing, and the new API. This is controlled via markdown-preview.useOriginalParser which is true by default)
  • notifications
  • settings-view

With that writeup done, it's important to note that this feature set covers everything that was original used across all core packages and puts it all together in a single API, that now any core package or community package can utilize. So while this is a bigger one, this is now ready for review, and once successfully reviewed we can merge #774 into this branch then bring everything into master. Thanks a ton to anyone willing to review!

Copy link
Contributor

@mauricioszabo mauricioszabo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First thing that I have to say: INCREDIBLE JOB!

I am really, really exited to use this on my own plug-in (that currently have to render markdown by itself, causing a whole lot of weird issues and different behavior than what's expected).

Now, I just added 3 requests for changes - they all relate to documentation. Because we're adding a new API, and we still don't have the API Docs anywhere, it's a good idea to add some JSDocs to avoid us forgetting about the new API we added.

With the JSDoc in place, I say: merge it, and let's go! :)

src/ui.js Outdated Show resolved Hide resolved
src/ui.js Outdated Show resolved Hide resolved
src/ui.js Outdated Show resolved Hide resolved
@confused-Techie
Copy link
Member Author

@mauricioszabo I really appreciate your review!
I'll go ahead and add some documentation here, because you are totally correct that it's very lacking currently, but thanks again shouldn't take too long to add it in

Copy link
Member

@DeeDeeG DeeDeeG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a "lite-approve" on this, as multiple people seem to really want it. I support it mainly for that reason.

Feels a bit beyond my level to properly review every bit of the diff, but I'd be willing to lend a hand troubleshooting it if anything were to go wrong post-merge.

spec/ui-spec.js Outdated
});
it("resolves incomplete root links", () => {
expect(atom.ui.markdown.render(
"[Hello](/readme.md)",
{ rootDomain: "https://github.com/pulsar-edit/pulsar" }
)).toBe('<p><a href="https://github.com/pulsar-edit/pulsar/readme.md">Hello</a></p>\n');
)).toBe('<p><a href="https://github.com/pulsar-edit/pulsar/blob/HEADreadme.md">Hello</a></p>\n');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Erm is this right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic catch, no, no it is not. Lemme fix that, but makes me wonder why the CI passed on it, since it shouldn't have

Copy link
Member

@DeeDeeG DeeDeeG Nov 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be HEAD/readme.md with the slash, not HEADreadme.md?

EDIT: I'm too slow typing this, didn't see you had replied before I refreshed the page, heh.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly, it's missing part of the path.
Lemme add the delimiter and see if specs are still passing

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the delimiter, and checked that the code should be making this change, so not sure why we got a reported success if that was not the case. But lets let the specs run and see if it was a fluke

Copy link
Member

@DeeDeeG DeeDeeG Nov 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you believe the spec failed on exactly this test? EDIT: to be clear, the passing CI run had a spec failure in it, actually.

https://github.com/pulsar-edit/pulsar/actions/runs/6780796150/job/18430013470#step:7:2435

Looks like our workflow (our test runner script, specifically?) kept going and ended up with an apparently erroneous "clean" exit code from some stuff after the failure? Hmmm.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DeeDeeG Even crazier, would you believe how it passed on the run immediately after this?

Finished in 0.039 seconds
0 tests, 0 assertions, 0 failures, 2283 skipped

Copy link
Member

@DeeDeeG DeeDeeG Nov 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the test runner script/run-tests.js was trying to filter to run again, but only for the failing tests, but maybe it filtered too hard and didn't leave any tests in at all? (A loose guess.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@confused-Techie
Copy link
Member Author

Thanks for the review both to @mauricioszabo and @DeeDeeG, and thanks for offering to help post merge @DeeDeeG!

Mauricio has confirmed on Discord the changes look good and we are a go for merge.

Lets get this one into core in time for the next release!

PS. Also big thanks to DeeDee for catching that last minute bug!

@confused-Techie confused-Techie merged commit 39391c1 into master Nov 9, 2023
99 checks passed
@confused-Techie confused-Techie deleted the ui-api branch November 9, 2023 04:48
@confused-Techie confused-Techie restored the ui-api branch November 9, 2023 04:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants