ast node editor

[felixroos]

It would be cool to be able to switch between code and node based editor. Because the transpiler is able to translate between textual (code) and structural (AST) formats of the same code, the AST could be another way to edit the code. And by edit I mean having some form of visual / node based UI that is able to make changes to the AST, which will automatically be applied to the code as well, so it's possible to switch back and forth. I approached this idea by generating a (read only) SVG of a small piece of strudel code:

n("0 1 2 3 4 5 6 7")
  .sometimes(x=>x.ply(2))
  .rarely(x=>x.speed("2 | -2"))
  .s('amencutup')

after parsing that with acorn into an AST + rendering that AST with SVGjs + d3-hierarchy, it looks like this:

Source: https://codesandbox.io/s/acorn-test-xv4zdq?file=/src/index.js

Of course, editing that monstrosity is not an option, but looking closer, some node patterns could be condensed to single nodes:

factory functions n("0 1 2 3 4 5 6 7")

chained function: .s('amencutup')

modifier function: x=>x.ply(2)

So if we replace all of those structures with a single node, the picture gets much clearer:

• 2-3 types of lines (or nodes) needed: chained connection, argument connection, (modifier argument connection?)
• maybe there is a way to make it simpler (pure data style), by only allowing argument connections
• it is clear that the ast editor will only support a subset of what's possible in JS, or at least many things won't be supported.

This is just an idea that I will stop to investigate for now, just dropping this here so I won't forget it.
Maybe someone else is interested in thinking about this.

Related:

• https://rohandrape.net/pub/blksc3/blksc3-wasm.html
• https://natto.dev/
• https://reactflow.dev/
• https://developers.google.com/blockly/
• https://www.youtube.com/watch?v=9iQVwHOdC7Y

[yaxu]

Oh lovely! This looks really interesting.

I'd forgot about that 'autocanute' experiment, thanks for digging that out.. I think the words are moving around randomly based on sound input. Seems I failed to capture sound output? Here's a version without the sound-reactive stuff: https://www.youtube.com/watch?v=kr4DdQuNAKc

The nice thing about that was visualising the patterns going between the functions, an earlier version visualised types there instead though, which I think worked well: https://youtu.be/5KtFlGEVFGE?t=216

Visually you can see that 'every' has arity of three, with different types. <~ has arity of 2, and you can see it has one of its parameters missing so is curried.

The functions are connected up based on proximity and type-compatibility. The earlier system was implemented in C, and I had to implement some of the haskell type system to make it work. The later system was implemented in Haskell, but I still had to implement some of the haskell type system because I couldn't work out how to use the haskell compiler as a library to help me work out what was compatible with what..

The proximity-based spatial syntax was inspired by the reactable, which also uses distance as a parameter. I intended to experiment with this but got distracted.

Honestly I find node-based editing quite fiddly in general, but when I see someone work who is fluent with e.g. Max, it is pretty amazing really. The reactable and more recent systems like the amazing bespoke show how what would be called esoteric programming language features in software engineering circles, really come into their own in the domain of music technology.

Anyway yes I'm very keen to explore this stuff, and also bring it into physical interfaces. I'm super inspired by Andean Khipu, and planning to host khipu e-textile/sound artist Paola Torres Nũnez del Prado to collaborate on exploring ideas for this. https://github.com/nebogeo/ has been prototyping a tangible interface for this.

I think Strudel is the perfect platform for exploring this stuff! My feeling from these past experiments (proof-of-concepts that weren't yet proved) is that strict types unlock a lot of possibilities visual interfaces. But my experience with C is that the lack of strict types in the host wasn't a barrier to getting them to work. It's a bit fiddly supporting currying but not that bad. Maybe some javascript libraries exist to help.

Anyway back to the topic.. Yes being able to switch between node editing and text editing would be really nice. I think it's good to think as both forms as being essentially based on connecting words. Like text editing is about connecting words by putting them adjacent to one another, and node editing is about connecting words with wires. Both are visual relationships, despite node editors often being called visual editors.

Some confusion comes from thinking about text in terms of letters of the alphabet, because that's how we write and edit the words, unless we're using a structure or block-based editor. Really both node and text editing are different types of text or code editing, we just don't have a good phrase for the latter that I can think of.

Well sorry for all that, I get a bit excited about this topic :)

A quick question as I try to understand your thinking here. In the original code you used x => x.ply(2), and later you have the simpler ply(2), was there a reason for that?

[yaxu]

Nice tree visualisation ideas here: https://treevis.net/

[felixroos]

Nodysseus + strudel : https://www.youtube.com/watch?v=Z1IIpvKx2yQ

[njourdane]

I just discovered this discussion and the suggested ast visualization described above is close to what I did with Jaffle.

For instance here is the Dino Funk example (https://roipoussiere.frama.io/jaffle/#dino_funk):

Graph made with d3.js for anyone wandering.

source code: https://github.com/roipoussiere/jaffle

[jarmitage]

Slightly tangential, but would it be possible to have an AST buffer, so that you could record and loop and pattern ASTs? (i.e. treat as data)

[felixroos]

not sure how ASTs would fit in with a recording feature.. what would be recorded?

[jarmitage]

The AST itself?