Meta-issue: Tree restructuring

[kddnewton]

In order to better enable a bunch of functionality in Syntax Tree and to take advantage of YARP, we need to restructure our tree. This is a meta-issue that is meant to track the overall status of each node being changed.

Issues that are tracked in this issue should target the structure branch so that we can continue to release from main in case there are any bugs.

YARP	Syntax Tree	Issue
AliasNode	AliasNode	#348
AlternationPatternNode	Binary	#349
AndNode	Binary	#350
ArgumentsNode	Args	#351
ArrayNode	ArrayLiteral, MRHS, QSymbols, QWords, Symbols, Words	#352
ArrayPatternNode	AryPtn	#353
AssocNode	Assoc
AssocSplatNode	AssocSplat
BeginNode	Begin, BodyStmt
BlockArgumentNode	ArgBlock
BlockNode	BlockNode
BlockParameterNode	BlockArg
BlockParametersNode	BlockVar, LambdaVar
BreakNode	Break
CallNode	ARef, ARefField, CallNode, Command, CommandCall, Field, Not, Unary, VCall	#355
CapturePatternNode	Binary	#354
CaseNode	Case
ClassNode	ClassDeclaration
ClassVariableReadNode	CVar
ClassVariableWriteNode	Assign, VarField
ConstantPathNode	ConstPathRef, TopConstRef
ConstantPathWriteNode	ConstPathField, TopConstField, VarField
ConstantReadNode	Const, ConstRef
DefNode	DefNode
DefinedNode	Defined
ElseNode	Else
EnsureNode	Ensure
FalseNode	Kw, VarRef
FindPatternNode	FndPtn
FloatNode	FloatLiteral
ForNode	For
ForwardingArgumentsNode	ArgsForward
ForwardingParameterNode	ArgsForward
ForwardingSuperNode	ZSuper
GlobalVariableReadNode	Backref, GVar
GlobalVariableWriteNode	Assign, VarField
HashNode	BareAssocHash, HashLiteral
HashPatternNode	HshPtn
HeredocNode	Heredoc
IfNode	Elsif, IfNode
ImaginaryNode	Imaginary
InNode	In
InstanceVariableReadNode	IVar
InstanceVariableWriteNode	Assign, VarField
IntegerNode	Int
InterpolatedRegularExpressionNode	RegexpLiteral
InterpolatedStringNode	StringLiteral, Word
InterpolatedSymbolNode	DynaSymbol, Word
InterpolatedXStringNode	XStringLiteral
KeywordParameterNode	Label
KeywordRestParameterNode	KwRestParam
LambdaNode	Lambda
LocalVariableReadNode	VarRef
LocalVariableWriteNode	VarField
MatchPredicateNode	RAssign
MatchRequiredNode	RAssign
MissingNode
ModuleNode	ModuleDeclaration
MultiWriteNode	MAssign, MLHS
NextNode	Next
NilNode	Kw, VarRef
NoKeywordsParameterNode	:nil
OperatorAndAssignmentNode	OpAssign
OperatorAssignmentNode	OpAssign
OperatorOrAssignmentNode	OpAssign
OptionalParameterNode	Ident
OrNode	Binary
ParametersNode	Params
ParenthesesNode	Paren
PinnedExpressionNode	PinnedBegin
PinnedVariableNode	PinnedVarRef
PostExecutionNode	ENDBlock
PreExecutionNode	BEGINBlock
ProgramNode	Program
RangeNode	RangeNode
RationalNode	RationalLiteral
RedoNode	Redo
RegularExpressionNode	RegexpLiteral
RequiredDestructuredParameterNode	MLHSParen
RequiredParameterNode	Ident
RescueModifierNode	RescueMod
RescueNode	Rescue, RescueEx
RestParameterNode	RestParam
RetryNode	Retry
ReturnNode	ReturnNode
Scope
SelfNode	Kw, VarRef
SingletonClassNode	SClass
SourceEncodingNode	Kw, VarRef
SourceFileNode	Kw, VarRef
SourceLineNode	Kw, VarRef
SplatNode	ArgStar
StatementsNode	Statements
StringConcatNode	StringConcat
StringInterpolatedNode	StringEmbExpr
StringNode	CHAR, StringLiteral, Word
SuperNode	Super
SymbolNode	Label, SymbolLiteral, Word
TernaryNode	IfOp
TrueNode	Kw, VarRef
UndefNode	Undef
UnlessNode	UnlessNode
UntilNode	UntilNode
WhenNode	When
WhileNode	WhileNode
XStringNode	XStringLiteral
YieldNode	YieldNode

These nodes are seen as comments and attached to the parse result. For that we should create the same kind of structure instead of adding them to the AST.

• Comment
• EmbDoc, EmbExprBeg, EmbExprEnd
• EndContent

The rest of the nodes in the AST can be removed in favor of just tracking source ranges. This is what YARP and the whitequark/parser gem does and should hopefully cut down on our allocations. These nodes are:

• ArgParen
• Backtick
• Comma
• EmbVar
• ExcessedComma
• HeredocBeg
• HeredocEnd
• Ident
• LBrace
• LBracket
• LParen
• LabelEnd
• MethodAddBlock
• Op
• Period
• QSymbolsBeg
• QWordsBeg
• RBrace
• RBracket
• RParen
• RegexpBeg
• RegexpContent
• RegexpEnd
• StringContent
• StringDVar
• SymBeg
• SymbolContent
• SymbolsBeg
• TLamBeg
• TLambda
• TStringBeg
• TStringContent
• TStringEnd
• VoidStmt
• WordsBeg
• XString

[egiurleo]

I'd like to work on this!

[wenderjean]

@egiurleo Do you need some help on these items? I could take one =)

[egiurleo]

@wenderjean I haven't worked on anything except #299 and #303! Please feel free to take anything else on the list.