4.8.0: API changes, require HHVM 4.8

This release:

• requires HHVM 4.8 due to changes in parser output
• renames HHAST\EditableNode to HHAST\Node, and HHAST\EditableList to HHAST\NodeList; 'editable' was a misleading historical artifact
• removes magic NodeList::getItems() and NodeList::getItemsOfType($classname) wrappers, which would behave differently depending on if the children were ListItems or not
• makes ListItem generic in terms of the single child class
• adds NodeList::getChildrenOfItems() as a clearer, type-safe, less magic replacement of NodeList::getItems(), which is only available if the NodeList is a NodeList<ListItem<T>>
• also added NodeList::getChildrenOfItemsOfType($classname) as a replacement for NodeList::getItemsOfType()
• makes vendor/bin/hhast-* hack executables instead of shell script wrappers

In most cases, code directly using HHAST (such as custom linters) can be migrated with:

find linters/ -type f | xargs sed -i 's/EditableList/NodeList/g'
find linters/ -type f | xargs sed -i 's/Editable//g'
find linters/ -type f | xargs sed -i 's/getItems/getChildrenOfItems/g'

For MacOS, replace sed with gsed from the gnu-sed homebrew package.

Target - and require - HHVM 4.7

Changes in this release

hhast-migrate now recognizes --empty-expression and --hhvm-4.6-to-4.7 - however, they can not be supported on 4.7; instead, they tell the user to use HHAST 4.6.1

Changes in 4.7.0

• target - and require - HHVM 4.7
• function naming linters are now auto-fixing linters

Add `--empty-expression` to `hhast-migrate`

This replaces empty($foo) with (!$foo ?? false)

Target current HHVM nightly builds (4.7-dev)

• target - and require - HHVM 4.7
• function naming linters are now auto-fixing linters

API changes, performance improvements

New AST Linter API

• AST Linters now have a 'context' ancestor node; this is used to simplify code, and to increase performance by reducing the number of nodes to look at for both the lint error and suppression comments. Script and CompoundStatement may be appropriate context nodes if nothing more specific is apparent.
• AST Linters and Lint Errors are no longer generic; instead, type constants are used

For example, in HHAST 4.5:

final class DontAwaitInALoopLinter extends ASTLinter<PrefixUnaryExpression> {
  <<__Override>>
  public function getTargetType(): classname<PrefixUnaryExpression> {
    return PrefixUnaryExpression::class;
  }

  <<__Override>>
  public function getLintErrorForNode(
    PrefixUnaryExpression $node,
  ): ?ASTLintError<PrefixUnaryExpression> {
    // ...
  }
}

In 4.6:

final class DontAwaitInALoopLinter extends ASTLinter {
  const type TContext = ILoopStatement;
  const type TNode = PrefixUnaryExpresison;

  <<__Override>>
  public function getLintErrorForNode(
    this::TContext $context,
    this::TNode $node,
  ): ?ASTLintError {
    // ...
  }
}

New auto-fixing AST linter API

->getFixedNode() is no longer part of the API, and will not be automatically called; instead, a callable can be provided to ASTLintError's constructor.

Migration

bin/hhast-migrate --hhast-4.5-linter-to-4.6 src/FooLinter.hack can be used to partially migrate a linter; this
is intended to do the majority of the work for migration, but a small amount of manual work is likely to be required.

New library entrypoint API

HHAST\from_file_async() is now the only entry point to HHAST's parsing layer, and:

• returns an Awaitable<Script> instead of Awaitable<EditableNode>
• takes an HHAST\File object; this can be produced with HHAST\File::fromPath() or HHAST\File::fromPathAndContents()

Performance improvements

• hh_parse output is now cached; src/foo.hack will create src/.foo.hack.hhast.parser-cache; it's recommended to add these files to source control and editor ignore lists.
• UTF-8 source files are now parsed faster
• getDescendantsOfType() is now much faster
• getAncestorsOfNode() has been replaced by getAncestorsOfDescendant(), which can be called on any node (for example, a linter context node). To retrieve all ancestors, call it on the root Script node.
• node parents/ancestors/ are passed in many fewer places; use getAncestorsOfDescendant() if needed
• namespace APIs now take the root node instead of ancestor lists
• many optimizations to specific linters; in general, this is avoiding use of ancestors, and replacing rewrite()/rewriteDescendants() with replace() or `without()
• re-compute and cache namespace resolution information
• add getFirstAncestorOfDescendantWhere() where a parent node is needed that matches a certain condition - for example, $script->getFirstAncestorOfDescendantWhere($node, $it ==> $it instanceof IStatement)

Other Changes

• fixed cases where suppression comments (e.g. /* HHAST_IGNORE_ERROR[DontAwaitInALoop] */) were not having an effect
• fixed cases where suppression comments were affecting more code than they should
• added Script::getTokens()
• hhast-migrate --await-precedence has been removed as the old behavior is no longer supported by hh_parse
• added UnparsedReplacementCodeNode: this allows creating fake 'AST nodes' with code provided as a string. This is useful if the replacement node structure is complicated and it is certain that there will not be additional changes needed to the replacement code such as other lint fixes or migration steps.

Add `--hhvm-4.5-to-4.6` flag to `hhast-migrate`

This flag is equivalent to --await-precedence, and will be removed in HHAST 4.6.

Make T covariant in `EditableList<T>`

For example, this allows passing EditableList<IExpression> to functions expecting an EditableList<EditableNode>.

More marker interfaces, wrapped name nodes

This release adds many more 'marker' interfaces to quickly identify and work with various kinds of nodes via type tests; the complete list is now:

• IClassBody
• IComment
• IContainer
• IControlFlowStatement
• IExpression
• IFunctionCallishExpression
• IFunctionishDeclaration
• IHackArray
• IHasAttributeSpec
• IHasFunctionBody
• IHasOperator
• IHasTypeHint
• ILambdaBody
• ILambdaSignature
• ILoopStatement
• INameishNode
• INamespaceBody
• INamespaceUseDeclaration
• IPHPArray
• IParameter
• ISimpleCreationSpecifier
• IStatement
• IStringLiteral
• ISwitchLabel
• ITraitUseItem
• ITypeSpecifier
• IXHPAttribute

There is one caveat: NameToken and QualifiedName can sometimes be expressions - when they represent a constant name - but sometimes aren't - for example, when they're representing a class name, such as in an instanceof or is expression. In cases where a field is an IExpression, a wrapper node is created - NameExpression. The underlying NameToken or QualifiedName can be retrieved with getWrappedNode(). This allows targetting IExpression without incorrectly acting on all name tokens and qualified names.

This release supports HHVM 4.3 and above.

Target HHVM 4.3.0

This release removes support for <> from linters and migrations, as that token is no longer produced by HHVM 4.3

Performance improvements, easier profiling, add `--hhvm-4.2-to-4.3` migration flag

This release:

• improves performance of linting extremely large files with lots of lint errors by ~ 65%
• adds the --xhprof-dot foo.dot parameter to hhast-lint, to generate a .dot file suitable for use with GraphViz
• add --hhvm-4.2-to-4.3 to hhast-migrate as an alias of --ltgt-to-ne, replacing the <> operator with !=
• fixes a race condition in the LSP server that could lead to a crash; crashes are extremely harmful to performance with a JIT runtime
• added rewriteChildren() as a shallow alternative to rewriteDescendants()

Performance advice for linter authors

• rewriting the AST is expensive; avoid if (fixed_node($in) !== $in) { raise_error() }-like code
• prefer replace() over rewriteChildren() over rewriteDescendants() over rewrite()