Update the docs for reconstruction

README.md

@@ -44,11 +44,11 @@ with open('foo.tf', 'r') as file:
 
 ### Parse Tree to HCL2 reconstruction
 
-With version 5.0.0 the possibility of HCL2 reconstruction from Lark Parse Tree was introduced.
+With version 5.x the possibility of HCL2 reconstruction from the Lark Parse Tree and Python dictionaries directly was introduced.
 
-Example of manipulating Lark Parse Tree and reconstructing it back into valid HCL2 can be found in [tree-to-hcl2-reconstruction.md](https://github.com/amplify-education/python-hcl2/blob/main/tree-to-hcl2-reconstruction.md) file.
+Documentation and an example of manipulating Lark Parse Tree and reconstructing it back into valid HCL2 can be found in [tree-to-hcl2-reconstruction.md](https://github.com/amplify-education/python-hcl2/blob/main/tree-to-hcl2-reconstruction.md) file.
 
-More details about reconstruction implementation can be found in this [PR](https://github.com/amplify-education/python-hcl2/pull/169).
+More details about reconstruction implementation can be found in PRs #169 and #177.
 
 ## Building From Source
 

tree-to-hcl2-reconstruction.md

@@ -1,13 +1,111 @@
-Given `example.tf` file with following content
+# Writing HCL2 from Python
+
+Version 5 of this library supports reconstructing HCL files directly from
+Python. This guide details how the reconstruction process takes place. See
+also: [Limitations](#limitations)
+
+There are three major phases:
+
+- [Building a Python Dictionary](#building-a-python-dictionary)
+- [Building an AST](#building-an-ast)
+- [Reconstructing the file from the AST](#reconstructing-the-file-from-the-ast)
+
+## Example
+
+To create the `example.tf` file with the following content:
 
 ```terraform
 resource "aws_s3_bucket" "bucket" {
   bucket = "bucket_id"
-  force_destroy   = true
+  force_destroy = true
+}
+```
+
+You can use the `hcl2.Builder` class like so:
+
+```python
+import hcl2
+
+example = hcl2.Builder()
+
+example.block(
+    "resource",
+    ["aws_s3_bucket", "bucket"],
+    bucket="bucket_id",
+    force_destroy=True,
+)
+
+example_dict = example.build()
+example_ast = hcl2.reverse_transform(example_dict)
+example_file = hcl2.writes(example_ast)
+
+print(example_file)
+# resource "aws_s3_bucket" "bucket" {
+#   bucket = "bucket_id"
+#   force_destroy = true
+# }
+#
+```
+
+This demonstrates a couple of different phases of the process worth mentioning.
+
+### Building a Python dictionary
+
+The `hcl2.Builder` class produces a dictionary that should be identical to the
+output of `hcl2.load(example_file, with_meta=True)`. The `with_meta` keyword
+argument is important here. HCL "blocks" in the Python dictionary are
+identified by the presence of `__start_line__` and `__end_line__` metadata
+within them. The `Builder` class handles adding that metadata. If that metadata
+is missing, the `hcl2.reconstructor.HCLReverseTransformer` class fails to
+identify what is a block and what is just an attribute with an object value.
+Without that metadata, this dictionary:
+
+```python
+{
+    "resource": [
+        {
+            "aws_s3_bucket": {
+                "bucket": {
+                    "bucket": "bucket_id",
+                    "force_destroy": True,
+                    # "__start_line__": -1,
+                    # "__end_line__": -1,
+                }
+            }
+        }
+    ]
 }
 ```
 
-below code will add a `tags` object to the S3 bucket definition. The code can also be used to print out readable representation of **any** Parse Tree (any valid HCL2 file), which can be useful when working on your own logic for arbitrary Parse Tree manipulation.
+Would produce this HCL output:
+
+```terraform
+resource = [{
+    aws_s3_bucket = {
+        bucket = {
+            bucket = "bucket_id"
+            force_destroy = true
+        }
+    }
+}]
+```
+
+(This output parses to the same datastructure, but isn't formatted in blocks
+as desired by the user. Therefore, using the `Builder` class is recommended.)
+
+### Building an AST
+
+The `hcl2.reconstructor.HCLReconstructor` class operates on an "abstract
+syntax tree" (`hcl2.AST` or `Lark.Tree`, they're the same.) To produce this AST
+from scratch in Python, use `hcl2.reverse_transform(hcl_dict)`, and to produce
+this AST from an existing HCL file, use `hcl2.parse(hcl_file)`.
+
+You can also build these ASTs manually, if you want more control over the
+generated HCL output. If you do this, though, make sure the AST you generate is
+valid within the `hcl2.lark` grammar.
+
+Here's an example, which would add a "tags" element to that `example.tf` file
+mentioned above.
 
 ```python
 from copy import deepcopy
@@ -128,3 +226,23 @@ if __name__ == "__main__":
     main()
 
 ```
+
+### Reconstructing the file from the AST
+
+Once the AST has been generated, you can convert it back to valid HCL using
+`hcl2.writes(ast)`. In the above example, that conversion is done in the
+`main()` function.
+
+## Limitations
+
+- Some formatting choices are impossible to specify via `hcl2.Builder()` and
+  require manual intervention of the AST produced after the `reverse_transform`
+  step.
+
+    - Most notably, this means it's not possible to generate files containing
+      comments (both inline and block comments)
+
+- Even when parsing a file directly and writing it back out, some formatting
+  information may be lost due to Terminals discarded during the parsing process.
+  The reconstructed output should still parse to the same dictionary at the end
+  of the day though.