some notes for refactor

TODO.txt

@@ -1,9 +1,25 @@
+Cross-compilation:
+
+Goal: switch from one emission of llvm to architecture-specific version of llvm.
+
+-- Add a compiler intermediate compilationunit, which maps function definition, global, constant, and everything that has a llvm_value field right now to the llvm value. 
+
+-- Remove emitllvm, typechecking now only handles generating types and invoking user callbacks, never emitting/optimizing llvm
+
+-- consider refactoring the SCC detection into it own object, or labelling each function with an identifier for the SCC it is in
+
+--  simplify the state for functions: untyped, in progress, typed, re-examine peektype, etc.
+--  see if extern functions do not need "initialization" anymore
+
+-- emitting llvm is now given a compilationunit, a root function to add to it, and archiecture
+-- include C creates a separate compilationunit, examine linking rules for including it.
+
 Language Implementation:
 
-Represent aggregate constants as aggregates rather than current method of casting
-switch construct, computed goto
-vector shuffle?
-cmpxchg, atomicrmw, fence
+-- Represent aggregate constants as aggregates rather than current method of casting
+-- switch construct, computed goto
+-- vector shuffle?
+-- cmpxchg, atomicrmw, fence (all but fence can be part of attrstore)
 
 Design Problems:
 -- how to save results for later (both Lua state and Terra JIT state)
@@ -14,45 +30,6 @@ Low-priority Implementation:
 -- pointer arithmetic on niltype?
 -- better handling of options to try for overloaded operators
 
-Implementation:
--- better handling of lua layout of structs with recursion (can we seperate the creation of structs for their layout to simplify the problem) -- test the compiler fo handling jitting of functions previously optimized with nojit = true
-   (look into the way LLVM handles pointers to opaque structs, can we delay layout of a struct until we need to access it similar to the way C allows opaque struct pointers).
-
--- tests for garbage collection of objects that mix LLVM state with Lua state to check for memory leaks from dangling references
-
---track down why some types RecordType structs do not have names, and fix the bugs that are caused by this. 
-
---refactor ctx object and terra.resolvetype it should be possible to canonicalize and resolve types without a ctx object
-  instead there should be a seperate diagnostic object for reporting errors, and resolvetype should be given an environment in which to run
-  getcanonical() should then take an _optional_ diagnostic object
-
--- casting lua functions to Terra functions when they return multiple arguments, and supporting calling Lua functions with structs values/expecting struct returns
-
-Making specialization eager and simplifying the compiler.
-
--- seperate specialization phase (e.g. variable resolution, select resolution, escape evaluation) from typechecking, specialization will be run eagerly
-
--- make the compilation graph a singleton, environments should be handled by function compilation individually. If something is re-entrant into compilation from lua code and there are compile errors stop compilation. Double-check that re-entrant compiler code can compile as long as it is in a different SCC from its compile-time caller. Remove ctx objects as arguments. fnvariant:gettype() and fnvariant:compile() should be seperate from the internal interfaces.
-
--- add asynchronous versions of :compile and :gettype that can be called when the compiler is running
--- support LLVM constant expressions: terralib.constant(`exp) (this will allow vtables etc. to be constructed)
-
--- (lower priority) allow codegeneration for opaque struct types when they are only used as pointers. this will require cooperation from the compiler to keep the struct opaque unless it is marked complete. compiler will need to be annotated to force completion of struct types when they are used as values (either directly or in an aggregate).
-
-Minitasks that came up during eager transform:
-
---luafunction is gone
---write the "documentor"
---the attribute macro is broken
--- getcanonical should cache whether it has been done on aggregates so it doesn't traverse types a bunch of times
-
-Things to write tests/audit for:
-
--- literals are only generated by the parser, constants handles all stuff from lua (including strings), add correct support for strings
--- select and var handling have moved
--- truncation behavior (since it was changed from a field to a node)
-
--- pretty print is being weird on dgemmpaper.t, add support for typedexpressionlist
 
 DONE:
 --function pointers
@@ -111,3 +88,43 @@ DONE:
 -- change parser rule to allow let rec form that can include  local/global structs/terra declarations/definitions together. modify terralib to accept this form and remove constructors for other forms.
 -- tag _all_ tree nodes with the filename and remove the complicated stack of files for error reporting (we can compress tree node size later)
 -- remove var syntax from lua. Instead use terralib.global to create global variables.
+
+Implementation:
+-- better handling of lua layout of structs with recursion (can we seperate the creation of structs for their layout to simplify the problem) -- test the compiler fo handling jitting of functions previously optimized with nojit = true
+   (look into the way LLVM handles pointers to opaque structs, can we delay layout of a struct until we need to access it similar to the way C allows opaque struct pointers).
+
+-- tests for garbage collection of objects that mix LLVM state with Lua state to check for memory leaks from dangling references
+
+--track down why some types RecordType structs do not have names, and fix the bugs that are caused by this. 
+
+--refactor ctx object and terra.resolvetype it should be possible to canonicalize and resolve types without a ctx object
+  instead there should be a seperate diagnostic object for reporting errors, and resolvetype should be given an environment in which to run
+  getcanonical() should then take an _optional_ diagnostic object
+
+-- casting lua functions to Terra functions when they return multiple arguments, and supporting calling Lua functions with structs values/expecting struct returns
+
+Making specialization eager and simplifying the compiler.
+
+-- seperate specialization phase (e.g. variable resolution, select resolution, escape evaluation) from typechecking, specialization will be run eagerly
+
+-- make the compilation graph a singleton, environments should be handled by function compilation individually. If something is re-entrant into compilation from lua code and there are compile errors stop compilation. Double-check that re-entrant compiler code can compile as long as it is in a different SCC from its compile-time caller. Remove ctx objects as arguments. fnvariant:gettype() and fnvariant:compile() should be seperate from the internal interfaces.
+
+-- add asynchronous versions of :compile and :gettype that can be called when the compiler is running
+-- support LLVM constant expressions: terralib.constant(`exp) (this will allow vtables etc. to be constructed)
+
+-- (lower priority) allow codegeneration for opaque struct types when they are only used as pointers. this will require cooperation from the compiler to keep the struct opaque unless it is marked complete. compiler will need to be annotated to force completion of struct types when they are used as values (either directly or in an aggregate).
+
+Minitasks that came up during eager transform:
+
+--luafunction is gone
+--write the "documentor"
+--the attribute macro is broken
+-- getcanonical should cache whether it has been done on aggregates so it doesn't traverse types a bunch of times
+
+Things to write tests/audit for:
+
+-- literals are only generated by the parser, constants handles all stuff from lua (including strings), add correct support for strings
+-- select and var handling have moved
+-- truncation behavior (since it was changed from a field to a node)
+
+-- pretty print is being weird on dgemmpaper.t, add support for typedexpressionlist