Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement kernel compilation in the assembler #1418

Merged
merged 198 commits into from
Jul 30, 2024
Merged
Show file tree
Hide file tree
Changes from 193 commits
Commits
Show all changes
198 commits
Select commit Hold shift + click to select a range
a94d52d
Introduce `ExternalNode`
plafer Jun 20, 2024
f8ad339
Replace `Assembler.node_id_by_digest` map
plafer Jun 20, 2024
06f421b
add TODOP
plafer Jun 20, 2024
bcf2a9b
Add `Host::get_mast_forest`
plafer Jun 20, 2024
980713d
Move kernel and entrypoint out of `MastForest`
plafer Jun 21, 2024
d2bbcd2
Add Host::get_mast_forest
plafer Jun 21, 2024
32e757e
Remove ProgramError
plafer Jun 21, 2024
f87217a
docs
plafer Jun 21, 2024
71f35f1
cleanup Program constructors
plafer Jun 21, 2024
a39f74f
fix docs
plafer Jun 21, 2024
f7e98af
Make `Program.kernel` an `Arc`
plafer Jun 21, 2024
b5538c5
fix executable
plafer Jun 21, 2024
2eab574
Merge remote-tracking branch 'origin/next' into plafer-object-store
plafer Jun 21, 2024
188651b
invoke_mast_root: fix external node creation logic
plafer Jun 23, 2024
b15263c
add failing test
plafer Jun 23, 2024
a94a095
don't make root in `combine_mast_node_ids` and `compile_body`
plafer Jun 23, 2024
7af0bfa
fix External docs
plafer Jun 23, 2024
a6fcf47
fmt
plafer Jun 23, 2024
c66db6f
fix `entrypoint` doc
plafer Jun 23, 2024
572fc7e
Rename `Program::new_with_kernel()`
plafer Jun 23, 2024
08ce2c7
Document `MastForestStore` and `MemMastForestStore`
plafer Jun 23, 2024
50e01e9
fix syscall
plafer Jun 23, 2024
071ab54
execute_* functions: use `MastForest`
plafer Jun 23, 2024
49de40d
`Program`: Remove `Arc` around kernel
plafer Jun 23, 2024
c28c876
remove `Arc` around `MastForest` in `Program`
plafer Jun 23, 2024
78b2b16
Return error on malformed host
plafer Jun 23, 2024
4883b44
Simplify `DefaultHost`
plafer Jun 23, 2024
155a798
`MastForest::add_node()`: add docs
plafer Jun 23, 2024
bc6d13e
fmt
plafer Jun 23, 2024
be24320
add failing `duplicate_procedure()` test
plafer Jun 23, 2024
32aedd6
Introduce `MastForestBuilder`
plafer Jun 23, 2024
088de82
Rename `mod tests` -> `testing`
plafer Jun 25, 2024
9d48fda
add `duplicate_node()` test
plafer Jun 25, 2024
6c62d9b
changelog
plafer Jun 25, 2024
039bba0
Program: use `assert!()` instead of `debug_assert!()`
plafer Jun 26, 2024
9c9e171
`MastForest::make_root()`: add assert
plafer Jun 26, 2024
8e4dc5e
Merge remote-tracking branch 'origin/next' into plafer-object-store
plafer Jun 26, 2024
c34e985
fmt
plafer Jun 26, 2024
c1b2693
Serialization for `MastNodeId`
plafer Jun 26, 2024
efc24fd
serialization for MastNode variants except basic block
plafer Jun 26, 2024
861d0d5
MastForest serialization scaffolding
plafer Jun 27, 2024
6173790
define `MastNodeType` constructor from `MastNode`
plafer Jun 27, 2024
e018cc8
test join serialization of MastNodeType
plafer Jun 27, 2024
6671afe
`MastNodeType` serialization of split
plafer Jun 27, 2024
babdd0c
Revert "serialization for MastNode variants except basic block"
plafer Jun 27, 2024
cd06527
add TODOP
plafer Jun 27, 2024
2dd4829
impl Deserializable for `MastForest` (scaffold)
plafer Jul 1, 2024
4be1401
mast_node_to_info() scaffold
plafer Jul 1, 2024
2695062
try_info_to_mast_node scaffold
plafer Jul 1, 2024
ef0a881
Rename `EncodedMastNodeType`
plafer Jul 1, 2024
dd89461
add info module
plafer Jul 1, 2024
91009b0
encode operations into `data` field
plafer Jul 1, 2024
d5ed108
decode operations
plafer Jul 1, 2024
0cf49ff
implement `BasicBlockNode::num_operations_and_decorators()`
plafer Jul 2, 2024
64c36ec
OperationOrDecoratorIterator
plafer Jul 2, 2024
794ebbb
basic block node: move tests in new file
plafer Jul 2, 2024
49673ae
operation_or_decorator_iterator test
plafer Jul 2, 2024
a5c324b
Implement `Operation::with_opcode_and_data()`
plafer Jul 2, 2024
c04bc90
encode decorators
plafer Jul 2, 2024
27e2783
implement `decode_decorator()`
plafer Jul 2, 2024
25fe82f
fix exec invocation
plafer Jul 2, 2024
36ecdd7
Merge remote-tracking branch 'origin/next' into plafer-object-store
plafer Jul 2, 2024
3c26bd6
no else blk special case
plafer Jul 2, 2024
34c2f7f
add procedure roots comment
plafer Jul 2, 2024
97d3de8
Merge remote-tracking branch 'origin/next' into plafer-object-store
plafer Jul 3, 2024
3166dc2
Merge branch 'plafer-object-store' into plafer-mast-forest-serialization
plafer Jul 3, 2024
ffc7c78
implement forgotten `todo!()`
plafer Jul 3, 2024
5504157
`serialize_deserialize_all_nodes` test
plafer Jul 3, 2024
93c4fca
`decode_operations_and_decorators`: fix bit check
plafer Jul 3, 2024
da14984
confirm_assumptions test scaffold
plafer Jul 3, 2024
4870b32
minor adjustments
plafer Jul 3, 2024
072b0b9
Introduce `StringTableBuilder`
plafer Jul 3, 2024
a04836c
Merge remote-tracking branch 'origin/next' into plafer-mast-forest-se…
plafer Jul 3, 2024
48aec6a
naming
plafer Jul 3, 2024
a4ef4b1
test confirm_operation_and_decorator_structure
plafer Jul 3, 2024
ca0e7fe
remove TODOP
plafer Jul 3, 2024
78e35fc
remove unused `MastNode::new_dyncall()`
plafer Jul 3, 2024
411f9f3
Remove `Error` type
plafer Jul 3, 2024
624984c
add TODOP
plafer Jul 3, 2024
858582a
complete test `serialize_deserialize_all_nodes`
plafer Jul 3, 2024
4e5efd3
check digest on deserialization
plafer Jul 3, 2024
3f522b8
remove TODOP
plafer Jul 3, 2024
161578d
safely decode mast node ids
plafer Jul 3, 2024
2ab1cf9
use method syntax in `MastNodeType` decoding
plafer Jul 3, 2024
c8cfa8e
TODOPs
plafer Jul 3, 2024
a360959
rewrite <= expression
plafer Jul 3, 2024
c8ba463
new `MastNodeType`
plafer Jul 3, 2024
ad18580
implement `Deserializable` for `MastNodeType`
plafer Jul 3, 2024
996498f
migrate tests to new
plafer Jul 3, 2024
e60bfc2
Use new MastNodeType
plafer Jul 3, 2024
578bda9
rename string_table_builder_ module
plafer Jul 3, 2024
d25671c
implement `BasicBlockDataBuilder`
plafer Jul 3, 2024
99f4544
add TODOP
plafer Jul 3, 2024
0d20629
BasicBlockDataDecoder
plafer Jul 3, 2024
b66f81b
use `BasicBlockDataDecoder`
plafer Jul 3, 2024
f9a3a0b
add headers
plafer Jul 3, 2024
df621c5
add `MastNodeInfo` method
plafer Jul 3, 2024
2afd588
return `Result` instead of `Option`
plafer Jul 3, 2024
9106a6f
Remove TODOP
plafer Jul 3, 2024
fe8b7a7
docs
plafer Jul 3, 2024
5869525
chore: add section separators and fix typos
bobbinth Jul 7, 2024
8cb3462
refactor: change type of the error code of u32assert2 from Felt to u3…
Fumuran Jul 10, 2024
5ec4826
impl `Serializable` for `Operation`
plafer Jul 10, 2024
d4a50e9
impl Deserializable for `Operation`
plafer Jul 10, 2024
3ce71b9
`StringTableBuilder`: switch to using blake 3
plafer Jul 10, 2024
db33dc7
`EncodedDecoratorVariant`: moved discriminant bit logic to `discrimin…
plafer Jul 10, 2024
10f02a6
Remove basic block offset
plafer Jul 10, 2024
421518d
Cargo: don't specify patch versions
plafer Jul 10, 2024
cf100c5
make deserialization more efficient
plafer Jul 10, 2024
f50073d
num-traits and num-derive: set default-features false
plafer Jul 10, 2024
aacb9e9
Merge branch 'next' into plafer-mast-forest-serialization
plafer Jul 11, 2024
956aac1
Remove `OperationData`
plafer Jul 11, 2024
9fe4e0e
`StringRef`: move string length to data buffer
plafer Jul 11, 2024
c525828
store offset in block
plafer Jul 11, 2024
2b332e1
Use `source.read_u32/u64()`
plafer Jul 12, 2024
9ca910a
Update `MastNodeInfo` docstring
plafer Jul 12, 2024
5c6f287
rename arguments in `encode_u32_pair`
plafer Jul 12, 2024
58df61e
Use basic block offset in deserialization
plafer Jul 12, 2024
1cdc419
`BasicBlockDataDecoder`: use `ByteReader::read_u16/32()` methods
plafer Jul 12, 2024
f45d0c5
`StringTableBuilder`: fix comment
plafer Jul 12, 2024
43cadeb
Remove `StringRef` in favor of `DataOffset`
plafer Jul 12, 2024
7dec428
cleanup `MastNodeType` serialization
plafer Jul 12, 2024
c081b00
derive `Copy` for `MastNodeType`
plafer Jul 12, 2024
c32ef22
`MastNodeType` tests
plafer Jul 12, 2024
c7ee9c8
add `MastNodeType` tests
plafer Jul 12, 2024
f9d2e59
use assert
plafer Jul 12, 2024
781fc73
fix asserts
plafer Jul 12, 2024
fa91716
`ModuleGraph::recompute()` reverse edge caller/callee
plafer Jul 16, 2024
1bd84ec
Implement `Assembler::assemble_library()`
plafer Jul 16, 2024
04e6b0a
Merge branch 'next' into plafer-mast-forest-serialization
plafer Jul 17, 2024
e6ef660
Merge branch 'plafer-mast-forest-serialization' into plafer-mast-libr…
plafer Jul 17, 2024
0a9cfad
changelog
plafer Jul 17, 2024
1c0e0a5
Merge branch 'plafer-mast-forest-serialization' into plafer-mast-libr…
plafer Jul 17, 2024
40cbbb9
fix docs
plafer Jul 17, 2024
727a62a
Merge branch 'plafer-mast-forest-serialization' into plafer-mast-libr…
plafer Jul 17, 2024
f405a2b
Introduce `CompiledFQDN`
plafer Jul 17, 2024
4236a84
Introduce `WrapperModule` to module graph
plafer Jul 17, 2024
c1947c2
split `ModuleGraph::add_module()`
plafer Jul 17, 2024
7878032
fix compile errors from API changes
plafer Jul 17, 2024
b3e337a
fix debug structs
plafer Jul 17, 2024
a4a5e17
fix `Assembler::get_module_exports()`
plafer Jul 18, 2024
b130167
fix `process_graph_worklist`
plafer Jul 18, 2024
1b5a1b5
fix procedure
plafer Jul 18, 2024
9bb9ed5
fix `NameResolver`
plafer Jul 18, 2024
f5699f1
move `CompiledModule`
plafer Jul 18, 2024
8d148ab
`CompiledLibrary::into_compiled_modules`
plafer Jul 18, 2024
2904e1e
`Assembler::add_compiled_library()`
plafer Jul 18, 2024
2adf48d
changelog
plafer Jul 18, 2024
da72cd0
Merge branch 'next' into plafer-mast-library-compilation
plafer Jul 18, 2024
f9d6c15
fix `assemble_library()` signature
plafer Jul 18, 2024
5061941
test `compiled_library()`
plafer Jul 18, 2024
4fba058
nits
plafer Jul 18, 2024
c287f73
register mast roots in `Assembler::add_compiled_library()`
plafer Jul 18, 2024
f5880e7
fix resolve
plafer Jul 18, 2024
b657031
`ModuleGraph::topological_sort_from_root`: only include AST procedures
plafer Jul 18, 2024
c0fb87a
`Assembler::resolve_target()`: look for digest in module graph first
plafer Jul 19, 2024
d3b856c
remove `AssemblyContext::allow_phantom_calls` flag
plafer Jul 19, 2024
096f289
remove TODOP
plafer Jul 19, 2024
f0efb06
`ResolvedProcedure` is no longer `Spanned`
plafer Jul 19, 2024
cca6fe2
improve test
plafer Jul 19, 2024
c83ba0b
remove TODOP
plafer Jul 19, 2024
db5a415
`CompiledProcedure` -> `ProcedureInfo`
plafer Jul 19, 2024
d0c1726
Document `CompiledLibrary`
plafer Jul 19, 2024
f5c8154
Rename `CompiledModule` -> `ModuleInfo`
plafer Jul 19, 2024
3456d9c
Refactor `ModuleInfo`
plafer Jul 19, 2024
7d7af17
`ModuleWrapper` -> `WrappedModule`
plafer Jul 19, 2024
033b741
Document `PendingModuleWrapper`
plafer Jul 19, 2024
7190323
document `Assembler::assemble_library()`
plafer Jul 19, 2024
1ee70da
fix TODOP
plafer Jul 19, 2024
9263fbb
rename
plafer Jul 19, 2024
c0ee866
Merge branch 'next' into plafer-mast-library-compilation
plafer Jul 19, 2024
f12ce36
Merge branch 'next' into plafer-mast-library-compilation
plafer Jul 24, 2024
5b81eec
Merge branch 'next' into plafer-mast-library-compilation
plafer Jul 25, 2024
a784b00
fix test
plafer Jul 25, 2024
98ae7bc
cleanup `ModuleGraph::topological_sort_from_root`
plafer Jul 25, 2024
250e86d
fix CI
plafer Jul 25, 2024
42a991a
re-implement `Spanned` for `ResolvedProcedure`
plafer Jul 25, 2024
8669dfc
reintroduce proper error message
plafer Jul 25, 2024
d444d67
remove unused methods
plafer Jul 25, 2024
48d59a9
Remove all `allow(unused)` methods
plafer Jul 25, 2024
770b25d
Document `unwrap_ast()` call
plafer Jul 26, 2024
53e6555
`NameResolver`: remove use of `unwrap_ast()`
plafer Jul 26, 2024
30ef4c5
Document or remove all calls to `WrappedModule.unwrap_ast()`
plafer Jul 26, 2024
2c43ed4
rename `PendingWrappedModule`
plafer Jul 26, 2024
258aa99
Add `ModuleGraph::add_compiled_modules()`
plafer Jul 26, 2024
49879de
Remove `ModuleGraph::add_module_info()`
plafer Jul 26, 2024
302c4cc
refactor: remove Assembler::compile_program() internal method
bobbinth Jul 28, 2024
fa4b79e
refactor: remove Assembler::assemble_with_options() internal method
bobbinth Jul 28, 2024
021e0cc
feat: implement Assembler:assemble_kernel()
bobbinth Jul 29, 2024
ede435a
chore: remove unused assembly errors
bobbinth Jul 29, 2024
52970ad
feat: add AssemblerError
bobbinth Jul 29, 2024
0f2cfe4
refactor: add ModuleGraph::with_kernel() constructor
bobbinth Jul 29, 2024
58dd399
chore: update changelog
bobbinth Jul 29, 2024
ea032fc
Merge branch 'next' into bobbin-assemble-kernel
bobbinth Jul 29, 2024
3b6db74
chore: fix lint
bobbinth Jul 29, 2024
a3fab87
refactor: introduce KernelLibrary
bobbinth Jul 29, 2024
a5a151d
chore: fix lints
bobbinth Jul 29, 2024
8292606
refactor: remove ModuleIndex param from Module::exported_procedures()
bobbinth Jul 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,16 @@
- Relaxed the parser to allow one branch of an `if.(true|false)` to be empty
- Added support for immediate values for `u32and`, `u32or`, `u32xor` and `u32not` bitwise instructions (#1362).
- Optimized `std::sys::truncate_stuck` procedure (#1384).
- Updated CI and Makefile to standardise it accross Miden repositories (#1342).
- Added serialization/deserialization for `MastForest` (#1370)
- Updated CI and Makefile to standardize it across Miden repositories (#1342).
- Add serialization/deserialization for `MastForest` (#1370)
- Assembler: add the ability to compile MAST libraries, and to assemble a program using compiled libraries (#1401)
- Updated CI to support `CHANGELOG.md` modification checking and `no changelog` label (#1406)
- Introduced `MastForestError` to enforce `MastForest` node count invariant (#1394)
- Added functions to `MastForestBuilder` to allow ensuring of nodes with fewer LOC (#1404)
- Make `Assembler` single-use (#1409)
- Remove `ProcedureCache` from the assembler (#1411).
- Add `Assembler::assemble_library()` (#1413)
- [BREAKING] Made `Assembler` single-use (#1409).
- Removed `ProcedureCache` from the assembler (#1411).
- Added `Assembler::assemble_library()` (#1413).
- Added `Assembler::assemble_kernel()` (#1418).

#### Changed

Expand Down
5 changes: 2 additions & 3 deletions assembly/src/assembler/basic_block_builder.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
use crate::AssemblyError;
use crate::{ast::Instruction, AssemblyError};

use super::{
mast_forest_builder::MastForestBuilder, BodyWrapper, Decorator, DecoratorList, Instruction,
ProcedureContext,
mast_forest_builder::MastForestBuilder, BodyWrapper, Decorator, DecoratorList, ProcedureContext,
};
use alloc::{borrow::Borrow, string::ToString, vec::Vec};
use vm_core::{mast::MastNodeId, AdviceInjector, AssemblyOp, Operation};
Expand Down
6 changes: 3 additions & 3 deletions assembly/src/assembler/instruction/mod.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
use super::{
ast::InvokeKind, mast_forest_builder::MastForestBuilder, Assembler, BasicBlockBuilder, Felt,
Instruction, Operation, ProcedureContext, ONE, ZERO,
Operation, ProcedureContext,
};
use crate::{diagnostics::Report, utils::bound_into_included_u64, AssemblyError};
use crate::{ast::Instruction, diagnostics::Report, utils::bound_into_included_u64, AssemblyError};
use core::ops::RangeBounds;
use vm_core::{mast::MastNodeId, Decorator};
use vm_core::{mast::MastNodeId, Decorator, ONE, ZERO};

mod adv_ops;
mod crypto_ops;
Expand Down
5 changes: 4 additions & 1 deletion assembly/src/assembler/instruction/procedures.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,10 @@ impl Assembler {
callee: proc.fully_qualified_name().clone(),
})
.and_then(|module| {
if module.is_kernel() {
// Note: this module is guaranteed to be of AST variant, since we have the
// AST of a procedure contained in it (i.e. `proc`). Hence, it must be that
// the entire module is in AST representation as well.
if module.unwrap_ast().is_kernel() {
Ok(())
} else {
Err(AssemblyError::InvalidSysCallTarget {
Expand Down
Loading
Loading