Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring of MVAU and VVAU #963

Closed
Closed
Show file tree
Hide file tree
Changes from 126 commits
Commits
Show all changes
131 commits
Select commit Hold shift + click to select a range
be1503a
First changes to custom_op for RTL-based MVAU
mmrahorovic Jan 3, 2023
8265985
Merge remote-tracking branch 'upstream/dev' into feature/dsp_packing
mmrahorovic Apr 5, 2023
afab9cd
[rtl custom op]: initial implementation of mvu_8sx9
mmrahorovic Apr 6, 2023
a94fc3b
[rtl custom op]: testbench for mvu_8sx9
mmrahorovic Apr 6, 2023
98f9acc
[rtl custom op]: initial implementation of flow control component for…
mmrahorovic Apr 6, 2023
96925a9
[rtl custom op]: implementation of replay buffer for mvu
mmrahorovic Apr 6, 2023
a3d1156
[rtl custom op]: testbench for mvu_8sx9_axi (including axi_wrapper & …
mmrahorovic Apr 6, 2023
2aea664
[rtl custom op]: initial implementation of verilog wrapper for mvu_8s…
mmrahorovic Apr 6, 2023
c92e4e3
Merge remote-tracking branch 'upstream/dev' into feature/dsp_packing
mmrahorovic Apr 6, 2023
8b57849
[rtl mvu]: fix tab indentation
mmrahorovic Apr 11, 2023
5e61f42
[rtl custom op]: fix to indentation
mmrahorovic Apr 12, 2023
cbee193
[rtl custom-op]: minor changes for compiler integration
mmrahorovic Apr 12, 2023
ba5e77b
[rtl custom op]: moved testbenches to separate directory
mmrahorovic Apr 12, 2023
69310b4
[rtl custom op]: fixed output width to ACCU_WIDTH
mmrahorovic Apr 12, 2023
cfcff00
[rtl custom op]: renamed file and added generic to switch between com…
mmrahorovic Apr 12, 2023
72b5196
[rtl custom op]: renamed file and added generic to switch between com…
mmrahorovic Apr 12, 2023
c068bb6
[rtl mvu]: added behavioral model DSP58
mmrahorovic May 8, 2023
18f94e7
[rtl mvu]: extended flow control wrapper with additional compute core…
mmrahorovic May 8, 2023
6d4a0a7
[rtl mvu]: fix to done_len flag when SIMD dimension fully unrolled an…
mmrahorovic May 8, 2023
90c547d
[rtl mvu tb]: updated testbench
mmrahorovic May 8, 2023
0c37f1f
[builder]: added specialize_to_rtl step and changed standalone thresh…
mmrahorovic May 8, 2023
5ccb016
[builder]: added specialize_to_rtl step
mmrahorovic May 8, 2023
f099f4b
[custom op]: added custom op MatrixVectorActivation_rtl
mmrahorovic May 8, 2023
9a3b0fd
[custom op]: added additional attribute to enable conversion to RTL (…
mmrahorovic May 8, 2023
38aa930
[custom op]: modified ip-stitching and code generation
mmrahorovic May 8, 2023
4e44934
[tests]: initial version of unit test for RTL custom op and specializ…
mmrahorovic May 8, 2023
cc361d9
[rtl mvu]: specialized compute core for 4-bit weights and activations…
mmrahorovic May 8, 2023
8eefb53
[rtl mvu]: specialized compute core for > 4-bit weights and activatio…
mmrahorovic May 8, 2023
e7109e7
[fpgadataflow transform]: initial specialize_to_rtl_layers-transform …
mmrahorovic May 8, 2023
d107b4d
Merge remote-tracking branch 'upstream/dev' into feature/dsp_packing
mmrahorovic May 9, 2023
5a868d1
[rtl mvu] fixes for latest memstream + linting
maltanar May 9, 2023
4a9cfa1
[rtl custom_op]: add support for external weights
mmrahorovic May 11, 2023
8a9ac1a
Specify clock and reset associations of bus interfaces.
preusser May 11, 2023
51bbe02
Merge remote-tracking branch 'upstream/dev' into feature/dsp_packing
mmrahorovic May 21, 2023
3d856b7
Merge branch 'dev' into feature/dsp_packing
preusser May 23, 2023
d9b9079
[rtlmvu] More fixes for memstream and param gen
maltanar May 15, 2023
a5f2a83
[Build] apply config to only FIFO nodes in step_set_fifo_depths
maltanar May 11, 2023
08cbdc5
Revised control interface attributes.
preusser May 24, 2023
48f0c5c
Merge branch 'dev' into feature/dsp_packing
preusser May 24, 2023
d058cc2
Mask device primitives from Verilator in favor of using behavioral code.
preusser May 24, 2023
a66f38f
[Deps] update qonnx
maltanar May 11, 2023
8f9bd04
Adding folding hints. Impl selection by case statement.
preusser May 24, 2023
8799707
Merge branch 'feature/verilator_workarounds' into feature/dsp_packing
preusser May 24, 2023
9de5ed6
Fixed behavioral sideband prediction.
preusser May 24, 2023
b6e92bb
Merge remote-tracking branch 'origin/feature/dsp_packing' into featur…
mmrahorovic May 24, 2023
239759a
[rtl mvu]: extension to allow selecting PE values that are not multip…
mmrahorovic May 24, 2023
8d3247c
[rtlmvu] Avoid unintentional verilator metacomments
maltanar May 24, 2023
ffc11d6
Merge remote-tracking branch 'origin/feature/dsp_packing' into featur…
mmrahorovic May 24, 2023
c866350
[rtl mvu]: extension to allow selecting PE values that are not multip…
mmrahorovic May 24, 2023
fd1e038
[rtl mvu axi]: updated comments on folding hints
mmrahorovic May 24, 2023
f60d4c6
[rtl custom op]: minor fixes to codegen
mmrahorovic Jun 2, 2023
a1ad304
[specialize-to-rtl]: add ram_style and rt_writeable_weights support
mmrahorovic Jun 2, 2023
2cbb68f
[rtllib]: change string type to parameter type due to Vivado error
mmrahorovic Jun 2, 2023
92eb0ed
[rtllib]: renamed variable for consistency
mmrahorovic Jun 2, 2023
471a221
Fix improper blocking assignment & linting.
preusser Jun 2, 2023
5c5dc09
[test rtl mvu]: modified/extended test cases
mmrahorovic Jun 2, 2023
b4eb9b6
[rtl mvu]: updated DSP58 >4-bit variant to lift SIMD%3==0 restriction
mmrahorovic Jun 30, 2023
ad63673
[rtl mvu]: bug fix for SIMD=1 init_leave_loads
mmrahorovic Jun 30, 2023
79e8a5e
[mvu rtl]: restrict index i to be less than 3 (within bounds of hi4)
mmrahorovic Jul 13, 2023
7be62b4
Merge remote-tracking branch 'upstream/dev' into feature/dsp_packing
mmrahorovic Jul 17, 2023
e3493c3
Rewrite replay_buffer for input elasticity.
preusser Jun 2, 2023
44fae0c
Merge remote-tracking branch 'upstream/dev' into feature/dsp_packing
mmrahorovic Jul 31, 2023
df51f11
Merge remote-tracking branch 'upstream/dev' into feature/dsp_packing
mmrahorovic Aug 16, 2023
2efba68
[to-rtl]: Infer unique node names after transformation is applied
mmrahorovic Sep 5, 2023
114ea1b
[mvu rtl]: add synthesis directive to handle 'X in simulation
mmrahorovic Sep 18, 2023
79fafdb
[replay buffer rtl]: minor fix to when LEN=1 (= AWIDTH=0)
mmrahorovic Sep 18, 2023
619d9db
[mvu lut]: LUT-based MVU compute core
mmrahorovic Sep 18, 2023
090f2ac
[custom op]: add preferred_backend attribute
mmrahorovic Sep 19, 2023
ac5e82d
Ensure a minimum of two buffer slots even for length-1 sequences.
preusser Sep 21, 2023
d5ff2a2
Merge pull request #1 from Xilinx/bugfix/replay_len1
mmrahorovic Sep 21, 2023
bb94092
Merge remote-tracking branch 'origin/feature/dsp_packing' into featur…
mmrahorovic Sep 21, 2023
8515693
[rtl mvu wrapper]: support for vvu layer and rename
mmrahorovic Sep 21, 2023
cf28d78
[mvu vvu tb]: modified testbench to also support testing VVU on DSP58
mmrahorovic Sep 21, 2023
2617c39
[axi wrapper]: minor modification to comment description
mmrahorovic Sep 21, 2023
8ca5fe7
[mvu axi]: add support for VVU on DSP58
mmrahorovic Sep 21, 2023
32d6338
[mvu vvu axi]: renamed file for consistency purposes
mmrahorovic Sep 21, 2023
031406d
[mvu 8sx9]: added support for VVU on DSP58, resolved PyVerilator-caus…
mmrahorovic Sep 21, 2023
e2c1f15
[mvu vvu 8sx9]: renamed compute core for consistency
mmrahorovic Sep 21, 2023
adb5869
[axi wrapper]: changed parameter to localparam
mmrahorovic Sep 21, 2023
f54d438
[axi]: added support for LUT-based VVU
mmrahorovic Sep 21, 2023
a4e2ac7
[mvu vvu 8sx9]: minor change to list of generics
mmrahorovic Sep 21, 2023
40ad0b4
[mvu lut]: added support for VVU
mmrahorovic Sep 21, 2023
30fcb5b
[mvu vvu lut]: renamed file for consistency
mmrahorovic Sep 21, 2023
cb43438
Revert to proper address truncation without generation bit.
preusser Sep 21, 2023
b4b69f3
remove deletd/renamed files
mmrahorovic Sep 21, 2023
14c5fa9
[mvu vvu 8sx9]: renamed for consistency
mmrahorovic Sep 21, 2023
3a37588
[mvu vvu axi]: changes for renamed module
mmrahorovic Sep 21, 2023
afe36ba
[mvu vvu wrapper]: convert localparam to param
mmrahorovic Sep 25, 2023
e4f2f9e
[mvau-rtl custom-op]: bugfix to instantiate memstreamer, modified ren…
mmrahorovic Sep 25, 2023
b49b79a
[specialize to rtl]: fix to changed attribute name and added support …
mmrahorovic Sep 25, 2023
9bdba03
Adding core for DSP48 backport.
preusser Sep 19, 2023
2cf1ef7
[mvu rtl core]: added support for signed activations for DSP48-based …
mmrahorovic Sep 25, 2023
ab8d4a8
[rtl mvu custom-op]: add upper bound to SEGMENTLEN equal to number of…
mmrahorovic Sep 25, 2023
5a429fc
[mvu_vvu dsp58]: change weight input to 2D instead of 3D array
mmrahorovic Oct 13, 2023
a4a18bb
[mvu_vvu axi]: re-wire weights appropriately for VVU DSP58
mmrahorovic Oct 13, 2023
cc0737b
[mvu_vvu axi wrapper]: fix to IS_MVU parameter
mmrahorovic Oct 13, 2023
c0eff0b
[mvu_vvu tb]: WIP -- changes to self-checker and shape of input data
mmrahorovic Oct 13, 2023
cf7f494
[mvu vvu axi]: minor bugfixes to enable VVU
mmrahorovic Nov 1, 2023
5ffc221
[mvu vvu axi]: minor fix -- define mvauin_weight_t
mmrahorovic Nov 20, 2023
d573043
Merge remote-tracking branch 'upstream/dev' into feature/dsp_packing
mmrahorovic Nov 27, 2023
40d652c
[rtl mvu op]: minor fix to chain length estimation and enabled behavi…
mmrahorovic Nov 29, 2023
6e98bac
[rtlsim]: use pyverilator util functions
mmrahorovic Dec 13, 2023
5dd74ad
[mvu vvu axi]: sign extend output tdata (byte-aligned)
mmrahorovic Dec 8, 2023
b20410b
[mvu core]: dsp48 convert unpacked array to packed array to work arou…
mmrahorovic Jan 8, 2024
1c2cc0c
[mvu axi]: update list of deduced parameters
mmrahorovic Jan 8, 2024
eeb3cea
[mvu custom-op]: remove lut-based implementation and update compute c…
mmrahorovic Jan 8, 2024
0813d14
[mvu axi]: remove LUT-based compute core
mmrahorovic Jan 8, 2024
4892d66
[hls custom-op]: enable reset in sim
mmrahorovic Jan 11, 2024
44f6e0f
[test mvu rtl]: updated test flow (DSP58 only)
mmrahorovic Jan 11, 2024
9b2cceb
[mvu vvu axi]: reworked flow control and backpressure handling by tpr…
mmrahorovic Jan 11, 2024
8b46c90
Merge remote-tracking branch 'origin/feature/dsp_packing' into refact…
mmrahorovic Jan 11, 2024
4ab6596
[hlsbackend]: update limit HLS axi streams (8k-1)
mmrahorovic Jan 11, 2024
3011144
Merge remote-tracking branch 'upstream/refactor/rtl_integration' into…
mmrahorovic Jan 18, 2024
c3ce0c6
Merge remote-tracking branch 'upstream/refactor/rtl_integration' into…
mmrahorovic Jan 26, 2024
72ccc83
[mvau hls]: refactored MVAU_hls custom_op
mmrahorovic Jan 26, 2024
7a9b82b
[refactor]: call to base_op_type method instead of custom_op type
mmrahorovic Jan 26, 2024
4556d2d
[hls custom-op]: add mvau_hls
mmrahorovic Jan 26, 2024
0b2fc98
[hw custom-op]: refactor MVAU
mmrahorovic Jan 26, 2024
1a40e6a
[VVAU hw custom-op]: add base_op_type method
mmrahorovic Jan 26, 2024
5e1ed9b
[transform]: add transformation to infer MVAU hw custom-op
mmrahorovic Jan 26, 2024
63c73c2
removed mvu rtl code to clean up PR
mmrahorovic Jan 26, 2024
472ce11
[test mvau]: modified to support new custom-ops
mmrahorovic Jan 26, 2024
31083d5
removed rtl refactoring steps
mmrahorovic Jan 26, 2024
0032743
removed old rtl custom-op
mmrahorovic Jan 26, 2024
04ec562
removed old specialize_to_rtl transform
mmrahorovic Jan 26, 2024
de778cf
removed rtl custom-op test
mmrahorovic Jan 26, 2024
4a47437
Merge remote-tracking branch 'upstream/refactor/rtl_integration' into…
mmrahorovic Feb 1, 2024
911239c
[vvau hls]: add custom op to dict
mmrahorovic Feb 1, 2024
b1ee540
[vvu hw-op]: refactored hw custom-op VVAU
mmrahorovic Feb 1, 2024
bc44a4d
[vvau hls-op]: refactored HLS custom-op VVAU
mmrahorovic Feb 1, 2024
faabc0f
[convert-to-hw]: added transformations to infer binary-MVAU and VVAU
mmrahorovic Feb 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/finn/analysis/fpgadataflow/res_estimation.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,8 @@ def res_estimation_complete(model):
res_dict = {}
for node in model.graph.node:
if is_fpgadataflow_node(node) is True:
op_type = node.op_type
inst = registry.getCustomOp(node)
op_type = inst.base_op_type()
if op_type == "MatrixVectorActivation" or op_type == "VectorVectorActivation":
orig_restype = inst.get_nodeattr("resType")
res_dict[node.name] = []
Expand Down
7 changes: 6 additions & 1 deletion src/finn/builder/build_dataflow_steps.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change looks like it has nothing to do with the new class hierarchy. Could you provide more information on this or take it out of the PR, please?

Copy link
Collaborator Author

@mmrahorovic mmrahorovic Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct, I'll remove it.
Nevertheless, do you think it's worth having this change upstream? My reasoning was that this particular transformation step (step_set_fifo_depths) should only affect the FIFOs as the name suggests. By passing an appropriate node_filter to the ApplyConfig transformation, we ensure that only StreamingFIFO-nodes are affected. This could potentially prevent confusion/bug when the folding config file has changed after the step_apply_folding_config step.
If so, I can create a separate PR with this change and a description.

Original file line number Diff line number Diff line change
Expand Up @@ -569,7 +569,12 @@ def step_set_fifo_depths(model: ModelWrapper, cfg: DataflowBuildConfig):
model = model.transform(GiveUniqueNodeNames())
model = model.transform(GiveReadableTensorNames())
if cfg.folding_config_file is not None:
model = model.transform(ApplyConfig(cfg.folding_config_file))
model = model.transform(
ApplyConfig(
cfg.folding_config_file,
node_filter=lambda x: x.op_type == "StreamingFIFO",
)
)

# extract the final configuration and save it as json
hw_attrs = [
Expand Down
2 changes: 2 additions & 0 deletions src/finn/custom_op/fpgadataflow/hls/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
from finn.custom_op.fpgadataflow.hls.streamingmaxpool_hls import StreamingMaxPool_hls
from finn.custom_op.fpgadataflow.hls.tlastmarker_hls import TLastMarker_hls
from finn.custom_op.fpgadataflow.hls.upsampler_hls import UpsampleNearestNeighbour_hls
from finn.custom_op.fpgadataflow.hls.matrixvectoractivation_hls import MatrixVectorActivation_hls

custom_op = dict()

Expand All @@ -73,3 +74,4 @@
custom_op["StreamingMaxPool_hls"] = StreamingMaxPool_hls
custom_op["TLastMarker_hls"] = TLastMarker_hls
custom_op["UpsampleNearestNeighbour_hls"] = UpsampleNearestNeighbour_hls
custom_op["MatrixVectorActivation_hls"] = MatrixVectorActivation_hls
Loading