Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LLVM] Basic scalable vector support draft #676

Draft
wants to merge 73 commits into
base: llvm
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
16d3cab
Disable python bindings for faster build
pramodk Nov 27, 2020
b1cfda6
Integrate LLVM into CMake build system
pramodk Nov 28, 2020
46d4779
Code infrastructure for LLVM code generation backend
pramodk Nov 28, 2020
51987dc
Azure CI fixes for LLVM build and README update
pramodk Nov 28, 2020
ae07ce4
Print build status after cmake configure stage
pramodk Nov 29, 2020
c540fb1
Adding test template for LLVM codegen
pramodk Nov 29, 2020
724605c
Initial LLVM codegen vistor routines (#457)
georgemitenkov Dec 22, 2020
b621d4e
FunctionBlock code generation and terminator checks (#470)
georgemitenkov Dec 25, 2020
917a7da
Add option to run LLVM optimisation passes (#471)
pramodk Dec 28, 2020
b261ba9
Add function call LLVM code generation (#477)
georgemitenkov Dec 30, 2020
7884de8
Support for IndexedName codegen (#478)
georgemitenkov Dec 30, 2020
dbda271
Improvements for code generation specific transformations (#483)
pramodk Jan 6, 2021
83abf60
nrn_state function generation in NMODL AST to help LLVM codegen (#484)
pramodk Jan 6, 2021
bcc091b
Running functions from MOD files via LLVM JIT (#482)
georgemitenkov Jan 8, 2021
838ed6f
Extended support for binary ops and refactoring (#489)
georgemitenkov Jan 12, 2021
eaeb7aa
Avoid converting LOCAL statement in all StatementBlocks (#492)
pramodk Jan 12, 2021
6d60ca9
Handle CodegenVarType type in JSON printer (#494)
pramodk Jan 13, 2021
5b32b31
Integrating LLVM helper into LLVM visitor (#497)
georgemitenkov Jan 25, 2021
bc305ba
LLVM code generation for if/else statements (#499)
georgemitenkov Jan 25, 2021
c8ea994
Added error handling for values not in scope (#502)
georgemitenkov Jan 26, 2021
a32f76b
Added support for WHILE statement (#501)
georgemitenkov Jan 26, 2021
bfaff72
Create mechanism instance struct in LLVM IR (#507)
iomaganaris Feb 1, 2021
b690961
Printf support in LLVM IR codegen (#510)
georgemitenkov Feb 3, 2021
a561c97
Fix issue error: ‘runtime_error’ is not a member of ‘std’ (#512)
iomaganaris Feb 15, 2021
5c80684
Move code gen specific InstanceStruct node to codegen.yaml (#526)
pramodk Mar 5, 2021
5b72bc3
* Improvements to codegen helper (Part I)
pramodk Feb 27, 2021
4b3e2fc
Addressing TODOs for Instance struct (#533) Part II
georgemitenkov Mar 6, 2021
c08eb22
Unit test for scalar state kernel generation in LLVM (#547)
georgemitenkov Mar 9, 2021
ec91271
Indexed name codegen improvements (#550)
georgemitenkov Mar 12, 2021
559d152
Add InstanceStruct test data generation helper and unit test (#546)
iomaganaris Mar 13, 2021
b5d152f
Add the remainder loop for vectorization of DERIVATIVE block (#534)
Mar 17, 2021
c7e5e28
Always initialize return variable in function block (#554)
Mar 19, 2021
20dc785
Running a kernel with NMODL-LLVM JIT (#549)
georgemitenkov Apr 9, 2021
f684466
Loop epilogue fix for LLVM visitor helper (#567)
georgemitenkov Apr 9, 2021
cc92fc2
Gather support and vectorisation fixes for LLVM code generation (#568)
georgemitenkov Apr 10, 2021
3803ae1
Verification and file utilities for LLVM IR codegen (#582)
georgemitenkov Apr 13, 2021
60e68c9
Add gather execution test (#591)
georgemitenkov Apr 16, 2021
8f1fbae
Fixed loop allocations (#590)
georgemitenkov Apr 17, 2021
53cd1cc
Benchmarking LLVM code generation (#583)
georgemitenkov Apr 17, 2021
db80372
Minor benchmarking improvement (#593)
pramodk Apr 18, 2021
2a699a8
Bug fix in codegen helper: delete LOCAL statement (#595)
pramodk Apr 19, 2021
630033c
LLVM 13 compatibility and fixing void* type (#603)
georgemitenkov Apr 20, 2021
fe3d856
Allow LOCAL variable inside StatementBlock for LLVM IR generation (#599)
pramodk Apr 20, 2021
dddffed
Update CI with LLVM v13 (trunk) (#605)
pramodk Apr 22, 2021
068ba5d
Integrating vector maths library into LLVM codegen (#604)
georgemitenkov Apr 22, 2021
98a88d5
Using shared libraries in LLVM JIT (#609)
georgemitenkov Apr 22, 2021
16504c7
Avoid local std::ofstream object causing segfault (#614)
pramodk Apr 24, 2021
dd2889d
Refactoring of runners' infrastructure and dumping object files (#620)
georgemitenkov Apr 30, 2021
93732e0
Optimisation levels for benchmarking (#623)
georgemitenkov May 7, 2021
150943c
Adding function debug information (#628)
georgemitenkov May 8, 2021
1802b74
Fixed using benchmarking_info in TestRunner (#631)
georgemitenkov May 8, 2021
3359ea3
Fixes to run CI with NVHPC/PGI compiler
pramodk May 8, 2021
524b292
Fixed addition of SOLVE block to kernel's FOR loop (#636)
georgemitenkov May 11, 2021
68639a7
IR builder redesign for LLVM IR code generation pipeline (#634)
georgemitenkov May 13, 2021
454a18f
Fixed initialisation of `CodegenAtomicStatement` (#642)
georgemitenkov May 13, 2021
53727df
Fix instance struct data generation for testing/benchmarking (#641)
pramodk May 13, 2021
23100e9
Basic scatter support (#643)
georgemitenkov May 13, 2021
bd05479
Benchmarking code re-organisation and minor improvements (#647)
pramodk May 16, 2021
5d126aa
Added attributes and metadata to LLVM IR compute kernels (#648)
georgemitenkov May 17, 2021
ee8bbdb
Added loaded value to the stack (#655)
georgemitenkov May 18, 2021
5ee761b
Basic predication support for LLVM backend (#652)
georgemitenkov May 20, 2021
8bee7de
Improvements for LLVM code generation and benchmarking (#661)
georgemitenkov May 20, 2021
1461d8f
Fixed `alloca`s insertion point for LLVM backend (#663)
georgemitenkov May 20, 2021
908779e
Fast math flags for LLVM backend (#662)
georgemitenkov May 21, 2021
2ca85e5
Avoid generating LLVM IR for Functions and Procedures if inlined (#664)
iomaganaris May 21, 2021
7fdbb4f
Fixed typo in benchmarking metrics (#665)
georgemitenkov May 21, 2021
4c585f3
Remove only inlined blocks from AST based on symtab properties (#668)
iomaganaris May 21, 2021
f0a3afc
Use VarName on the RHS of assignment expression (#669)
pramodk May 25, 2021
2609f87
[LLVM] SLEEF and libsystem_m vector libraries support (#674)
georgemitenkov May 30, 2021
7300eb9
Added scalable type support
georgemitenkov May 31, 2021
77881fa
Clang format
georgemitenkov May 31, 2021
885fd55
Added scalable induction variable support
georgemitenkov May 31, 2021
86aa20d
Added scalable vector constant support
georgemitenkov May 31, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Added scalable induction variable support
  • Loading branch information
georgemitenkov committed May 31, 2021
commit 885fd5563f731c144fd3cfa8aa3624482df52e74
78 changes: 58 additions & 20 deletions src/codegen/llvm/codegen_llvm_helper_visitor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -550,17 +550,33 @@ void CodegenLLVMHelperVisitor::visit_function_block(ast::FunctionBlock& node) {
}

/**
* Create loop increment expression `id = id + width`
* \todo : same as int_initialization_expression()
* Create loop increment expression
* \todo : llvm.vscale.i32 is currently hardcoded. This can be done in a more elegant way.
*/
static std::shared_ptr<ast::Expression> loop_increment_expression(const std::string& induction_var,
int vector_width) {
// first create id + x
int vector_width,
bool scalable) {
const auto& id = create_varname(induction_var);
const auto& inc = new ast::Integer(vector_width, nullptr);
if (scalable) {
// For scalable vectorized code generation, the increment is
// id = id + vscale * vector_width
const auto& call_for_factor =
new ast::FunctionCall(new ast::Name(new ast::String("llvm.vscale.i32")), {});
const auto& min_vector_width = new ast::Integer(vector_width, /*macro=*/nullptr);
const auto& actual_width = new ast::BinaryExpression(
call_for_factor, ast::BinaryOperator(ast::BOP_MULTIPLICATION), min_vector_width);
const auto& inc_expr =
new ast::BinaryExpression(id, ast::BinaryOperator(ast::BOP_ADDITION), actual_width);
return std::make_shared<ast::BinaryExpression>(id->clone(),
ast::BinaryOperator(ast::BOP_ASSIGN),
inc_expr);
}

// Otherwise, the increment is
// id = id + vector_width
const auto& increment = new ast::Integer(vector_width, /*macro=*/nullptr);
const auto& inc_expr =
new ast::BinaryExpression(id, ast::BinaryOperator(ast::BOP_ADDITION), inc);
// now create id = id + x
new ast::BinaryExpression(id, ast::BinaryOperator(ast::BOP_ADDITION), increment);
return std::make_shared<ast::BinaryExpression>(id->clone(),
ast::BinaryOperator(ast::BOP_ASSIGN),
inc_expr);
Expand All @@ -569,16 +585,16 @@ static std::shared_ptr<ast::Expression> loop_increment_expression(const std::str
/**
* Create loop count comparison expression
*
* Based on if loop is vectorised or not, the condition for loop
* is different. For example:
* - serial loop : `id < node_count`
* - vector loop : `id < (node_count - vector_width + 1)`
* Serial loop: `id < node_count`
* Fixed vector width loop : `id < node_count - (vector_width - 1)`
* Scalable vector width loop : `id < node_count - (vscale * vector_width - 1)`
*
* \todo : same as int_initialization_expression()
* \todo : llvm.vscale.i32 is currently hardcoded. This can be done in a more elegant way.
*/
static std::shared_ptr<ast::Expression> loop_count_expression(const std::string& induction_var,
const std::string& node_count,
int vector_width) {
int vector_width,
bool scalable) {
const auto& id = create_varname(induction_var);
const auto& mech_node_count = create_varname(node_count);

Expand All @@ -589,8 +605,26 @@ static std::shared_ptr<ast::Expression> loop_count_expression(const std::string&
mech_node_count);
}

// For vectorised loop, the condition is id < mech->node_count - vector_width + 1
const auto& remainder = new ast::Integer(vector_width - 1, /*macro=*/nullptr);
// For fixed vector width, the condition is id < mech->node_count - vector_width + 1
if (!scalable) {
const auto& remainder = new ast::Integer(vector_width - 1, /*macro=*/nullptr);
const auto& count = new ast::BinaryExpression(mech_node_count,
ast::BinaryOperator(ast::BOP_SUBTRACTION),
remainder);
return std::make_shared<ast::BinaryExpression>(id->clone(),
ast::BinaryOperator(ast::BOP_LESS),
count);
}

// For scalable vector width, the condition is id < mech->node_count - vscale * vector_width + 1
const auto& call_for_factor =
new ast::FunctionCall(new ast::Name(new ast::String("llvm.vscale.i32")), {});
const auto& min_vector_width = new ast::Integer(vector_width, /*macro=*/nullptr);
const auto& actual_width = new ast::BinaryExpression(
call_for_factor, ast::BinaryOperator(ast::BOP_MULTIPLICATION), min_vector_width);
const auto& one = new ast::Integer(1, /*macro=*/nullptr);
const auto& remainder =
new ast::BinaryExpression(actual_width, ast::BinaryOperator(ast::BOP_SUBTRACTION), one);
const auto& count = new ast::BinaryExpression(mech_node_count,
ast::BinaryOperator(ast::BOP_SUBTRACTION),
remainder);
Expand Down Expand Up @@ -683,8 +717,9 @@ void CodegenLLVMHelperVisitor::visit_nrn_state_block(ast::NrnStateBlock& node) {
{
/// loop constructs : initialization, condition and increment
const auto& initialization = int_initialization_expression(INDUCTION_VAR);
const auto& condition = loop_count_expression(INDUCTION_VAR, NODECOUNT_VAR, vector_width);
const auto& increment = loop_increment_expression(INDUCTION_VAR, vector_width);
const auto& condition =
loop_count_expression(INDUCTION_VAR, NODECOUNT_VAR, vector_width, scalable);
const auto& increment = loop_increment_expression(INDUCTION_VAR, vector_width, scalable);

/// clone it
auto local_loop_block = std::shared_ptr<ast::StatementBlock>(loop_block->clone());
Expand All @@ -711,9 +746,12 @@ void CodegenLLVMHelperVisitor::visit_nrn_state_block(ast::NrnStateBlock& node) {
/// remainder loop possibly vectorized on vector_width
if (vector_width > 1) {
/// loop constructs : initialization, condition and increment
const auto& condition =
loop_count_expression(INDUCTION_VAR, NODECOUNT_VAR, /*vector_width=*/1);
const auto& increment = loop_increment_expression(INDUCTION_VAR, /*vector_width=*/1);
const auto& condition = loop_count_expression(INDUCTION_VAR,
NODECOUNT_VAR,
/*vector_width=*/1,
/*scalable=*/false);
const auto& increment =
loop_increment_expression(INDUCTION_VAR, /*vector_width=*/1, /*scalable=*/false);

/// rename local variables to avoid conflict with main loop
rename_local_variables(*loop_block);
Expand Down
9 changes: 7 additions & 2 deletions src/codegen/llvm/codegen_llvm_helper_visitor.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,9 @@ class CodegenLLVMHelperVisitor: public visitor::AstVisitor {
/// explicit vectorisation width
int vector_width;

/// target scalable ISAs
bool scalable;

/// newly generated code generation specific functions
CodegenFunctionVector codegen_functions;

Expand Down Expand Up @@ -134,8 +137,10 @@ class CodegenLLVMHelperVisitor: public visitor::AstVisitor {
static const std::string VOLTAGE_VAR;
static const std::string NODE_INDEX_VAR;

CodegenLLVMHelperVisitor(int vector_width)
: vector_width(vector_width) {}
CodegenLLVMHelperVisitor(int vector_width,
bool scalable = false)
: vector_width(vector_width)
, scalable(scalable) {}

const InstanceVarHelper& get_instance_var_helper() {
return instance_var_helper;
Expand Down
8 changes: 7 additions & 1 deletion src/codegen/llvm/codegen_llvm_visitor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -746,6 +746,12 @@ void CodegenLLVMVisitor::visit_function_call(const ast::FunctionCall& node) {
if (func) {
create_function_call(func, name, node.get_arguments());
} else {
// If generating scalable vectorized IR, process the call to `vscale` separately.
if (name == "llvm.vscale.i32" && scalable) {
ir_builder.create_vscale_call(*module);
return;
}

auto symbol = sym_tab->lookup(name);
if (symbol && symbol->has_any_property(symtab::syminfo::NmodlType::extern_method)) {
create_external_function_call(name, node.get_arguments());
Expand Down Expand Up @@ -832,7 +838,7 @@ void CodegenLLVMVisitor::visit_program(const ast::Program& node) {
// - convert function and procedure blocks into CodegenFunctions
// - gather information about AST. For now, information about functions
// and procedures is used only.
CodegenLLVMHelperVisitor v{vector_width};
CodegenLLVMHelperVisitor v{vector_width, scalable};
const auto& functions = v.get_codegen_functions(node);
instance_var_helper = v.get_instance_var_helper();
sym_tab = node.get_symbol_table();
Expand Down
4 changes: 4 additions & 0 deletions src/codegen/llvm/codegen_llvm_visitor.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,9 @@ class CodegenLLVMVisitor: public visitor::ConstAstVisitor {
/// Explicit vectorisation width.
int vector_width;

/// Generate scalable vectorized IR.
bool scalable;

public:
CodegenLLVMVisitor(const std::string& mod_filename,
const std::string& output_dir,
Expand All @@ -112,6 +115,7 @@ class CodegenLLVMVisitor: public visitor::ConstAstVisitor {
, opt_passes(opt_passes)
, vector_width(vector_width)
, vector_library(vec_lib)
, scalable(scalable)
, add_debug_information(add_debug_information)
, ir_builder(*context, use_single_precision, vector_width, fast_math_flags, scalable)
, debug_builder(*module)
Expand Down
7 changes: 7 additions & 0 deletions src/codegen/llvm/llvm_ir_builder.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,13 @@ void IRBuilder::create_intrinsic(const std::string& name,
}
}

void IRBuilder::create_vscale_call(llvm::Module& module) {
llvm::Function* vscale_function =
llvm::Intrinsic::getDeclaration(&module, llvm::Intrinsic::vscale, get_i32_type());
llvm::Value* vscale = builder.CreateCall(vscale_function);
value_stack.push_back(vscale);
}

void IRBuilder::set_kernel_attributes() {
// By convention, the compute kernel does not free memory and does not throw exceptions.
current_function->setDoesNotFreeMemory();
Expand Down
3 changes: 3 additions & 0 deletions src/codegen/llvm/llvm_ir_builder.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,9 @@ class IRBuilder {
void create_scalar_or_vector_alloca(const std::string& name,
llvm::Type* element_or_scalar_type);

/// Creates a call to llvm.vscale.i32().
void create_vscale_call(llvm::Module& module);

/// Generates LLVM IR for the given unary operator.
void create_unary_op(llvm::Value* value, ast::UnaryOp op);

Expand Down