Any plans for wasm64 support? #185
Comments
|
Yes; the main work right now is happening in upstream LLVM (eg. here). There will also need to be some amount of work to port wasi-libc. All the code in wasi-libc is written with a wasm64 target in mind. We'll need a 64-bit version of the WASI APIs as well, and the code for that is also all written with a wasm64 target in mind. So in theory this should be a straightforward process. |
|
Exciting! I have a strong vested interest in seeing WASM succeed as a server-side plugin (and write-once, run-anywhere code distribution) platform. There are a few missing pieces:
but it seems that all 4 areas are seeing a lot of active work, which means 2022 will be an exciting year for WASM. |
|
It's been a few years and the clang link seems to have been resolved. Any updates? |
|
I don't see anyone working on this too closely at the moment: are you interested in adding support for |
|
If it isn't a bunch of work. Are there assumptions being made that make this a big effort, or is it a simple change? |
|
I would hope its not too much work. I imagine most things will just work. The biggest change is likely that would we would need to start shipping all of our pre-built libraries in wsam64 variants. |
ac000
added a commit
to ac000/unit
that referenced
this issue
Aug 9, 2023
This adds the core of runtime WebAssmebly[0] support. Future commits will enable this in the Unit core and expose the configuration. This introduces a new src/wasm directory for storing this source. We are initially using wasmtime[0] as the WebAssmebly runtime, however this has been designed with the ability to use different runtimes in mind. src/wasm/nxt_wasm.[ch] is the main interface to Unit. src/wasm/nxt_rt_wasmtime.c is the wasmtime runtime support. This is nicely insulated from any knowledge of internal Unit workings. wasmtime is what loads and runs the WASM modules. The WASM modules can export functions wasmtime can call and wasmtime can export functions that the module can call. We make use of both. The terminology used is that function exports are what the WASM module exports and function imports are what the WASM runtime exports to the module. We currently have four function imports (functions exported by the runtime to be called by the WASM module). 1) nxt_wasm_get_init_mem_size This allows WASM modules to get the size of the initially allocated shared memory. This is the size allocated at Unit startup and what the WASM modules can assume they have access to (in reality this shared memory will likely be larger). The amount of memory allocated at startup is NXT_WASM_MEM_SIZE which as of this commit is 32MiB. We do actually allocate NXT_WASM_MEM_SIZE + NXT_WASM_PAGE_SIZE at startup which is an extra 64KiB (the smallest allocation unit), this is to allow room for the response structure and so module developers can just assume they have the full 32MiB for their actual response. 2) nxt_wasm_send_headers This allows WASM modules to send their headers. 3) nxt_wasm_send_response This allows WASM modules to send their response. 4) nxt_wasm_response_end This allows WASM modules to inform Unit they have finished sending their response. This calls nxt_unit_request_done() Then there are currently up to eight functions that a module can export. Three of which are required. These function can be named anything. I'll use the Unit configuration names to refer to them 1) request_handler The main driving function. This may be called multiple times for a single HTTP request if the request is larger than the shared memory. 2) malloc_handler Used to allocate a chunk of memory at language module startup. This memory is allocated from the WASM modules address space and is what is sued for communicating between the WASM module (the guest) and Unit (the host). 3) free_handler Used to free the memory from above at language module shutdown. Then there are the following optional handlers 1) module_init_handler If set, called at language module startup. 2) module_end_handler If set, called at language module shutdown. 3) request_init_handler If set, called at the start of request. Called only once per HTTP request. 4) request_end_handler If set, called once all of a request has been sent to the WASM module. 5) response_end_handler If set, called at the end of a request, once the WASM module has sent all its headers and data. 32bits We currently support 32bit WASM modules, I.e wasm32-wasi. Newer version of clang, 13+[2], do seem to have support for wasm64 as a target (which uses a LP64 model). However it's not entirely clear if the WASI SDK fully supports[3] this and by extension WASI libc/wasi-sysroot. 64bit support is something than can be explored more thoroughly in the future. As such in structures that are used to communicate between the host and guest we use 32bit ints. Even when a single byte might be enough. This is to avoid issues with structure layout differences between a 64bit host and 32bit guest (I.e WASM module) and the need for various bits of structure padding depending on host architecture. Instead everything is 4-byte aligned. [0]: <https://webassembly.org/> [1]: <https://wasmtime.dev/> [2]: <https://reviews.llvm.org/rG670944fb20b226fc22fa993ab521125f9adbd30a> [3]: <WebAssembly/wasi-sdk#185> Signed-off-by: Andrew Clayton <[email protected]>
ac000
added a commit
to ac000/unit
that referenced
this issue
Aug 9, 2023
This adds the core of runtime WebAssembly[0] support. Future commits will enable this in the Unit core and expose the configuration. This introduces a new src/wasm directory for storing this source. We are initially using Wasmtime[0] as the WebAssembly runtime, however this has been designed with the ability to use different runtimes in mind. src/wasm/nxt_wasm.[ch] is the main interface to Unit. src/wasm/nxt_rt_wasmtime.c is the Wasmtime runtime support. This is nicely insulated from any knowledge of internal Unit workings. Wasmtime is what loads and runs the Wasm modules. The Wasm modules can export functions Wasmtime can call and Wasmtime can export functions that the module can call. We make use of both. The terminology used is that function exports are what the Wasm module exports and function imports are what the Wasm runtime exports to the module. We currently have four function imports (functions exported by the runtime to be called by the Wasm module). 1) nxt_wasm_get_init_mem_size This allows Wasm modules to get the size of the initially allocated shared memory. This is the size allocated at Unit startup and what the Wasm modules can assume they have access to (in reality this shared memory will likely be larger). The amount of memory allocated at startup is NXT_WASM_MEM_SIZE which as of this commit is 32MiB. We do actually allocate NXT_WASM_MEM_SIZE + NXT_WASM_PAGE_SIZE at startup which is an extra 64KiB (the smallest allocation unit), this is to allow room for the response structure and so module developers can just assume they have the full 32MiB for their actual response. 2) nxt_wasm_send_headers This allows WASM modules to send their headers. 3) nxt_wasm_send_response This allows WASM modules to send their response. 4) nxt_wasm_response_end This allows WASM modules to inform Unit they have finished sending their response. This calls nxt_unit_request_done() Then there are currently up to eight functions that a module can export. Three of which are required. These function can be named anything. I'll use the Unit configuration names to refer to them 1) request_handler The main driving function. This may be called multiple times for a single HTTP request if the request is larger than the shared memory. 2) malloc_handler Used to allocate a chunk of memory at language module startup. This memory is allocated from the WASM modules address space and is what is sued for communicating between the WASM module (the guest) and Unit (the host). 3) free_handler Used to free the memory from above at language module shutdown. Then there are the following optional handlers 1) module_init_handler If set, called at language module startup. 2) module_end_handler If set, called at language module shutdown. 3) request_init_handler If set, called at the start of request. Called only once per HTTP request. 4) request_end_handler If set, called once all of a request has been sent to the WASM module. 5) response_end_handler If set, called at the end of a request, once the WASM module has sent all its headers and data. 32bits We currently support 32bit WASM modules, I.e wasm32-wasi. Newer version of clang, 13+[2], do seem to have support for wasm64 as a target (which uses a LP64 model). However it's not entirely clear if the WASI SDK fully supports[3] this and by extension WASI libc/wasi-sysroot. 64bit support is something than can be explored more thoroughly in the future. As such in structures that are used to communicate between the host and guest we use 32bit ints. Even when a single byte might be enough. This is to avoid issues with structure layout differences between a 64bit host and 32bit guest (I.e WASM module) and the need for various bits of structure padding depending on host architecture. Instead everything is 4-byte aligned. [0]: <https://webassembly.org/> [1]: <https://wasmtime.dev/> [2]: <https://reviews.llvm.org/rG670944fb20b226fc22fa993ab521125f9adbd30a> [3]: <WebAssembly/wasi-sdk#185> Signed-off-by: Andrew Clayton <[email protected]>
ac000
added a commit
to ac000/unit
that referenced
this issue
Aug 16, 2023
This adds the core of runtime WebAssembly[0] support. Future commits will enable this in the Unit core and expose the configuration. This introduces a new src/wasm directory for storing this source. We are initially using Wasmtime[0] as the WebAssembly runtime, however this has been designed with the ability to use different runtimes in mind. src/wasm/nxt_wasm.[ch] is the main interface to Unit. src/wasm/nxt_rt_wasmtime.c is the Wasmtime runtime support. This is nicely insulated from any knowledge of internal Unit workings. Wasmtime is what loads and runs the Wasm modules. The Wasm modules can export functions Wasmtime can call and Wasmtime can export functions that the module can call. We make use of both. The terminology used is that function exports are what the Wasm module exports and function imports are what the Wasm runtime exports to the module. We currently have four function imports (functions exported by the runtime to be called by the Wasm module). 1) nxt_wasm_get_init_mem_size This allows Wasm modules to get the size of the initially allocated shared memory. This is the size allocated at Unit startup and what the Wasm modules can assume they have access to (in reality this shared memory will likely be larger). The amount of memory allocated at startup is NXT_WASM_MEM_SIZE which as of this commit is 32MiB. We do actually allocate NXT_WASM_MEM_SIZE + NXT_WASM_PAGE_SIZE at startup which is an extra 64KiB (the smallest allocation unit), this is to allow room for the response structure and so module developers can just assume they have the full 32MiB for their actual response. 2) nxt_wasm_send_headers This allows WASM modules to send their headers. 3) nxt_wasm_send_response This allows WASM modules to send their response. 4) nxt_wasm_response_end This allows WASM modules to inform Unit they have finished sending their response. This calls nxt_unit_request_done() Then there are currently up to eight functions that a module can export. Three of which are required. These function can be named anything. I'll use the Unit configuration names to refer to them 1) request_handler The main driving function. This may be called multiple times for a single HTTP request if the request is larger than the shared memory. 2) malloc_handler Used to allocate a chunk of memory at language module startup. This memory is allocated from the WASM modules address space and is what is sued for communicating between the WASM module (the guest) and Unit (the host). 3) free_handler Used to free the memory from above at language module shutdown. Then there are the following optional handlers 1) module_init_handler If set, called at language module startup. 2) module_end_handler If set, called at language module shutdown. 3) request_init_handler If set, called at the start of request. Called only once per HTTP request. 4) request_end_handler If set, called once all of a request has been sent to the WASM module. 5) response_end_handler If set, called at the end of a request, once the WASM module has sent all its headers and data. 32bits We currently support 32bit WASM modules, I.e wasm32-wasi. Newer version of clang, 13+[2], do seem to have support for wasm64 as a target (which uses a LP64 model). However it's not entirely clear if the WASI SDK fully supports[3] this and by extension WASI libc/wasi-sysroot. 64bit support is something than can be explored more thoroughly in the future. As such in structures that are used to communicate between the host and guest we use 32bit ints. Even when a single byte might be enough. This is to avoid issues with structure layout differences between a 64bit host and 32bit guest (I.e WASM module) and the need for various bits of structure padding depending on host architecture. Instead everything is 4-byte aligned. [0]: <https://webassembly.org/> [1]: <https://wasmtime.dev/> [2]: <https://reviews.llvm.org/rG670944fb20b226fc22fa993ab521125f9adbd30a> [3]: <WebAssembly/wasi-sdk#185> Signed-off-by: Andrew Clayton <[email protected]>
ac000
added a commit
to ac000/unit
that referenced
this issue
Aug 17, 2023
This adds the core of runtime WebAssembly[0] support. Future commits will enable this in the Unit core and expose the configuration. This introduces a new src/wasm directory for storing this source. We are initially using Wasmtime[0] as the WebAssembly runtime, however this has been designed with the ability to use different runtimes in mind. src/wasm/nxt_wasm.[ch] is the main interface to Unit. src/wasm/nxt_rt_wasmtime.c is the Wasmtime runtime support. This is nicely insulated from any knowledge of internal Unit workings. Wasmtime is what loads and runs the Wasm modules. The Wasm modules can export functions Wasmtime can call and Wasmtime can export functions that the module can call. We make use of both. The terminology used is that function exports are what the Wasm module exports and function imports are what the Wasm runtime exports to the module. We currently have four function imports (functions exported by the runtime to be called by the Wasm module). 1) nxt_wasm_get_init_mem_size This allows Wasm modules to get the size of the initially allocated shared memory. This is the size allocated at Unit startup and what the Wasm modules can assume they have access to (in reality this shared memory will likely be larger). The amount of memory allocated at startup is NXT_WASM_MEM_SIZE which as of this commit is 32MiB. We do actually allocate NXT_WASM_MEM_SIZE + NXT_WASM_PAGE_SIZE at startup which is an extra 64KiB (the smallest allocation unit), this is to allow room for the response structure and so module developers can just assume they have the full 32MiB for their actual response. 2) nxt_wasm_send_headers This allows WASM modules to send their headers. 3) nxt_wasm_send_response This allows WASM modules to send their response. 4) nxt_wasm_response_end This allows WASM modules to inform Unit they have finished sending their response. This calls nxt_unit_request_done() Then there are currently up to eight functions that a module can export. Three of which are required. These function can be named anything. I'll use the Unit configuration names to refer to them 1) request_handler The main driving function. This may be called multiple times for a single HTTP request if the request is larger than the shared memory. 2) malloc_handler Used to allocate a chunk of memory at language module startup. This memory is allocated from the WASM modules address space and is what is sued for communicating between the WASM module (the guest) and Unit (the host). 3) free_handler Used to free the memory from above at language module shutdown. Then there are the following optional handlers 1) module_init_handler If set, called at language module startup. 2) module_end_handler If set, called at language module shutdown. 3) request_init_handler If set, called at the start of request. Called only once per HTTP request. 4) request_end_handler If set, called once all of a request has been sent to the WASM module. 5) response_end_handler If set, called at the end of a request, once the WASM module has sent all its headers and data. 32bits We currently support 32bit WASM modules, I.e wasm32-wasi. Newer version of clang, 13+[2], do seem to have support for wasm64 as a target (which uses a LP64 model). However it's not entirely clear if the WASI SDK fully supports[3] this and by extension WASI libc/wasi-sysroot. 64bit support is something than can be explored more thoroughly in the future. As such in structures that are used to communicate between the host and guest we use 32bit ints. Even when a single byte might be enough. This is to avoid issues with structure layout differences between a 64bit host and 32bit guest (I.e WASM module) and the need for various bits of structure padding depending on host architecture. Instead everything is 4-byte aligned. [0]: <https://webassembly.org/> [1]: <https://wasmtime.dev/> [2]: <https://reviews.llvm.org/rG670944fb20b226fc22fa993ab521125f9adbd30a> [3]: <WebAssembly/wasi-sdk#185> Signed-off-by: Andrew Clayton <[email protected]>
ac000
added a commit
to ac000/unit
that referenced
this issue
Aug 17, 2023
This adds the core of runtime WebAssembly[0] support. Future commits will enable this in the Unit core and expose the configuration. This introduces a new src/wasm directory for storing this source. We are initially using Wasmtime[0] as the WebAssembly runtime, however this has been designed with the ability to use different runtimes in mind. src/wasm/nxt_wasm.[ch] is the main interface to Unit. src/wasm/nxt_rt_wasmtime.c is the Wasmtime runtime support. This is nicely insulated from any knowledge of internal Unit workings. Wasmtime is what loads and runs the Wasm modules. The Wasm modules can export functions Wasmtime can call and Wasmtime can export functions that the module can call. We make use of both. The terminology used is that function exports are what the Wasm module exports and function imports are what the Wasm runtime exports to the module. We currently have four function imports (functions exported by the runtime to be called by the Wasm module). 1) nxt_wasm_get_init_mem_size This allows Wasm modules to get the size of the initially allocated shared memory. This is the size allocated at Unit startup and what the Wasm modules can assume they have access to (in reality this shared memory will likely be larger). The amount of memory allocated at startup is NXT_WASM_MEM_SIZE which as of this commit is 32MiB. We do actually allocate NXT_WASM_MEM_SIZE + NXT_WASM_PAGE_SIZE at startup which is an extra 64KiB (the smallest allocation unit), this is to allow room for the response structure and so module developers can just assume they have the full 32MiB for their actual response. 2) nxt_wasm_send_headers This allows WASM modules to send their headers. 3) nxt_wasm_send_response This allows WASM modules to send their response. 4) nxt_wasm_response_end This allows WASM modules to inform Unit they have finished sending their response. This calls nxt_unit_request_done() Then there are currently up to eight functions that a module can export. Three of which are required. These function can be named anything. I'll use the Unit configuration names to refer to them 1) request_handler The main driving function. This may be called multiple times for a single HTTP request if the request is larger than the shared memory. 2) malloc_handler Used to allocate a chunk of memory at language module startup. This memory is allocated from the WASM modules address space and is what is sued for communicating between the WASM module (the guest) and Unit (the host). 3) free_handler Used to free the memory from above at language module shutdown. Then there are the following optional handlers 1) module_init_handler If set, called at language module startup. 2) module_end_handler If set, called at language module shutdown. 3) request_init_handler If set, called at the start of request. Called only once per HTTP request. 4) request_end_handler If set, called once all of a request has been sent to the WASM module. 5) response_end_handler If set, called at the end of a request, once the WASM module has sent all its headers and data. 32bits We currently support 32bit WASM modules, I.e wasm32-wasi. Newer version of clang, 13+[2], do seem to have support for wasm64 as a target (which uses a LP64 model). However it's not entirely clear if the WASI SDK fully supports[3] this and by extension WASI libc/wasi-sysroot. 64bit support is something than can be explored more thoroughly in the future. As such in structures that are used to communicate between the host and guest we use 32bit ints. Even when a single byte might be enough. This is to avoid issues with structure layout differences between a 64bit host and 32bit guest (I.e WASM module) and the need for various bits of structure padding depending on host architecture. Instead everything is 4-byte aligned. [0]: <https://webassembly.org/> [1]: <https://wasmtime.dev/> [2]: <https://reviews.llvm.org/rG670944fb20b226fc22fa993ab521125f9adbd30a> [3]: <WebAssembly/wasi-sdk#185> Reviewed-by: Alejandro Colomar <[email protected]> Signed-off-by: Andrew Clayton <[email protected]>
|
I'd be interested in seeing wasi-sdk support wasm64, if everything is ready for it. Seems that the first thing to do would be to make wasi-libc support wasm64 ; should we open an issue there ? |
|
There's been some work already on that in WebAssembly/wasi-libc#444 but there are concerns about how that is implemented so I think some discussion is needed. @loganek is also interested in making this happen and has added an agenda item to the WASI subgroup meeting to have that discussion: WebAssembly/meetings#1479. Can you join that? |
|
Thanks, I did checked the issues but not the PRs. I just joined the WebAssembly CG and filled the form for the meeting. I'll be there. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Are there any plans to support 64-bit builds?
It looks to me like wasm64 (also known as "memory64") uses a LP64 model (
intis 32 bits,long,long long, and pointers are 64 bits), which is the same as the memory model on 64-bit Unix-y platforms (not Windows, wherelongis still 32 bits even onx86_64).The text was updated successfully, but these errors were encountered: