-
Notifications
You must be signed in to change notification settings - Fork 4
FFI communication between native and meta chain #106
Comments
@wafflespeanut: Thanks for opening an issue, it is currently awaiting triage. The triage/accepted label can be added by foundation members by writing /triage accepted in a comment. DetailsI am a bot created to help the DeFiCh developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository. |
@wafflespeanut: There are no 'area' labels on this issue. Adding an appropriate label will greatly expedite the process for us. You can add as many area as you see fit. If you are unsure what to do you can ignore this! You can add area labels by leaving a DetailsI am a bot created to help the DeFiCh developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository. |
I think it's sound given if both options are provided with a shared Interfacing for the sake of testing with the ability to force
Although #45 is important, it isn't sane for masternode operators to operate on a host machine they can't trust. It's more of an additional security good practice IMO, as I don't think they should not be operating their masternode over the network.
From the current literature on our threading model, I think #94 requires a renewal of our existing threading model for MC integration rather than the bottleneck of integration. |
While I'm okay with supporting RPC for the purpose of testing, the whole aim of using FFI is to prepackage the binaries and to not worry about having RPC calls in production or needing to secure them, but if we're planning to have it in the near future anyway, then I don't think the FFI way serves any purpose. @prasannavl Need your opinion here.
Is the performance and storage requirements the only constraint for not being able to run MC alongside NC?
This is not specific to MC though, right? Wouldn't it apply to NC as well? |
Yup, DMC is the first part of a series of changes to modulize the DeFiChain blockchain into several components that were initially proposed in DFIP 2111-B. DMC is the first of many execution planes we want to integrate into a shared network plane. The goal is to modularize the blockchain to make upgrading easier. It was explained in a video on a generative approach to adding more modularized components with a shared network plane.
Part of it. With DMC embedded, the "server profile" and server maintenance's operational flow must be changed if we force
Yup, it would; it's not specific to MC or NC, just running servers or validators in general. IMO, additional security cushions more for the uninitiated server maintainers. |
Btw, @mambisi has brought it to my attention going through the FFI route will not be as easy as it sounds because we won't get the context from substrate outside of wasm environment. Until we have a clear direction there, this is how we've currently decided to go: graph
subgraph DeFiCh/ain
nc[Native chain consensus]
nb[Native chain bootstrap]
nb --> nc
nb --" FFI "--> mb
nc --" FFI "--> nrc
nb --" FFI "--> lock
subgraph "DeFiCh/libain-rs"
pt["Protobuf spec"]
ngc["gRPC client"]
nrc["RPC client"]
pt --> ngc
pt --> nrc
end
subgraph DeFiCh/metachain
mb[Metachain bootstrap]
mc[Metachain consensus]
mprc[RPC server]
lock[Random number agreement]
mprc --> lock
lock --> mprc
mb --> mc
mprc --> mc
nrc --" JSON RPC "--> mprc
end
end
|
Yup sounds good, I'm afraid of that as well.
Yup, this is great; it's important that it's optional. #97 (comment) Does this also mean instead of starting up MetaChain, we could also provide an URL with a port number?
Could we use JWT instead if it's not too troublesome? Minor, I'm also very comfortable with this design for now, but it might require changes. Let's see, it's not important now.
Sounds good. |
I didn't have this in mind, but it shouldn't be hard to implement with the current design.
Agreed, let's get both defid and metachain running in a single node. I'm sure we can improve the mechanism later on. |
What would you like to be added:
As an alternative to JSON RPC communication (#80), I propose invoking functions through FFI. Metachain (MC) can be compiled as a staticlib, which then gets linked to defid (NC). The PoC for this approach exists in
libmc
branch in metachain andain
repository. CLI args from NC will be passed to MC, which will be bootstrapped and then defid will follow.Why is this needed:
This way, we can eliminate the security concerns for running MC as an independent process and communicating through RPC (#45), which also reduces TCP overhead and avoids us packaging and running two binaries which will most probably exist in the same machine. #94 wouldn't have any adverse impact on this decision as the source of the bottleneck will still be in defid. It should be noted that MC and NC will not have any knowledge of what the other is doing and neither will one manage the other. However, when MC fails and exits, it will shutdown defid as well (at this point).
As for e2e testing, we can still expose RPC from MC, because essentially they'll be the same FFI calls.
cc @prasannavl @fuxingloh @DieHard073055
The text was updated successfully, but these errors were encountered: