Consolidates building a node tutorial #156

ConorOkus · 2022-08-08T12:34:09Z

This uses the new CodeSwitcher so we can have one Building a node tutorial but have code examples for multiple languages in one place. For instance, clicking the "Java" tab changes all the code examples on the page to be Java. This PR includes Rust, Java, Kotlin. Swift + JavaScript/TypeScript to follow

It also reworks the content structure to be more in line with how we currently explain LDK.

TO-DO

Includes minor changes to the navigation layout.

Preview

netlify · 2022-08-08T12:34:16Z

✅ Deploy Preview for lightningdevkit ready!

Name	Link
🔨 Latest commit	`cd008da`
🔍 Latest deploy log	https://app.netlify.com/sites/lightningdevkit/deploys/65c515a876ed010008a518cd
😎 Deploy Preview	https://deploy-preview-156--lightningdevkit.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

moneyball · 2022-08-09T02:34:42Z

Concept ACK

ConorOkus · 2023-01-25T14:55:04Z

docs/tutorials/building-a-node-with-ldk/setting-up-a-channel-manager.md

+
+There are a few dependencies needed to get this working. Let's walk through setting up each one so we can plug them into our `ChannelManager`.
+
+### Initialize the `FeeEstimator`


We should provide an example using BDK's new fee estimator API.

docs/tutorials/building-a-node-with-ldk/setting-up-a-channel-manager.md

tnull

Did a quick scroll through and annotated what stuck out to me. Feel free to disregard if many of the things are still WIP.

docs/tutorials/building-a-node-with-ldk/introduction.md

docs/tutorials/building-a-node-with-ldk/connect-to-peers.md

docs/tutorials/building-a-node-with-ldk/setting-up-a-channel-manager.md

docs/tutorials/building-a-node-with-ldk/opening-a-channel.md

tnull

Did a ~full high-level pass.

While it's nice to have a single flow, I'm not fully convinced that a unified tutorial improves readability. I think at times it can be confusing where the APIs or implementation behavior of the different languages diverge slightly.

docs/tutorials/building-a-node-with-ldk/introduction.md

tnull · 2023-10-20T10:31:52Z

docs/tutorials/building-a-node-with-ldk/introduction.md

+3. Payments & Routing, ability to create and pay invoices.
+
+To make the above work we also need to setup a series of supporting modules, including:
+1. A [`FeeEstimator`](https://docs.rs/lightning/*/lightning/chain/chaininterface/trait.FeeEstimator.html)


I think it would be nice if each of these list items would hold a short description giving some context what the respective modules are used/required for.

It would mostly be repeating what is in the Rust docs though so I'd rather just lean on that.

docs/tutorials/building-a-node-with-ldk/setting-up-a-peer-manager.md

tnull · 2023-10-20T11:54:55Z

docs/tutorials/building-a-node-with-ldk/handling-events.md

+  Event::PaymentFailed { payment_hash, rejected_by_dest } => {
+          // Handle failed payment
+  }
+  Event::FundingGenerationReady { .. } =>


I think we need to at least mention PaymentClaimable,PendingHTLCsForwardable, SpendableOutputs here, as they really need to be implemented. Possibly also BumpTransaction if users use Anchors.

I'd prefer if they were mentioned in context like - https://deploy-preview-156--lightningdevkit.netlify.app/tutorials/building-a-node-with-ldk/opening-a-channel/

jkczyz

Did a ~full high-level pass.

While it's nice to have a single flow, I'm not fully convinced that a unified tutorial improves readability. I think at times it can be confusing where the APIs or implementation behavior of the different languages diverge slightly.

I have similar concerns. Perhaps separate pages should be dedicated to components where the interfaces diverge (i.e., pull each of those components out of the ChannelManager setup guide).

With that in mind, it seems there's so much needed in terms of step-by-step instructions to set up ChannelManager. But a good number of the examples are just skeleton implementations of traits or simply demonstrate how to pass parameters to preexisting implementations. Whether here or later, maybe we should take an opportunity to pare back that portion of the guide to have fewer and more consolidated examples. The previous suggestion may help as well on cutting down the size.

jkczyz · 2024-02-05T21:14:16Z

docs/building-a-node-with-ldk/setting-up-a-channel-manager.md

+```rust
+let network_graph_path = format!("{}/network_graph", ldk_data_dir.clone());
+let network_graph = Arc::new(disk::read_network(Path::new(&network_graph_path), args.network, logger.clone()));
+
+let scorer_path = format!("{}/scorer", ldk_data_dir.clone());
+let scorer = Arc::new(RwLock::new(disk::read_scorer(
+    Path::new(&scorer_path),
+    Arc::clone(&network_graph),
+    Arc::clone(&logger),
+)));
+
+```


This is using utilities from the sample, which aren't available in LDK. Similarly above for gossip sync.

For the sake of keeping the code snippets short maybe it's ok?

jkczyz · 2024-02-05T21:14:42Z

docs/building-a-node-with-ldk/handling-events.md

+
+</CodeSwitcher>
+
+References: [Rust `Event` docs](https://docs.rs/lightning/0.0.114/lightning/util/events/enum.Event.html), [Java/Kotlin `Event` bindings](https://github.com/lightningdevkit/ldk-garbagecollected/blob/main/src/main/java/org/ldk/structs/Event.java)


We'll want to use * or latest instead of version numbers. Same elsewhere.

Fixed in 39e95b4

jkczyz · 2024-02-05T21:18:47Z

docs/building-a-node-with-ldk/handling-events.md

+```rust
+use lightning::util::events::{Event};
+
+// In the event handler passed to BackgroundProcessor::start


Looks like BackgroundProcessor was removed from the guide. We'll want to have a section on it as it drives everything. Probably somewhere in the introduction.

Fixed in 43b12ca

jkczyz · 2024-02-05T21:20:17Z

docs/building-a-node-with-ldk/introduction.md

+1. A [`PeerManager`](https://docs.rs/lightning/*/lightning/ln/peer_handler/struct.PeerManager.html), for establishing TCP/IP connections to other nodes on the lightning network.
+2. A [`ChannelManager`](https://docs.rs/lightning/*/lightning/ln/channelmanager/struct.ChannelManager.html), to open and close channels.


May want to switch the order of these two given the order they are covered.

Fixed in 31f155f

jkczyz · 2024-02-05T21:29:55Z

docs/building-a-node-with-ldk/setting-up-a-channel-manager.md

+To add a `ChannelManager` to your application, run:
+
+<CodeSwitcher :languages="{rust:'Rust', kotlin:'Kotlin', swift:'Swift'}">
+  <template v-slot:rust>
+
+```rust
+use lightning::ln::channelmanager;
+
+let channel_manager = ChannelManager::new(
+  &fee_estimator,
+  &chain_monitor,
+  &broadcaster,
+  &router,
+  &logger,
+  &entropy_source,
+  &node_signer,
+  &signer_provider,
+  user_config,
+  chain_params,
+  current_timestamp
+);
+```


This alone isn't very valuable and requires that prerequisites that follow. Additionally, it is repeated in greater detail later on under "Initialize the ChannelManager".

Because the ChannelManager is so "involved" I find it useful to start with what one might end up with as it helps shape the mental model of a critical piece of the API. I feel like when you see this the next logical question is well how do I set up a FeeEstimator etc arguably it's already covered in the intro. Slight rewording d26a8e8

jkczyz · 2024-02-05T21:33:22Z

docs/building-a-node-with-ldk/setting-up-a-channel-manager.md

+2. Tell LDK what your best known block header and height is.
+3. Call `channel_manager_constructor.chain_sync_completed(..)` to complete the initial sync process.
+
+**References:** [Rust `Confirm` docs](https://docs.rs/lightning/*/lightning/chain/trait.Confirm.html), [Rust `Listen` docs](https://docs.rs/lightning/*/lightning/chain/trait.Listen.html), [Rust `lightning_block_sync` module docs](https://docs.rs/lightning-block-sync/*/lightning_block_sync/), [Rust `lightning_transaction_sync` module docs](https://docs.rs/lightning-transaction-sync/*/lightning_transaction_sync/)


s/module/crate

Fixed in 5787ac6

docs/building-a-node-with-ldk/setting-up-a-channel-manager.md

jkczyz · 2024-02-05T21:34:51Z

docs/building-a-node-with-ldk/setting-up-a-channel-manager.md

+**Full Blocks or BIP 157/158**
+
+If you are connecting full blocks or using BIP 157/158, then it is recommended to use
+LDK's `lightning_block_sync` sample module as in the example above: the high-level steps that must be done for both `ChannelManager` and each `ChannelMonitor` are as follows:


s/lightning_block_sync sample module/lightning-block-sync crate

Fixed in 74a2f49

docs/building-a-node-with-ldk/setting-up-a-channel-manager.md

jkczyz · 2024-02-05T21:44:30Z

docs/building-a-node-with-ldk/setting-up-a-channel-manager.md

+  &fee_estimator,
+  &chain_monitor,
+  &broadcaster,
+  &router,


The most recent version of LDK has ChannelManager take Router, so this is correct. But a later section with a similar examples is using an older version of LDK that does not take a Router.

Fixed in 8f276a0

This reverts commit 63d3ead.

ConorOkus · 2024-02-07T19:01:06Z

Did a ~full high-level pass.
While it's nice to have a single flow, I'm not fully convinced that a unified tutorial improves readability. I think at times it can be confusing where the APIs or implementation behavior of the different languages diverge slightly.

I have similar concerns. Perhaps separate pages should be dedicated to components where the interfaces diverge (i.e., pull each of those components out of the ChannelManager setup guide).

With that in mind, it seems there's so much needed in terms of step-by-step instructions to set up ChannelManager. But a good number of the examples are just skeleton implementations of traits or simply demonstrate how to pass parameters to preexisting implementations. Whether here or later, maybe we should take an opportunity to pare back that portion of the guide to have fewer and more consolidated examples. The previous suggestion may help as well on cutting down the size.

I feel like it is quite a common pattern/layout in SDK's that offer multiple languages, some examples include.

The structure is supposed to reflect someone who never built an LDK node before and wants to build the simplest node that can perform the basic functions. More detailed/advanced examples can follow.

tnull

Did another pass.

tnull · 2024-02-08T13:18:54Z

docs/building-a-node-with-ldk/introduction.md

+Let's start by looking at the core components we'll need to make this node work for the tasks we outlined above.
+
+1. A [`ChannelManager`](https://docs.rs/lightning/*/lightning/ln/channelmanager/struct.ChannelManager.html), to open and close channels.
+2. A [`PeerManager`](https://docs.rs/lightning/*/lightning/ln/peer_handler/struct.PeerManager.html), for establishing TCP/IP connections to other nodes on the lightning network.


nit: Technically it's not the PeerManager, but the network stack, i.e., lightning-net-tokio that establishes the connections. The PeerManager just keeps track of them and the associated peer state.

Updated d41fecb

docs/building-a-node-with-ldk/installation.md

docs/building-a-node-with-ldk/setting-up-a-channel-manager.md

tnull · 2024-02-08T14:34:54Z

docs/building-a-node-with-ldk/setting-up-a-channel-manager.md

+
+</CodeSwitcher>
+
+**Implementation notes:** this struct is not required if you are providing your own routes. It will be used internally in `ChannelManager` to build a `NetworkGraph`. Other networking options are: `LDKNetwork_Bitcoin`, `LDKNetwork_Regtest` and `LDKNetwork_Testnet`


The networking options here are really bindings specific types, won't work in Rust as described. Also not sure what 'other' is referring to, and Signet seems missing.

Updated in eb7ca72

docs/building-a-node-with-ldk/setting-up-a-peer-manager.md

tnull · 2024-02-08T14:37:28Z

docs/building-a-node-with-ldk/setting-up-a-channel-manager.md

+  let router = DefaultRouter::new(
+    network_graph.clone(),
+    logger.clone(),
+    keys_manager.get_secure_random_bytes(),


I think by now it takes an EntropySource (i.e., KeysManager reference) directly.

This snippet is from ldk-node and ldk-sample. I don't see an example with EntropySource

Ah, I think this was part of the bindings fixes for the last release, i.e., they have not 'officially' released on the Rust side yet. But probably makes sense to already update to the upcoming API versions here to not have the guide immediately outdated.

Co-authored-by: Elias Rohrer <[email protected]>

ConorOkus force-pushed the building-a-node-with-ldk-intro branch 2 times, most recently from e81534c to 2a05a55 Compare September 8, 2022 16:38

ConorOkus force-pushed the building-a-node-with-ldk-intro branch from 2a05a55 to c597f31 Compare October 5, 2022 14:48

ConorOkus force-pushed the building-a-node-with-ldk-intro branch from c597f31 to bf123ea Compare January 9, 2023 15:54

ConorOkus force-pushed the building-a-node-with-ldk-intro branch from 8cda8b0 to 95ca4ce Compare January 25, 2023 14:54

ConorOkus commented Jan 25, 2023

View reviewed changes

docs/tutorials/building-a-node-with-ldk/setting-up-a-channel-manager.md Outdated Show resolved Hide resolved

tnull reviewed Jan 25, 2023

View reviewed changes

ConorOkus force-pushed the building-a-node-with-ldk-intro branch from de5e256 to 4bd2cde Compare March 24, 2023 21:47

ConorOkus force-pushed the building-a-node-with-ldk-intro branch from 4bd2cde to c1fef0b Compare September 14, 2023 16:53

ConorOkus force-pushed the building-a-node-with-ldk-intro branch from 52c58df to f8f7404 Compare October 19, 2023 14:52

tnull reviewed Oct 20, 2023

View reviewed changes

jkczyz self-requested a review October 20, 2023 14:41

ConorOkus force-pushed the building-a-node-with-ldk-intro branch from a7fa74b to fe72a99 Compare January 3, 2024 19:01

Conor Okus added 14 commits February 2, 2024 16:56

Adds tutorial introudction page

9c54349

Consolidate building a node tutorials

ce32f6e

Adds P2PGossip Sync docs

c069152

Adds setting up a PeerManager page

7d94135

Rebase

db4dc1d

Adds initializing networking to connect peers page

c99e9f1

Adds opening a channel page

a9ded9a

Move return type from register_output

b3b90e4

Adds opening a channel page

a508b5c

Trigger build

b5e80e9

Adds FundingGenerationReady Event Handling section

35a3bcf

Improve fee estimator code examples

05e2898

Rename variables in broadcaster code samples

a619831

Removes channel_watch instructions

e9549da

Conor Okus added 4 commits February 2, 2024 20:40

Point to API's

69e0ce1

Restructure nav

5044bdc

Trigger Build

4e9d35f

Trigger Build

5e80f4e

ConorOkus force-pushed the building-a-node-with-ldk-intro branch from 6c08290 to 5e80f4e Compare February 5, 2024 17:36

jkczyz reviewed Feb 5, 2024

View reviewed changes

Conor Okus added 8 commits February 6, 2024 16:32

Use latest instead of version numbers

39e95b4

Adds back in background processing

43b12ca

Switch component ordering

31f155f

Copy update for setting up a channel manager

d26a8e8

s/module/crate

5787ac6

Electrum/Esplora available in transaction-sync

a5504a1

s/module/crate

74a2f49

Trigger build

63d3ead

ConorOkus force-pushed the building-a-node-with-ldk-intro branch from 0f797b1 to 63d3ead Compare February 6, 2024 22:57

Conor Okus added 4 commits February 6, 2024 18:04

Revert "Trigger build"

387ac95

This reverts commit 63d3ead.

Remove unused files

09ee30d

Adds missing link

7831e7e

Removes older version of channel manager initialisation

8f276a0

tnull reviewed Feb 8, 2024

View reviewed changes

Conor Okus and others added 7 commits February 8, 2024 10:30

Copy update

d41fecb

Copy updates for installation page

5763d65

Update docs/building-a-node-with-ldk/setting-up-a-channel-manager.md

d745489

Co-authored-by: Elias Rohrer <[email protected]>

Copy update

eb7ca72

Update docs/building-a-node-with-ldk/setting-up-a-peer-manager.md

88fed6f

Co-authored-by: Elias Rohrer <[email protected]>

Update docs/building-a-node-with-ldk/setting-up-a-peer-manager.md

9ea1442

Co-authored-by: Elias Rohrer <[email protected]>

Update docs/building-a-node-with-ldk/setting-up-a-peer-manager.md

cd008da

Co-authored-by: Elias Rohrer <[email protected]>

ConorOkus merged commit 20dc0a5 into lightningdevkit:main Feb 9, 2024
5 checks passed

ConorOkus deleted the building-a-node-with-ldk-intro branch February 9, 2024 16:01


		There are a few dependencies needed to get this working. Let's walk through setting up each one so we can plug them into our `ChannelManager`.

		### Initialize the `FeeEstimator`


		</CodeSwitcher>

		References: [Rust `Event` docs](https://docs.rs/lightning/0.0.114/lightning/util/events/enum.Event.html), [Java/Kotlin `Event` bindings](https://github.com/lightningdevkit/ldk-garbagecollected/blob/main/src/main/java/org/ldk/structs/Event.java)

		1. A [`PeerManager`](https://docs.rs/lightning/*/lightning/ln/peer_handler/struct.PeerManager.html), for establishing TCP/IP connections to other nodes on the lightning network.
		2. A [`ChannelManager`](https://docs.rs/lightning/*/lightning/ln/channelmanager/struct.ChannelManager.html), to open and close channels.


		</CodeSwitcher>

		Implementation notes: this struct is not required if you are providing your own routes. It will be used internally in `ChannelManager` to build a `NetworkGraph`. Other networking options are: `LDKNetwork_Bitcoin`, `LDKNetwork_Regtest` and `LDKNetwork_Testnet`

Consolidates building a node tutorial #156

Consolidates building a node tutorial #156

Conversation

ConorOkus commented Aug 8, 2022 • edited Loading

netlify bot commented Aug 8, 2022 • edited Loading

✅ Deploy Preview for lightningdevkit ready!

moneyball commented Aug 9, 2022

Choose a reason for hiding this comment

tnull left a comment

Choose a reason for hiding this comment

tnull left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkczyz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ConorOkus commented Feb 7, 2024

tnull left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ConorOkus commented Aug 8, 2022 •

edited

Loading

netlify bot commented Aug 8, 2022 •

edited

Loading