Allow LDK node to send payjoin transactions #295

jbesraa · 2024-05-21T07:07:11Z

Partially addresses #177

This adds the ability for the on chain wallet to send payjoin transactions as described in BIP 77 https://github.com/bitcoin/bips/blob/d7ffad81e605e958dcf7c2ae1f4c797a8631f146/bip-0077.mediawiki

The payjoin receiver part will be added in a separate PR with e2e tests with this pr code.

tnull

Cool! Let me know when this is ready for a first round of review.

Until then only one early comment:

tnull · 2024-05-21T09:25:54Z

Cargo.toml

@@ -57,7 +57,7 @@ lightning-liquidity = { version = "0.1.0-alpha.4", features = ["std"] }

 bdk = { version = "0.29.0", default-features = false, features = ["std", "async-interface", "use-esplora-async", "sqlite-bundled", "keys-bip39"]}

-reqwest = { version = "0.11", default-features = false, features = ["json", "rustls-tls"] }
+reqwest = { version = "0.11", default-features = false, features = ["json", "rustls-tls", "blocking"] }


FWIW, I'd prefer to stick with the async variant, as the blocking variant introduces even more downstream dependencies, some of which just recently had some security issues, see #283 (comment).

jbesraa · 2024-05-22T12:35:09Z

@tnull this is ready for review

tnull

Cool, did a first pass!

tnull · 2024-05-22T12:37:59Z

Cargo.toml

@@ -68,6 +68,7 @@ tokio = { version = "1", default-features = false, features = [ "rt-multi-thread
 esplora-client = { version = "0.6", default-features = false }
 libc = "0.2"
 uniffi = { version = "0.26.0", features = ["build"], optional = true }
+payjoin = { version = "0.15.0", features = ["send", "receive", "v2"] }


Why do we need receive if this is only the sending side?

yea we dont, it should be fixed in the next payjoin crate release payjoin/rust-payjoin#258

tnull · 2024-05-22T12:41:55Z

src/builder.rs

@@ -248,6 +257,12 @@ impl NodeBuilder {
 		self
 	}

+	/// Configures the [`Node`] instance to enable sending payjoin transactions.
+	pub fn set_payjoin_sender_config(&mut self, payjoin_relay: payjoin::Url) -> &mut Self {


To expose this in bindings we can't use Url as-is, we should likely just take a String here.

nit: Also possibly just?:

Suggested change

pub fn set_payjoin_sender_config(&mut self, payjoin_relay: payjoin::Url) -> &mut Self {

pub fn set_payjoin_relay(&mut self, relay: payjoin::Url) -> &mut Self {

I think this naming would be confusing when we add the payjoin receiver.
the payjoin receiver config will also require payjoin_relay argument.

the receiver config will be:

{ payjoin_directory: Url, payjoin_relay: Url, ohttp_keys: Option<OhttpKeys> }

tnull · 2024-05-22T12:44:24Z

src/builder.rs

@@ -454,6 +471,11 @@ impl ArcedNodeBuilder {
 		self.inner.write().unwrap().set_gossip_source_p2p();
 	}

+	/// Configures the [`Node`] instance to enable sending payjoin transactions.
+	pub fn set_payjoin_sender_config(&self, payjoin_relay: payjoin::Url) {


Same as above: could use a rename and we need to use a String.

Updated to String, please see the comment above regarding the naming

tnull · 2024-05-22T12:44:58Z

src/builder.rs

@@ -524,6 +546,7 @@ fn build_with_store_internal(
 	gossip_source_config: Option<&GossipSourceConfig>,
 	liquidity_source_config: Option<&LiquiditySourceConfig>, seed_bytes: [u8; 64],
 	logger: Arc<FilesystemLogger>, kv_store: Arc<DynStore>,
+	payjoin_sender_config: Option<&PayjoinSenderConfig>,


Let's move this argument up to the other _configs, i.e., before seed_bytes.

tnull · 2024-05-22T12:49:06Z

src/builder.rs

@@ -973,6 +996,16 @@ fn build_with_store_internal(
 	};

 	let (stop_sender, _) = tokio::sync::watch::channel(());
+	let payjoin_sender = if let Some(payjoin_sender_config) = payjoin_sender_config {


Lets use payjoin_sender_config.as_ref().and_then(|psc| { .. }) pattern as for liquidity_source above.

tnull · 2024-05-22T13:39:25Z

src/payjoin_sender.rs

+use std::ops::Deref;
+use std::sync::Arc;
+use std::time::Instant;
+use tokio::time::sleep;


nit: Please don't import this globally, but rather use tokio::time::sleep in the code itself. Otherwise it's a bit confusing which sleep is meant and the blocking one from std really can't be used in async contexts.

tnull · 2024-05-22T13:43:34Z

src/payjoin_sender.rs

+	) -> Option<Vec<u8>> {
+		let duration = std::time::Duration::from_secs(3600);
+		let sleep = || sleep(std::time::Duration::from_secs(10));
+		loop {


Let's use the dedicated tokio macros for these control flows: tokio::select/tokio::time::timeout/tokio::time::interval, etc.

not sure I fully follow how tokio::select can be utilised here, should I add a ticker similar to how its implement in the lib.rs and replace the loop with tokio::select ?

fixed the imports

Yeah, if you figure out a way to do so, a combination of select and interval similar to lib.rs would be preferable to loop. If not, it's not that important.

Still think we should clean this up using interval or similar.

tnull · 2024-05-22T13:45:48Z

src/payjoin_sender.rs

+		}
+
+		// get original inputs from original psbt clone (ocean_psbt)
+		let mut original_inputs = input_pairs(&mut ocean_psbt).peekable();


tnull · 2024-05-22T13:46:28Z

src/payjoin_sender.rs

+		let mut ocean_psbt = ocean_psbt.clone();
+		// for BDK, we need to reintroduce utxo from original psbt.
+		// Otherwise we wont be able to sign the transaction.
+		fn input_pairs(


Not sure why we need this?

sorry, here is a link explaining why this is needed bitcoindevkit/bdk-cli#156 (comment)

Thanks for the link but I mostly meant the local function? Given the complex function definition and the additional Box, couldn't we just psbt.unsigned_tx.input.iter().zip(&mut psbt.inputs) directly where it's used?

tnull · 2024-05-22T13:48:40Z

src/payjoin_sender.rs

+			}
+		}
+
+		let mut sign_options = SignOptions::default();


As above, I think these specifics should live in the Wallet-side methods.

jbesraa · 2024-05-22T15:22:20Z

Thanks @tnull
Ill address the review fully tomorrow.

src/payjoin_sender.rs

jbesraa · 2024-05-23T16:44:30Z

src/payment/payjoin.rs

+		let sender = Arc::clone(&self.sender);
+		let (mut original_psbt, request, context) =
+			sender.create_payjoin_request(payjoin_uri, amount, fee_rate)?;
+		let response = sender.fetch(&request).await;


the idea here is that we try to fetch a response first, if we dont get one after the timeout we set on the request, we will start the background process to wait for the receiver response. I think we could emit an event in line 63

I think it's kind of confusing to provide two separate control flows to the user. Can't we just tell the user to check back regularly instead of polling for an ~arbitrary amount of time after which we'll fail quietly?

If we really think we need the background polling, why not avoid returing anything here and just return the necessary information via an event? This would also solve the async/sync issue above.

tnull

A few more comments, have yet to conduct a full top-bottom review of the current state.

src/types.rs

tnull · 2024-07-22T09:15:38Z

src/uniffi_types.rs

+	type Builtin = String;
+
+	fn into_custom(val: Self::Builtin) -> uniffi::Result<Self> {
+		Ok(ScriptBuf::from_hex(&val).map_err(|_| Error::InvalidPublicKey)?)


Is this supposed to be a PublicKey? What if we'd ever want to use ScriptBuf outside of the narrow context you're using it for currently?

tnull · 2024-07-22T09:18:46Z

src/wallet.rs

+	) -> Result<Psbt, Error> {
+		let fee_rate = self
+			.fee_estimator
+			.get_est_sat_per_1000_weight(ConfirmationTarget::OutputSpendingFee) as f32;


I think we really need to define a new LDK-Node specific ConfirmationTarget type that wraps LDK's but also adds other/custom variants. As this is mostly independent from this PR, it should probably happen in a separate prefactor PR. Let me know if you want to pick this up, otherwise I'm happy to do this in the coming days.

tnull · 2024-07-22T09:30:51Z

src/wallet.rs

+				if proposed_txin.previous_output == original_txin.previous_output {
+					proposed_psbtin.witness_utxo = original_psbtin.witness_utxo.clone();
+					proposed_psbtin.non_witness_utxo = original_psbtin.non_witness_utxo.clone();
+					original_inputs.next();


I think only advancing this if proposed_txin.previous_output == original_txin.previous_output means that we require the same ordering for original and proposed PSBTs. Is this a known requirement?

tnull · 2024-07-22T09:31:55Z

src/payment/payjoin/mod.rs

+								break;
+							},
+							Ok(None) => {
+								dbg!("Payjoin request sent, waiting for response...");


Please remove dbg lines before pushing.

jbesraa · 2024-07-22T14:24:32Z

@tnull bear with me a moment please as I am adding tests and polishing the pr to make it more clear

tnull · 2024-07-22T14:26:13Z

@tnull bear with me a moment please as I am adding tests and polishing the pr to make it more clear

Of course, please ping me whenever you feel it's ready for the next round of review!

jbesraa · 2024-07-25T09:16:11Z

@tnull Thanks for the ongoing review.

Few things done:

Broke down the commit into smaller commits, hopefully it will be easier to review. Maybe before merging its worth squashing the last four commits into a single one as they are all around the Payjoin integration/implementation.
Instead of introducing PayjoinTransaction struct in order to be able to track the transaction status, I added a couple of new properties to PaymentStore that would make it possible to track onchain transactions with the Confirm trait. I tried to not break the current API by setting both new properties, txid and best_block to None in the insert function.
Reduced the number of Payjoin specific events to 3 after your last review.
Added tests to cover the different responses the Payjoin receiver can communicate with the Payjoin sender.
Updated the documentation to be more comprehensive.

There are still few things to cover/discuss, but I would appreciate a review over the usage of the PaymentStore, the Confirm trait and the overall integration of the PayjoinPayment in the code. I hope to get most of the commits ACK'd and to eventually focus on reviewing cd30d62 as its introducing the main BIP77 implementation.

tnull

Did another ~half pass, have yet to take a closer look at the PayjoinHandler and Confirm logic (note we should probably also implement Listen while we're at it, as syncing full blocks is coming up).

tnull · 2024-08-09T10:00:37Z

src/payment/store.rs

NACK on this change adding fields that only make sense in payjoin context to PaymentDetails directly. Please rather add a PaymentKind::Payjoin variant for payjoin transactions as we had discussed before.

tnull · 2024-08-09T10:05:59Z

src/config.rs

@@ -40,6 +40,15 @@ pub(crate) const RESOLVED_CHANNEL_MONITOR_ARCHIVAL_INTERVAL: u32 = 6;
 // The time in-between peer reconnection attempts.
 pub(crate) const PEER_RECONNECTION_INTERVAL: Duration = Duration::from_secs(10);

+// The time before a payjoin http request is considered timed out.


nit: s/http/HTTP/ here and below.

tnull · 2024-08-09T10:07:26Z

src/config.rs

+// The duration between retries of a payjoin http request.
+pub(crate) const PAYJOIN_RETRY_INTERVAL: Duration = Duration::from_secs(3);
+
+// The total duration of retrying to send a payjoin http request.


This comment isn't very helpful. What does 'total duration of retrying' mean? Do we abort the flow afterwards, or do we just give up?

tnull · 2024-08-09T10:09:36Z

src/event.rs

+	PayjoinPaymentSuccessful {
+		/// This can refer to the original PSBT or to the finalised Payjoin transaction.
+		///
+		/// If [`is_original_psbt_modified`] field is `true`, this refers to the finalised Payjoin


Seems this doc link is broken.

tnull · 2024-08-09T10:10:05Z

src/event.rs

+	/// This event is emitted only after one onchain confirmation. To determine the current number
+	/// of confirmations, refer to [`PaymentStore::best_block`]:.
+	///
+	/// [`PaymentStore::best_block`]: crate::payment::store::PaymentStore::best_block


Seems this doc link is broken, PaymentStore doesn't (and shouldn't) hold the best block.

tnull · 2024-08-09T10:40:24Z

src/payment/mod.rs

 mod spontaneous;
 pub(crate) mod store;
 mod unified_qr;

+pub use self::payjoin::PayjoinPayment;


Suggested change

pub use self::payjoin::PayjoinPayment;

pub use payjoin::PayjoinPayment;

tnull · 2024-08-09T10:41:55Z

src/payment/payjoin/handler.rs

@@ -0,0 +1,265 @@
+use bitcoin::address::NetworkChecked;


Please move these imports down after the crate/lightning ones.

tnull · 2024-08-09T10:43:05Z

src/payment/payjoin/handler.rs

+
+use lightning::chain::{BestBlock, Confirm, Filter};
+use lightning::ln::channelmanager::PaymentId;
+use lightning::log_error;


Please import logging-specific things via the logger module, i.e., use crate::logger::{log_error, Logger, FilesystemLogger}

tnull · 2024-08-09T10:45:15Z

src/payment/payjoin/handler.rs

+
+impl Confirm for PayjoinHandler {
+	fn transactions_confirmed(&self, header: &Header, txdata: &TransactionData, height: u32) {
+		self.internal_transactions_confirmed(header, txdata, height);


Why not inline these internal_ methods here? Do we gain something from them being separate?

Maybe they would be reusabe for implementing Listen, which we should probably also do?

tnull · 2024-08-09T10:49:47Z

src/payment/payjoin/mod.rs

+			)?;
+		let original_psbt = payjoin_handler.start_request(payjoin_uri.clone())?;
+		let payjoin_handler = Arc::clone(payjoin_handler);
+		let runtime = rt_lock.as_ref().unwrap();


I know we do this in other places, too, but might be best to avoid this unwrap. Could do this above:

let runtime = if let Some(runtime) = rt_lock.as_ref() { runtime } else { return Err(Error::NotRunning); };

Payjoin [`BIP77`] implementation. Compatible with previous Payjoin version [`BIP78`]. Should be retrieved by calling [`Node::payjoin_payment`]. Payjoin transactions can be used to improve privacy by breaking the common-input-ownership heuristic when Payjoin receivers contribute input(s) to the transaction. They can also be used to save on fees, as the Payjoin receiver can direct the incoming funds to open a lightning channel, forwards the funds to another address, or simply consolidate UTXOs. In a Payjoin transaction, both the sender and receiver contribute inputs to the transaction in a coordinated manner. The Payjoin mechanism is also called pay-to-endpoint(P2EP). The Payjoin receiver endpoint address is communicated through a [`BIP21`] URI, along with the payment address and an optional amount parameter. In the Payjoin process, parties edit, sign and pass iterations of the transaction between each other, before a final version is broadcasted by the Payjoin sender. [`BIP77`] codifies a protocol with 2 iterations (or one round of interaction beyond address sharing). [`BIP77`] Defines the Payjoin process to happen asynchronously, with the Payjoin receiver enrolling with a Payjoin Directory to receive Payjoin requests. The Payjoin sender can then make requests through a proxy server, Payjoin Relay, to the Payjoin receiver even if the receiver is offline. This mechanism requires the Payjoin sender to regularly check for response from the Payjoin receiver as implemented in [`Node::payjoin_payment::send`]. A Payjoin Relay is a proxy server that forwards Payjoin requests from the Payjoin sender to the Payjoin receiver subdirectory. A Payjoin Relay can be run by anyone. Public Payjoin Relay servers are: - https://pj.bobspacebkk.com A Payjoin directory is a service that allows Payjoin receivers to receive Payjoin requests offline. A Payjoin directory can be run by anyone. Public Payjoin Directory servers are: - https://payjo.in

The `Confirm` trait is implemented in order to track the Payjoin transaction(s). We track two different transaction: 1. Original PSBT, which is the initial transaction sent to the Payjoin receiver. The receiver can decide to broadcast this transaction instead of finishing the Payjoin flow. Those we track it. 2. Final Payjoin transaction. The transaction constructed after completing the Payjoin flow, validated and broadcasted by the Payjoin sender.

Payjoin transactions

jbesraa · 2024-12-09T08:30:25Z

It seems that rust-payjoin introduced breaking changes in latest updates. I am a bit hesitant to update the crate version yet, until it includes the functionality we need (payjoin/rust-payjoin#336) as that version might also introduce breaking changes.

tnull · 2024-12-09T08:34:01Z

It seems that rust-payjoin introduced breaking changes in latest updates. I am a bit hesitant to update the crate version yet, until it includes the functionality we need (payjoin/rust-payjoin#336) as that version might also introduce breaking changes.

Alright. Your choice when to bump here and in the other PR. Feel free to go the path of least effort/lowest likelihood of errors.

DanGould · 2024-12-09T16:17:07Z

The latest release is planned to be the final breaking BIP 77 wire protocol changes. I recommend beginning to upgrade since it's possible to make a working solution based on the reference implementation in payjoin-cli (using Receiver.id() and Sender.pj_url() as the id for the Sender.)

Since these are short-lived session IDs, even if the data identifier is changed after merge here (which it won't be) there's no critical persisted. data lost. I'll ask @nothingmuch what he thinks about this issue in more detail now that we've firmed up the wire protocol

tnull reviewed May 21, 2024

View reviewed changes

jbesraa force-pushed the pj-sender branch 4 times, most recently from 71f7ce5 to b444fab Compare May 21, 2024 16:42

DanGould mentioned this pull request May 21, 2024

Building fail with v2, send features @ v0.15 payjoin/rust-payjoin#258

Closed

jbesraa force-pushed the pj-sender branch 2 times, most recently from 79f8b3f to 532c6ef Compare May 22, 2024 09:14

jbesraa mentioned this pull request May 22, 2024

Depend on bitcoin/rand for rand payjoin/rust-payjoin#261

Merged

jbesraa force-pushed the pj-sender branch from 532c6ef to da4a464 Compare May 22, 2024 09:40

jbesraa mentioned this pull request May 22, 2024

Payjoin integration #257

Closed

jbesraa force-pushed the pj-sender branch 2 times, most recently from e929b4f to 0dea51e Compare May 22, 2024 12:02

jbesraa marked this pull request as ready for review May 22, 2024 12:32

tnull reviewed May 22, 2024

View reviewed changes

jbesraa force-pushed the pj-sender branch from 0dea51e to 14e9c40 Compare May 22, 2024 15:12

jbesraa force-pushed the pj-sender branch 10 times, most recently from 21d1070 to c050240 Compare May 23, 2024 16:32

jbesraa commented May 23, 2024

View reviewed changes

src/payjoin_sender.rs Outdated Show resolved Hide resolved

jbesraa commented May 23, 2024

View reviewed changes

jbesraa force-pushed the pj-sender branch 3 times, most recently from 8cdcfc0 to 9ce890d Compare July 19, 2024 13:17

tnull reviewed Jul 22, 2024

View reviewed changes

jbesraa force-pushed the pj-sender branch 2 times, most recently from db47845 to aad3e09 Compare July 22, 2024 11:28

jbesraa force-pushed the pj-sender branch 8 times, most recently from a152d29 to 9c6b018 Compare July 25, 2024 08:58

jbesraa mentioned this pull request Jul 25, 2024

Implement Writeable and Readable for bitcoin::Address lightningdevkit/rust-lightning#3205

Closed

tnull reviewed Aug 9, 2024

View reviewed changes

jbesraa force-pushed the pj-sender branch from 9c6b018 to e9cde47 Compare November 8, 2024 07:37

jbesraa added 5 commits December 5, 2024 15:07

Add NodeBuilder::PayjoinConfig to allow ..

ce87a44

Payjoin transactions

Expose PayjoinPayment module

56b1660

f

cf788ef

jbesraa force-pushed the pj-sender branch from e9cde47 to cf788ef Compare December 5, 2024 14:27

f

fbc83fd

	pub fn set_payjoin_sender_config(&mut self, payjoin_relay: payjoin::Url) -> &mut Self {
	pub fn set_payjoin_relay(&mut self, relay: payjoin::Url) -> &mut Self {

	pub use self::payjoin::PayjoinPayment;
	pub use payjoin::PayjoinPayment;

Allow LDK node to send payjoin transactions #295

Are you sure you want to change the base?

Allow LDK node to send payjoin transactions #295

Conversation

jbesraa commented May 21, 2024 • edited Loading

tnull left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbesraa commented May 22, 2024

tnull left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbesraa May 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbesraa commented May 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tnull left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbesraa commented Jul 22, 2024

tnull commented Jul 22, 2024

jbesraa commented Jul 25, 2024

tnull left a comment

Choose a reason for hiding this comment

tnull Aug 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbesraa commented Dec 9, 2024

tnull commented Dec 9, 2024

DanGould commented Dec 9, 2024

jbesraa commented May 21, 2024 •

edited

Loading

jbesraa May 22, 2024 •

edited

Loading

tnull Aug 9, 2024 •

edited

Loading