Automated retries

[ekzyis]

Description

close #1492 based on #1785, #1787

All failed invoices are returned only once to exactly one client via a new query failedInvoices. To avoid an invoice being returned but never retried by a client for any reason, a visibility timeout as described in #1492 (comment) is used: for each returned invoice, we set a new Invoice.lockedAt column to the current time and then unset it when we can assume that the client did not and will not attempt a retry of this invoice.

To stop after three payment attempts (= two retries), a new integer column Invoice.paymentAttempt tracks at which payment attempt we are. This number is increased when we retry an invoice that has been locked. This is only the case when we start a new payment attempt. If we retry the first invoice of a payment attempt, it is not locked and thus the new invoice will have the same number in its paymentAttempt column. This means payment attempts are isolated from each other by giving each invoice of the same payment attempt the same value in their paymentAttempt column. But this also means that if a payment attempt is interrupted after the first invoice was retried, we cannot continue it but will increase the counter. I think that's okay.

TODO:

• retry returned failed invoices on client
• count retries so we can stop returning them after we hit a limit
• show failed invoices that have exhausted retries in notifications
• don't show failed invoices that haven't exhausted retries in notifications
• don't retry invoices that have been manually cancelled?

added userCancel column, see #1785

• don't return failed invoices to clients that don't have send wallets

client only polls when it has send wallets

• don't return intermediate failed invoices that will be retried due to sender or receiver fallbacks

added "cancelledAt" < now() - interval '${WALLET_RETRY_AFTER_MS} milliseconds' filter

• ⚠️ make sure when this is deployed, this will not start retrying old failed invoices ⚠️

added WALLET_RETRY_BEFORE_MS used in this filter:

AND now()::timestamp <@ tsrange(
  "cancelledAt" + interval '${WALLET_RETRY_AFTER_MS} milliseconds',
  "cancelledAt" + interval '${WALLET_RETRY_BEFORE_MS} milliseconds'
)

Additional Context

• this will still retry invoices that have been created before deployment and failed if they aren't older than a day
• if a client detaches all wallets after they tried a payment for the first time, it will never show up in notifications as failed since it wasn't retried often enough. I consider this to be an edge case we don't need to worry about.
• see https://github.com/stackernews/stacker.news/pull/1776/files#r1907791409

Checklist

Are your changes backwards compatible? Please answer below:

tbd

On a scale of 1-10 how well and how have you QA'd this change and any features it might affect? Please answer below:

7. Tested automated retries for posting and comment with this patch:

diff --git a/components/use-qr-payment.js b/components/use-qr-payment.js
index dddc53e9..de766618 100644
--- a/components/use-qr-payment.js
+++ b/components/use-qr-payment.js
@@ -20,7 +20,7 @@ export default function useQrPayment () {
       let paid
       const cancelAndReject = async (onClose) => {
         if (!paid && cancelOnClose) {
-          const updatedInv = await invoice.cancel(inv, { userCancel: true })
+          const updatedInv = await invoice.cancel(inv, { userCancel: false })
           reject(new InvoiceCanceledError(updatedInv))
         }
         resolve(inv)

To decrease time between retries, use this patch:

diff --git a/lib/constants.js b/lib/constants.js
index 43d892ac..e467ce2e 100644
--- a/lib/constants.js
+++ b/lib/constants.js
@@ -197,7 +197,7 @@ export const WALLET_CREATE_INVOICE_TIMEOUT_MS = 45_000
 // When should failed invoices be returned to a client to retry?
 // This must be high enough such that intermediate failed invoices that will be retried
 // by the client due to sender or receiver fallbacks are not returned to the client.
-export const WALLET_RETRY_AFTER_MS = 60_000
+export const WALLET_RETRY_AFTER_MS = 3_000

Tested automated retries for p2p zaps with this patch that makes forwards fail and disables the fallback to CCs:

diff --git a/api/paidAction/index.js b/api/paidAction/index.js
index 8feabc30..031e4d7e 100644
--- a/api/paidAction/index.js
+++ b/api/paidAction/index.js
@@ -355,6 +355,7 @@ export async function retryPaidAction (actionType, args, incomingContext) {
       invoiceArgs = { bolt11, wrappedBolt11, wallet, maxFee }
     } catch (err) {
       console.log('failed to retry wrapped invoice, falling back to SN:', err)
+      throw err
     }
   }
diff --git a/worker/paidAction.js b/worker/paidAction.js
index 9b3ecb5a..b9172ebf 100644
--- a/worker/paidAction.js
+++ b/worker/paidAction.js
@@ -268,7 +268,7 @@ export async function paidActionForwarding ({ data: { invoiceId, ...args }, mode
     payViaPaymentRequest({
       lnd,
       request: bolt11,
-      max_fee_mtokens: String(maxFeeMsats),
+      max_fee_mtokens: String(0),
       pathfinding_timeout: LND_PATHFINDING_TIMEOUT_MS,
       confidence: LND_PATHFINDING_TIME_PREF_PPM,
       max_timeout_height: maxTimeoutHeight

For frontend changes: Tested on mobile, light and dark mode? Please answer below:

n/a

Did you introduce any new environment variables? If so, call them out explicitly here:

no

[ekzyis]

This useMemo and the one in useSendWallets is pretty important since it avoids that waitForWalletPayment changes between renders which would mean that we run the useEffect for retries with the same failed invoices multiple times.

In general, this means it is very important that waitForWalletPayment is stable across renders.

[ekzyis]

RetryHandler isn't a context provider, but I think we should use more providers inside context providers nonetheless. There are too many in pages/_app.js.

For example, we could put all providers for the carousel (price, block height, chain fee) into the same component.