Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

payout_transactions syncs too much duplicate data #63

Open
idris opened this issue Apr 20, 2020 · 0 comments
Open

payout_transactions syncs too much duplicate data #63

idris opened this issue Apr 20, 2020 · 0 comments

Comments

@idris
Copy link

idris commented Apr 20, 2020

I am using Stitch and replicating Stripe every 30 minutes. As you can see in the screenshot below, for the last 12 hours, the payout_transactions table has replicated 8,701 rows every 30 minutes.

My guess is that it's selecting the latest payout (because it's >= the created) of the latest replicated one), and then querying Stripe for all balance_transactions related to that payout. In my opinion, it should either (a) use > for payouts, instead of >= or (b) keep the latest payout.id in addition to the created date, and if we've already replicated that id, don't bother fetching the balance_transactions to fill in the payout_transactions sub-stream.

The impact of the current behavior is (1) waste of requests to Stripe and (2) a lot of unnecessary additional cost on Stitch.

Screen Shot 2020-04-20 at 10 15 44 AM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant