Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP API #688

Open
3 tasks
Changaco opened this issue Sep 7, 2017 · 47 comments
Open
3 tasks

HTTP API #688

Changaco opened this issue Sep 7, 2017 · 47 comments

Comments

@Changaco
Copy link
Member

Changaco commented Sep 7, 2017

We need a proper HTTP API so that external apps can interface with Liberapay, and for any deep integration we need to become an OAuth provider so that apps can access and/or modify a user's donations.

Technical checklist:

  • Money amounts should be represented as strings in JSON (HTTP API #688 (comment)).
  • The API should support CBOR, or at least be implemented in a way that is compatible with supporting CBOR.
  • Authentication tokens should have prefixes so that they can be easily recognized (see this blog post and RFC 8959 for more details).
@aggsol
Copy link
Contributor

aggsol commented Sep 15, 2017

Why would any external app interface with Liberapay? What usecase do you see for creator and donors?

@Changaco
Copy link
Member Author

@aggsol An unauthenticated API could be used for discovery, e.g. figuring out if a person or project has a Liberapay account. Technically we already have that but it's not documented and not versioned.

OAuth could be used by:

  • another platform to allow its users to connect their Liberapay account in the same way Liberapay users can connect their elsewhere accounts (GitHub, Mastodon, etc)
  • a generic alternative frontend (mobile, desktop, or even web)
  • a specialized app, for example to deal with accounting and taxes

@Changaco
Copy link
Member Author

Some preliminary thoughts:

  • Should the API be part of the main app? Or should it be a separate app?
  • https://api.liberapay.com/ or https://liberapay.com/api? The subdomain is probably better in the long run because it's easy to route it separately from the main app.
  • Should we use GraphQL? Here's an interesting comparison: GraphQL vs. REST.

@aggsol
Copy link
Contributor

aggsol commented Sep 21, 2017

I vote for:

  • a separated app makes it easier to move around, isolate, throttle etc
  • https://api.liberapay.com
  • REST is well know, easy to test (curl and jq ir more than enough) and liberapay is not complex enough IMO to benefit from GraphQL

@Changaco Changaco mentioned this issue Nov 4, 2017
2 tasks
@Changaco
Copy link
Member Author

Changaco commented Dec 4, 2017

@elegaanz
Copy link
Contributor

elegaanz commented Dec 4, 2017

I tried GraphQL once, and the first problem I had is there is no official solution for authentication. You can hack something, but it seems difficult to limit authentication to some resources. And as @aggsol said, GraphQL would probably be too much for Liberapay anyway.

@timpritlove
Copy link
Contributor

I want to voice my support for an API and I want to give an example what kind of software could integrate such an API: online banking software (e.g. MoneyMoney which provides an internal API itself https://moneymoney-app.com/api/webbanking/).

The goal would be to integrate Liberapay into banking apps as if they were a real bank. To the user, there is not a big difference:

  • there is authenticated access to an account
  • the account has a balance
  • there are transactions going on (payouts and withdrawals)

So if LP could be treated like a bank account, it would automatically integrate into online banking software that is itself has an open API. An even more useful step would be to actually provide FINTS [1] compatibility which would make LP compatible with all online banking software in Germany.

However, a simple HTTP-based API that provides balance and uniquely identified transactions would be a good start.

[1] http://www.hbci-zka.de/english/

@ddevault
Copy link
Contributor

ddevault commented Dec 9, 2017

I'm looking for an API, too. I want to use LiberaPay to accept donations, but I also accept donations from Patreon and a self-hosted (also FOSS) platform. On the latter I have an aggregated summary of donations from both sources: https://drewdevault.com/donate/

I would use the API to pull that data and display it alongside the other sources here.

@Changaco
Copy link
Member Author

@SirCmpwn If all you need is basic public information about your account you can get that from https://liberapay.com/$userid/public.json, e.g. https://liberapay.com/Liberapay/public.json.

@ddevault
Copy link
Contributor

Nice, that'll do it.

@jelv
Copy link
Contributor

jelv commented Jul 23, 2018

It would be cool if sites like The Ocean Cleanup could use the oauth for there fund widget like https://www.theoceancleanup.com/fund/. Step 1 set funding and step 2 register or connect.

@ColinFinck
Copy link

I would like to stress the importance of an authenticated API to get information about the balance and transactions.
The ReactOS Project has recently started accepting donations over Liberapay due to popular demand. As the treasurer of the German non-profit organization backing ReactOS, I have to book each and every donation going to the project. When PayPal donations started to create an overwhelming amount of work, I wrote a plugin for our bookkeeping software to automate the import: https://www.reactos.org/blogs/inside-reactos-deutschland-ev-donations-and-how-automate-them

Currently, booking the few Liberapay donations is still possible manually, albeit tedious.
However, this puts us in a paradoxical situation: If Liberapay becomes too popular, we have to stop supporting it, because booking hundreds of donations manually isn't feasible. An API to automate this task could instantly resolve the situation.

@Changaco
Copy link
Member Author

Technical note: representing money amounts as integers in JSON is not a good idea (I'm looking at you, Stripe), because it can easily result in an end-user being billed 100 times more (or less) than intended. We're currently representing money amounts as strings in our undocumented endpoints, and we should continue to use strings.

@ddevault
Copy link
Contributor

ddevault commented Oct 20, 2018

No no no, don't represent them as floats or strings, that's even more error prone. Representing money as cents and integers is industry standard.

@Changaco
Copy link
Member Author

@SirCmpwn I disagree, my personal experience is that integers are much more problematic. In python dealing with money amounts represented as strings is a piece of cake: Decimal(amount) when parsing and str(amount) when serializing. With integers the parsing and serializing involve figuring out whether or not the amount needs to be multiplied/divided by 100, which is very error prone and isn't guaranteed to be the same for all APIs (because each one may have a different list of "zero-decimal" currencies).

@ddevault
Copy link
Contributor

Not all languages have good decimal support like that. Consider JavaScript - where it's harder to fuck up integers than floats. If you always work in cents you're far more likely to undercharge than overcharge.

I mean, it's a matter of personal opinion for you. But it's a matter of industry standard for everyone else. Don't let your personal judgement make your API surprising and error-prone.

@Changaco
Copy link
Member Author

Accidental floating point math errors don't result in numbers that are off by a factor of one hundred, they're far less dangerous than integers. Making an API more difficult to use from Python in order to make it easier to (mis)use from JavaScript seems like a bad trade-off to me.

If you always work in cents

They're not always cents, some currencies are "zero-decimal".

@ddevault
Copy link
Contributor

Well, cents are a US dollar thing. There is no demonination smaller than 1 Yen. The idea is still basically the same, it's storing integers at a precision specific enough to include the smallest demonination.

@Changaco
Copy link
Member Author

It's not always clear what the "smallest denomination" of a currency is, and it can change over time.

By the way, PayPal's REST API uses strings for money amounts, not integers.

@ddevault
Copy link
Contributor

Well, I've made my point. I'm not convinced but it's your API.

@danhunsaker
Copy link

A decimal would work fine as long as the payload also includes the power of ten to adjust it by. US$ would, for example, have a value of 2 passed along with the amount, and the value can then be calculated internally as amount / (10 ^ places). Zero-decimal currencies would pass 0 for the adjusted places, and since 10 ^ 0 == 1, everything works as expected.

Strings, however, have this information built in to a single value. Language-specific precision issues can be bypassed by grabbing the integer form of the amount with the decimal point already removed (it's generally fairly simple to remove all instances of a given character from a string), and counting the number of characters after the decimal point in the string value to determine the places adjustment.

Really, it's all sixes.

@danhunsaker
Copy link

Having chimed in on the technical side, I have another use case to propose.

Patreon permits OAuth logins to be used to unlock backer-only content on external sites. Most often, from what I've seen, this amounts to early access to things the general public will get to see later. My specific use case is exactly that – offering early access to web comic content for folks who back or otherwise donate, as a "thank you" rather than a purchase. I'd love to integrate Liberapay the same way, so that creators can enjoy this functionality without needing Patreon at all (or possibly even alongside it).

@Changaco
Copy link
Member Author

@danhunsaker That use case is forbidden by the current terms of service. Donors must not receive any reward: no early access, no publicity, nothing. The discussion about modifying the terms of service is liberapay/liberapay.org#12, I've copied your comment there.

@danhunsaker
Copy link

That's an entirely fair point. FLO as an ideology really doesn't support that use case, after all.

Luckily, other use cases presented here are still valid!

@decentral1se
Copy link

Hi folks, I'd like to chime in here and show a use case that is taking off in the world of Javascript. The open collective install banner: https://github.com/nuxt/opencollective. When you install a package, it shows the creator's project page and some meta data etc. I want to create this for liberapay but I'd need API access to get started.

@decentral1se
Copy link

decentral1se commented Apr 3, 2019

btw, if there is some way forward, I wouldn't mind putting time into the implementation. Perhaps a first step to show some minimial metadata for the profile. That would probably cover my needs right now anyway ...

Silly me, I've just seen #688 (comment), sorry for noise!

@nektro
Copy link

nektro commented Jul 10, 2020

Being able to add a webhook for successful transactions would allow external count tracking for uses such as donor-rewards for reaching milestones, etc.

@12people
Copy link

12people commented Oct 9, 2020

@Changaco Just curious — is this being worked on? If so, how far along is it?

@Changaco
Copy link
Member Author

Changaco commented Oct 9, 2020

@12people I haven't started implementing this.

@12people
Copy link

Ok, thanks for the info!

@MalteKiefer
Copy link

Is there any new status here?

@Changaco
Copy link
Member Author

Changaco commented Dec 30, 2021

No. Having a proper HTTP API would be nice, but it still isn't a priority.

I encourage you to share why you want an API, what you would like to do with it.

@MalteKiefer
Copy link

MalteKiefer commented Dec 30, 2021 via email

@Changaco
Copy link
Member Author

Thank you for the clarification. An Android app has been discussed before in liberapay/salon#214. I still think that a native app for Liberapay is too much work, and that working on #1000 is more efficient and effective.

@kairusds
Copy link

No. Having a proper HTTP API would be nice, but it still isn't a priority.

I encourage you to share why you want an API, what you would like to do with it.

Having a proper API would be great if you want to get a list of the supporters/donators & your revenue

@Changaco
Copy link
Member Author

If you're only looking for basic public information, you can find it in https://liberapay.com/<username>/patrons/public.csv and https://liberapay.com/<username>/public.json.

If you need the full private list of patrons, you can find the links in your Patrons page, but it's difficult to automate the downloading of these private CSV files, because Liberapay currently only supports authentication tokens which eventually expire.

@brunoais
Copy link

authentication tokens which eventually expire.

As long as there's a way to get a new authentication token, that's not a problem. How can a new token be obtained?

@kastwey
Copy link

kastwey commented Feb 25, 2023

Hello everyone.

I am the administrator of a Mastodon instance (mastodon.cat). I wanted to integrate a bot that we have, to send messages when we receive donations, but I see that you still don't have an API that allows this.
I have seen that there are endpoints to get public data, but I imagine that this does not include donations that people make privately, right?
Is there a way for me to know when I get a donation, whatever it is, a webhook maybe?

Thanks!

@Changaco
Copy link
Member Author

@kastwey There is currently no way to be notified when you gain a new donor. If you want to know when you receive a payment, you could use Stripe's webhooks, however you won't be able to distinguish initial payments from renewal payments.

You can easily get the list of your public donors, but currently there is no clean way to automate getting the list of your private donors.

@brunoais
Copy link

@Changaco How much work would be needed to allow automation to get the list of private donors?

@Changaco
Copy link
Member Author

@brunoais Not much. The only thing missing is read-only authentication tokens which never expire.

However, since rewarding patrons is forbidden, the list of private donors has very few valid use cases.

@brunoais
Copy link

brunoais commented Feb 27, 2023

However, since rewarding patrons is forbidden,

Took me a while to find it but you mean this, right?
image

I failed to spot that earlier.

I wish it was changed into liberapay allowing those but they are completely unrelated to liberapay in terms of responsibilities. Is there such suggestion or a place where I can write such suggestion?

@Changaco
Copy link
Member Author

@brunoais You can share your use case for rewards in liberapay/liberapay.org#12.

@osapon
Copy link
Contributor

osapon commented Dec 2, 2024

It would be nice to just have a link to download the CSV with a bearer token, etc.

Now, I am running a process that scrapes the login process, downloads the CSV, and imports it into my system, but sometimes I can't log in, as if some kind of scraping check is working.

@Changaco
Copy link
Member Author

Changaco commented Dec 5, 2024

@osapon Your automated log-in attempts are probably hitting the new checks I added last week in response to a wave of abuse. I've started working on a proper mechanism for indefinite read-only access to an account. I should be able to land it before the end of next week since it's a relatively small change.

@Changaco
Copy link
Member Author

It's now possible to obtain stable credentials for use in scripts: https://liberapay.com/about/me/access/constant-session

@osapon
Copy link
Contributor

osapon commented Dec 11, 2024

@Changaco Excellent!!
I'm glad I consulted. Thank you very much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests