All repos for an organisation

[tinydylan]

My organisation has hundreds of GitHub repositories under a single GitHub Organisation. This tap only extracts data a single repository - we would like this tap to extract data for all repositories in our Organisation.

[luandy64]

tap-github/tap_github.py

Line 863 in 35ac5df

repositories = list(filter(None, config['repository'].split(' ')))

tap-github/tap_github.py

Line 868 in 35ac5df

for repo in repositories:

I think according to these lines the tap is able to run for multiple repositories

[tinydylan]

Thanks, @luandy64.

So, it sounds like the tap would just need to be altered so that it can read the list of repositories from a GitHub API call (https://api.github.com/orgs/ORG_NAME/repos).

[osterman]

Btw, I went down the route but quickly ran into rate limits. Without #63, this cannot be done reliably.

[tinydylan]

In stitchdata, I have the github integration running against a hard-coded list of 350 repositories. I hit rate limits for a few hours, then it's fine after that.

Getting all repos for an org in one setting is no different to what I'm doing, it just means that I don't have to keep regenerating the list of repos every time it changes.

While the rate limit does get in the way, it gets in the way already and I would argue for going ahead with this requirement independendly of graceful rate-limit handling.