Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Impossible to set PK on Reports #45

Open
judahrand opened this issue Jan 23, 2020 · 3 comments
Open

Impossible to set PK on Reports #45

judahrand opened this issue Jan 23, 2020 · 3 comments

Comments

@judahrand
Copy link

judahrand commented Jan 23, 2020

This is an issue which I've noticed across a few taps but first here so I'll raise the issue here and take it from there.

In my use case I am pulling static reports from various taps and using a target to store them in a database. Because the reports are static I know what the primary key (PK) should be.

It would be good to be able to set the PK in a similar fashion as was just merged into tap-adwords (singer-io/tap-adwords#58)

This would involve changing this line to:

    singer.write_schema(report_stream.stream,
                                     report_schema,
                                     metadata.get(stream_metadata, (), 'tap-bing-ads.report-key-properties') or [])
   )

This change would, of course, need the relevant documentation additions.

Please do let me know if such a PR would be approved and I'll put it in ASAP.

@judahrand
Copy link
Author

I've realised there is an issue with this approach... This tap seems not really support the current Singer Spec as the generated metadata does not seem to include an empty breadcrumb entry. This would also need to be added. And while one was at it they could make it use the current select criteria from the metadata for the stream too.

@judahrand
Copy link
Author

Can someone look at this?

@ilkkapeltola
Copy link

Two years later, this still bugs me out.
If I take this forward to a target that doesn't have any information about primary keys, but needs to, this fails.
Can you please fix this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants