Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ability to persist/merge hashed target data into reconcilation DB #18

Closed
chadlwilson opened this issue Oct 4, 2021 · 0 comments
Closed

Ability to persist/merge hashed target data into reconcilation DB #18

chadlwilson opened this issue Oct 4, 2021 · 0 comments
Assignees
@chadlwilson
Labels
enhancement New feature or request size:M medium items

Comments

@chadlwilson
Copy link
Collaborator

Context / Goal

In #6 we implemented initial ability to persist the source data into the reconciliation DB. We now need to extend this to query the target database with its dataset definition, and augment the data with that from the target database.

Expected Outcome

  • be able to load data from a target DB using its dataset query
  • insert new rows where a row containing the MigrationKeyis not found already
  • update rows with the hashed target data if the row already exists

Out of Scope

  • generating a report of differences between the two datasets

Additional context / implementation notes
@chadlwilson chadlwilson added the enhancement New feature or request label Oct 4, 2021
@chadlwilson chadlwilson mentioned this issue Oct 12, 2021
Closed
@chadlwilson chadlwilson self-assigned this Oct 18, 2021
chadlwilson added a commit that referenced this issue Oct 19, 2021
@chadlwilson
#18 Augment MigrationRecords in the recce DB with the hashes from t…
b54b868 
…he target DB

- Currently runs completely sequentially - all source rows first, then update|insert target rows
- Needs to be refactored out for improved testability and clarity
chadlwilson added a commit that referenced this issue Oct 19, 2021
@chadlwilson
#18 Move support for running query against a source|target DB to comm…
707e758 
…on `DataLoadDefinition`
chadlwilson added a commit that referenced this issue Oct 19, 2021
@chadlwilson
#18 Use more of a Kotlin style for inOrder verifications
5bda0a7
This was referenced Oct 25, 2021
Closed
Open
@chadlwilson chadlwilson added the size:M medium items label Nov 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@chadlwilson chadlwilson

Labels
enhancement New feature or request size:M medium items
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@chadlwilson