Filter out deleted accounts #143
Conversation
There was a problem hiding this comment.
@fivetran-avinash I have a changelog suggestion and also a question!
When I ran the consistency tests with seed data, I get the below failure.
When I dug into it, there was a change in the adjusted+amount.
Is this expected given the changes?
| ## Upstream Source Package Updates | ||
| - Deleted records have been previously been brought into the `dbt_quickbooks` package. To ensure accuracy in reporting, the following updates were made in the [v0.11.0 release](https://github.com/fivetran/dbt_quickbooks_source/releases/tag/v0.11.0) of `dbt_quickbooks_source`: | ||
| - We introduced the `_fivetran_deleted` field to filter out deleted records from the following staging models: | ||
| - `stg_quickbooks__account` | ||
| - `stg_quickbooks__bundle` | ||
| - `stg_quickbooks__customer` | ||
| - `stg_quickbooks__department` | ||
| - `stg_quickbooks__item` | ||
| - `stg_quickbooks__vendor` |
There was a problem hiding this comment.
Can you add which end models could potentially be affected by this change? This would help customers check their data if they need to.
There was a problem hiding this comment.
It's impacting all models haha, so just tweaked line 13.
|
@fivetran-avinash I realized I pasted the wrong screenshot yesterday. It's fixed now! |
|
@fivetran-avinash as we discussed the failure is expected because of the udpates and after I redid tests there is no discrepancy between warehouses. Just some minor doc updates are left! |
| ## Upstream Source Package Updates | ||
| - Deleted records have been previously been brought into the `dbt_quickbooks` package. To ensure accuracy in reporting, the following updates were made in the [v0.11.0 release](https://github.com/fivetran/dbt_quickbooks_source/releases/tag/v0.11.0) of `dbt_quickbooks_source`: | ||
| - We introduced the `_fivetran_deleted` field to filter out deleted records from the following staging models: | ||
| - `stg_quickbooks__account` | ||
| - `stg_quickbooks__bundle` | ||
| - `stg_quickbooks__customer` | ||
| - `stg_quickbooks__department` | ||
| - `stg_quickbooks__item` | ||
| - `stg_quickbooks__vendor` |
There was a problem hiding this comment.
It's impacting all models haha, so just tweaked line 13.
PR Overview
This PR will address the following Issue/Feature: #55 in dbt_quickbooks_source
This PR will result in the following new package version: 0.16.0
We are removing deleted records, changing the transformation reporting, which requires a breaking change.
Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:
PR Checklist
Basic Validation
Please acknowledge that you have successfully performed the following commands locally:
Before marking this PR as "ready for review" the following have been applied:
Detailed Validation
Please share any and all of your validation steps:
All integration validation tests passed.
Upstream Source Package Updates
dbt_quickbookspackage. To ensure accuracy in reporting, the following updates were made in the v0.11.0 release ofdbt_quickbooks_source:_fivetran_deletedfield to filter out deleted records from the following staging models:stg_quickbooks__accountstg_quickbooks__bundlestg_quickbooks__customerstg_quickbooks__departmentstg_quickbooks__itemstg_quickbooks__vendordbt_quickbooks.dbt_quickbooksreports, these are breaking changes for this package.Documentation update
_fivetran_deletedfield to the above corresponding seed files in integration tests.If you had to summarize this PR in an emoji, which would it be?
🧢