Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use total population in CDBG #1359

Merged
merged 4 commits into from
Jan 3, 2025
Merged

use total population in CDBG #1359

merged 4 commits into from
Jan 3, 2025

Conversation

damonmcc
Copy link
Member

@damonmcc damonmcc commented Jan 2, 2025

related to #1348

according to HUD's data dictionary, the column LOWMOD_PCT isn't what we expect because the denominator LOWMODUNIV is "Persons with the potential for being deemed Low-, Moderate- and Middle-income" rather than the total population

so we have to get total population from a different dataset

changes

  1. add dcp_censusdata_blocks to get the total population of each census block
  2. change where census blocks are grouped to census block groups.
    • This helped make the new data usable because it doesn't have a column with GEOID values to parse and determine the census block group. So joining all census block data before grouping seem best.
  3. use new total population data

@damonmcc damonmcc force-pushed the dm-cdbg-total-pop branch 3 times, most recently from 23ea7b4 to fc4429c Compare January 2, 2025 20:13
@damonmcc damonmcc marked this pull request as ready for review January 2, 2025 21:17
@damonmcc damonmcc requested a review from fvankrieken January 2, 2025 21:40
Copy link
Contributor

@fvankrieken fvankrieken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this all seems fine for the incremental change needed. I do think we should sit down with some quick white boarding to simplify the repo

@fvankrieken
Copy link
Contributor

Dbt tests currently failing though

@damonmcc
Copy link
Member Author

damonmcc commented Jan 3, 2025

Dbt tests currently failing though

a column in dcp_censusdata_blocks is named pop1.1 and sqluff seems to think it "refers to table/view not found in the FROM clause or found in ancestor statement"

gonna try different single/double quote combos (locally first)

edit: might have to ignore it for now, couldn't fix it with quotes

a column name with a period in it seems unfixable with single/double quotes
@damonmcc damonmcc merged commit 509daca into main Jan 3, 2025
17 checks passed
@fvankrieken
Copy link
Contributor

Dbt tests currently failing though

a column in dcp_censusdata_blocks is named pop1.1 and sqluff seems to think it "refers to table/view not found in the FROM clause or found in ancestor statement"

gonna try different single/double quote combos (locally first)

edit: might have to ignore it for now, couldn't fix it with quotes

Yeah that seems like a sqlfluff bug

@damonmcc damonmcc deleted the dm-cdbg-total-pop branch January 3, 2025 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants