Skip to content

Commit

Permalink
[pre-commit.ci] auto fixes from pre-commit.com hooks
Browse files Browse the repository at this point in the history
for more information, see https://pre-commit.ci
  • Loading branch information
pre-commit-ci[bot] committed Jul 2, 2024
1 parent 1411002 commit 376e216
Show file tree
Hide file tree
Showing 18 changed files with 406 additions and 281 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/gh-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,4 @@ jobs:
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
uses: actions/deploy-pages@v4
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,4 @@ repos:
types_or: [python, pyi, jupyter]
args: [--fix]
- id: ruff-format
types_or: [python, pyi, jupyter]
types_or: [python, pyi, jupyter]
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -382,4 +382,4 @@ Exhibit B - "Incompatible With Secondary Licenses" Notice
---------------------------------------------------------

This Source Code Form is "Incompatible With Secondary Licenses", as
defined by the Mozilla Public License, v. 2.0.
defined by the Mozilla Public License, v. 2.0.
34 changes: 16 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,40 @@
## Indicative data use cases for Resilient Kerala

In recent years, the Government of Kerala has invested significantly in data , recognizing it as a critical resource that will advance policy decisions and effect efficient resource-sharing and building amongst the government, businesses and communities.
In recent years, the Government of Kerala has invested significantly in data , recognizing it as a critical resource that will advance policy decisions and effect efficient resource-sharing and building amongst the government, businesses and communities.

Currently, there are several platforms that host these open data:
- [Open Government Data Portal of Kerala](https://kerala.data.gov.in/) : intended for use by Departments and Organizations of Government of Kerala to publish datasets, documents, services, tools and applications collected by them for public use.
- [Open Government Data Portal of Kerala](https://kerala.data.gov.in/) : intended for use by Departments and Organizations of Government of Kerala to publish datasets, documents, services, tools and applications collected by them for public use.
- [Map Kerala](https://map.opendatakerala.org/) : an Open Data Kerala initiative that hosts downloadable geospatial information for Kerala districts.
- [Kerala State Spatial Data Infrastructure (KSDI)](https://opensdi.kerala.gov.in/): an Internet based Geo-spatial Data Directory for the state that facilitates users of the system to share and explore data related to political and administrative boundaries, natural resources, transportation and infrastructure, demography, agro and socio economy etc., of the state.

The team built several demos to illustrate use cases with open data that industry practitioners and governments can use as part of evidence based policymaking.
The team built several demos to illustrate use cases with open data that industry practitioners and governments can use as part of evidence based policymaking.

## Analysing Heatwaves and Air Quality in Kerala
Using Open Data retrieved from public platforms, the team studied the impact of heatwaves and air quality in Kerala, specifically in Kochi (also known as Cochin) and Trivandrum (also known as Thiruvananthapuram).
Using Open Data retrieved from public platforms, the team studied the impact of heatwaves and air quality in Kerala, specifically in Kochi (also known as Cochin) and Trivandrum (also known as Thiruvananthapuram).

Air pollution is one of the greatest environmental risk to health, contributing to respiratory and cardiovascular diseases, cancer, and premature death. Heatwaves, on the other hand, impact human health through heat-related illnesses such as heatstroke, dehydration and heat exhaustion.
Air pollution is one of the greatest environmental risk to health, contributing to respiratory and cardiovascular diseases, cancer, and premature death. Heatwaves, on the other hand, impact human health through heat-related illnesses such as heatstroke, dehydration and heat exhaustion.
In addition to health impacts, air pollution and heatwaves can lead to droughts, wildfires, displacement and migration, having environmental, economic and social impacts as well.
Tracking heatwaves and air pollution levels can help governments and policy makers make informed policies and decisions to improve public and environmental health.

## Who is this for
Data Scientists, Public Health Officials, Emergency Responders, Policy Makers, Urban Planners and Local Government Officials, Environment Scientists and Researchers, General Public.

## How will this work
The analysis will be presented in the form a Data Good that can replicated and reused in other geographic contexts as well. More about data goods [here](https://datapartnership.org/heatwaves-in-india/docs/introduction_to_data_goods.html).
The analysis will be presented in the form a Data Good that can replicated and reused in other geographic contexts as well. More about data goods [here](https://datapartnership.org/heatwaves-in-india/docs/introduction_to_data_goods.html).

## Future Work
## Future Work

This work is a demonstration of what we could do with currently available open access data. In the forthcoming versions, it can be expanded in multiple ways.
This work is a demonstration of what we could do with currently available open access data. In the forthcoming versions, it can be expanded in multiple ways.

### I. Understanding Impact of Heat in Cities of Kerala

Urban Heat Island (UHI) Effect has known to be the reason for the increased vulnerability of cities to heat risk {cite}`doi:10.1596/40771`. Studying the effect of UHI is critical in mitigation and adaptation to heat risk especially in India, which is considered to be highly vulnerable to it. To this end, the team proposes supplimenting the sample Land Surface Temperature Analysis with a few components -
Urban Heat Island (UHI) Effect has known to be the reason for the increased vulnerability of cities to heat risk {cite}`doi:10.1596/40771`. Studying the effect of UHI is critical in mitigation and adaptation to heat risk especially in India, which is considered to be highly vulnerable to it. To this end, the team proposes supplimenting the sample Land Surface Temperature Analysis with a few components -

Component 1: Citizen Science Heat Data Collection Campaigns: Measuring accurate indoor and outdoor temperature can help understand the difference in temperature across schools and hospitals allowing policymakers to take necessary interventions to mitigate impacts. This will also allow for an accurate climate map of the entire city which can become the basis for modelling scenarios under varying climate pathways.
Component 1: Citizen Science Heat Data Collection Campaigns: Measuring accurate indoor and outdoor temperature can help understand the difference in temperature across schools and hospitals allowing policymakers to take necessary interventions to mitigate impacts. This will also allow for an accurate climate map of the entire city which can become the basis for modelling scenarios under varying climate pathways.

Component 2: Overlaying heat risk with existing sociodemographic information to determine the people and infrastructure most impacted by rising temperatures. This will allow for adaptation interventions to be put in place for neighbrohoods that are worse impacted.
Component 2: Overlaying heat risk with existing sociodemographic information to determine the people and infrastructure most impacted by rising temperatures. This will allow for adaptation interventions to be put in place for neighbrohoods that are worse impacted.

Component 3: Supplimenting data from satellite imagery and remote sensing: The existing analysis from LANDSAt will be supplimented with additional data from Copernicus ERA5, and any other temperature data available through the Indian Space Research Organization. Although satellite imagery is not the most accurate representation of reality, it is a dataset that is easily available, highly granular and can be updated at a high frequency. This makes the data useful once initial calibration is conducted using data obtained through citizen science campaigns.
Component 3: Supplimenting data from satellite imagery and remote sensing: The existing analysis from LANDSAt will be supplimented with additional data from Copernicus ERA5, and any other temperature data available through the Indian Space Research Organization. Although satellite imagery is not the most accurate representation of reality, it is a dataset that is easily available, highly granular and can be updated at a high frequency. This makes the data useful once initial calibration is conducted using data obtained through citizen science campaigns.


### II. Understanding Impact of Air Pollution in Kerala
Expand All @@ -47,12 +47,12 @@ Ambient air pollution poses considerable health risk. Additionally, with rising

### IV. Analysing Urban Space Usage

Understanding how populations use urban space is critical for assessing economic activity, societal resilience, and infrastructure accessibility. However, measuring urban space usage at the frequency and granularity required to support these assessments can be challenging, especially when relying only on traditional survey methods. To overcome this challenge, the team proposes using mobility data -- i.e., telemetry data derived from mobile devices – to generate high-frequency indices of urban space usage (e.g., retail centers, construction sites, manufacturing zones, financial centers, residential areas, etc.). This data can be used to deisgn two components -
Understanding how populations use urban space is critical for assessing economic activity, societal resilience, and infrastructure accessibility. However, measuring urban space usage at the frequency and granularity required to support these assessments can be challenging, especially when relying only on traditional survey methods. To overcome this challenge, the team proposes using mobility data -- i.e., telemetry data derived from mobile devices – to generate high-frequency indices of urban space usage (e.g., retail centers, construction sites, manufacturing zones, financial centers, residential areas, etc.). This data can be used to deisgn two components -

Component 1: Methodological Research to Measure Urban Space Usage: The team will develop a methodology for processing and analyzing mobility data to generate urban space activity level indicators, or “index”. The output will be a detailed methodological guide that can be replicated in varied contexts and a suite of interactive visualizations that communicate mobility patterns and their correlations with economic indicators. The index will support informed urban planning and design decisions, particularly in relation to climate resilience.

Component 1: Methodological Research to Measure Urban Space Usage: The team will develop a methodology for processing and analyzing mobility data to generate urban space activity level indicators, or “index”. The output will be a detailed methodological guide that can be replicated in varied contexts and a suite of interactive visualizations that communicate mobility patterns and their correlations with economic indicators. The index will support informed urban planning and design decisions, particularly in relation to climate resilience.

Component 2: Urban Climate Shock Response Analysis: The team will employ the methodologies and tools developed in the first part, using the Kerala as a case study, to answer research questions around climate-driven shifts in urban space usage, regional comparison of these shifts, correlation between urban space usage and climate shocks, and predictive analysis of future impacts. The research, disseminated in the form of a policy research working paper, will support policy and planning decisions related to urban climate resilience in Kerala. The climate shocks can include floods and heatwaves.

Component 2: Urban Climate Shock Response Analysis: The team will employ the methodologies and tools developed in the first part, using the Kerala as a case study, to answer research questions around climate-driven shifts in urban space usage, regional comparison of these shifts, correlation between urban space usage and climate shocks, and predictive analysis of future impacts. The research, disseminated in the form of a policy research working paper, will support policy and planning decisions related to urban climate resilience in Kerala. The climate shocks can include floods and heatwaves.

A sample of a similar analysis can be found in the team's work for [Turkiye Earthquake response](https://datapartnership.org/turkiye-earthquake-impact/notebooks/mobility/activity.html#id1).

Expand Down Expand Up @@ -88,5 +88,3 @@ The team proposes the use of different definitions of heatwaves applicable to cr
One important impact of heatwaves is the triggering of cascading natural disasters – floods and droughts. To account for this, two indicators are identified. These two indicators allow for the spatial mapping of areas that are flood-prone and drought-prone due to heatwaves, thus making them more vulnerable.
The following indicators are proposed to assess the impact of heatwaves across sectors - The following indicators will be calculated at the granularity of the lowest administrative boundary data available, preferably a city. The impact will be measured between the years 2015 and 2022. -->


2 changes: 1 addition & 1 deletion data/delhi-shapefiles/Delhi-polygon.cpg
Original file line number Diff line number Diff line change
@@ -1 +1 @@
UTF-8
UTF-8
2 changes: 1 addition & 1 deletion data/delhi-shapefiles/Delhi-polygon.prj
Original file line number Diff line number Diff line change
@@ -1 +1 @@
GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]]
GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]]
1 change: 0 additions & 1 deletion docs/_toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,3 @@ parts:
- caption: Acknowledgements
chapters:
- file: docs/team

2 changes: 1 addition & 1 deletion docs/bibliography.bib
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ @article{rahman2022effects
pages={1117--1127},
year={2022},
publisher={American Thoracic Society}
}
}
2 changes: 1 addition & 1 deletion docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
jupyter-book>=0.12.0
gunicorn
nbconvert
nbconvert
1 change: 0 additions & 1 deletion docs/team.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,3 @@ The Data Lab would like to express our sincere gratutude and appreciation for th
| [Yim Chew Kloe](mailto:[email protected]) | Data Scientist | DECCS, World Bank, Washington DC |
| [Sahiti Sarva](mailto:[email protected]) | Data Scientist | Data Lab, DECDG, World Bank, Washington DC |
| Shashank Jayakumar | Climate Change Consultant | URS, World Bank, India |

20 changes: 7 additions & 13 deletions notebooks/00-landsat.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -40,16 +40,9 @@
},
"outputs": [],
"source": [
"import asyncio\n",
"import json\n",
"import os\n",
"from typing import Optional\n",
"\n",
"import geopandas\n",
"import h3\n",
"import pandas as pd\n",
"import shapely\n",
"from aiohttp import ClientSession\n",
"from shapely.geometry import Polygon"
]
},
Expand All @@ -63,8 +56,8 @@
},
"outputs": [],
"source": [
"kochi = geopandas.read_file('../data/shapefiles/kochi.shp')\n",
"trivandrum = geopandas.read_file('../data/shapefiles/trivandrum.shp')"
"kochi = geopandas.read_file(\"../data/shapefiles/kochi.shp\")\n",
"trivandrum = geopandas.read_file(\"../data/shapefiles/trivandrum.shp\")"
]
},
{
Expand All @@ -78,6 +71,8 @@
"outputs": [],
"source": [
"import geopandas\n",
"\n",
"\n",
"def get_h3_tessellation(\n",
" gdf: geopandas.GeoDataFrame, name=\"shapeName\", resolution=10\n",
") -> geopandas.GeoDataFrame:\n",
Expand Down Expand Up @@ -132,9 +127,8 @@
},
"outputs": [],
"source": [
"import shapely\n",
"kochi_h3 = get_h3_tessellation(kochi, name=\"NAME\", resolution=9)\n",
"kochi_h3.to_file('../data/shapefiles/kochi_h3.shp', format='ESRI Shapefile')"
"kochi_h3.to_file(\"../data/shapefiles/kochi_h3.shp\", format=\"ESRI Shapefile\")"
]
},
{
Expand All @@ -147,8 +141,8 @@
},
"outputs": [],
"source": [
"trivandrum_h3 = get_h3_tessellation(trivandrum, name = 'NAME', resolution =9)\n",
"trivandrum_h3.to_file('../data/shapefiles/trivandrum_h3.shp', format='ESRI Shapefile')"
"trivandrum_h3 = get_h3_tessellation(trivandrum, name=\"NAME\", resolution=9)\n",
"trivandrum_h3.to_file(\"../data/shapefiles/trivandrum_h3.shp\", format=\"ESRI Shapefile\")"
]
},
{
Expand Down
40 changes: 32 additions & 8 deletions notebooks/00-problem-understanding.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
}
],
"source": [
"df = pd.read_excel('../data/emdat_public_2022_07_14_query_uid-QgPAAC.xlsx', skiprows = 6)"
"df = pd.read_excel(\"../data/emdat_public_2022_07_14_query_uid-QgPAAC.xlsx\", skiprows=6)"
]
},
{
Expand Down Expand Up @@ -219,7 +219,20 @@
}
],
"source": [
"heatwaves_year = df[df['Disaster Subtype']=='Heat wave'][['Year', 'Country', 'Total Deaths', 'Total Affected',\"Total Damages, Adjusted ('000 US$)\"]].groupby('Year').sum().reset_index()\n",
"heatwaves_year = (\n",
" df[df[\"Disaster Subtype\"] == \"Heat wave\"][\n",
" [\n",
" \"Year\",\n",
" \"Country\",\n",
" \"Total Deaths\",\n",
" \"Total Affected\",\n",
" \"Total Damages, Adjusted ('000 US$)\",\n",
" ]\n",
" ]\n",
" .groupby(\"Year\")\n",
" .sum()\n",
" .reset_index()\n",
")\n",
"heatwaves_year"
]
},
Expand Down Expand Up @@ -343,9 +356,17 @@
}
],
"source": [
"heatwaves = df[df['Disaster Subtype']=='Heat wave'][['Year', 'Country', 'Total Deaths', 'Total Affected',\"Total Damages, Adjusted ('000 US$)\"]]\n",
"heatwaves_country = heatwaves.groupby(['Year', 'Country']).sum().reset_index()\n",
"heatwaves_country[heatwaves_country['Country']=='India']"
"heatwaves = df[df[\"Disaster Subtype\"] == \"Heat wave\"][\n",
" [\n",
" \"Year\",\n",
" \"Country\",\n",
" \"Total Deaths\",\n",
" \"Total Affected\",\n",
" \"Total Damages, Adjusted ('000 US$)\",\n",
" ]\n",
"]\n",
"heatwaves_country = heatwaves.groupby([\"Year\", \"Country\"]).sum().reset_index()\n",
"heatwaves_country[heatwaves_country[\"Country\"] == \"India\"]"
]
},
{
Expand Down Expand Up @@ -373,7 +394,7 @@
}
],
"source": [
"heatwaves_country['Country'].unique()"
"heatwaves_country[\"Country\"].unique()"
]
},
{
Expand Down Expand Up @@ -456,7 +477,7 @@
}
],
"source": [
"heatwaves_country[heatwaves_country['Country']=='United States of America (the)']"
"heatwaves_country[heatwaves_country[\"Country\"] == \"United States of America (the)\"]"
]
},
{
Expand Down Expand Up @@ -549,7 +570,10 @@
}
],
"source": [
"heatwaves_country[heatwaves_country['Country']=='United Kingdom of Great Britain and Northern Ireland (the)']"
"heatwaves_country[\n",
" heatwaves_country[\"Country\"]\n",
" == \"United Kingdom of Great Britain and Northern Ireland (the)\"\n",
"]"
]
},
{
Expand Down
16 changes: 10 additions & 6 deletions notebooks/01-weather-tomorrow.io.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -584,9 +584,12 @@
"\n",
" async def post(self, url, json, params={}, headers={}):\n",
" params[\"apikey\"] = self.token\n",
" async with self.semaphore, self.session.post(\n",
" url, json=json, params=params, headers=headers\n",
" ) as response:\n",
" async with (\n",
" self.semaphore,\n",
" self.session.post(\n",
" url, json=json, params=params, headers=headers\n",
" ) as response,\n",
" ):\n",
" return await response.json()"
]
},
Expand Down Expand Up @@ -725,8 +728,7 @@
"outputs": [],
"source": [
"async with TomorrowAPIClient(token=TOMORROW_API_KEY) as client:\n",
"\n",
" url = f\"https://api.tomorrow.io/v4/historical\"\n",
" url = \"https://api.tomorrow.io/v4/historical\"\n",
" headers = {\"Accept\": \"application/json\", \"Content-Type\": \"application/json\"}\n",
"\n",
" futures = [client.post(url, json=payload, headers=headers) for payload in payloads]\n",
Expand Down Expand Up @@ -3836,7 +3838,9 @@
}
],
"source": [
"dataframes = [pd.json_normalize(item[\"data\"][\"timelines\"][0][\"intervals\"]) for item in data]"
"dataframes = [\n",
" pd.json_normalize(item[\"data\"][\"timelines\"][0][\"intervals\"]) for item in data\n",
"]"
]
},
{
Expand Down
Loading

0 comments on commit 376e216

Please sign in to comment.