asf_welsh_energy_consultation

This repo contains code for producing charts for ASF's April 2023 response to the Welsh Government's consultation on Wales' renewable energy targets.

The remainder of the charts in the response can be produced from code in the repo asf_senedd_response, as these are based on charts originally produced for a previous call for evidence.

Setup

• Meet the data science cookiecutter requirements, in brief:
  • Install: direnv and conda
• Clone the repo: git clone [email protected]:nestauk/asf_welsh_energy_consultation.git
• Navigate to your local repo folder
• Checkout the correct branch if not working on dev
• Run direnv allow
• Run make install to configure the development environment. This will:
  • Setup the conda environment
  • Configure pre-commit
  • Install packages listed in requirements.txt
• Activate conda environment: conda activate asf_welsh_energy_consultation
• Run make inputs-pull to pull the zipped supplementary data from S3 and put it in /inputs/data. There will be one folder per historical analysis containing the supplementary data files as listed in the Historical analysis section below.
• Run python asf_welsh_energy_consultation/analysis/produce_plots_and_stats.py --local_data_dir <YOUR_LOCAL_DIR>. You need to specify the path to the local directory where your local copy of the EPC data is/will be saved by replacing <YOUR_LOCAL_DIR> with the path to your "ASF_data" directory or equivalent. If you don't have a local directory for ASF core data, you can create a folder called "ASF_data" in your home directory.
  • You can specify which batch of EPC data to download and MCS data to load from S3 by passing the --epc_batch and --mcs_batch arguments, both default to downloading/loading the newest data from S3, respectively. Check the output info logs or set the batches manually to ensure expected batches used.
  • You can specify which supplementary data folder to use by passing the --supp_data argument. It defaults to using the latest supplementary data folder.
  • To recreate the full October 2023 analysis, set the --calculate_average_installations argument to True. This will calculate some additional numbers on MCS installations per year included in the October 2023 response. For other historical analyses, this argument is not required and defaults to False.
  • Run python asf_welsh_energy_consultation/analysis/produce_plots_and_stats.py -h for more info.

The script should generate the following seven plots which will be saved in your local repo in outputs/figures:

• cumulative_retrofits.html
• electric_tenure.html
• installations_by_gas_status.html
• installations_by_rurality.html
• new_build_hp_cumulative.html
• new_build_hp_proportion.html
• total_cumulative_installations.html

It should generate a further 10 plots, five in English and five in Welsh, saved in outputs/figures/english and outputs/figures/welsh, respectively:

• age_prop[_welsh].png
• epc_all[_welsh].html
• epc_hp_private_retrofit[_welsh].html
• epc_hp_private[_welsh].html
• hp_tenure[_welsh].html

It will also generate a stats.txt text file containing some summary statistics.

Skeleton folder structure

asf_welsh_energy_consultation/
├─ analysis/
│  ├─ produce_plots_and_stats.py - produces plots
│  ├─ unused_plots.py - unused plotting functions from August '22
├─ config
│  ├─ base.yaml - global variables
│  ├─ translation_config.py - English to Welsh translations for producing figures translated into Welsh
├─ getters/
│  ├─ get_data.py - getters for raw data
├─ pipeline/
│  ├─ plotting.py - functions for plotting
│  ├─ process_data.py - functions to process and enhance raw data
│  ├─ unused_processing.py - unused processing functions from August '22
inputs/
├─ data/
│  ├─ data_[YYYYMM]/ - data files, a mixture of csv, xlsx and ods
│  │  ├─ postcodes/ - individual subfolders for each postcode region
outputs/
├─ figures/ - where charts are saved

Historical analysis

Versions of data used for previous analysis are listed below.

Analysis*	EPC1	MCS processing date2	Postcodes3	Postcode to OA4	Off gas postcodes5	Rural-urban classification6	Dwellings7	Tenure8
November 2024	2024 Q1 complete (preprocessed, and preprocessed and deduplicated)	241113	February 2024	February 2024	September 2024	2011	2021 census	2021 census
October 2023	2023 Q2 complete (preprocessed, and preprocessed and deduplicated)	231009	August 2023	May 2022	2022	2011	2021 census	2021 census
April 2023	2022 Q4 complete (preprocessed)	230315	ONS postcode directory. Date unknown.	ONS data. Date unknown.	2022	2011	Not used	ONS data. Date unknown.

*This column refers to the date the analysis was run.

Data attributions

1. Energy Performance Certificate (EPC) Register data. England and Wales data available here and Scotland data available here. Data has undergone cleaning and processing before analysis - the processing code is available in this Github repository
2. Microgeneration Certification Scheme (MCS) Installations Database (MID). The numbers in this column represent the internal processing date. The analysis requires mcs_installations_YYMMDD.csv; mcs_installations_most_relevant_YYMMDD.csv; and mcs_installations_epc_full_YYMMDD.csv.
3. ONS Postcode Directory for the UK. Contains OS data © Crown copyright and database right 2024. Contains Royal Mail data © Royal Mail copyright and database right 2024. Source: Office for National Statistics licensed under the Open Government Licence v.3.0
4. ONS Postcode to OA to LSOA to MSOA to LAD Best Fit Lookup in the UK. Contains OS data © Crown copyright and database right 2024. Contains Royal Mail data © Royal Mail copyright and database right 2024. Source: Office for National Statistics licensed under the Open Government Licence v.3.0
5. Off-gas Postcode Register from Xoserve.
6. Rural Urban Classification Lookup table. Contains public sector information licensed under the Open Government Licence v3.0.
7. Number of dwellings by housing characteristics in England and Wales. Contains public sector information licensed under the Open Government Licence v3.0.
8. Accommodation type by type of central heating in household by tenure. Contains public sector information licensed under the Open Government Licence v3.0.

Contributor guidelines

Technical and working style guidelines

---

Project based on Nesta's data science project template (Read the docs here).