Skip to content

Commit

Permalink
Update data
Browse files Browse the repository at this point in the history
  • Loading branch information
actions-user committed Dec 1, 2024
1 parent a75de4c commit d4ab164
Show file tree
Hide file tree
Showing 9 changed files with 1,154 additions and 484 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Combining lists of UK Government organisations to make useful CSV files. Automat
Data on UK Government organisations is published in a couple of places under the Open Government Licence:

* [gov.uk/government/organisations](https://www.gov.uk/government/organisations) - this is an HTML page listing <!--GB-GOVUK-->607<!--END GB-GOVUK--> departments, agencies and public bodies. It isn't clear how up-to-date this source is. It has a note if the organisation has a separate website (separate to gov.uk/government/organisations) although these separate website URLs can only be found by following the links and extracting the link from the subsequent webpage. These "real" URLs are saved in [gov.uk_government_organisations-replacement-urls.csv](gov.uk_government_organisations-replacement-urls).
* [gov.uk/api/organisations](https://www.gov.uk/api/organisations) - this is a paged JSON endpoint that provides more data on <!--GB-GOVUKAPI-TOTAL-->1207<!--END GB-GOVUKAPI-TOTAL--> organisations including some which are no longer operating. It provides an ID that seems to match [GB-GOR](http://org-id.guide/list/GB-GOR) as well as useful metadata about update/closed dates.
* [gov.uk/api/organisations](https://www.gov.uk/api/organisations) - this is a paged JSON endpoint that provides more data on <!--GB-GOVUKAPI-TOTAL-->1211<!--END GB-GOVUKAPI-TOTAL--> organisations including some which are no longer operating. It provides an ID that seems to match [GB-GOR](http://org-id.guide/list/GB-GOR) as well as useful metadata about update/closed dates.

It isn't clear if these use the same source or are independently maintained.

Expand All @@ -16,7 +16,7 @@ It isn't clear if these use the same source or are independently maintained.
The [get.pl](get.pl) perl script compiles the data from the two sources (HTML/JSON) into several output files. The [gov.uk_government_organisations-replacement-urls.csv](gov.uk_government_organisations-replacement-urls) file can be manually updated to add replacement URLs if gov.uk/government/organisations doesn't know them. The output files are:

* [gov.uk_api_organisations.json](gov.uk_api_organisations.json) - a JSON blob that contains the results from paging through gov.uk/api/organisations.
* [gov.uk_api_organisations-augmented.csv](gov.uk_api_organisations-augmented.csv) & [gov.uk_api_organisations-augmented.tsv](gov.uk_api_organisations-augmented.tsv) - CSV/TSV versions of the results from gov.uk/api/organisations augmented with the "real" URL from gov.uk/government/organisations. It doesn't include any organisations with `closed_at` set so has <!--GB-GOVUKAPI-CURRENT-->1076<!--END GB-GOVUKAPI-CURRENT--> organisations.
* [gov.uk_api_organisations-augmented.csv](gov.uk_api_organisations-augmented.csv) & [gov.uk_api_organisations-augmented.tsv](gov.uk_api_organisations-augmented.tsv) - CSV/TSV versions of the results from gov.uk/api/organisations augmented with the "real" URL from gov.uk/government/organisations. It doesn't include any organisations with `closed_at` set so has <!--GB-GOVUKAPI-CURRENT-->1079<!--END GB-GOVUKAPI-CURRENT--> organisations.
* [gov.uk_government_organisations.html](gov.uk_government_organisations.html) - the HTML page from gov.uk/government/organisations.
* [gov.uk_government_organisations.csv](gov.uk_government_organisations.csv) - a CSV version of gov.uk/government/organisations with the unique URL slug used as an ID along with the title, gov.uk URL, and the "real" URL (which may be identical).
* [gov.uk_government_organisations-augmented.csv](gov.uk_government_organisations-augmented.csv) & [gov.uk_government_organisations-augmented.tsv](gov.uk_government_organisations-augmented.tsv) - CSV/TSV versions of gov.uk/government/organisations that has had IDs, type, and updated date added. It has <!--GB-GOVUK-->607<!--END GB-GOVUK--> organisations listed.
556 changes: 424 additions & 132 deletions docs/gov.uk_api_organisations.json

Large diffs are not rendered by default.

87 changes: 45 additions & 42 deletions gov.uk_api_organisations-augmented.csv

Large diffs are not rendered by default.

87 changes: 45 additions & 42 deletions gov.uk_api_organisations-augmented.tsv

Large diffs are not rendered by default.

556 changes: 424 additions & 132 deletions gov.uk_api_organisations.json

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions gov.uk_government_organisations-augmented.csv
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ D1382,"Department for Business &amp; Trade",Ministerial department,https://gov.u
D1383,"Department for Culture, Media &amp; Sport",Ministerial department,https://gov.uk/government/organisations/department-for-culture-media-and-sport,https://gov.uk/government/organisations/department-for-culture-media-and-sport,2024-05-01T14:49:09.000+01:00
D6,"Department for Education",Ministerial department,https://gov.uk/government/organisations/department-for-education,https://gov.uk/government/organisations/department-for-education,2024-10-07T09:59:54.000+01:00
D1380,"Department for Energy Security &amp; Net Zero",Ministerial department,https://gov.uk/government/organisations/department-for-energy-security-and-net-zero,https://gov.uk/government/organisations/department-for-energy-security-and-net-zero,2024-08-07T11:45:27.000+01:00
D7,"Department for Environment Food &amp; Rural Affairs",Ministerial department,https://gov.uk/government/organisations/department-for-environment-food-rural-affairs,https://gov.uk/government/organisations/department-for-environment-food-rural-affairs,2023-08-25T13:43:51.000+01:00
D1381,"Department for Science, Innovation &amp; Technology",Ministerial department,https://gov.uk/government/organisations/department-for-science-innovation-and-technology,https://gov.uk/government/organisations/department-for-science-innovation-and-technology,2024-05-01T14:49:49.000+01:00
D7,"Department for Environment, Food &amp; Rural Affairs",Ministerial department,https://gov.uk/government/organisations/department-for-environment-food-rural-affairs,https://gov.uk/government/organisations/department-for-environment-food-rural-affairs,2024-11-15T16:37:52.000+00:00
D1381,"Department for Science, Innovation &amp; Technology",Ministerial department,https://gov.uk/government/organisations/department-for-science-innovation-and-technology,https://gov.uk/government/organisations/department-for-science-innovation-and-technology,2024-11-08T12:19:01.000+00:00
D9,"Department for Transport",Ministerial department,https://gov.uk/government/organisations/department-for-transport,https://gov.uk/government/organisations/department-for-transport,2023-08-08T14:39:04.000+01:00
D10,"Department for Work &amp; Pensions",Ministerial department,https://gov.uk/government/organisations/department-for-work-pensions,https://gov.uk/government/organisations/department-for-work-pensions,2024-10-08T11:38:36.000+01:00
D12,"Department of Health &amp; Social Care",Ministerial department,https://gov.uk/government/organisations/department-of-health-and-social-care,https://gov.uk/government/organisations/department-of-health-and-social-care,2022-08-17T11:57:06.000+01:00
Expand Down
4 changes: 2 additions & 2 deletions gov.uk_government_organisations-augmented.tsv
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ D1382 Department for Business &amp; Trade Ministerial department https://gov.uk/
D1383 Department for Culture, Media &amp; Sport Ministerial department https://gov.uk/government/organisations/department-for-culture-media-and-sport https://gov.uk/government/organisations/department-for-culture-media-and-sport 2024-05-01T14:49:09.000+01:00
D6 Department for Education Ministerial department https://gov.uk/government/organisations/department-for-education https://gov.uk/government/organisations/department-for-education 2024-10-07T09:59:54.000+01:00
D1380 Department for Energy Security &amp; Net Zero Ministerial department https://gov.uk/government/organisations/department-for-energy-security-and-net-zero https://gov.uk/government/organisations/department-for-energy-security-and-net-zero 2024-08-07T11:45:27.000+01:00
D7 Department for Environment Food &amp; Rural Affairs Ministerial department https://gov.uk/government/organisations/department-for-environment-food-rural-affairs https://gov.uk/government/organisations/department-for-environment-food-rural-affairs 2023-08-25T13:43:51.000+01:00
D1381 Department for Science, Innovation &amp; Technology Ministerial department https://gov.uk/government/organisations/department-for-science-innovation-and-technology https://gov.uk/government/organisations/department-for-science-innovation-and-technology 2024-05-01T14:49:49.000+01:00
D7 Department for Environment, Food &amp; Rural Affairs Ministerial department https://gov.uk/government/organisations/department-for-environment-food-rural-affairs https://gov.uk/government/organisations/department-for-environment-food-rural-affairs 2024-11-15T16:37:52.000+00:00
D1381 Department for Science, Innovation &amp; Technology Ministerial department https://gov.uk/government/organisations/department-for-science-innovation-and-technology https://gov.uk/government/organisations/department-for-science-innovation-and-technology 2024-11-08T12:19:01.000+00:00
D9 Department for Transport Ministerial department https://gov.uk/government/organisations/department-for-transport https://gov.uk/government/organisations/department-for-transport 2023-08-08T14:39:04.000+01:00
D10 Department for Work &amp; Pensions Ministerial department https://gov.uk/government/organisations/department-for-work-pensions https://gov.uk/government/organisations/department-for-work-pensions 2024-10-08T11:38:36.000+01:00
D12 Department of Health &amp; Social Care Ministerial department https://gov.uk/government/organisations/department-of-health-and-social-care https://gov.uk/government/organisations/department-of-health-and-social-care 2022-08-17T11:57:06.000+01:00
Expand Down
6 changes: 3 additions & 3 deletions gov.uk_government_organisations.csv
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ department-for-business-and-trade,"Department for Business &amp; Trade",https://
department-for-culture-media-and-sport,"Department for Culture, Media &amp; Sport",https://gov.uk/government/organisations/department-for-culture-media-and-sport,https://gov.uk/government/organisations/department-for-culture-media-and-sport
department-for-education,"Department for Education",https://gov.uk/government/organisations/department-for-education,https://gov.uk/government/organisations/department-for-education
department-for-energy-security-and-net-zero,"Department for Energy Security &amp; Net Zero",https://gov.uk/government/organisations/department-for-energy-security-and-net-zero,https://gov.uk/government/organisations/department-for-energy-security-and-net-zero
department-for-environment-food-rural-affairs,"Department for Environment Food &amp; Rural Affairs",https://gov.uk/government/organisations/department-for-environment-food-rural-affairs,https://gov.uk/government/organisations/department-for-environment-food-rural-affairs
department-for-environment-food-rural-affairs,"Department for Environment, Food &amp; Rural Affairs",https://gov.uk/government/organisations/department-for-environment-food-rural-affairs,https://gov.uk/government/organisations/department-for-environment-food-rural-affairs
department-for-science-innovation-and-technology,"Department for Science, Innovation &amp; Technology",https://gov.uk/government/organisations/department-for-science-innovation-and-technology,https://gov.uk/government/organisations/department-for-science-innovation-and-technology
department-for-transport,"Department for Transport",https://gov.uk/government/organisations/department-for-transport,https://gov.uk/government/organisations/department-for-transport
department-for-work-pensions,"Department for Work &amp; Pensions",https://gov.uk/government/organisations/department-for-work-pensions,https://gov.uk/government/organisations/department-for-work-pensions
Expand Down Expand Up @@ -109,7 +109,6 @@ civil-justice-council,"",,
civil-nuclear-police-authority,"",,
civil-procedure-rules-committee,"",,
civil-service-commission,"",,
the-coal-authority,"",,
college-of-policing,"",,
commission-for-countering-extremism,"",,
commission-on-human-medicines,"",,
Expand Down Expand Up @@ -172,7 +171,6 @@ evaluation-task-force,"",,
the-executive-office-northern-ireland,"",,
exmoor-national-park-authority,"",,
export-guarantees-advisory-council,"",,
fcdo-services,"",,
family-justice-council,"",,
family-procedure-rule-committee,"",,
financial-conduct-authority,"",,
Expand Down Expand Up @@ -279,6 +277,7 @@ medicines-and-healthcare-products-regulatory-agency,"",,
met-office,"",,
migration-advisory-committee,"",,
military-engineering-experimental-establishment,"",,
mining-remediation-authority,"",,
single-financial-guidance-body,"",,
geffrye-museum,"",,
nhs-blood-and-transplant,"",,
Expand Down Expand Up @@ -558,6 +557,7 @@ office-of-the-parliamentary-counsel,"",,
offshore-petroleum-regulator-for-environment-and-decommissioning,"",,
open-public-services,"",,
civil-service-operational-delivery-profession,"",,
packuk,"",,
civil-service-policy-profession,"",,
preventing-sexual-violence-in-conflict-initiative,"",,
probation-service,"",,
Expand Down
Loading

0 comments on commit d4ab164

Please sign in to comment.