stat neighbours added to export

README.md

@@ -5,7 +5,7 @@ Published: https://data-to-insight.github.io/ofsted-ilacs-scrape-tool/
 ### We're aware that the daily automated update of the ILACS overview page is not running. For now we're running it manually on a weekly basis.  
 
 ## Brief overview
-This project is based on a proof-of-concept, 'can we do this' basis. As such it's supplied very much with the disclaimer of please check the vitals if you're embedding it into something more critical, and likewise pls feel free to feedback into the project with suggestions. The structure of the code and processes have much scope for improvement, but some of the emphasis was on maintaining a level of readability so that others might have an easier time of taking it further. That said, we needed to take some of the scrape/cleaning processes further than anticipated due to inconsistencies in the source site/data and this has ultimately impacted the  intended mvp approach to codifying a solution for the original problem. 
+This project is based on a proof-of-concept, 'can we do this' basis. As such it's supplied very much with the disclaimer of please check the vitals if you're embedding it into something more critical, and likewise pls feel free to feedback into the project with suggestions. The structure of the code and processes have much scope for improvement, but some of the initial emphasis was on maintaining a level of readability so that others might have an easier time of taking it further. That said, we needed to take some of the scrape/cleaning processes further than anticipated due to inconsistencies in the source site/data and this has ultimately impacted the intended 're-usable mvp' approach to codifying a solution for the original problem. 
 
 The results structure and returned data is based almost entirely on the originating ILACS Summary produced/refreshed periodically by the ADCS; the use of which has previously underpinned several D2I projects. We're aware of several similar collections of longer-term work on and surrounding the Ofsted results theme, and would be happy to hear from those who perhaps also have bespoke ideas for changes here that would assist their own work. 
 

index.html

@@ -26,7 +26,7 @@
         <h1>Ofsted ILACS Summary</h1>
         <p>Summarised outcomes of published short and standard ILACS inspection reports by Ofsted, refreshed daily.<br/>An expanded version of the shown summary sheet, refreshed concurrently, is available to <a href='ofsted_childrens_services_overview.xlsx'>download here</a> as an .xlsx file. <br/>Data summary is based on the original <i>ILACS Outcomes Summary</i> published periodically by the ADCS:<a href='https://adcs.org.uk/inspection/article/ilacs-outcomes-summary'>https://adcs.org.uk/inspection/article/ilacs-outcomes-summary</a>.</p>
         <p>Disclaimer: This summary is built from scraped data direct from https://reports.ofsted.gov.uk/ and the published PDF inspection report files. As a result of the nuances|variance within the inspection report content or pdf encoding, we're noting some problematic data extraction for a small number of LA's*.<br/> <a href="mailto:[email protected]?subject=Ofsted-Scrape-Tool">Feedback</a> on specific problems|inaccuracies|suggestions welcomed.<br/>**LA reports with issues: southend-on-sea, [overall, help_and_protection_grade,care_leavers_grade], nottingham,[inspection_framework, inspection_date], redcar and cleveland,[inspection_framework, inspection_date], knowsley,[inspector_name], stoke-on-trent,[inspector_name]</p>
-        <p><b>Last updated: 08 04 2024 12:05</b></p>
+        <p><b>Last updated: 18 04 2024 08:15</b></p>
 
         <div class="container">
     <table border="1" class="dataframe">
@@ -1671,15 +1671,15 @@ <h1>Ofsted ILACS Summary</h1>
       <td>SE</td>
       <td>E10000025</td>
       <td>Oxfordshire</td>
-      <td><a href="https://files.ofsted.gov.uk/v1/file/50004449">ofsted.gov.uk/50004449</a></td>
+      <td><a href="https://files.ofsted.gov.uk/v1/file/50243682">ofsted.gov.uk/50243682</a></td>
+      <td>good</td>
+      <td>standard</td>
+      <td>Nicki Shaw</td>
+      <td>12/02/2024</td>
+      <td>good</td>
+      <td>good</td>
       <td>good</td>
-      <td>short</td>
-      <td>Donna Marriott</td>
-      <td>16/04/2018</td>
       <td>good</td>
-      <td>requires improvement</td>
-      <td>requires improvement</td>
-      <td>inspection_pre_dates_judgement</td>
     </tr>
     <tr>
       <td>80537</td>

ofsted_childrens_services_inspection_scrape.py

@@ -1540,7 +1540,7 @@ def extract_words_weights(topic_string):
 local_authorities_lookup_df['urn'] = pd.to_numeric(local_authorities_lookup_df['urn'], errors='coerce')
 
 # Define what data is required to be merged in
-additional_data_cols = ['la_code', 'region_code', 'ltla23cd']
+additional_data_cols = ['la_code', 'region_code', 'ltla23cd', 'stat_neighbours']
 ilacs_inspection_summary_df = merge_and_select_columns(ilacs_inspection_summary_df, local_authorities_lookup_df, key_col, additional_data_cols)
 
 # re-organise column structure now with new col(s)