Merge pull request #181 from data-to-insight/feat/extract_SQL

ssd refresh from dtoi-1555

CHANGELOG.md

@@ -1,6 +1,7 @@
 # Change log
 SSD Data Item Changes:
-The complete change history for SSD data items in reverse chronological order, with pending|expected changes showing first.Agreed data item-level changes are assigned an identifier. A sub-set of the change details for the most recent change (if any) also appears within each objects metadata block(YAML).The current change log contains only sample data until we deploy the first pilot release.Note: Object-level change tracking is not yet available/in progress; feedback/suggestions welcomed.Last updated: 02/01/2024 13:32
+The complete change history for SSD data items in reverse chronological order, with pending|expected changes showing first.Agreed data item-level changes are assigned an identifier. A sub-set of the change details for the most recent change (if any) also appears within each objects metadata block(YAML).The current change log contains only sample data until we deploy the first pilot release.Note: Object-level change tracking is not yet available/in progress; feedback/suggestions welcomed.Last updated: 02/01/2024 13:38
+
 
 | item_ref   | old_item_ref   | release_datetime   | change_id   | change_object_name    | change_impact_title     | change_status   | change_type   | change_source   | change_impact_notes                                    |
 |:-----------|:---------------|:-------------------|:------------|:----------------------|:------------------------|:----------------|:--------------|:----------------|:-------------------------------------------------------|

docs/README.html

@@ -103,7 +103,7 @@ <h3>Data objects</h3>
     cms_table:
     - liquid_logic:&lt;ll_tbl_name&gt;
     - mosaic: &lt;mosaic_tbl_name&gt;
-    guidance: &lt;DfE or similar guidance txt&gt;
+    guidance: &lt;DfE of similar guidance txt&gt;
     metadata:
       release_datetime: 
       change_id: 
@@ -140,7 +140,7 @@ <h3><a href="#current-phase">Current phase</a></h3>
 <tr><td>2</td><td>Continuous Deployment</td><td>Maintenance roadmap and framework agreement towards iterative improvement-driven approach.</td></tr>
 </tbody></table>
 <h3>Local Authorities guiding development</h3>
-<p><strong>Hertfordshire CC</strong> : Bid Lead | <strong>Knowsley CC</strong> : Steering Group | <strong>Data2Insight</strong> : Project Lead | <strong>East Sussex CC</strong> : Host Authority | Essex CC : Project Management | <strong>ADCS North West</strong> (hosted by <strong>Stockport Council</strong>)</p>
+<p><strong>Hertfordshire CC</strong> : Bid Lead | <strong>Knowsley CC</strong> : Steering Group | <strong>Data2Insight</strong> : Project Lead | <strong>East Sussex CC</strong> : Host Authority | Essex CC : Project Management and Mosaic Pilot Development | <strong>ADCS North West</strong> (hosted by <strong>Stockport Council</strong>) | <strong>Blackpool CC</strong> : Mosaic Pilot Development | <strong>East Riding CC</strong> : Azeus Pilot Development  </p>
 <p>Repo forks, and direct involvement with the project are welcomed and you can find more information about Data2Insight on our website <a href="https://www.datatoinsight.org/">https://www.datatoinsight.org/</a></p>
 <h2>Repo workflow</h2>
 <p>In brief, folders <a href="./data/">data</a>, <a href="./docs/">docs</a> &amp; <a href="./sql/">sql</a> are output folders. Python scripts within the tools folder generate those files with the exception of <a href="./docs/admin">docs/admin</a> which contains the import csv definitions of all data objects &amp; relationships. <a href="./tools/">tools/*.py</a> are numbered to dictate their required run order when updating the specification. This workflow enables the entire project and all outputs to be instantly updated enabling full development transparency and ease of later updates both minor modifications and new modules/objects. It is anticipated that most LA's will only need to access the generated extract <a href="./sql/">SQL files</a> for their particular CMS. </p>

docs/guidance.html

@@ -1,6 +1,6 @@
 <html><head><style>body { margin: 20px; }table { border-collapse: collapse; width: 100%; margin: auto; table-layout: fixed; }th, td { text-align: left; padding: 8px; word-wrap: break-word; border: 1px solid #ddd; }th { background-color: #f2f2f2; }th.image-column { width: 300px; }.last-updated-container { display: flex; align-items: center; }.last-updated-text { font-weight: bold; margin-right: 5px; }.repo-link { text-decoration: none; }.link-section a { margin-left: 10px; margin-right: 10px; }.guidance-notes { color: orange; font-size: 18px; }</style></head><body><script>
 var returns_categories = {"Existing": {"colour": "#CCCCCC", "description": "Current returned data"}, "Local": {"colour": "#C5E625", "description": "Recorded locally but not currently included in any data collections"}, "1aDraft": {"colour": "#1CFCF2", "description": "Suggested new item for SSD"}, "1bDraft": {"colour": "#F57C1D", "description": "Suggested new item for one of the 1b projects"}, "1bSpecified": {"colour": "#FFC91E", "description": "Final specified item for one of the 1b projects"}, "1aRemove": {"colour": "#FF2114", "description": "Specified item rejected from final spec."}, "1aNotInCMS": {"colour": "#F08080", "description": "Currently exists but commonly outside Social Care CMS. (SSD==Placeholder data)"}};
-</script><h1>Standard Safeguarding Dataset: Data Item Guidance</h1><p></p><div style='padding: 2px;'><div class='last-updated-container'><span class='last-updated-text'>Last updated:</span><span class='last-updated-date'>08-12-2023 16:52</span><div class='link-section'><a href='https://data-to-insight.github.io/ssd-data-model/README.html' class='repo-link'> | SSD Github</a><a href='https://data-to-insight.github.io/ssd-data-model/index.html' class='repo-link'> | Object Concept Model</a><a href='https://data-to-insight.github.io/ssd-data-model/guidance.html' class='repo-link'> | Data Item Guidance</a><a href='https://data-to-insight.github.io/ssd-data-model/existingreturnsmap.html' class='repo-link'> | Existing returns maps</a><a href='https://forms.office.com/e/UysrcGApJ1' class='repo-link'> | Submit Change Request</a><a href='https://data-to-insight.github.io/ssd-data-model/object_definitions.pdf' class='repo-link'> | Object Spec download(PDF)</a></div></div><h1>Objects Overview:  <span class='guidance-notes'>(+DfE Data Item Guidance Texts)</span></h1><p>This page shows detailed item-level guidance on data items within the Standard Safeguarding Dataset. Right click and open the image in a new browser tab to zoom/magnify/scroll object level detail. Data item id numbers [AAA000A] enable specific item/field referencing.</p><p>This is work in progress subject to frequent change; Data objects and item definitions are published here to support iterative development. Diagrams consisdered as conceptual interpretations, not a true relational/representational model</p><div id='table-container'><img id="main-image" src="imgs/ssd_erd_diagram.png" alt="Data Objects Overview" style="max-width: 100%; margin-bottom: 20px;"><div id='legend-container' style='margin-top: 20px;'><h3>Object/Data item key:</h3><ul id='legend' style='background: rgba(255, 255, 255, 0.7); padding: 10px; border-radius: 5px; list-style: none; margin-top: 2px;'><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #CCCCCC;'></div>Existing - Current returned data</li><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #C5E625;'></div>Local - Recorded locally but not currently included in any data collections</li><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #1CFCF2;'></div>1aDraft - Suggested new item for SSD</li><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #F57C1D;'></div>1bDraft - Suggested new item for one of the 1b projects</li><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #FFC91E;'></div>1bSpecified - Final specified item for one of the 1b projects</li><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #FF2114;'></div>1aRemove - Specified item rejected from final spec.</li><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #F08080;'></div>1aNotInCMS - Currently exists but commonly outside Social Care CMS. (SSD==Placeholder data)</li></ul><p>Note: Colour identifiers on Overview image indicate presence of <i>at least</i> one known change. See below tables for the item level changes/additions with granular colour-coding.</p></div></div></div><h1>Object Data Points Overview<span class='guidance-notes'>(DfE Data Item Guidance Notes)</span>:</h1><div><h2 style='text-align: left; margin-top: 20px;'>Object name: ssd_ehcp_named_plan</h2><div style='display: flex; align-items: flex-start;'><img src="imgs/erd_data_objects/ssd_ehcp_named_plan.png" alt="ssd_ehcp_named_plan" style="width: 300px; margin-right: 20px;"><div style='padding-bottom: 50px;'><table style='border-collapse: collapse; border: none;'><colgroup><col style='width: 12% !important;'/><col style='width: 15% !important;'/><col style='width: 43% !important;'/><col style='width: 15% !important;'/><col style='width: 15% !important;'/></colgroup><tr><th class='item-ref-column'>Item Ref</th><th class='data-item-column'>Data Item Name</th><th class='guidance-column'>Guidance</th><th class='returns-column'>Returns</th><th class='changelog-column'>Change Log</th></tr><tr class="row-EHCN001A"><td>EHCN001A</td><td>ehcn_named_plan_id</td><td>EHCP named plan unique ID from system or auto-generated as part of export. This module collects information on the content of the EHC plan, i.e. what is in Section I. It should be completed for all existing active EHC plans.
+</script><h1>Standard Safeguarding Dataset: Data Item Guidance</h1><p></p><div style='padding: 2px;'><div class='last-updated-container'><span class='last-updated-text'>Last updated:</span><span class='last-updated-date'>02-01-2024 13:39</span><div class='link-section'><a href='https://data-to-insight.github.io/ssd-data-model/README.html' class='repo-link'> | SSD Github</a><a href='https://data-to-insight.github.io/ssd-data-model/index.html' class='repo-link'> | Object Concept Model</a><a href='https://data-to-insight.github.io/ssd-data-model/guidance.html' class='repo-link'> | Data Item Guidance</a><a href='https://data-to-insight.github.io/ssd-data-model/existingreturnsmap.html' class='repo-link'> | Existing returns maps</a><a href='https://forms.office.com/e/UysrcGApJ1' class='repo-link'> | Submit Change Request</a><a href='https://data-to-insight.github.io/ssd-data-model/object_definitions.pdf' class='repo-link'> | Object Spec download(PDF)</a></div></div><h1>Objects Overview:  <span class='guidance-notes'>(+DfE Data Item Guidance Texts)</span></h1><p>This page shows detailed item-level guidance on data items within the Standard Safeguarding Dataset. Right click and open the image in a new browser tab to zoom/magnify/scroll object level detail. Data item id numbers [AAA000A] enable specific item/field referencing.</p><p>This is work in progress subject to frequent change; Data objects and item definitions are published here to support iterative development. Diagrams consisdered as conceptual interpretations, not a true relational/representational model</p><div id='table-container'><img id="main-image" src="imgs/ssd_erd_diagram.png" alt="Data Objects Overview" style="max-width: 100%; margin-bottom: 20px;"><div id='legend-container' style='margin-top: 20px;'><h3>Object/Data item key:</h3><ul id='legend' style='background: rgba(255, 255, 255, 0.7); padding: 10px; border-radius: 5px; list-style: none; margin-top: 2px;'><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #CCCCCC;'></div>Existing - Current returned data</li><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #C5E625;'></div>Local - Recorded locally but not currently included in any data collections</li><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #1CFCF2;'></div>1aDraft - Suggested new item for SSD</li><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #F57C1D;'></div>1bDraft - Suggested new item for one of the 1b projects</li><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #FFC91E;'></div>1bSpecified - Final specified item for one of the 1b projects</li><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #FF2114;'></div>1aRemove - Specified item rejected from final spec.</li><li style='margin-bottom: 5px; font-size: 12px;'><div style='display: inline-block; width: 20px; height: 20px; margin-right: 5px; background: #F08080;'></div>1aNotInCMS - Currently exists but commonly outside Social Care CMS. (SSD==Placeholder data)</li></ul><p>Note: Colour identifiers on Overview image indicate presence of <i>at least</i> one known change. See below tables for the item level changes/additions with granular colour-coding.</p></div></div></div><h1>Object Data Points Overview<span class='guidance-notes'>(DfE Data Item Guidance Notes)</span>:</h1><div><h2 style='text-align: left; margin-top: 20px;'>Object name: ssd_ehcp_named_plan</h2><div style='display: flex; align-items: flex-start;'><img src="imgs/erd_data_objects/ssd_ehcp_named_plan.png" alt="ssd_ehcp_named_plan" style="width: 300px; margin-right: 20px;"><div style='padding-bottom: 50px;'><table style='border-collapse: collapse; border: none;'><colgroup><col style='width: 12% !important;'/><col style='width: 15% !important;'/><col style='width: 43% !important;'/><col style='width: 15% !important;'/><col style='width: 15% !important;'/></colgroup><tr><th class='item-ref-column'>Item Ref</th><th class='data-item-column'>Data Item Name</th><th class='guidance-column'>Guidance</th><th class='returns-column'>Returns</th><th class='changelog-column'>Change Log</th></tr><tr class="row-EHCN001A"><td>EHCN001A</td><td>ehcn_named_plan_id</td><td>EHCP named plan unique ID from system or auto-generated as part of export. This module collects information on the content of the EHC plan, i.e. what is in Section I. It should be completed for all existing active EHC plans.
 It is possible that multiple plans may be recorded for a single person. For example, if an EHC plan has previously ceased and a further plan has later been issued following a new needs assessment. Changes may occur to this section from one year to the next for the same person, for example where an establishment named on the EHC plan is changed.</td><td>1aNotInCMS</td><td></td></tr><tr class="row-EHCN002A"><td>EHCN002A</td><td>ehcn_ehcp_asmt_id</td><td>See EAM001A</td><td>1aNotInCMS</td><td></td></tr><tr class="row-EHCN003A"><td>EHCN003A</td><td>ehcn_named_plan_start_date</td><td>Date of current EHC plan.</td><td>1aNotInCMS, SEN2</td><td></td></tr><tr class="row-EHCN004A"><td>EHCN004A</td><td>ehcn_named_plan_cease_date</td><td>Please provide the date the EHC plan ended or the date the EHC plan was transferred to another local authority. Do not record the date of the decision to cease. Local authorities must continue to maintain the EHC plan until the time has passed for bringing an appeal or, when an appeal has been registered, until it has been concluded.</td><td>1aNotInCMS, SEN2</td><td></td></tr><tr class="row-EHCN005A"><td>EHCN005A</td><td>ehcn_named_plan_cease_reason</td><td>Please provide the reason the EHC plan ended from the list below
 1 – Reached maximum age (this is the end of the academic year during which the young person turned 25)
 2 – Ongoing educational or training needs being met without an EHC plan