Triage missing LDDs and data types for fields updating per re-indexing sweeper

[jordanpadams]

💡 Description

Here are the logs. we need to triage whether we need LDD JSONs generated, or these are expected.

2024-10-15 00:56:10,851::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_collection is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,851::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_collection_secondary is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,851::pds.registrysweepers.reindexer.main::WARNING::Property ops:Harvest_Info/ops:harvest_version is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,854::pds.registrysweepers.reindexer.main::WARNING::Property ops:Provenance/ops:registry_sweepers_provenance_version is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,860::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_associate is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,866::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_document is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,867::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_errata is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,867::pds.registrysweepers.reindexer.main::WARNING::Property pds:Affiliation/pds:organization_name is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,867::pds.registrysweepers.reindexer.main::WARNING::Property pds:Affiliation/pds:organization_rorid is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,867::pds.registrysweepers.reindexer.main::WARNING::Property pds:Funding_Acknowledgement/pds:funding_acknowledgement_text is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,867::pds.registrysweepers.reindexer.main::WARNING::Property pds:Funding_Acknowledgement/pds:funding_source is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,867::pds.registrysweepers.reindexer.main::WARNING::Property pds:Person/pds:family_name is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,867::pds.registrysweepers.reindexer.main::WARNING::Property pds:Person/pds:given_name is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,867::pds.registrysweepers.reindexer.main::WARNING::Property pds:Person/pds:person_orcid is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,868::pds.registrysweepers.reindexer.main::WARNING::Property pds:Person/pds:sequence_number is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,889::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_calibration is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,889::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_browse is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,891::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_data is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,892::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_instrument is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,892::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_instrument_host is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,892::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_investigation is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,893::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_target is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,935::pds.registrysweepers.reindexer.main::WARNING::Property @timestamp is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,935::pds.registrysweepers.reindexer.main::WARNING::Property @version is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,958::pds.registrysweepers.reindexer.main::WARNING::Property pds:Acronym_Index_Entry/pds:acronym is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,958::pds.registrysweepers.reindexer.main::WARNING::Property pds:Acronym_Index_Entry/pds:priority_order is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,958::pds.registrysweepers.reindexer.main::WARNING::Property pds:Alias_Index_Entry/pds:name is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,958::pds.registrysweepers.reindexer.main::WARNING::Property pds:Alias_Index_Entry/pds:priority_order is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,958::pds.registrysweepers.reindexer.main::WARNING::Property pds:Instrument_Host/pds:terse_description is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,963::pds.registrysweepers.reindexer.main::WARNING::Property pds:Investigation/pds:terse_description is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,965::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_agency is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,965::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_archivist is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,965::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_manager is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,965::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_operator is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:10,965::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_personnel is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:11,029::pds.registrysweepers.reindexer.main::WARNING::Property ref_lid_set is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:11,030::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_set is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:11,046::pds.registrysweepers.reindexer.main::WARNING::Property pds:File/pds:file_URL is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:11,052::pds.registrysweepers.reindexer.main::WARNING::Property cart:Geographic/cart:geographic_description is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:11,053::pds.registrysweepers.reindexer.main::WARNING::Property pds:Encoded_External/pds:encoding_standard_id is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:11,053::pds.registrysweepers.reindexer.main::WARNING::Property pds:Encoded_External/pds:offset is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:16,850::pds.registrysweepers.reindexer.main::WARNING::Property mgn:Magellan_Parameters/mgn:product_type is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:16,878::pds.registrysweepers.reindexer.main::WARNING::Property pds:Affiliation/pds:sequence_number is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:16,932::pds.registrysweepers.reindexer.main::WARNING::Property pds:Funding_Acknowledgement/pds:funding_award is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:16,938::pds.registrysweepers.reindexer.main::WARNING::Property ref_lid_product is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 00:56:16,938::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_product is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:18,963::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_geometry is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:18,968::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_package is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:18,970::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_bundle is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:19,040::pds.registrysweepers.reindexer.main::WARNING::Property ref_lidvid_resource is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:19,050::pds.registrysweepers.reindexer.main::WARNING::Property vg1:Voyager1/vg1:mission_phase_name is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:19,051::pds.registrysweepers.reindexer.main::WARNING::Property vg1:Voyager1/vg1:spacecraft_clock_start_count is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:19,051::pds.registrysweepers.reindexer.main::WARNING::Property vg1:Voyager1/vg1:spacecraft_clock_stop_count is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:19,053::pds.registrysweepers.reindexer.main::WARNING::Property vg2:Voyager2/vg2:mission_phase_name is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:19,054::pds.registrysweepers.reindexer.main::WARNING::Property vg2:Voyager2/vg2:spacecraft_clock_start_count is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:19,054::pds.registrysweepers.reindexer.main::WARNING::Property vg2:Voyager2/vg2:spacecraft_clock_stop_count is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:19,057::pds.registrysweepers.reindexer.main::WARNING::Property v:Voyager1/v:mission_phase_name is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:19,057::pds.registrysweepers.reindexer.main::WARNING::Property v:Voyager1/v:spacecraft_clock_start_count is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"
2024-10-15 01:11:19,057::pds.registrysweepers.reindexer.main::WARNING::Property v:Voyager1/v:spacecraft_clock_stop_count is missing from the index mappings and does not have an entry in the data dictionary index - defaulting to type "keyword"

2024-10-15 01:11:19,131::pds.registrysweepers.reindexer.main::INFO::Detected 1186952 docs with 78 missing mappings and 0 mappings conflicting with the DD, out of a total of 1186952 docs
2024-10-15 01:11:19,131::pds.registrysweepers.reindexer.main::WARNING::Problems were detected with docs having harvest timestamps between 2022-10-09T08:41:51.430200 and 2024-10-09T00:59:55.734228
2024-10-15 01:11:19,131::pds.registrysweepers.reindexer.main::WARNING::Problems were detected with docs having harvest versions ['3.8.2', '3.9.0-SNAPSHOT', '4.1.0-SNAPSHOT']
2024-10-15 00:56:11,081::pds.registrysweepers.reindexer.main::INFO::Property lro:LRO_Parameters/lro:mission_phase_name will be updated to type "keyword" from data dictionary
2024-10-15 01:11:18,970::pds.registrysweepers.reindexer.main::INFO::Property pds:Checksum_Manifest/pds:object_length will be updated to type "long" from data dictionary
2024-10-15 01:11:18,970::pds.registrysweepers.reindexer.main::INFO::Property pds:Archival_Information_Package/pds:description will be updated to type "text" from data dictionary
Query 8a2042: 100%|████████████████▉| 1183964/1186952 [15:11<00:01, 2564.80it/s]2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Manifest_SIP_Deep_Archive/pds:offset will be updated to type "long" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Information_Package_Component_Deep_Archive/pds:aip_label_checksum will be updated to type "keyword" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Manifest_SIP_Deep_Archive/pds:description will be updated to type "text" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Manifest_SIP_Deep_Archive/pds:field_delimiter will be updated to type "keyword" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Manifest_SIP_Deep_Archive/pds:name will be updated to type "keyword" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Manifest_SIP_Deep_Archive/pds:local_identifier will be updated to type "keyword" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Manifest_SIP_Deep_Archive/pds:object_length will be updated to type "long" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:SIP_Deep_Archive/pds:provider_site_id will be updated to type "keyword" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Manifest_SIP_Deep_Archive/pds:record_delimiter will be updated to type "keyword" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Information_Package_Component_Deep_Archive/pds:aip_lidvid will be updated to type "keyword" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Information_Package_Component_Deep_Archive/pds:checksum_type will be updated to type "keyword" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Information_Package_Component_Deep_Archive/pds:manifest_url will be updated to type "keyword" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Manifest_SIP_Deep_Archive/pds:records will be updated to type "long" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Manifest_SIP_Deep_Archive/pds:parsing_standard_id will be updated to type "keyword" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:SIP_Deep_Archive/pds:description will be updated to type "text" from data dictionary
2024-10-15 01:11:18,982::pds.registrysweepers.reindexer.main::INFO::Property pds:Information_Package_Component_Deep_Archive/pds:manifest_checksum will be updated to type "keyword" from data dictionary

⚔️ Parent Epic / Related Tickets

No response