Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improvements to XMatch #426

Merged
merged 23 commits into from
Apr 23, 2024
Merged

Improvements to XMatch #426

merged 23 commits into from
Apr 23, 2024

Conversation

albireox
Copy link
Member

@albireox albireox commented Apr 23, 2024 •
edited
Loading

Makes several improvements to XMatch:

  • Removes some hard-coded use of SQL files. These should not be necessary and if they were, we should get to the bottom of why they are ...
  • Store intermediate results in a sandboxed catalog_to_XXX table. This allows to run a full dry-run cross-match for a single catalogue.
  • Phase 1: distinct only on model_pk.
  • Phase 3: Require using only best matches.

Additionally, this PR add the too carton.

@albireox
Allow to read XMatchPlanner configuration from a dictionary
ca9a2c4
@albireox
Use rich handler for logger
33550ae
@albireox
Allow _temp_count=0
0c0e685
@albireox
Allow to override the logger used by the XMatchPlanner
b0c971c
@albireox
Linting indentation fix
ff7605f
@albireox
Add added_by_phase column to relational model
79ab55e
@albireox
Revert Felipe's use of SQL files for phase 1
3b17417
@albireox
Phase 1: distinct only on model_pk
67564fa 
This should prevent duplicates when a target in the intermediate
table has been cross-matched to multiple catalogids.
@albireox
Phase 2: allow to cross-match with temporary and real catalog
cb23df4
@albireox
Phase 3: require using only best matches
8b5308f 
This is an imporant bug fix. Before we were missing targets
that where associated with a catalogid in phase 2 but only as
a secondary match.
@albireox
Revert Felipe's use of SQL files for phase 3
f30eeb5
@albireox
Consolidate line
0311d19
@albireox
Revert using rich logger
8de1845
@albireox
Delete phase1_range option from XMatchPlanner
0768bff
@albireox
Deepcopy config file
3d4eba5
@albireox
Add mode parameter to load method of BaseCarton
9199426
@albireox
Add ToO_Carton
f0c0388
@albireox
Always use self.log instead of log in BaseCarton
21bc645
@albireox
Import CatalogToToO_Target inside build_query
405fc2d
@albireox
Go back to importing CatalogToToO_Target at top
6d3f778
@albireox
Add IF NOT EXISTS to the creation of the q3c index
0d5d216
@albireox
Store intermediate catalog_to_XXX results in sandbox
52389c7
@albireox
Linting
fb91aa1
@albireox albireox merged commit fb697c6 into main Apr 23, 2024
1 check passed
@albireox albireox deleted the too-xmatch branch April 23, 2024 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@albireox