You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Implement the original code at https://github.com/jlvdb/yet_another_wizz to fit into RAIL and improve the performance to match the scale of the future datasets.
Main challenges:
Implementing the estimator efficiently (potentially extending the scipy.spatial.cKDTree to handle inverse distance weights)
Parallelization (probably spatial splitting of the data set)
The text was updated successfully, but these errors were encountered:
Given that RAIL now speaks ceci, it might be worth looking into what existing infrastructure from TXPipe could potentially be called in a RAIL stage specifically for the cross-correlation step. @joezuntz has thought about this a bunch and probably has additional ideas for efficiency.
If I remember correctly that YAW is similar to the-wizz in switching from actual correlation function calculations to KDTree matching, then we probably don't need any of the actual TreeCorr stuff. But, I think we do need to figure out how to do proper mask generation/use and generating randoms in a way that is consistent with how other DESC people are doing those operations.
Good points, Alex and Sam, I will check TXPipe and talk to Joe if necessary. I'll try to use as much existing infrastructure as possible unless that requires changing too much of the existing code base.
Implement the original code at
https://github.com/jlvdb/yet_another_wizz
to fit into RAIL and improve the performance to match the scale of the future datasets.Main challenges:
scipy.spatial.cKDTree
to handle inverse distance weights)The text was updated successfully, but these errors were encountered: