Created new dataset North Uganda 2020

[yashgadhiya10]

North Uganda 2020

[yashgadhiya10]

@ivanzvonkov - When the data pipeline check is initiated, there is an error related to micromamba setup which is failing the check. Let us know on how to fix this.

[yashgadhiya10]

@adebowaledaniel - we are facing some micromamba related errors in our data-pipeline. Your help would be appreciated in moving ahead with this issue.

[yashgadhiya10]

@ivanzvonkov . We are still facing the micromamba issue with the data pipeline. You help will be appreciated for sorting this out.

[ivanzvonkov]

Hey @yashgadhiya10 sorry for the delayed response.
I've been dealing with the micromamba issue and a host of other issues here: #406
Basically many things that crop-mask depended on: micromamba, GEE Sentinel-2 collection in OpenMapFlow, xarray, we're recently deprecated and that has caused some issues. I'm finishing up a fix today.

[ivanzvonkov]

Still dealing with OpenMapFlow issue: https://github.com/nasaharvest/openmapflow/actions/runs/11941328078/job/33285918436

[ivanzvonkov]

Fixed OpenMapFlow issue, now exporting new Senegal data and fixing area test issue, I think it has something to do with GDAL here: #406

[ivanzvonkov]

@yashgadhiya10 Okay, the data pipeline + all tests (except area) are working now. The data will take a day or so to get from Google Earth Engine

[ivanzvonkov]

@yashgadhiya10 There's a dataset duplicates test failure here.
This is because you've added samples from:

• ceo-UNHCR-North-Uganda-Feb-2019---Feb-2020-(Set-1)-sample-data-2024-10-01.csv
• ceo-UNHCR-North-Uganda-Feb-2019---Feb-2020-(Set-2)-sample-data-2024-10-01.csv

But these samples are already included in UgandaNorthCEO2019

• ceo-UNHCR-North-Uganda-Feb-2019---Feb-2020-(Set-1)-sample-data-2024-03-12.csv
• ceo-UNHCR-North-Uganda-Feb-2019---Feb-2020-(Set-2)-sample-data-2024-03-12.csv

What is the differencebetween the data you added and this data that already existed in UgandaNorthCEO2019?
cc @MsPixels

[yashgadhiya10]

@ivanzvonkov Thank you so much for your help!

So it was just a manual error of placing the 2019-20 csv for 2020 dataset. I fixed the code and made another commit within the same branch. However it seems it is still giving the duplication error. So, I believe we have to remove the already present dataset from openmapflow cloud based on my understanding, do correct me if I am wrong but I am not sure how to remove those.

ceo-UNHCR-North-Uganda-Feb-2019---Feb-2020-(Set-1)-sample-data-2024-10-01.csv
ceo-UNHCR-North-Uganda-Feb-2019---Feb-2020-(Set-2)-sample-data-2024-10-01.csv

So, we won't need these csv files and we can keep the orginial ones.

[ivanzvonkov]

@yashgadhiya10 since that dataset was already made in data/datasets it wasn't being created again. I've removed it which retriggers the pipeline

[yashgadhiya10]

Latest code changes have 2020-21 file as expected.

[yashgadhiya10]

Thank you for the clarification, @ivanzvonkov. I can see the latest code having the correct files, so we should be merging the pull request after your approval on this request. Do correct me if I missed anything. Again, thank you!