update donors list handling

[drfaustusfade]

as per previous works

• to compile fe donors with db ones ( current db https://docs.google.com/spreadsheets/d/1aCXr6SqC4OK-CvcH8IS9m-01Wljf94UvoKuD6EsBQWE/edit?usp=sharing )

• to migrate to API based: country cluster donor_id donor_name

• proposal to add a default value for the cluster, admin0 for use when clusters, admin0 not provided the lists

• to have stocks donors also per cluster

[fakhrihawari]

Link Donor from FE ,some don't have cluster so i make it to 'all'
https://docs.google.com/spreadsheets/d/1UI7NJ-01MvUgLO4fdhSNcriExgQ2kMHVeHjNVLC2bzw/edit?usp=sharing

[drfaustusfade]

that's exactly right

[fakhrihawari]

for the API the filter is just country cluster ? or donor_id donor_name too?
engine: https://github.com/fakhrihawari/ngm-reportEngine/tree/migrate-newdonor

[drfaustusfade]

for the api contry cluster should be enough, but in general like for activities country list should be fetched + default ALL country list, on FE, project country cluster list should be used, if no cluster default country ALL cluster, if no country list default ALL global list

[fakhrihawari]

for the api contry cluster should be enough, but in general like for activities country list should be fetched + default ALL country list, on FE, project country cluster list should be used, if no cluster default country ALL cluster, if no country list default ALL global list

updated

[drfaustusfade]

for the api contry cluster should be enough, but in general like for activities country list should be fetched + default ALL country list, on FE, project country cluster list should be used, if no cluster default country ALL cluster, if no country list default ALL global list

updated

alright, why would use aggregate rather than find? also, should there be FE updates?

[fakhrihawari]

for the api contry cluster should be enough, but in general like for activities country list should be fetched + default ALL country list, on FE, project country cluster list should be used, if no cluster default country ALL cluster, if no country list default ALL global list

updated

alright, why would use aggregate rather than find? also, should there be FE updates?

You mean inside native?

i just take a look from other that using native its using aggregate
and i already change from aggregate to find now.
is it more fast using find??
link FE: https://github.com/fakhrihawari/ngm-reportHub/tree/migrate-newdonor-api

[fakhrihawari]

Updated link:
https://github.com/fakhrihawari/ngm-reportHub/tree/migrate-newdonor-api
https://github.com/fakhrihawari/ngm-reportEngine/tree/migrate-newdonor

and for stock donor just need to update in FE right? coz it use same data that use in project plan right?

[drfaustusfade]

right, that should be the same, but as stock cluster could be changed donors should be updated on change

[fakhrihawari]

right, that should be the same, but as stock cluster could be changed donors should be updated on change

okay , updated

[drfaustusfade]

please make pull so that i would check, and please prepare file and script for import, getStockLists: function (org) org param is it report or organization?

[fakhrihawari]

please make pull so that i would check, and please prepare file and script for import, getStockLists: function (org) org param is it report or organization?

i chenge it again to same as before so now is just admin0pcode when set stock lists

[fakhrihawari]

please make pull so that i would check, and please prepare file and script for import, getStockLists: function (org) org param is it report or organization?

file:
https://drive.google.com/file/d/1turEG34jseQZBGbI-EfN8gHCmvIQX3Z5/view?usp=sharing

script:
mongoimport -d ngmHealthCluster -c donors --drop --headerline --type csv --file /home/ubuntu/nginx/www/ngm-reportEngine/donors.csv

PR:
drfaustusfade/ngm-reportEngine#26
drfaustusfade/ngm-reportHub#129

[drfaustusfade]

• there are project donors duplicates

for the api contry cluster should be enough, but in general like for activities country list should be fetched + default ALL country list, on FE, project country cluster list should be used, if no cluster default country ALL cluster, if no country list default ALL global list

• is it working like next? if cluster list use it, if previous not defined, use all country list, if previous not defined use global all cluster list, if previous not defined use global all admin0, for example, get AF, ENSFI; if not get AF, ALL; if not get ALL, ENSFI; if not get ALL, ALL

right, that should be the same, but as stock cluster could be changed donors should be updated on change

• that's not correct, should be filtered to cluster ones (probably with custom filter) and reset on change, as a possible solution; because updating on different stock records affect other selection lists, as the common list is used

[fakhrihawari]

• there are project donors duplicates

for the api contry cluster should be enough, but in general like for activities country list should be fetched + default ALL country list, on FE, project country cluster list should be used, if no cluster default country ALL cluster, if no country list default ALL global list

• is it working like next? if cluster list use it, if previous not defined, use all country list, if previous not defined use global all cluster list, if previous not defined use global all admin0, for example, get AF, ENSFI; if not get AF, ALL; if not get ALL, ENSFI; if not get ALL, ALL

right, that should be the same, but as stock cluster could be changed donors should be updated on change

• that's not correct, should be filtered to cluster ones (probably with custom filter) and reset on change, as a possible solution; because updating on different stock records affect other selection lists, as the common list is used

Point 1
for project donor duplicate what should we do ? i dont how we should merge it coz some have different cluster id or admin0pcode for example :

wash | danida | DANIDA
esnfi | danida | DANIDA

it become [wash,esnfi] | danida | DANIDA (array) or wash,esnfi | danida | DANIDA (string);

Point 2
rightnow in the last commit
if get AF ESNFI => then get donors with AF and ALL with cluster ESNFI; if not get AF then just ALL with cluster ESNFI

point 3. so we need to make different list for each stock record right?

[drfaustusfade]

1. that should be fixed if 2 is correct ( that is no duplicates per cluster list )
2. is it as per logic below next? please also reference the code block to check

If ESNFI
1 get AF ESNFI
2 or (if not 1) get AF ALL
3 or (if not 2) get ALL ESNFI
4 or (if not 3) get ALL ALL

3. or filter donors by stock cluster ( if 2 logic, then with angularjs custom filter )

[fakhrihawari]

1. that should be fixed if 2 is correct ( that is no duplicates per cluster list )
2. is it as per logic below next? please also reference the code block to check

If ESNFI
1 get AF ESNFI
2 or (if not 1) get AF ALL
3 or (if not 2) get ALL ESNFI
4 or (if not 3) get ALL ALL

1. or filter donors by stock cluster ( if 2 logic, then with angularjs custom filter )

Updated:
file:
https://drive.google.com/file/d/1FDfqlSBWZoSbVhTKRbj9xJaSJx49GF_X/view?usp=sharing

script:
mongoimport -d ngmHealthCluster -c donors --drop --headerline --type csv --file /home/ubuntu/nginx/www/ngm-reportEngine/donors.csv

PR:
drfaustusfade/ngm-reportEngine#26
drfaustusfade/ngm-reportHub#129

[drfaustusfade]

alright, for multi-country role users will that return corresponding donors or of their registered country?

[fakhrihawari]

it based on country that user have coz i used admin0pcode from user profie

[drfaustusfade]

alright, let's do multicountry in another issue,

for AF donors are already on db, but they are without admin0pcode, that should be included

also for

If ESNFI
1 get AF ESNFI
2 or (if not 1) get AF ALL
3 or (if not 2) get ALL ESNFI
4 or (if not 3) get ALL ALL

that should apply when setting lists for project or stocks, I think right now on BE does it check for cluster if not in params? as project or stock can have a cluster to it, I guess with the current setup default global list and country-specific could be requested, and then filtered correspondingly

[fakhrihawari]

You mean this one?
https://docs.google.com/spreadsheets/d/1aCXr6SqC4OK-CvcH8IS9m-01Wljf94UvoKuD6EsBQWE/edit?usp=sharing

if yes , i already put it but i set all to admin0pcode as 'ALL'

[drfaustusfade]

Yes, this is actually used by AF

[fakhrihawari]

alright, let's do multicountry in another issue,

for AF donors are already on db, but they are without admin0pcode, that should be included

also for

If ESNFI
1 get AF ESNFI
2 or (if not 1) get AF ALL
3 or (if not 2) get ALL ESNFI
4 or (if not 3) get ALL ALL

that should apply when setting lists for project or stocks, I think right now on BE does it check for cluster if not in params? as project or stock can have a cluster to it, I guess with the current setup default global list and country-specific could be requested, and then filtered correspondingly

yes its check cluster params and also for filter like this:

If ESNFI
1 get AF ESNFI
2 or (if not 1) get AF ALL
3 or (if not 2) get ALL ESNFI
4 or (if not 3) get ALL ALL

i already do that in BE

[drfaustusfade]

if we define for example one donor admin0pcode AF and cluster protection as somedonor we will get on protection project somedonor and other checkboxes?

[fakhrihawari]

other will be add on getDonors function on ngmClusterLists

[fakhrihawari]

updated file:
https://drive.google.com/file/d/1wETW2hBp8aXCVGCOHQBMy6b5_XPGfq9k/view?usp=sharing

updated branch
drfaustusfade/ngm-reportEngine#26
drfaustusfade/ngm-reportHub#129

[drfaustusfade]

Alright, looks good,
Minor comment, why having run sequence, rather than simple conditional like check (AF ESNFI) than if no donors length than check ( AF all )...
I think also we could later make a general 4 rule fn for other list use

[fakhrihawari]

Alright, looks good,
Minor comment, why having run sequence, rather than simple conditional like check (AF ESNFI) than if no donors length than check ( AF all )...
I think also we could later make a general 4 rule fn for other list use

just to make sure donors not return empty string;