Release daa.analytics v0.7.0 · pepfar-datim/daa-analytics

New Features

Updated the code to accommodate the new disaggregations of 65+ for specific indicators.
We included a timestamp functionality to allow us to keep track on the last time S3 bucket was updated.
Through a DATIM API, we were able to get more facility level data for analysis, that is data for DSD and TA.
Implemented a new formula for calculating the global concordance, this will enable countries compare how they're performing against the global results for a specific indicator and year.
Included a descriptive information for country users under the "Reporting Rates" tab for countries that do not have either PEPFAR or MOH data for the current fiscal year.
On the Reporting Rates tab, we added raw numbers on top of each bar chart to show actual figures of each indicator.

We changed the R code settings on the Rstudio platform to allow more people to log in simultaneously and pull data, this enhancement increased the number of active sessions from 3 to 5.

On the raw data download, we removed duplicate columns "emr_present" and maintained one of them which we then verified that the information provided for a facility if it has EMR in that site, correlates with what is on DATIM, we used a DATIM api to pull all facilities and compared to all facilities we had on the Explorer App for all PEPFAR supported countries and confirmed that the presence of EMR is similar on both datasets for all facilities.
Fixed an issue with the dates under subtitles whereby when a user changes a filter date, the subtitle with the date was not changing but rather stayed static, now it is responsive of what the user selects on the filter side. This mainly affected the Reporting Rates tab and the Concordance Graph.

Improved the performance of the overall application by optimizing specific functions responsible for taking up a huge load time. The function "get_daa_data" pulls millions of rows, we removed unnecessary function loops and used inbuilt recommended R functions to make it more efficient. This reduced our time to pull all data for all countries from 24 minutes to 20 minutes.

We adjusted the size of the scatterplot in the Site Scatterplot tab to make it fill up more of the page and not be bunched up at the top.
We adjusted the size of the concordance graph in the Concordance Graph tab to make it more visible to users.
We removed the raw numbers of tx_pvls_n and tx_pvls_d from the raw data download by removing the code handling those figures on S3 bucket. Since we have them as indicators now, we do not need them as individual columns on the raw data files.
We updated the global summary file by including a column which shows the global concordance percentage of all countries for each year and each indicator. We also included two new suggested columns for further analysis and re-formatted the naming of the columns.
On the site data tab, we removed a column called Weighted Concordance(OU Level) from the table and we also made the column called Difference to be absolute values and not have negatives.

There is still room for improvement on the load time the system takes to load all data. We can make further improvements on caching especially for data coming from the S3 bucket specifically the dataset handling VLS and EMR called pvls_emr_raw.
We can also see way to further optimize the functions that fetch data from DATIM.

We welcome your feedback on any suggestions for improvements or issues encountered.

[Christian Onyango - Software Engineer, ICF.]