New field in 2020 data call

[hgerritsen]

Hello,
The 2020 data call asks for a new field: Anonymized vessel id.
From the 2019 WGFSD report i understand that this means that for rows with 2 vessels or less should be disaggregated by vessel (if there are 2) and a vessel id should be provided.
I have put together some code, that i think could do the trick and could be added to the workflow script (around line 512). I am not 100% sure that it is ok and there may be a better way but here is my attempt:

VE_lut <- data.frame(VE_REF=unique(c(table1$VE_REF,table2$VE_REF)))
VE_lut$VE_ID <- paste0('IRL',sprintf("%03d", 1:nrow(VE_lut))) # use relevant country code!
## check that there are fewer than 999 unique vessels!
nrow(VE_lut)

table1 <- left_join(table1,VE_lut)
table2 <- left_join(table2,VE_lut)


table1 <- table1 %>%
    group_by(RT, VE_COU,
             Year, Month,
             Csquare, LENGTHCAT,
             LE_GEAR, LE_MET) %>% 
    mutate(n_vessels = n_distinct(VE_REF, na.rm = TRUE))
table1$VE_ID[table1$n_vessels>2] <- NA #only provide vessel id for cells with 1 or 2 vessels


## Aggregation of VMS data:
table1Save <- table1 %>%
    group_by(RT, VE_COU,
             Year, Month,
             Csquare, LENGTHCAT,
             LE_GEAR, LE_MET, VE_ID) %>%
    summarise(sum_intv = sum(INTV, na.rm = TRUE),
              sum_kwHour = sum(kwHour, na.rm = TRUE),
              sum_le_kg_tot = sum(LE_KG_TOT, na.rm = TRUE),
              sum_le_euro_tot = sum(LE_EURO_TOT, na.rm = TRUE),
              mean_si_sp = mean(SI_SP, na.rm = TRUE),
              mean_ve_len = mean(VE_LEN, na.rm = TRUE),
              mean_ve_kf = mean(VE_KW, na.rm = TRUE),
              n_vessels = n_distinct(VE_REF, na.rm = TRUE)) %>%
    as.data.frame()

table2 <- table2 %>%
    group_by(RT, VE_COU,
             Year, Month,
             LE_RECT, LE_GEAR,
             LE_MET, LENGTHCAT,
             tripInTacsat) %>%
    mutate(n_vessels = n_distinct(VE_REF, na.rm = TRUE))
table2$VE_ID[table2$n_vessels>2] <- NA #only provide vessel id for cells with 1 or 2 vessels

## Aggregation of LogBook data:
table2Save <- table2 %>%
    group_by(RT, VE_COU,
             Year, Month,
             LE_RECT, LE_GEAR,
             LE_MET, VE_ID, LENGTHCAT,
             tripInTacsat) %>%
    summarise(sum_intv = sum(INTV, na.rm = TRUE),
              sum_kwDays = sum(kwDays, na.rm = TRUE),
              sum_le_kg_tot = sum(LE_KG_TOT, na.rm = TRUE),
              sum_le_euro_tot = sum( LE_EURO_TOT, na.rm = TRUE),
              n_vessels = n_distinct(VE_REF, na.rm = TRUE)) %>%
    as.data.frame()

[colinpmillar]

Thanks Hans - will review and add in tonight - can I add you as author of the commit?

[hgerritsen]

Yes, that would be good. I didn’t want to go messing with the code, so I raised it as an issue, rather than requesting write access to the github but if you think it makes sense I would be happy to add it to the script. I also informed the WGSFD chairs by email. Hans From: Colin Millar [mailto:[email protected]] Sent: Wednesday 26 February 2020 15:43 To: ices-eg/wg_WGSFD <[email protected]> Cc: Hans Gerritsen <[email protected]>; Author <[email protected]> Subject: Re: [ices-eg/wg_WGSFD] New field in 2020 data call (#23) Thanks Hans - will review and add in tonight - can I add you as author of the commit? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#23?email_source=notifications&email_token=AHA5GPNYVG43YGXNT6HTH2DRE2EWTA5CNFSM4K4IKGV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENAXI6Y#issuecomment-591492219>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AHA5GPPZ5S4WYWTMHEA5ZJ3RE2EWTANCNFSM4K4IKGVQ>.

[colinpmillar]

Actually you are in the WGSFD sharepoint group - so I can give you write access. That might be easier, as you have actual data to work with - I just have mocked up data.

If are up for that - would you mind looking at adding it to the https://github.com/ices-eg/wg_WGSFD/tree/test-workflow branch?

[hgerritsen]

OK

[hgerritsen]

Done.

[colinpmillar]

thanks so much!!