2GB character string limit: integer overflow corp_text_rbindlist

[edmondac]

Using corp_text_rbindlist with enough sufficiently large input corp_text objects, results in this error:

Error in paste(sapply(x, corp_get_text), collapse = " ") :
  result would exceed 2^31-1 bytes
Calls: <Anonymous> -> corp_text -> paste
In addition: Warning message:
In CorporaCoCo::corp_text_rbindlist(corp_list) :
  integer overflow in 'cumsum'; use 'cumsum(as.numeric(.))'
Execution halted

Indeed https://stat.ethz.ch/R-manual/R-devel/library/base/html/Memory-limits.html says that "The number of bytes in a character string is limited to 2^31 - 1"

[edmondac]

Trivial example:

> a<-"test"
> while (TRUE) { a<-paste0(a, a); print(nchar(a)) }
[1] 8
[1] 16
[1] 32
[1] 64
[1] 128
[1] 256
[1] 512
[1] 1024
[1] 2048
[1] 4096
[1] 8192
[1] 16384
[1] 32768
[1] 65536
[1] 131072
[1] 262144
[1] 524288
[1] 1048576
[1] 2097152
[1] 4194304
[1] 8388608
[1] 16777216
[1] 33554432
[1] 67108864
[1] 134217728
[1] 268435456
[1] 536870912
[1] 1073741824
Error in paste0(a, a) : result would exceed 2^31-1 bytes