Extending OpenMS for compress support

[ypriverol]

Multiple users are using quantms in AWS for FLQ and TMT analysis. One of the feature requests is to decrease the size of the intermediate files, ideXML, and consensusXML. We have been discussing this internally and would be great to have native support in OpenMS tools including the proteomicsLFQ, and the Search Adapters for compressed files. Please provide your feedback before starting the implementation @timosachsenberg.

Related to issue bigbio/quantms#393

[jpfeuffer]

Reading should work already if the file type is registered as a possible filetype for id and cxml.

Writing should also be easy by adding something like this:

https://techoverflow.net/2020/01/13/how-to-gzip-compress-on-the-fly-in-c-using-boostiostreams/

to the writeTo methods of affected xmlhandlers.
We could probably abstract this across all handlers if we let them accept non std ostreams.

Alternatively you can probably just use named pipes on any unix system and compress on the fly right now without any changes in OpenMS.

[ypriverol]

@jpfeuffer we have one student working on this issue. I was talking to @timosachsenberg today about this feature and he agreed is a good thing to do. What do you think about the feature? I think in AWS is crucial to have smaller files to transfer because you have to pay for space and bandwidth.

[jpfeuffer]

Yes, useful feature in general. You can try with unix tools first to see how much it helps and how much it affects the runtime.

[timosachsenberg]

I think the mzML handler is one example that already implements this

[jpfeuffer]

Hmm I'm not sure if we support writing gzipped mzml. I did not find any relevant code in a quick search.

[timosachsenberg]

Ah ok then its only in the reader...