Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement redirection operators for file- and byte-iterators #46

Open
holgerbrandl opened this issue Nov 11, 2015 · 2 comments
Open

Implement redirection operators for file- and byte-iterators #46

holgerbrandl opened this issue Nov 11, 2015 · 2 comments
Labels

Comments

@holgerbrandl
Copy link
Contributor

The following examples from the README.md are not working

Seq(file1, file2) >: file3   
Seq(file1, file2) >>: file3
cat(file1) >>: file

This is because both operators just allow for String as input and not for Iterable[File] or Seq[Iterator[Byte]].

There's also a typo in the first example >: should be BACKTICK>:BACKTICK

Also the File.append* API does not allow for an efficient concatenation of Files. To do so we need to write something like:

val chunkResults: Iterable[File] = ...

chunkResults.map(_.contentAsString).foreach(combinedFile.appendLines(_))
// or
chunkResults.map(_.bytes.toArray).foreach(combinedBlastOut.append)

Both approaches load the complete input files into memory and don't stream/iterate through them which will cause problems for larger files.

Suggestion:

/**
 * Overwrite destination with this file's contents
 *
 * @param destination
 * @return destination
 */
def >(destination: File): File = ???

/**
 * Append this file's contents to destination
 *
 * @param destination
 * @return destination
 */
def `>>`(destination: File): File = ???
@pathikrit pathikrit changed the title Redirection operators don't work for file- and byte-iterators Implement redirection operators for file- and byte-iterators Nov 11, 2015
@mpollmeier
Copy link
Contributor

Any updates on when this this will be fixed?
Concatenating files and writing them to a new file is the most basic thing a file library should do IMO. Is there a workaround? I used unix4j in the end.

@pathikrit
Copy link
Owner

This issue is only for the operators DSL. You can append by calling append just fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants