Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shell-style piping for transformers #358

Open
dmitriy-serdyuk opened this issue Jul 27, 2016 · 5 comments · May be fixed by #359
Open

Shell-style piping for transformers #358

dmitriy-serdyuk opened this issue Jul 27, 2016 · 5 comments · May be fixed by #359
Labels

Comments

@dmitriy-serdyuk
Copy link
Contributor

I propose to add a syntactic sugar to easier manipulate streams, we could use some infix operator to combine transformers (like | or >). | would look like linux piping:

stream = Mapping(Flatten(DataStream.default_stream(dataset)), my_func)

becomes

stream = dataset | DataStream.default_stream() | Flatten() | Mapping(my_func)

Less parenthesis, transformers are applied in the direct order.

Implementation looks very straightforward.

What do you guys think?

@nouiz
Copy link
Contributor

nouiz commented Jul 27, 2016

Does python support that syntax? I never heard of that.

On Wed, Jul 27, 2016 at 3:44 PM, dmitriy-serdyuk [email protected]
wrote:

I propose to add a syntactic sugar to easier manipulate streams, we could
use some infix operator to combine transformers (like | or >). | would
look like linux piping:

stream = Mapping(Flatten(DataStream.default_stream(dataset)), my_func)

becomes

stream = dataset | DataStream.default_stream() | Flatten() | Mapping(my_func)

Less parenthesis, transformers are applied in the direct order.

Implementation looks very straightforward.

What do you guys think?


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#358, or mute the thread
https://github.com/notifications/unsubscribe-auth/AALC-5BnbP2B47e3c5WW3sknfSHBmz7Hks5qZ7UXgaJpZM4JWjd0
.

@dwf
Copy link
Contributor

dwf commented Jul 27, 2016

I like it a lot. Current syntax gets quite hairy.

Is it going to be easy to do this without breaking literally everyone's
code?

I imagine we could make the first argument optional via a decorator like
@lazy and make self.stream or whatever a property that raises an
informative message. Not sure how to get Mapping(func) to work without
passing as a kwarg though.

On Wed, Jul 27, 2016, 3:44 PM dmitriy-serdyuk [email protected]
wrote:

I propose to add a syntactic sugar to easier manipulate streams, we could
use some infix operator to combine transformers (like | or >). | would
look like linux piping:

stream = Mapping(Flatten(DataStream.default_stream(dataset)), my_func)

becomes

stream = dataset | DataStream.default_stream() | Flatten() | Mapping(my_func)

Less parenthesis, transformers are applied in the direct order.

Implementation looks very straightforward.

What do you guys think?


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#358, or mute the thread
https://github.com/notifications/unsubscribe-auth/AADrLqYtcDY_mpH1sdfvNddbl7RC4b3eks5qZ7UXgaJpZM4JWjd0
.

@dmitriy-serdyuk
Copy link
Contributor Author

@nouiz , it is a bit-wise or operator which can be overridden with __or__.

I think about implementing a proxy object in the case if transformer was created without the datastream argument. The object should store a link to the transformer and create a "real" one as soon as the pipe is constructed.

@dmitriy-serdyuk
Copy link
Contributor Author

Why not make it always optional? It shouldn't break others' code.

When an outside user decides to use piping syntax, she just rewrites her custom transformers.

@dmitriy-serdyuk dmitriy-serdyuk linked a pull request Jul 28, 2016 that will close this issue
@dmitriy-serdyuk
Copy link
Contributor Author

What if create a class factory method for lazy constructor? Like Mapping.lazy(func) or Mapping.pipe(func).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants