Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Statelful nodes do not work with when batch size is set to all #4

Open
daniyall opened this issue Feb 19, 2018 · 0 comments
Open

Statelful nodes do not work with when batch size is set to all #4

daniyall opened this issue Feb 19, 2018 · 0 comments

Comments

@daniyall
Copy link
Owner

When batch_size is set to BATCH_SIZE_ALL on a stateful node in multithreaded execution, some data is dropped and input buffers are not combined before being passed to the node. For example,

gen = Generate("gen", size=5)
double = Double("double")
printer = Printer("printer", batch_size=Node.BATCH_SIZE_ALL) 
p = Pipeline(gen | double | printer, n_threads=4)

Expected result is that the output buffer from all double nodes is combined before being passed to the printer and the printer will print [0, 2, 4, 6, 8] (order may be different). However, actual result is different whenever run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant