Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement merging with empty Welford objects #2

Open
lschmelzeisen opened this issue Jul 31, 2021 · 0 comments
Open

Implement merging with empty Welford objects #2

lschmelzeisen opened this issue Jul 31, 2021 · 0 comments

Comments

@lschmelzeisen
Copy link

Hey, just found this library and want to say that it covers exactly my use case. Thanks!

However, I found a minor annoyance in the API. If I have multiple Welford objects w1, w2, and so on, and want to merge them into one

v_merged = None
for w in [w1, w2, ...]:
    w_merged.merge(w)

will give the following error:

Traceback (most recent call last):
  File "/home/lschmelzeisen/test/test.py", line 16, in <module>
    w_merged.merge(w)
  File "/home/lschmelzeisen/test/.venv/lib64/python3.9/site-packages/welford/welford.py", line 139, in merge
    delta = self.__m - other.__m
TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

Causing me to have to write a workaround like the following:

import numpy as np
from welford import Welford

w1 = Welford()
w1.add(np.array([1.0]))
w1.add(np.array([2.0]))
w1.add(np.array([3.0]))

w2 = Welford()
w2.add(np.array([4.0]))
w2.add(np.array([5.0]))
w2.add(np.array([6.0]))

w_merged = None
for w in [w1, w2]:
    if w_merged is None:
        w_merged = w
    else:
        w_merged.merge(w)

print(w_merged.mean)
print(w_merged.var_s)
print(w_merged.var_p)

(In my actual use case I can't do something simple like w_merged = w1 at the start because I need to have the variable in a larger scope.)

Could a check be added that tests if w_merged does not contain any data and then simply overwrites it with the values of w instead of trying to add them?

While I'm talking about this, could a check be added so that something like w1.add(1.0) works instead of manually needing to wrap the value on input in something like w1.add(np.array([1.0]))?

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant