Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failure for test_multiply in pytorch-latest CI job #65

Closed
rgommers opened this issue Nov 9, 2023 · 3 comments
Closed

Test failure for test_multiply in pytorch-latest CI job #65

rgommers opened this issue Nov 9, 2023 · 3 comments

Comments

@rgommers
Copy link
Member

rgommers commented Nov 9, 2023

Observed in gh-59 and gh-61 with the same test case:
array_api_tests/test_operators_and_elementwise_functions.py::test_multiply[__imul__(x1, x2)]
just with different values. The reproducers:

FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_multiply[__imul__(x1, x2)] - AssertionError: x1=(2+67138304j), but should be roughly (x1 * x2)=67138308j [__imul__()]
  x1=(9638+9638j), x2=(3483+3483j)
assert False
 +  where False = isclose(2.0, 0.0, 3.4028234663852886e+38)
 +    where 2.0 = (2+67138304j).real
 +    and   0.0 = 67138308j.real
Falsifying example: test_multiply(
    ctx=BinaryParamContext(<__imul__(x1, x2)>),
    data=data(...),
)
Draw 1 (x1): tensor(9638.+9638.j)
Draw 2 (x2): tensor(3483.+3483.j)

You can reproduce this example by temporarily adding @reproduce_failure('6.86.2', b'AXicY2eAA9Vl2JgMvLORmQA8GwLu') as a decorator on your test case

and

FAILED array_api_tests/test_operators_and_elementwise_functions.py::test_multiply[__imul__(x1, x2)] - AssertionError: x1=(-5+92034536j), but should be roughly (x1 * x2)=(-3+92034537j) [__imul__()]
  x1=(15339089+15339090j), x2=(3+3j)
assert False
 +  where False = isclose(-5.0, -3.0, 3.4028234663852886e+38)
 +    where -5.0 = (-5+92034536j).real
 +    and   -3.0 = (-3+92034537j).real
Falsifying example: test_multiply(
    ctx=BinaryParamContext(<__imul__(x1, x2)>),
    data=data(...),
)
Draw 1 (x1): tensor(15339089.+15339090.j)
Draw 2 (x2): tensor(3.+3.j)

You can reproduce this example by temporarily adding @reproduce_failure('6.86.1', b'AXicY2eAgVd8gQhmEAMCMCMzAVPvAqE=') as a decorator on your test case

A comment on gh-59 has a full traceback.

@asmeurer
Copy link
Member

asmeurer commented Nov 9, 2023

Probably the same sort of issue discussed at data-apis/array-api-tests#190. The test suite isn't checking complex number closeness correctly.

@asmeurer
Copy link
Member

asmeurer commented Nov 9, 2023

We're going to need to add this to the xfails (even if the test failure were legitimate, we can't do anything about operator methods). I think it's set up right now so it won't release unless the tests pass, so this has to be done.

@asmeurer
Copy link
Member

asmeurer commented Mar 8, 2024

This was fixed (CI is now passing).

@asmeurer asmeurer closed this as completed Mar 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants