allow notebook env

[olivergiles]

Added the possibiltiy to use functions define in the notebook inside the tests. You add an additional notebook_env=False and then you can pass functions into the tests!

[gmanchon]

good idea to use dill, I suggest to systematically use dill and only fallback to pickle, WDYT ?

[gmanchon]

rather than introducing a second use case for the package, why not systematically use dill so that lambdas are supported and the content writers do not need to wonder what they will be saving (which in time might be a source of bugs) in order to deal with that additional notebook_env parameter ?

if we keep a single use case, then the code only needs to deal with legacy content :

save :

• save as dill file
• remove pickle file if exists (no risk when the students run the code)

load :

• load from dill file
• fallback to load from pickle file

... then we can amend the package in the future when no content uses pickle anymore

WDYT ?

[olivergiles]

Dill can load any file created with pickle so I don't think we need the fallback. Are you suggesting we abstract away the saving of the notebook env?

[gmanchon]

I miss read the dump_session function 🙌

it is unclear to me why we need to save the state of the student __main__ module : should not the content of the tested functions be independent from the state ?

[olivergiles]

If you do this

custom_ones = lambda x: np.ones(x)

you could not then pass this function alone

[gmanchon]

I do not see the issue (I expect the code loading the dill files to deal with the package imports)

[olivergiles]

I think it is specific to how pytest operates

[gmanchon]

ok I get the issue with the numpy import when using the ChallengeResult class. Even if I manually import numpy the np.ones in the lambda fails to find numpy

stackoverflow also points to dump_session

[gmanchon]

@ogiles1999 I found the ugliest hack to have pytest work with the numpy reference in the lambda 🙈

test_solvers.py :

from nbresult import ChallengeResultTestCase
import numpy as np

import __main__
__main__.np = np

class TestSolvers(ChallengeResultTestCase):
    def test_custom_ones(self):
        self.assertEqual([[1.0], [1.0], [1.0]], self.result.custom_ones([3, 1]).tolist())

Test Dill.ipynb :

import numpy as np

custom_ones = lambda x: np.ones(x)

custom_ones([3, 1])

from nbresult import ChallengeResult

result = ChallengeResult(
    'solvers',
    custom_ones=custom_ones
)
result.write()
print(result.check())

[olivergiles]

Haha that is very hacky 😓

[gmanchon]

One part of me is not proud 😁