Task/update fork method preserve owner parameter

[shaikh-ma]

As informed by Crunch team, the special properties of the "Personal" folder are about to be replaced with a regular project folder that behaves like any other Crunch folder, at some point approaching the end of CY 2024.

Recently added functionality to create a fork in a target location will still allow to use preserve_owner and hence might lead to creating the fork in either the "Personal" folder or keeping the source dataset's location.

This MR proactively attempts to push towards the best-practice approach here by updating the fork method.
The behaviour of the preserve_owner parameter has been updated as described below:

• If preserve_owner=False and a project is not provided, we want to raise and indicate that an explicit project location is expected.
• If preserve_owner=False and a project is provided, this is consistent with what we want to acheive and the code can be executed as is.
• If preserve_owner=True and a project is provided, raise and indicate that this combination is ambiguous, only one of the two can be used AND that preserve_owner will be deprecated soon.
• Regardless of the value given to preserve_owner, log a DeprecationWarning that the preserve_owner parameter will be removed soon in favour of providing a project or not (in which case the behavior will be as it is currently if preserve_owner=True and project=None).

Project passed	Preserve_owner = True
Yes
No

Project passed	Preserve_owner = False
Yes
No

[alexbuchhammer]

Hey @jjdelc - what's your opinion on the suggested changes? We do not yet want to get rid of the preserve_owner parameter to protect users from breaking scripts. Let us know!

Cheers
Alex

[jjdelc]

This looks in the right direction, but I worry that the change may come sooner than expected. I will check with @rchacon1 on what the status of this is in the backend.

[rchacon1]

A project will always be required in the future state as far the API is concerned.

[jamesrkg]

The idea here is, as an abstraction over the API, the user can either give a project or not, and if they don't, then the fork will be created in the same location as the dataset being forked. What happens below the signature of the method isn't something we want users to think about. In this way, it will be possible for a user to say ds.fork(..., project=None), even though that's not what is implemented internally.

[rchacon1]

if you want to keep the preserve_owner abstraction you can future proof it by sending resource.body.owner as the project in the payload.

[jamesrkg]

Thanks, yep that's what happening here:

scrunch/scrunch/datasets.py

Lines 2420 to 2427 in 930905a

	if preserve_owner:
	if project:
	raise ValueError(
	"Cannot pass 'project' or 'owner' when preserve_owner=True."
	)
	else:
	# Create fork in source dataset path.
	body["project"] = self.resource.body.owner

[rchacon1]

you can replace body["owner"] with body["project"] to future proof this. same goes for any other place in this function where owner is added to payload.

[shaikh-ma]

Done

[rchacon1]

[shaikh-ma]

Hi @rchacon1 @jjdelc,

Please could you push these changes in a tag release. 🙇

[rchacon1]

version 0.18.6 published to pypi