Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Avoid extra stream copy to memory/temp if we can determine the size #38407

Closed
wants to merge 2 commits into from

Conversation

juliusknorr
Copy link
Member

@juliusknorr juliusknorr commented May 23, 2023

On my local setup the stream was seekable and reported the proper size, so I think we can actually avoid the extra effort to copy the stream beginning to memory to determin which upload mechanism to use for cases where a size is reported. The fallback is still in place if the stream reports 0 or null as the size.

Introduced with https://github.com/nextcloud/server/pull/27877/files#diff-030e38a9e621019981acf91654c1d8d3a12ab3150dbd9fcc4c280479b8c853dbR144-R148

Test case needed adjustment since the implementation of a NonSeekableStream falsely reported to be seekable on its metadata.

@juliusknorr juliusknorr force-pushed the bugfix/avoid-extra-stream-copy branch from 0277e70 to a2b37a8 Compare May 23, 2023 14:44
@juliusknorr juliusknorr changed the title fix: Avoid extra stream copy to memory if we can determine the size fix: Avoid extra stream copy to memory/temp if we can determine the size May 23, 2023
@juliusknorr juliusknorr marked this pull request as ready for review May 23, 2023 18:19
@juliusknorr juliusknorr added 3. to review Waiting for reviews and removed 2. developing Work in progress labels May 23, 2023
$buffer->seek(0);
if ($buffer->getSize() < $this->putSizeLimit) {
$streamSize = $psrStream->getSize();
if ($psrStream->isSeekable() && (int)$streamSize !== 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find it a bit weird to cast here and not above.
It should work though because $buffer->getSize will overwrite it later.

Copy link
Contributor

@kesselb kesselb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good stuff 👍

@kesselb kesselb requested a review from artonge May 25, 2023 21:12
Copy link
Member Author

@juliusknorr juliusknorr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking as this has been reported to still take quite some round trips on some setup

@juliusknorr juliusknorr removed the 3. to review Waiting for reviews label May 30, 2023
@szaimen szaimen added the 2. developing Work in progress label Jun 12, 2023
@juliusknorr
Copy link
Member Author

#38763 might be the reason why this wasn't working as expected on the mentioned setup

@tsdicloud
Copy link
Contributor

tsdicloud commented Jul 6, 2023

Man, this would really help. Where did the stream behavior change? Is there a chance that I bakcport it already for our Next (V25) release or later - as MagentaCLOUD will hopefully move fast to V27 in fall?

If I understand it right the behavior change comes from #34232, which is not merged to stable 24. I assume if I backport both, this could work, right?

Thanks.

@skjnldsv skjnldsv mentioned this pull request Nov 1, 2023
This was referenced Nov 6, 2023
@blizzz blizzz mentioned this pull request Nov 14, 2023
This was referenced Mar 12, 2024
This was referenced Mar 20, 2024
@skjnldsv skjnldsv mentioned this pull request Mar 28, 2024
81 tasks
@skjnldsv skjnldsv modified the milestones: Nextcloud 29, Nextcloud 30 Mar 28, 2024
This was referenced Jul 30, 2024
This was referenced Aug 5, 2024
@skjnldsv skjnldsv mentioned this pull request Aug 13, 2024
@skjnldsv skjnldsv closed this Aug 14, 2024
@skjnldsv skjnldsv removed this from the Nextcloud 31 milestone Aug 14, 2024
@skjnldsv skjnldsv deleted the bugfix/avoid-extra-stream-copy branch August 30, 2024 07:30
@juliusknorr
Copy link
Member Author

#49352 is going into a similar direction and was merged now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants