filesize remains pending after marie accepts the federated shared file. #189

navid-shokri · 2023-06-09T11:52:24Z

when Marie accepts the shared folder/file the pending file remains beside it.

thepeak99 · 2023-06-13T11:26:36Z

Blocked by #194 and #193

navid-shokri · 2023-06-14T10:57:24Z

ℹ️
So, I could not reproduce the #194 , #193 but when I shared my file with the recipient group and group member accepted the share I was facing with this error when I was trying to open the file. I guess it can help you.

{"reqId":"5a7AEXWHMwgAGAe99L4R","level":3,"time":"2023-06-14T10:53:24+00:00","remoteAddr":"172.18.0.6","user":"marie","app":"PHP","method":"POST","url":"\/index.php\/apps\/files_sharing\/api\/externalShares","message":"Undefined index: password at \/var\/www\/html\/apps\/files_sharing\/lib\/External\/Storage.php#92"}

Also, the pending flag is shown just for the folder and the files can indicate their size as you can see below image:

yasharpm · 2023-06-15T11:56:07Z

After researching #68 it is clear to me that the file size remains pending if the webda PROPFIND call to the share provider fails. So this is most likely same as #169

yasharpm · 2023-06-22T09:38:48Z

After a debugging session with @thepeak99 here is our findings:

This problem exists independent from our code. There is something wrong with our dev environment setup but we need to know what it is in order to make sure it won't happen in a real world setup and also to be able to test our code properly.
All webdav calls result in connection refused error.
But the provider actually receives the call and returns the expected result.
If we make the same call that OC makes to webdav using curl, we get the response no problem.
If the call is modified to use http instead of https, it works fine.

The piece of code we were specifically starting at is this.

yasharpm · 2023-06-22T11:06:52Z

Our setup has no problem with webdav on NextCloud. I am comparing what is different between the 2 regarding the use of webdav:

OC uses php version 7.4 but NC uses php version 8.0 . Our dev environment (on the oc-opencloudmesh-testing branch that we are currently using) uses php version 7.4 for OC and php version 8.0 for NC.
OC uses guzzlehttp/guzzle version ^7.5 in composer.json and version 7.5.0 in composer.lock. NC uses guzzlehttp/guzzle version ^7.5.0 in composer.json and ^6.5.8 || ^7.4.5 in composer.lock
The code that gets the shared file content is identical for both OC and NC.
The ClientService that both OC and NC use for opening connections are functionally identical. Both of them make use of a CertificateManager which comparing the code of the two under the path lib/private/Security/CertificateManager.php seems like they basically do the same thing.

yasharpm · 2023-06-22T11:37:30Z

I tried to run OC on php 8.0 by changing the docker file header to FROM apache-php-8.0 but I got this error:

This version of ownCloud is not compatible with PHP 8.0
You are currently running PHP 8.1.2-1ubuntu2.11.

yasharpm · 2023-06-22T11:40:33Z

OwnCloud installation manual states here:

Ubuntu 22.04 only provides PHP 8.x which is currently not supported by ownCloud. Therefore PHP 7.4 is installed via PPA.

The greatest difference between OC and NC that I detected about the webdav was the php version. :-(

yasharpm · 2023-06-22T12:08:17Z

Trying to inspect the source of connection refused exception in the guzzlehttp library.

The stack trace for connection refused up to the point the stack reaches OC is as follows:


#0 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/Handler\\\/Proxy.php(48): GuzzleHttp\\\\Handler\\\\StreamHandler->__invoke()
#1 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/PrepareBodyMiddleware.php(35): GuzzleHttp\\\\Handler\\\\Proxy::GuzzleHttp\\\\Handler\\\\{closure}(*** sensitive parameters replaced ***)
#2 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/Middleware.php(31): GuzzleHttp\\\\PrepareBodyMiddleware->__invoke()
#3 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/RedirectMiddleware.php(71): GuzzleHttp\\\\Middleware::GuzzleHttp\\\\{closure}(*** sensitive parameters replaced ***)
#4 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/Middleware.php(63): GuzzleHttp\\\\RedirectMiddleware->__invoke()
#5 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/HandlerStack.php(75): GuzzleHttp\\\\Middleware::GuzzleHttp\\\\{closure}(*** sensitive parameters replaced ***)
#6 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/Client.php(331): GuzzleHttp\\\\HandlerStack->__invoke()
#7 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/Client.php(168): GuzzleHttp\\\\Client->transfer()
#8 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/Client.php(187): GuzzleHttp\\\\Client->requestAsync()
#9 \\\/var\\\/www\\\/html\\\/lib\\\/composer\\\/guzzlehttp\\\/guzzle\\\/src\\\/ClientTrait.php(44): GuzzleHttp\\\\Client->request()
#10 \\\/var\\\/www\\\/html\\\/lib\\\/private\\\/Http\\\/Client\\\/Client.php(184): GuzzleHttp\\\\Client->get()

yasharpm · 2023-06-22T12:13:17Z

Local path inside the docker image to the guzzlehttp source is:
/var/www/html/lib/composer/guzzlehttp/guzzle/src

The github repository for guzzlehttp version 7.5.1 that is present in composer.lock of the container is:
https://github.com/guzzle/guzzle/tree/b964ca597e86b752cd994f27293e9fa6b6a95ed9/src

yasharpm · 2023-06-22T12:55:39Z

The first trace Handler/Proxy.php(48) is ambiguous as it is a callable call to a closure which we don't know what it is. I put a log on the call that creates the closure that makes the close on Proxy.php(47) to print the stack trace and figure out what is being called. It is quite a wraps happening but from the logs, I can tell the first call in the queue is failing so fingers crossed the failing first call is calling wrap here:

called wrap here: 
#0 /var/www/html/lib/composer/guzzlehttp/guzzle/src/Utils.php(102): GuzzleHttp\\Handler\\Proxy::wrapStreaming()\n
#1 /var/www/html/lib/composer/guzzlehttp/guzzle/src/HandlerStack.php(49): GuzzleHttp\\Utils::chooseHandler()\n
#2 /var/www/html/lib/composer/guzzlehttp/guzzle/src/Client.php(60): GuzzleHttp\\HandlerStack::create()\n
#3 /var/www/html/lib/private/Http/Client/ClientService.php(56): GuzzleHttp\\Client->__construct()\n
#4 /var/www/html/apps/federatedfilesharing/lib/DiscoveryManager.php(57): OC\\Http\\Client\\ClientService->newClient()\n
#5 /var/www/html/apps/federatedfilesharing/lib/AppInfo/Application.php(210): OCA\\FederatedFileSharing\\DiscoveryManager->__construct()\n
#6 /var/www/html/apps/federatedfilesharing/lib/AppInfo/Application.php(195): OCA\\FederatedFileSharing\\AppInfo\\Application->initFederatedShareProvider()\n
#7 /var/www/html/apps/federatedfilesharing/lib/AppInfo/Application.php(281): OCA\\FederatedFileSharing\\AppInfo\\Application->getFederatedShareProvider()\n
#8 /var/www/html/apps/federatedfilesharing/appinfo/app.php(41): OCA\\FederatedFileSharing\\AppInfo\\Application->registerListeners()\n
#9 /var/www/html/lib/private/legacy/app.php(253): require_once('/var/www/html/a...')\n
#10 /var/www/html/lib/private/legacy/app.php(192): OC_App::requireAppFile()\n
#11 /var/www/html/lib/private/legacy/app.php(125): OC_App::loadApp()\n
#12 /var/www/html/lib/private/legacy/util.php(148): OC_App::loadApps()\n
#13 /var/www/html/lib/private/Files/Filesystem.php(262): OC_Util::setupFS()\n
#14 /var/www/html/apps/oc-opencloudmesh/opencloudmesh/lib/Files_Sharing/Hooks.php(52): OC\\Files\\Filesystem::getMountManager()\n
#15 /var/www/html/apps/oc-opencloudmesh/opencloudmesh/lib/AppInfo/Application.php(210): OCA\\OpenCloudMesh\\Files_Sharing\\Hooks->__construct()\n
#16 /var/www/html/lib/composer/pimple/pimple/src/Pimple/Container.php(122): OCA\\OpenCloudMesh\\AppInfo\\Application->OCA\\OpenCloudMesh\\AppInfo\\{closure}()\n
#17 /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php(107): Pimple\\Container->offsetGet()\n
#18 /var/www/html/apps/oc-opencloudmesh/opencloudmesh/lib/AppInfo/Application.php(344): OC\\AppFramework\\Utility\\SimpleContainer->query()\n
#19 /var/www/html/apps/oc-opencloudmesh/opencloudmesh/appinfo/app.php(27): OCA\\OpenCloudMesh\\AppInfo\\Application->registerEvents()\n
#20 /var/www/html/lib/private/legacy/app.php(253): require_once('/var/www/html/a...')\n
#21 /var/www/html/lib/private/legacy/app.php(192): OC_App::requireAppFile()\n
#22 /var/www/html/lib/private/legacy/app.php(125): OC_App::loadApp()\n
#23 /var/www/html/lib/base.php(883): OC_App::loadApps()\n
#24 /var/www/html/index.php(54): OC::handleRequest()\n
#25 {main}

michielbdejong · 2023-06-22T13:24:59Z

I could not reproduce this with latest dev-stock main branch.

Be careful! There is the word 'pending' in the 'file size' column! Don't be distracted by that :)

Please reopen if you have tested this on the latest dev-stock main branch and you still see it happening there

michielbdejong · 2023-06-22T14:31:56Z

ah sorry, after reading the comments I realised that this issue is actually about the filesize! :)
Confirmed that this is still happening, both for ocm-to-user and ocm-to-group, in dev-stock main branch:

yasharpm · 2023-06-28T10:32:19Z

The checkRedirect function in the RedirectMiddleware class seems to be the source of our problems. I could verify that the receiver's get call to the provider is successful and gets a response. This response then gets further processed by the redirect middle and somehow it ends up with a connection refused error. The thread enters the function but it never leaves it!

Here is the function code:
https://github.com/guzzle/guzzle/blob/b964ca597e86b752cd994f27293e9fa6b6a95ed9/src/RedirectMiddleware.php#L80

yasharpm · 2023-06-28T11:07:49Z

I was wrong. The thread safely exits the redirect middleware.

yasharpm · 2023-06-28T15:33:18Z

My best guess atm is that the await() function here throws the exception.

yasharpm · 2023-06-28T15:53:34Z

My guess seems to be correct. In which case the await call on the FulfilledPromise class throws an exception the second(?) time the code tries to send a request.

FulfilledPromise.php is located at /var/www/html/lib/composer/guzzlehttp/promises/src on the local server and https://github.com/guzzle/promises/blob/2.0/src/FulfilledPromise.php on Github.

yasharpm · 2023-06-28T16:25:28Z

A FullfilledPromise does nothing in the wait function. It already has the value and just returns it.

yasharpm · 2023-06-28T16:49:51Z

I confirmed. Upon refreshing the page to get the received file content. The first call is GET https://oc1.docker/ocm-provider/ which is successful and the second call is GET https://oc1.docker/public.php/webdav/ which returns a RejectedPromise and throws the exception on wait.

yasharpm · 2023-06-29T14:13:33Z

Finally closed down on the culprit:
https://github.com/guzzle/guzzle/blob/b964ca597e86b752cd994f27293e9fa6b6a95ed9/src/Handler/StreamHandler.php#L328

yasharpm · 2023-06-29T14:18:55Z

This means that the root of the issue is:

$context = [
	"http" =>
		[
			"method" => "GET",
			"header" => "Authorization: Basic UzZxd0MwdDdPYVpNRU12Og==\\r\\nHost: oc1.docker\\r\\nUser-Agent: ownCloud Server Crawler\\r\\nContent-Length: 0\\r\\nConnection: close",
			"protocol_version" => "1.1",
			"ignore_errors" => true,
			"follow_location" => 0
		],
	"ssl" =>
		[
			"peer_name" => "oc1.docker",
			"cafile" => "\\/var\\/www\\/html\\/data\\/files_external\\/rootcerts.crt",
			"verify_peer" => true,
			"verify_peer_name" => true,
			"allow_self_signed" => false
		]
];
$params = [];
$contextResource = \stream_context_create($context, $params);
return @\fopen('https://oc1.docker/public.php/webdav/', 'r', false, $contextResource);

This returns false while it shouldn't. Going to verify.

yasharpm · 2023-06-29T15:27:46Z

Progress!
Setting cafile config to point to /etc/ssl/certs/ca-certificates.crt fixes the issue. Next is to figure out where the current config comes from and how to change it.

yasharpm · 2023-06-29T15:31:10Z

So adding

$context['ssl']['cafile'] = '/etc/ssl/certs/ca-certificates.crt';

to line 318 of /Handler/StreamHandler.php fixes the issue.

yasharpm · 2023-06-29T16:11:54Z

Here is where in OC code that the ca file option is being set:

		if ($this->certificateManager->listCertificates() !== []) {
			$options[RequestOptions::VERIFY] = $this->certificateManager->getAbsoluteBundlePath();
		} else {
			// If the instance is not yet setup we need to use the static path as
			// $this->certificateManager->getAbsoluteBundlePath() tries to instantiate
			// a view
			if ($this->config->getSystemValue('installed', false) && !\OCP\Util::needUpgrade()) {
				$options[RequestOptions::VERIFY] = $this->certificateManager->getAbsoluteBundlePath(null);
			} else {
				$options[RequestOptions::VERIFY] = \OC::$SERVERROOT . '/resources/config/ca-bundle.crt';
			}
		}

yasharpm · 2023-07-04T11:34:06Z

The certificate manager is located at lib/private/Security/CertificateManager.php. It gathers certificates from the /files_external/uploads/ folder and /var/www/html/resources/config/ca-bundle.crt and /files_external/rootcerts.crt files.

yasharpm · 2023-07-10T16:34:48Z

I fixed it by appending the contents of /etc/ssl/certs/ca-certificates.crt to /var/www/html/resources/config/ca-bundle.crt inside the docker image on dev-stock in this commit.

michielbdejong · 2023-09-15T17:57:05Z

Apparently this is happening in production now:

yasharpm · 2023-09-21T21:19:49Z

I was not able to reproduce this on our test environment. However what got my attention is that apparently the share provider is not actually sharing.
From my understanding, the left image is a screenshot from the provider and the right one is from the receiver. But this is how the share provider should see their shared files:

Notice the "shared" label next to the share icon.
This symptom is probably has been caused by something far different from what we resolved on this issue. My guess is that our SRAM controller is calling the wrong functions to create a share. So the OCM notifications are being send to the receiver but share rows are not being created on the provider side. This is highly unusual. Maybe is it another case of master slave database architecture that SURF is using on their OwnCloud servers.

@michielbdejong the following questions are vital to resolving this issue:

The share label the receiver side says "tomw@...". Is it a user share?
Is the share created via the SRAM?
I could not reproduce this error. Is it possible to wait for SURF to provide their test environment to us first?

michielbdejong · 2023-09-22T05:35:08Z

Thanks! That helps, I'll talk to Tom!

michielbdejong · 2023-09-25T14:16:04Z

The share label the receiver side says "tomw@...". Is it a user share?

That would be the sender, right? In our dev env it would usually say "[email protected]" there when Maria views it, right?

yasharpm · 2023-09-26T06:42:31Z

Correct!

thepeak99 changed the title ~~the pendding flag remains after marie accept th federated shared file.~~ the pending flag remains after marie accept the federated shared file. Jun 12, 2023

thepeak99 added the milestone 1.1 label Jun 13, 2023

thepeak99 assigned yasharpm Jun 14, 2023

michielbdejong closed this as not planned Won't fix, can't repro, duplicate, stale Jun 22, 2023

michielbdejong changed the title ~~the pending flag remains after marie accept the federated shared file.~~ filesize remains pending after marie accepts the federated shared file. Jun 22, 2023

michielbdejong reopened this Jun 22, 2023

michielbdejong mentioned this issue Jun 22, 2023

error when trying to open shared file #169

Closed

yasharpm closed this as completed Jul 10, 2023

This was referenced Aug 14, 2023

Download of files gives HTTP 500 sciencemesh/nc-sciencemesh#34

Closed

Cannot open text file that was shared from remote server owncloud/core#40594

Closed

michielbdejong reopened this Sep 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

filesize remains pending after marie accepts the federated shared file. #189

filesize remains pending after marie accepts the federated shared file. #189

navid-shokri commented Jun 9, 2023

thepeak99 commented Jun 13, 2023

navid-shokri commented Jun 14, 2023 •

edited

Loading

yasharpm commented Jun 15, 2023

yasharpm commented Jun 22, 2023

yasharpm commented Jun 22, 2023

yasharpm commented Jun 22, 2023

yasharpm commented Jun 22, 2023

yasharpm commented Jun 22, 2023

yasharpm commented Jun 22, 2023

yasharpm commented Jun 22, 2023

michielbdejong commented Jun 22, 2023

michielbdejong commented Jun 22, 2023

yasharpm commented Jun 28, 2023

yasharpm commented Jun 28, 2023

yasharpm commented Jun 28, 2023

yasharpm commented Jun 28, 2023

yasharpm commented Jun 28, 2023

yasharpm commented Jun 28, 2023

yasharpm commented Jun 29, 2023

yasharpm commented Jun 29, 2023 •

edited

Loading

yasharpm commented Jun 29, 2023

yasharpm commented Jun 29, 2023

yasharpm commented Jun 29, 2023

yasharpm commented Jul 4, 2023

yasharpm commented Jul 10, 2023

michielbdejong commented Sep 15, 2023

yasharpm commented Sep 21, 2023

michielbdejong commented Sep 22, 2023

michielbdejong commented Sep 25, 2023

yasharpm commented Sep 26, 2023

filesize remains pending after marie accepts the federated shared file. #189

filesize remains pending after marie accepts the federated shared file. #189

Comments

navid-shokri commented Jun 9, 2023

thepeak99 commented Jun 13, 2023

navid-shokri commented Jun 14, 2023 • edited Loading

yasharpm commented Jun 15, 2023

yasharpm commented Jun 22, 2023

yasharpm commented Jun 22, 2023

yasharpm commented Jun 22, 2023

yasharpm commented Jun 22, 2023

yasharpm commented Jun 22, 2023

yasharpm commented Jun 22, 2023

yasharpm commented Jun 22, 2023

michielbdejong commented Jun 22, 2023

michielbdejong commented Jun 22, 2023

yasharpm commented Jun 28, 2023

yasharpm commented Jun 28, 2023

yasharpm commented Jun 28, 2023

yasharpm commented Jun 28, 2023

yasharpm commented Jun 28, 2023

yasharpm commented Jun 28, 2023

yasharpm commented Jun 29, 2023

yasharpm commented Jun 29, 2023 • edited Loading

yasharpm commented Jun 29, 2023

yasharpm commented Jun 29, 2023

yasharpm commented Jun 29, 2023

yasharpm commented Jul 4, 2023

yasharpm commented Jul 10, 2023

michielbdejong commented Sep 15, 2023

yasharpm commented Sep 21, 2023

michielbdejong commented Sep 22, 2023

michielbdejong commented Sep 25, 2023

yasharpm commented Sep 26, 2023

navid-shokri commented Jun 14, 2023 •

edited

Loading

yasharpm commented Jun 29, 2023 •

edited

Loading