NC29:Fulltextsearch live crash

[PesekLasy]

Hello, 2 days ago my service for fultextsearch live crash and since this day does not start. When i run occ fulltextsearch:live by hand
./occ fulltextsearch:live
is returning error to stdout:

Error: Typed property OC\FullTextSearch\Model\IndexDocument::$access must not be accessed before initialization in /home/ncloud/lib/private/FullTextSearch/Model/IndexDocument.php:775
Stack trace:
#0 [internal function]: OC\FullTextSearch\Model\IndexDocument->jsonSerialize()
#1 /home/ncloud/apps/files_fulltextsearch/vendor/artificial-owl/my-small-php-tools/lib/Traits/Nextcloud/nc22/TNC22Logger.php(192): json_encode(Array)
#2 /home/ncloud/apps/files_fulltextsearch/vendor/artificial-owl/my-small-php-tools/lib/Traits/Nextcloud/nc22/TNC22Logger.php(177): OCA\Files_FullTextSearch\Provider\FilesProvider->log(0, '[debug] updateD...', false, Array)
#3 /home/ncloud/apps/files_fulltextsearch/lib/Provider/FilesProvider.php(315): OCA\Files_FullTextSearch\Provider\FilesProvider->debug('[debug] updateD...', Array)
#4 /home/ncloud/apps/fulltextsearch/lib/Service/IndexService.php(409): OCA\Files_FullTextSearch\Provider\FilesProvider->updateDocument(Object(OCA\FullTextSearch\Model\Index))
#5 /home/ncloud/apps/fulltextsearch/lib/Command/Live.php(313): OCA\FullTextSearch\Service\IndexService->updateDocument(Object(OCA\FullTextSearch_Elasticsearch\Platform\ElasticSearchPlatform), Object(OCA\Files_FullTextSearch\Provider\FilesProvider), Object(OCA\FullTextSearch\Model\Index))
#6 /home/ncloud/apps/fulltextsearch/lib/Command/Live.php(262): OCA\FullTextSearch\Command\Live->liveCycle()
#7 /home/ncloud/3rdparty/symfony/console/Command/Command.php(298): OCA\FullTextSearch\Command\Live->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /home/ncloud/core/Command/Base.php(177): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /home/ncloud/3rdparty/symfony/console/Application.php(1040): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#10 /home/ncloud/3rdparty/symfony/console/Application.php(301): Symfony\Component\Console\Application->doRunCommand(Object(OCA\FullTextSearch\Command\Live), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#11 /home/ncloud/3rdparty/symfony/console/Application.php(171): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /home/ncloud/lib/private/Console/Application.php(213): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /home/ncloud/console.php(102): OC\Console\Application->run()
#14 /home/ncloud/occ(11): require_once('/home/ncloud...')
#15 {main}

NC version 29.0.2

[PesekLasy]

Hello,
I was fix this issue with reset index and then is possible run again liveindex.

[vasyugan]

I'm seeing this error on two instances immediately after running occ fulltextsearch:reset and freshly reindexing everything.

Here is the console output:

Memory: 50 MB                                                                                                                                        
┌─ Indexing  ────                                                          
│ Action: indexing
│ Provider: Files                Account: **                                                                                                          
│ Document:                                                                                                                                           
│ Info:                                                                                                                                               
│ Title:                                                                   
│ Content size:                                                                                                                                       
└──                                                                                                                                                   
┌─ Results ────                                                                                                                                       
│ Result:      0/0                                                                                                                                    
│ Index:                                                                                                                                              
│ Status:                                                                                                                                             
│ Message:                                                                                                                                            
│                                                                                                                                                     
│                                                                                                                                                     
└──                                                                                                                                                   
┌─ Errors ────                                                             
│ Error:    144/144                                                        
│ Index: files:9176428              
│ Exception: OCA\FullTextSearch_Elasticsearch\Vendor\Elastic\Elasticsearch\Exception\ClientResponseException                                          
│ Message: unknown error             
│                                    
│                                    
└──                                  
## x:first result ## c/v:prec/next result ## b:last result                 
## f:first error ## h/j:prec/next error ## d:delete error ## l:last error  
## q:quit ## p:pause                 
An unhandled exception has been thrown:                                    
Error: Typed property OC\FullTextSearch\Model\IndexDocument::$access must not be accessed before initialization in /var/www/nextcloud/lib/private/FullTextSearch/Model/IndexDocument.php:775                                     
Stack trace:      
#0 [internal function]: OC\FullTextSearch\Model\IndexDocument->jsonSerialize()                                                                        
#1 /var/www/nextcloud/apps/files_fulltextsearch/vendor/artificial-owl/my-small-php-tools/lib/Traits/Nextcloud/nc22/TNC22Logger.php(192): json_encode()
#2 /var/www/nextcloud/apps/files_fulltextsearch/vendor/artificial-owl/my-small-php-tools/lib/Traits/Nextcloud/nc22/TNC22Logger.php(177): OCA\Files_FullTextSearch\Provider\FilesProvider->log()                                  
#3 /var/www/nextcloud/apps/files_fulltextsearch/lib/Provider/FilesProvider.php(315): OCA\Files_FullTextSearch\Provider\FilesProvider->debug()         
#4 /var/www/nextcloud/apps/fulltextsearch/lib/Service/IndexService.php(409): OCA\Files_FullTextSearch\Provider\FilesProvider->updateDocument()        
#5 /var/www/nextcloud/apps/fulltextsearch/lib/Command/Live.php(313): OCA\FullTextSearch\Service\IndexService->updateDocument()                        
#6 /var/www/nextcloud/apps/fulltextsearch/lib/Command/Live.php(262): OCA\FullTextSearch\Command\Live->liveCycle()                                     
#7 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(298): OCA\FullTextSearch\Command\Live->execute()                                   
#8 /var/www/nextcloud/core/Command/Base.php(177): Symfony\Component\Console\Command\Command->run()                                                    
#9 /var/www/nextcloud/3rdparty/symfony/console/Application.php(1040): OC\Core\Command\Base->run()                                                     
#10 /var/www/nextcloud/3rdparty/symfony/console/Application.php(301): Symfony\Component\Console\Application->doRunCommand()                           
#11 /var/www/nextcloud/3rdparty/symfony/console/Application.php(171): Symfony\Component\Console\Application->doRun()                                  
#12 /var/www/nextcloud/lib/private/Console/Application.php(213): Symfony\Component\Console\Application->run()                                         
#13 /var/www/nextcloud/console.php(102): OC\Console\Application->run()     
#14 /var/www/nextcloud/occ(11): require_once('...')                        
#15 {main}

[ralfi]

yes, same here, but fulltextsearch:index seams to work

[vasyugan]

Still happening. After a restart of the server I had to reset and recreate the indexes for two of my instances. This seems to affect every NC instance I manage, across multiple servers.

[ralfi]

Complete reset and rebuild the index solves the issue at me, including PDF indexing

[vasyugan]

Complete reset and rebuild the index solves the issue at me, including PDF indexing

Until after a while it fails again. That's what I am experiencing.

[XueSheng-GIT]

I'm also seeing that issue occasionally on one of my instances (NC29.0.7). On this specific instance, there's a lot of work done in groupfolders. Some of them using ACL. My assumption is that this is part of the issue (maybe the the live process fails on files which are restricted for some users/groups!?).
If the issue occurs, a reset and index does solve it (at least for a couple of hours or days... depending on the case).

[pmarini-nc]

Hello maintainers, what is the root cause of this issue? Is there a stable fix? I applied the code change proposed here nextcloud/fulltextsearch#849 (comment), but it would be great to have a proper solution. Thanks!

[PesekLasy]

Many months since report and still doesn't fix.

[XueSheng-GIT]

I'm now able to reproduce the issue on NC30.0.2rc2. It would be great if some of you could double check whether you can reproduce it.

Steps:

1. Create user "Max"
2. Create Group "Group1"
3. Assign Max to Group1
4. Create Groupfolder "TestFolder", Group: Group1 (write, share, delete), ACL: Group1, Screenshot below
5. Login as Max
6. Change ACL of TestFolder (restrict share for Group1), Screenshot below
7. Stop live index: occ fulltextsearch:stop
8. Start live index: occ fulltextsearch:live
9. Open TestFolder as Max in webif
10. Create a new file (e.g. markdown).

Expected behavior:
Created file is added to fulltextsearch index.

Actual behavior:
Live index fails with error mentioned in original issue report. #375 (comment)

Screenshots of config:

Futher details:
Generally, it seems that this is a permission issue.
It seems that updateDocument (see following link) does not return a valid FilesDocument so that finally json_encode of $document fails at some point. Whereas the property $access itself does not seem to be the root cause.
https://github.com/nextcloud/files_fulltextsearch/blob/d3472a4584fef46152a815c17856fad840e316bc/lib/Service/FilesService.php#L582-L590

Example how $document looks like in case of such an error (without $access to be able to get that output):

{\"id\":\"2939448\",\"providerId\":\"files\",\"modifiedTime\":0,\"title\":\"\",\"link\":\"\",\"index\":{\"ownerId\":\"Max.Mustermann\",\"providerId\":\"files\",\"collection\":\"local\",\"source\":\"\",\"documentId\":\"2939448\",\"lastIndex\":0,\"errors\":[],\"errorCount\":0,\"status\":60,\"options\":[]},\"source\":\"\",\"info\":[],\"hash\":\"\",\"contentSize\":0,\"tags\":[],\"metatags\":[],\"subtags\":[],\"more\":[],\"excerpts\":[],\"score\":\"\"}"}

[sbaecker]

I think the real problem is a debug line carelessly trying to access an uninitialized document variable.

#3 /home/ncloud/apps/files_fulltextsearch/lib/Provider/FilesProvider.php(315): OCA\Files_FullTextSearch\Provider\FilesProvider->debug('[debug] updateD...', Array)

I commented out that line and the error is gone...

[PesekLasy]

I think the real problem is a debug line carelessly trying to access an uninitialized document variable.

#3 /home/ncloud/apps/files_fulltextsearch/lib/Provider/FilesProvider.php(315): OCA\Files_FullTextSearch\Provider\FilesProvider->debug('[debug] updateD...', Array)

I commented out that line and the error is gone...

Hello, which number of line you are commented - 315?

315: $this->debug('updateDocument result', ['index' => $index, 'document' => $document]);

I think, that is necessery make change in git, becouse after upgrade everything changes will be gone. Line number 315 look as not important for function of plugin. Can you test wraped this line to "if"?

if (isset($document)) {
    $this->debug('updateDocument result', ['index' => $index, 'document' => $document]);
}

FYI: I'm not programer

[XueSheng-GIT]

I think the real problem is a debug line carelessly trying to access an uninitialized document variable.

#3 /home/ncloud/apps/files_fulltextsearch/lib/Provider/FilesProvider.php(315): OCA\Files_FullTextSearch\Provider\FilesProvider->debug('[debug] updateD...', Array)

I commented out that line and the error is gone...

This is as good as the formerly mentioned "workaround". nextcloud/fulltextsearch#849 (comment)

This may fix the crash itself but the question remains whether this uninitialized $access variable is really expected or still needed at some other place?

[EnderArchery]

I also have this issue.
Does anybody have some news on it?
Can we make the debug line at least not crash?
It seems to be fine after commenting out line 757 in /mnt/md0/webroot/nextcloud/lib/private/FullTextSearch/Model/IndexDocument.php

[EnderArchery]

Issues that I think might arise. Maybe it's fine during indexing but now is missing said attribute when trying to access a file from search. Will test said behavior.

[XueSheng-GIT]

There is still no confirmation how to reproduce. At least I have provided steps how I can reproduce (#375 (comment)) but so far nobody confirmed or provided other steps how to reproduce.

[EnderArchery]

There is still no confirmation how to reproduce. At least I have provided steps how I can reproduce (#375 (comment)) but so far nobody confirmed or provided other steps how to reproduce.

Ah I see, I don't have any groups or group folders, it seems to happen under different circumstances as well.
If it really is a permissions issue... like this, maybe it also happens when a user shares a file without any forwarding privileges?

In that case further logging of the path that has been attempted to to be indexed might help. I'm not an expert in PHP development though, maybe it's possible by wrapping the return statement in line 754 into a try-catch statement?
But how would I output it to the nextcloud log? (That would make it easier to troubleshoot, by getting a bigger list of affected files)

PS: I'm sorry if I'm not the most helpful commentator.
I'm however trying my best, so if you want me to go and do/change something, just tell me.

[EnderArchery]

Issues that I think might arise. Maybe it's fine during indexing but now is missing said attribute when trying to access a file from search. Will test said behavior.

Created a test-file with a weird long string inside of it, was able to find and access it through the search while line 757 was commented out.