Wrong folder size in webview / truncate oc_filecache corrupts share structure in db

[deMattin]

First, sorry for my english ... ;)
The main problem is, that the folder size isn't calculated correct in OC 7 in webview.
And I need some help to repair this.
The try to repair this caused another problem that I only could "repair" with a restoring of my database backup.

Steps to reproduce

I did the following steps:

1. activated encryption (don't know if this does matter)
2. deactivated encryption (don't know if this does matter)
3. folder size in webview is shown wrong/less (not systematic, sometimes the folder size is the size of a single file in the folder, sometimes calculated correct, sometimes a part of all files).
   Filesize itself is always shown correct!
4. rescanned filesystem (/path/of/owncloud/console.php files:scan --all) -> nothing changed
5. truncated oc_filecache -> folder size now is shown correct but all shares are corrupt.
   Some of my files are shown as shared by others, my shares link to other files that shouldn't have been shared, ...
   The whole sharing structure is linked wrong in database with no chance to repair it in the webview.
   There was NO dataloss (>10GB of Data)!

Expected behaviour

Foldersize will show the sum of all files in the folder
AND
truncate filecache shouldn't corrupt shareing structure in database

Actual behaviour

foldersize is shown wrong
"truncate oc_filecache;" corrupts database

Server configuration

Operating system:
Debian 7
Web server:
Apache 2
Database:
MySQL 5.0
PHP version:
5.4.4
ownCloud version: (see ownCloud admin page)
7.0.0.8
Updated from an older ownCloud or fresh install:
updated from well working 6.0.4
List of activated apps:
default
+videoviewer
The content of config/config.php:

'............', 'passwordsalt' => '.........', 'trusted_domains' => array ( 0 => '192.168.192.210', 1 => 'domain.tld', ), 'datadirectory' => '/media/sf_Owncloud-Data', 'dbtype' => 'mysql', 'version' => '7.0.0.8', 'dbname' => 'ocdb', 'dbhost' => 'localhost', 'dbtableprefix' => 'oc_', 'dbuser' => '......', 'dbpassword' => '..................', 'installed' => true, 'forcessl' => true, "overwritehost" => "domain.tld", "overwriteprotocol" => "https", "overwritewebroot" => "owncloud", 'trashbin_retention_obligation' => 180, 'trashbin_auto_expire' => true, 'loglevel' => '2', "remember_login_cookie_lifetime" => 60_60_24_30, 'enable_previews' => true, 'theme' => '', 'maintenance' => false, ); *_Are you using external storage, if yes which one:*\* local/smb/sftp/... no **Are you using encryption:** yes/no no (but had activated it before) ### Client configuration **Browser:** Google Chrome **Operating system:** Windows 7 ### Logs #### Web server error log nothing special #### ownCloud log (data/owncloud.log) some errors like: Error PHP Undefined offset: 2 at /var/www/owncloud/lib/private/share/share.php#1339 2014-07-29T20:48:18+00:00 Error PHP Undefined offset: 1 at /var/www/owncloud/lib/private/share/share.php#1339 2014-07-29T20:42:42+00:00 ... #### Browser log nothing special

[DeepDiver1975]

1. truncated oc_filecache -> folder size now is shown correct but all shares are corrupt.

don't truncate that table! Truncating the file cache was a workaround back in the days of OC4!

[deMattin]

Ok, if it's not allowed to truncate this table, how is it possible to force a rebuild of its content?
It has to be possible to rescan the data and rebuild all dynamic and cached database content for maintenance!
Or (just in my case): How do I force a recalculation of folder sizes?

I could delete and recreate the files and then I'm shure the database would be "repaired".
But that's no real solution because I have some user with slow connections and a resync with the mirall client would be a pain.

[DeepDiver1975]

rescanned filesystem (/path/of/owncloud/console.php files:scan --all) -> nothing changed

using files:scan is the way to go - please note that in some cases this is not possible - e.g. some mounted filesystems or in case encryption is enabled.

but even without files:scan the filesizes will populate themselves - espacially in your case with that many files it can take some time because the file sizes have to be propagated to the file root

[deMattin]

The rescanning works - it crawles through the whole filesystem.
But the wrong folder sizes were not corrected this way. The server has had time enough (whole night without users loged in).
FYI.: The data filesystem is a mounted NTFS Volume. But this should not be the cause.
As I already said: clearing the oc_filecache repairs the wrong shown sizes (here I see how it populates the folder sizes one after the other) but has the disadvantage of corrupting the shares.
So the problem is the WRONG folder size in db, which will never be corrected, because ownCloud "thinks" it has got the correct size.

And to be clear: There is no problem with the FILE size - here everything is fine!

Is it possible to delete only the cached size of the folders in db and force a recalculation this way?
I think, it seems to work like this:
No Information about (folder) size -> foldersize will be (re)calculated
Wrong information about filesize in db -> not checking or recalculating folder size

Edit:

I had a look in database and found this:
column "size" is populated
column "encrypted" is always "0" on any file and folder
column "unencrypted_size" is sometimes populated (with wrong sizes), but mostly "0"

And here it comes:
The webview uses the value of the WRONG "unencrypted_size" column for the folder size and not the (seeming to be) correct value of the column "size"!
In column "size" the folder size seems to be correct!

Would it be save or possible to delete the whole column "unencrypted_size" if encryption is disabled?
Shouldn't deactivating the encryption has have done this?
I had activated encryption for testing but deactivated it now.
The problem seems to be caused by the activated and deactivated encryption and there is a bug, now using the wrong db-column for showing the FOLDER size.

[deMattin]

Ok, the values in "unencrypted_size" have been the clue.
I have set them all to "0" (~450 files of more than 17500) and now the folder size seems to be shown correct.

Although I have solved my problem now, there has to be a bug in activating and deactivating the encryption app.
There has to be a final step setting all "unencrypted_size" values to 0 in Database or there needs to be a check, if there are some left.
I don't know why in the database for some files this value hasn't been set to 0 in my case after deactivating encryption.

Or the safest way:
"unencrypted_size" has always to be ignored and "size" used instead, if "encrypted" is set to "0"!

One last thing:
Does the column "unencrypted_size" exist in databases if encryption never has been activated?
Am I right, not to be allowed to delete this column?

[PVince81]

CC @schiesbn

[PVince81]

Size propagation has issues on OC 7.0.4 with shared folders: #14596

@deMattin the encryption app normally calculates unencrypted_size during the initial encryption of every user. If you set it back to 0 it will only recompute it the next time the file is accessed/modified.
This is not a situation the encryption app was designed for, so not really a bug.

To recover from this particular situation, one idea is to reset the migration flag in the database for all users: update oc_preferences set configvalue=0 where appid='files_encryption' and configkey='migration_status';

The next time a user will login, the unencrypted_size should be recomputed.

Closing as duplicate of #14596 (the size propagation ticket)