Crash during initial upload

[kallekn]

I got the plugin working on my Mac, but after loading a bit more than 1 000 files it crashed, I just got repeated error messages. I closed Obsidian and opened it again, the plugin seems to be running but no new uploads are happening. Is there any way to restart the uploads, other than selecting "upload all", which would mean deleting the alreadu uploaded 1 000 files.

I have more than 6 000 files to upload, so this seems a bit shaky.

Also, after this the whole app started misbehaving. Impossible to display notes. When closing down the app I got following:

Uncaught Exception:
TypeError: Cannot read properties of undefined (reading 'origin')
at Function. (/Users/kalle.kniivila/Applications/Obsidian.app/Contents/Resources/obsidian.asar/main.js:27:9882)

After starting the app the problem persists, I can read one note, but when I try to switch to another one, I just get the title of the other note, with the content of the first one. Then nothing will work at all.

Turning the plugin off didn't seem to help. After removing the plugin files Obsidian seems to work normally again. But no sync :-(

The file that caused the crash might be a sound file, as that is the last file uploaded and it is zero bytes.

[stravo1]

I have personally not tested the plug-in for vaults over 1000 files, will do it soon.

And no you don't need to re-upload all files, if you edit a file (notes only, not attachments) which was not uploaded it will ask for a confirmation and upload automatically, however if it's an attachment which was not uploaded, you need to make use of the "upload" button to upload them manually.

I will try and add a functionality to resume the upload process if the initial upload process fails, so that this type of issues can be avoided.

And for the crashing part, please enable the "Verbose Logging" and "Error Logging" in the plug-in settings and attach them here.

[stravo1]

Also would suggest deleting the 0 byte file from Drive (delete from Drive's Bin as well) and trying to use the plug-in again.

[kallekn]

I have personally not tested the plug-in for vaults over 1000 files, will do it soon.

And no you don't need to re-upload all files, if you edit a file (notes only, not attachments) which was not uploaded it will ask for a confirmation and upload automatically, however if it's an attachment which was not uploaded, you need to make use of the "upload" button to upload them manually.

I will try and add a functionality to resume the upload process if the initial upload process fails, so that this type of issues can be avoided.

And for the crashing part, please enable the "Verbose Logging" and "Error Logging" in the plug-in settings and attach them here.

Thanks for the quick answer. Looks like I'll need to reload everything. Too much work doing edits in 5000 files...

Some options for what should be synced would be great in the future. Like "sync stuff which was modified during last month".

But maybe I'll just put some of the old notes in an archive vault...

[kallekn]

Ok, I got it working on my Mac when I reduced the vault to approximately 500 files.

However, on my Android phone the initial download process crashes after approximately 400 files. I just get ERROR: Something went wrong! Sync is paused. - over and over again.

Also, it is a bit irritating that the warning totally covers the top menu, so it is difficult to hit the "settings" icon in order to turn off the plugin.

[stravo1]

Yes, the infinite error is a bug i am looking into. Will keep you updated.

[stravo1]

Also can you please include the verbose and error logs.

[stravo1]

Yes, the infinite error is a bug i am looking into. Will keep you updated.

fixed at https://github.com/stravo1/obsidian-gdrive-sync/releases/tag/v0.9.9-beta-7-1

[kallekn]

Thanks. Seems to be working now, I succeeded in syncing the files to my phone.

However, there seems to be a problem with metadata: the "created" date is not the original date, so I cannot sort the notes according to how old or new they are.

Also, all attachment files seem to have been renamed, so all the links in my notes are broken :-(

[kallekn]

Also can you please include the verbose and error logs.

I'll do that next time I get errors.

[kallekn]

So, here are some error logs. Now I tried to sync the preexisting notes to my other Mac laptop. Not very successful, I'm afraid.

error-log-gdrive-plugin.md
verbose-log-gdrive-plugin.md

[stravo1]

Thanks. Seems to be working now, I succeeded in syncing the files to my phone.

However, there seems to be a problem with metadata: the "created" date is not the original date, so I cannot sort the notes according to how old or new they are.

Also, all attachment files seem to have been renamed, so all the links in my notes are broken :-(

sorry for not mentioning it in the README file, yes the attachments are renamed when they are uploaded because unlike notes there's no "lastSync" tag that the plug-in can read from the note's content, so to differentiate between which attachments are synced and which are not it's renamed to "attachment_name-synced". i am actively trying to create a workaround for this, but it will take time.

there're a lot of bugs which too might lead to data loss, please do make a copy of the vault before installing the plug-in.

will make sure to add this warning on the README page.

[stravo1]

So, here are some error logs. Now I tried to sync the preexisting notes to my other Mac laptop. Not very successful, I'm afraid.

error-log-gdrive-plugin.md
verbose-log-gdrive-plugin.md

took a look into the logs, couldn't decipher much, but seems like there's some conflict in the folder structure, if there's nothing sensitive in the vault which gave these errors, can you share the vault structure as well as a screenshot of the Drive folder where this vault was uploaded.

[kallekn]

There might have been a conflict with another syncing app that was running. I'll test again and come back to you with the results.

[kallekn]

Still not working properly. Created new note on my phone, made an audio recording in it. After sync the audio recording cannot be found.

Tue Apr 02 2024 20:47:16 GMT+0200 (Keski-Euroopan kesäaika)-TypeError-Cannot read properties of null (reading '0')-TypeError: Cannot read properties of null (reading '0')
at driveSyncPlugin.uploadNewAttachment (plugin:obsidian-gdrive-sync:3004:161)
at async eval (plugin:obsidian-gdrive-sync:3318:13)

[stravo1]

The recording was probably synced and was renamed to:
"Recording....-synced.webm" because as I mentioned earlier attachments are renamed to keep track which are synced and which are not.

The error occurs while trying to show you a pop-up which tells you to rename the attachment in your notes file as well. I have identified the issue and will try and fix it as soon as possible.

[kallekn]

Yes, there is a sound file with the new name. This is highly impractical, as all links to files in my notes are broken. I do hope you can find another way to do this.

[kallekn]

This is what I get trying to sync my laptop now:

Wed Apr 03 2024 09:02:32 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:39 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:46 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:52 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:58 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:03:04 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:03:10 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:03:12 GMT+0200 (Keski-Euroopan kesäaika)-Error-FATAL ERROR: Too many errors within a minute.-Error: FATAL ERROR: Too many errors within a minute.
at eval (plugin:obsidian-gdrive-sync:2782:42)

[stravo1]

the attachments are renamed when they are uploaded because unlike notes there's no "lastSync" tag that the plug-in can read from the note's content, so to differentiate between which attachments are synced and which are not it's renamed to "attachment_name-synced". i am actively trying to create a workaround for this, but it will take time.

I understand, for new vaults it's an annoyance but for big existing vaults it's a deal breaker. for the This hack works for small vaults that I personally use without too many attachments so I did not see the need to give too much attention to this issue. Now that others are facing this issue, I will try and figure out a proper solution, however it will take time.

[stravo1]

This is what I get trying to sync my laptop now:

Wed Apr 03 2024 09:02:32 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:39 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:46 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:52 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:58 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:03:04 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:03:10 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:03:12 GMT+0200 (Keski-Euroopan kesäaika)-Error-FATAL ERROR: Too many errors within a minute.-Error: FATAL ERROR: Too many errors within a minute. at eval (plugin:obsidian-gdrive-sync:2782:42)

its the same issue as in the error logs, there's some conflict between folders, can't diagnose only from the error logs unfortunately. you can contact me on my email [email protected] for further assistance on this issue, i might need to look at the vault structure.

[stravo1]

The recording was probably synced and was renamed to: "Recording....-synced.webm" because as I mentioned earlier attachments are renamed to keep track which are synced and which are not.

The error occurs while trying to show you a pop-up which tells you to rename the attachment in your notes file as well. I have identified the issue and will try and fix it as soon as possible.

fixed: https://github.com/stravo1/obsidian-gdrive-sync/releases/tag/v0.9.9-beta-7-2

[kallekn]

Thanks, will test!

[kallekn]

Still getting similar error messages. A note that was created on my Android phone seems to have been correctly downloaded to my Mac, but then I get these repeated error messages. Can't see anything odd with the file structure, it's just mydrive -> obsidian -> (name of vault)

Thu Apr 04 2024 16:48:06 GMT+0200 (centraleuropeisk sommartid)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 16:48:11 GMT+0200 (centraleuropeisk sommartid)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 16:48:18 GMT+0200 (centraleuropeisk sommartid)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 16:48:31 GMT+0200 (centraleuropeisk sommartid)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 16:48:33 GMT+0200 (centraleuropeisk sommartid)-Error-FATAL ERROR: Too many errors within a minute.-Error: FATAL ERROR: Too many errors within a minute.
at eval (plugin:obsidian-gdrive-sync:2782:42)

[kallekn]

Might this be the problem? In my computer he attachments (imported from Evernote) are in a specific folder called _resources
In Gdrive there is no such folder. Instead, the filename of the attachments begins with _resources/

[stravo1]

as i mentioned before the plug-in doesn't try to imitate the vault's folder structure on Drive, instead it stores the files with their "relative paths", so the attachments being uploaded as _resources/.... is an expected behaviour.

however this folder might be the cause of the "Folder Already Exists" issue. can you just give me an idea as how the folder structure for the _resources folder might be?

also if you can copy the vault and delete the folder, do you face the same issues? trying doing it with a smaller vault.

i will personally make a folder whose name starts with an underscore (_) to see if there's any issue with folders having such names.

thank you for the info :)

[kallekn]

There is no additional structure such as subfolders. Just media files in the folder.

[kallekn]

Now I tried to sync my other Mac, with another vault. There is no folder for that vault in the obsidian folder in my Gdrive. Still, the plugin says it is trying to download missing files. Is it looking for files that correspond to the ones already in the vault on my Mac?

I can sync more than one vault at the same time, right?

Then I get this in the error log. Should I just try syncing a vault with no attachments and no folders to see how that works?

Thu Apr 04 2024 20:38:59 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:06 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:13 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:20 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:25 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:31 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:37 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:39 GMT+0200 (Keski-Euroopan kesäaika)-Error-FATAL ERROR: Too many errors within a minute.-Error: FATAL ERROR: Too many errors within a minute.
at eval (plugin:obsidian-gdrive-sync:2782:42)

[stravo1]

if the vault names were the same then the plug-in would assume that you're trying to sync a common vault between two devices and would start downloading files from the folder named after that vault.

yes you should be able to sync multiple vaults, they need to have different names.

thank you for reporting all these errors!
i have been a little busy with my exams and internships, will look into these errors as and when i get time.

[kallekn]

It looks like the plugin is trying to sync the same files over and over again. For example, this file exists in one original version and 4 synced versions which seem to have 2 different names. Seemingly both in GDrive and my laptop.

Also I'm getting the same error:
Mon Apr 08 2024 09:10:05 GMT+0200 (centraleuropeisk sommartid)-Error-FATAL ERROR: Too many errors within a minute.-Error: FATAL ERROR: Too many errors within a minute.
at eval (plugin:obsidian-gdrive-sync:2782:42)

[stravo1]

that's interesting! will look into it soon :)

[stravo1]

the attachments are renamed when they are uploaded because unlike notes there's no "lastSync" tag that the plug-in can read from the note's content, so to differentiate between which attachments are synced and which are not it's renamed to "attachment_name-synced". i am actively trying to create a workaround for this, but it will take time.

I understand, for new vaults it's an annoyance but for big existing vaults it's a deal breaker. for the This hack works for small vaults that I personally use without too many attachments so I did not see the need to give too much attention to this issue. Now that others are facing this issue, I will try and figure out a proper solution, however it will take time.

attachment issues are now fixed at beta-13