CardDAV on macOS

[ScottPeterJohnson]

(This issue was imported from Gitea)
rnkn on March 11, 2021:
Thanks for putting in all the work it must have taken to get CardDAV going.

Unfortunately I haven't been able to get a sync happening between macOS Contacts.app and the server.

What I've tried so far:

• Deleting all contacts both locally and via Roundcube, then importing via Roundcube -- the contacts do not populate in Contacts.app.

• Then the same as above but importing via Contacts.app -- the contacts do not populate in Roundcube.

I can connect to the WebDAV server via Finder.app, and see that /webdav/NNN/carddav/default is populated with .vcf files of contacts imported via Roundcube.

If I alter a contact in Contacts.app, these changes do not propagate to Roundcube, even after selecting Settings > CardDAV > Force immediate synchronization with server. Also I can QuickLook the .vcf files accessible via WebDAV and see that changes have not propagated there either, and file modtime has not changed.

This suggests changes in Contacts.app are not propagating to the server at all.

If I go to the Purelymail account management page /manage/user/NNN I can see that my webdav app password was used "moments ago". This seems to suggest that authentication is working okay. I've tried all this with the master password too.

I have 2FA required for my user.

Here are the settings I've tried in Contacts.app:

User Name: [email protected]
Password: <APP PASSWORD> || <MASTER PASSWORD>
Server Address: purelymail.com
Server Path: /webdav/NNN/users/NNN || /webdav/NNN/carddav/

Something odd I've noticed is that /webdav/NNN/users/NNN is a 0B file.

Sorry for the bad news. Any ideas?

(I wouldn't blame you if you threw WebDAV out entirely and went with LDAP...)

[ScottPeterJohnson]

Comment by Scott on March 12, 2021:
You may be experiencing a known issue with Mac OS's Contacts app. The gist is that on Mac OS, the contacts app has absolutely no way of dealing with a user ever having multiple address books on a CardDAV account. You are likely only seeing the one you aren't expecting, probably Automatic Addresses.

I'm not sure how to fix this on my end yet. (And I don't know what Apple is doing, they have this working fine on iOS.)

Something odd I’ve noticed is that /webdav/NNN/users/NNN is a 0B file.

That's expected actually, it's a "virtual" file that only exists to have some WebDAV useful properties.

[ScottPeterJohnson]

Comment by rnkn on March 12, 2021:
Ah! Yes this is what it is. I saw the mention about the macOS problem with multiple address books and thought I'd checked, but I didn't wait long enough to see the .vcf files populate in /webdav/NNN/carddav/automaticallyCollected

At least it's good to know the address book is propagating to the server.

Possibly a silly suggestion, but could this be solved by renaming the paths to something like this?

/webdav/NNN/carddav/0-default
/webdav/NNN/carddav/1-automaticallyCollected

[ScottPeterJohnson]

Comment by Scott on March 13, 2021:
I'll try ordering (since other people with this issue also only needed to see Default). If it comes up that someone needs to see automatically collected, I might add some kind of hacky pragma to username logins that only shows one.

[ScottPeterJohnson]

Comment by rnkn on March 13, 2021:
The macOS Mail.app has its own previous recipients database so there should be no need for anyone to access automaticallyCollected via CardDAV. The only scenario I can think of is someone who alternates between Roundcube and Mail.app wanting to autocomplete a previous recipient they've only messaged via Roundcube, in which case it's not too hard to just search in /Sent and click that recipient's address and New Email.

I think you can quite reasonably say that only Default address book is available via CardDAV on macOS due to Apple bugs. You've got better things to spend your time on than working around Apple's laziness!

[ScottPeterJohnson]

Comment by Scott on June 4, 2021:
Sorry about the delay, but I finally figured out how to work around this correctly- by just hiding the Automatically Collected addressbook for MacOS clients (detected by a header they send). (Reordering didn't seem to do anything.)

Try it and let me know if it seems to pick up the correct addressbook for you?

[ScottPeterJohnson]

Comment by rnkn on June 6, 2021:
Sorry, unfortunately I'm seeing the same results as before.

I deleted all contacts in Roundcube and in Contacts.app, re-added the purelymail.com CardDAV account, then re-impored my contacts into Contacts.app. I created a test contact via Roundcube.

I can see via WebDAV the .vcf files are still populating in /carddav/automaticallyCollected and the test contact created via Roundcube is in /carddav/default.

In Contacts.app the automaticallyCreated group is shown as "Default".

Are there any other steps I should take to ensure there's nothing amiss on my end?

[ScottPeterJohnson]

Comment by Scott on June 7, 2021:
Hmm, maybe the header I'm detecting on changed between MacOS versions. The emulator I was using is on Catalina, what version of MacOS are you using?

[ScottPeterJohnson]

Comment by rnkn on June 7, 2021:
I was on 10.13 High Sierra but recently "upgraded" to 11.4 Big Sur.

This is pure speculation, but given macOS users are connecting via /webdav/NNN/users/NNN instead of /webdav/791/carddav/ could that be a solution?

[ScottPeterJohnson]

Comment by Scott on June 8, 2021:

I was on 10.13 High Sierra but recently "upgraded" to 11.4 Big Sur.

Ah, the emulator I was testing against is on Catalina. I'll see if I can figure out how to update it (the normal process seems broken).

This is pure speculation, but given macOS users are connecting via /webdav/NNN/users/NNN instead of /webdav/791/carddav/ could that be a solution?

I don't think so. Basically, the reason I advise using the /NNN/users/NNN URL on Mac is because the contacts app is oddly hardcoded to use the user principal URL, get the CardDAV home directory from that, then pick the "first" addressbook it finds in the CardDAV home directory as the one to sync with.

[ScottPeterJohnson]

Comment by rnkn on June 8, 2021:
Cool, thanks. Please do let me know if there is anything I can do from my machine to assist, e.g. check logs for the headers in question.

[ScottPeterJohnson]

Comment by Scott on June 10, 2021:
Having acquired a suitable Mac VM, I discovered they did change the header I was trying to detect between Catalina and Big Sur.

I've deployed another version of the workaround that detects both. Let me know if you have any better luck (you may have to readd the account again) and sorry for all the back and forth.

Catalina:
X-Mme-Client-Info <iMac10,1> <Mac OS X;10.15.7;19H15> <com.apple.AppleAccount/1.0 (com.apple.accountsd/113)>

Big Sur:
X-Mme-Client-Info: <iMacPro1,1> <macOS;11.4;20F71> <com.apple.AppleAccount/1.0 (com.apple.accountsd/113)>

[ScottPeterJohnson]

Comment by rnkn on June 10, 2021:
Works! Thanks!

Also I found that groups seem to need to be KIND=group (RFC6350) to show up in Contacts.app.

You should have an additional charge for CardDAV use for all the extra work you've put into it.

[ScottPeterJohnson]

Comment by rnkn on June 10, 2021:
p.s. I didn't need to re-add the acount but something I should mention is that all my contacts disappeared. This could be disconcerting to others unaware of what's going on, so you may get bug reports.

The easy solution is to open the WebDAV connection in Finder and copy all the .vcf files from /carddav/automaticallyCollected to carddav/default (which is quite slow because macOS's WebDAV is garbage).

[ScottPeterJohnson]

Comment by rnkn on July 26, 2021:
Just a quick followup note to again express my gratitute for adding this feature and getting it working on macOS now without a hitch. I'm finding it much more useful than I first anticipated :)