Can't implement list, purge_all, and restore_all on MacOS

[ArturKovacs]

It seems that there's no publicly available API on MacOS that is needed to list the contents of the trash and to permanently remove or restore items from the trash. These would be needed for list, purge_all, and restore_all accordingly, which are the features planned for version 2. These all may be possible to implement by parsing and editing the DS_Store file but as its structure is proprietary, I wouldn't want to take that approach.

All the three aforementioned functions and thereby all the added functionality of version 2 is currently implemented for both Windows and Linux. To allow users to take advantage of the new functionality on those platforms, version 2 will be released but it won't provide those functions on MacOS.

[NilsIrl]

Regardless of whether it is possible to implement these features on MacOS, I think that the current API (for v2) with restore, purge and list behind the linux_windows module is a bad idea.

[NilsIrl]

I don't know if you've seen this, but it may be able to help: https://stackoverflow.com/questions/18707618/put-back-items-from-trash-programmatically

[ArturKovacs]

Regardless of whether it is possible to implement these features on MacOS, I think that the current API (for v2) with restore, purge and list behind the linux_windows module is a bad idea.

Why do you think it's a bad idea and what would be the alternative you'd propose?

I don't know if you've seen this, but it may be able to help: https://stackoverflow.com/questions/18707618/put-back-items-from-trash-programmatically

Hmm I actually don't remember seeing this. It does look promising but I don't have access to a Mac so I won't be able to work on the Mac implementation right now.

I appreciate your interest in this crate and I will spend some time making sure that at least the Windows and Linux implementations are adequate and I will merge those into the master once there's an agreement on how Macs should be handled.

[NilsIrl]

what would be the alternative you'd propose?

please refer to #13

I appreciate your interest in this crate and I will spend some time making sure that at least the Windows and Linux implementation

Do not feel obliged to do anything. The only reason I'm here is because the only trash implementations I could fine are written in python and bash.

making sure that at least the Windows and Linux implementations are adequate

If there's anything that I can help with let me know.

I don't have access to a Mac so I won't be able to work on the Mac implementation right now

I don't either so, I won't be able to help for this.

[cjbassi]

I can spend some time going through the Linux implementation on the dev branches to make sure it looks good. Can you explain the differences between the 3 dev branches?

Also, I'm sure we can figure out some way to get macOS to work eventually, so maybe it would be good to try to merge those branches ASAP so they don't continue to get left behind if either the Linux or windows implementation is ready to be merged.

[ArturKovacs]

v2-dev is the main development branch. If you would like to contribute to version 2, make a PR agains this branch. v2-linux is where the linux (freedesktop.org Trash) implementation of v2 gets made and v2-mac is where the macOS implementation of v2 gets made.

I can spend some time going through the Linux implementation on the dev branches to make sure it looks good.

That would be appreciated. I'd like to ask you to estimate the objective value when considering to make a PR. Reviewing and discussing PRs is time consuming so try to make sure that the added objective value is worth the effort to process the PR.

maybe it would be good to try to merge those branches ASAP so they don't continue to get left behind if either the Linux or windows implementation is ready to be merged.

Unfortunately the windows implementation can't be merged because it is to this day waiting on a PR for winapi to get merged. This one to be exact: retep998/winapi-rs#834

If you'd need version 2's features for macOS urgently I can make an attempt at an implementation but otherwise I'm really quite buisy with other stuff.

[urwrstkn8mare]

Since osascript is already used, I think this might be promising:
applescript-finder-guide.pdf (see PG91: Trash-Object)

I don't have a mac right now, so I can't play around with it, but hopefully I can set up a VM or something along those lines.

[urwrstkn8mare]

Alr well I didn't get far with that but I managed to get some info from .DS_Store (original file loc) and the rest I can get from finder I'm assuming.

The issue is that with MacOS Put Back feat alr being a little buggy Finder the .DS_Store file being up to date is also a little buggier. (Mainly sometimes the file will lag one trash action behind. For example when u move a file to trash it might not show up in the file but after moving another file the first file will show up but not the second)

This makes using the file to implement list() produce sometimes incomplete results.

I see several options:

1. Implement it and accept the bugginess on MacOS's side. (Probably will have to allow some fields to be null)

2. Implement it but basically do the work of the os if the os doesn't do it. Basically this involves updating the DS_Store ourselves if Finder does not. This means writing a new .DS_Store reader and writer library. (Note that changing the file doesn't change any functionality of Put Back as it seems Apple doesn't depend on .DS_Store anymore but still keeps it around for legacy purposes I'm guessing.) Which is why the Put Back button can be available but the file loc isn't visible in the .DS_Store.

3. Forget it, leave it until the future if Apple ever decides to make any sort of API for this.

[Byron]

Thanks for the analysis and the writeup, it's much appreciated!

I wonder if there are ways to mimic what the OS is already able to do and get a working implementation that way. After all, the .DS_Store file seems to be broken enough to rather not use it.

Mimicking the OS could work if it solely relies on files on disk that could be read, maybe easily, maybe with some reverse engineering. If MacOS doesn't really seem to care about .DS_Store, why should we? That could be option 4.

[urwrstkn8mare]

Agreed. How to even begin to reverse engineer Put Back is beyond me. Despite that I'll give it a go but if anyone knows at least where to begin that'd be nice and I'll help however I can.

[ArturKovacs]

My opinion is that it's not worth the effort reverse engineering this feature, as the OS's implementation might change at any point, especially because it is undocumented.

Only implementing list (and restore) with an AppleScript (as @samit43, you suggested) would be a more reasonable approach, if that's possible. If that's not possible then I would wait for Apple to create some sort of API for these things.

[urwrstkn8mare]

Yep totally agree, upon I've quickly realised that for me reverse engineering this is borderline impossible and plus very susceptible to be broken by any update.

Looking at the applescript docs I think it may be possible but to be honest I don't understand the details enough to make it work. There is a Trash structure with many properties but I don't know how to even access them. I invite anyone with more experience to give it a go but other than that sorry for not really accomplishing anything.

[bestlem]

See https://github.com/ali-rantakari/trash for a C version of using Finder to trash the file which provides the Put Back Information

[jeff-hykin]

Personally as a Mac user, I'd be more than happy for trash to just have its own temp trash folder, and delete/restore from that list entirely seperate from the offical MacOS trash folder. I just want to have a cross platform rm equivlent that isn't instantly permanent/destructive.

If the builtin trash is going to be used I agree AppleScript is the way to go. ChatGPT has decent knowledge of applescript since Apple did provide quite a bit of online documentation. Here's a (tested on my mac) example for restoring a "my_file.txt" programmatically:

osascript -e ' 
tell application "Finder"
    set trashItems to items of trash
    repeat with currentItem in trashItems
        if name of currentItem is "my_file.txt" then
            try
                move currentItem to desktop
            end try
        end if
    end repeat
end tell
'

If applescript seems like the most promising direction I'm happy to test scripts to help get it implemented

[Byron]

Thanks for your input and for trying a way forward.

To implement these operations correctly, tapping into operating system functionality seems to be the way to go. I just wonder if using apple script is full of undesirable side-effects like windows popping up or the finder becoming the foreground application (please note that I didn't try the script mentioned here).

Also I wonder if there are ObjectiveC or Swift APIs that one can tap into instead that would skip a layer or two.

[jeff-hykin]

A lot of stack overflow answers with AppleScript involve windows popping up, but I think that's just because it's easy to not look up the API and instead just use the GUI buttons. The script above doesn't need any windows, and just as a sanity check, I tested it with closing all the finder windows and with having some finder windows open to make sure it didn't modify them, both worked fine.

I'm sure an Objective-C API would be much more runtime efficient, but I'm actually skeptical that there is an API for restoring files from the trash. I believe Finder is the only one with that capability, so I think regardless AppleScript is going to be required to talk to Finder and ask it to restore files.

It's a lot easier to test the AppleScript so at least for a 1.0 it might be nice to use it.

[urwrstkn8mare]

Personally as a Mac user, I'd be more than happy for trash to just have its own temp trash folder, and delete/restore from that list entirely seperate from the offical MacOS trash folder. I just want to have a cross platform rm equivlent that isn't instantly permanent/destructive.

If the builtin trash is going to be used I agree AppleScript is the way to go. ChatGPT has decent knowledge of applescript since Apple did provide quite a bit of online documentation. Here's a (tested on my mac) example for restoring a "my_file.txt" programmatically:

osascript -e ' 
tell application "Finder"
    set trashItems to items of trash
    repeat with currentItem in trashItems
        if name of currentItem is "my_file.txt" then
            try
                move currentItem to desktop
            end try
        end if
    end repeat
end tell
'

If applescript seems like the most promising direction I'm happy to test scripts to help get it implemented

If it helps this could be done with MacOS APIs. Cause it just takes an item from trash and moves it like moving any file to anywhere. I think the idea was to use MacOSs native Put Back functional (the button in right-click menu). If we just want to move a file out of trash we could create our own put back functionality by maintaining our own list of source paths. (Maybe even use the DS_Store approach and use this as a fall back).

[bestlem]

Personally as a Mac user, I'd be more than happy for trash to just have its own temp trash folder, and delete/restore from that list entirely seperate from the offical MacOS trash folder. I just want to have a cross platform rm equivlent that isn't instantly permanent/destructive.
If the builtin trash is going to be used I agree AppleScript is the way to go. ChatGPT has decent knowledge of applescript since Apple did provide quite a bit of online documentation. Here's a (tested on my mac) example for restoring a "my_file.txt" programmatically:

osascript -e ' 
tell application "Finder"
    set trashItems to items of trash
    repeat with currentItem in trashItems
        if name of currentItem is "my_file.txt" then
            try
                move currentItem to desktop
            end try
        end if
    end repeat
end tell
'

If applescript seems like the most promising direction I'm happy to test scripts to help get it implemented

If it helps this could be done with MacOS APIs. Cause it just takes an item from trash and moves it like moving any file to anywhere. I think the idea was to use MacOSs native Put Back functional (the button in right-click menu). If we just want to move a file out of trash we could create our own put back functionality by maintaining our own list of source paths. (Maybe even use the DS_Store approach and use this as a fall back).

The latter part is the reason to use oascript - it is not possible to allow Finder to a put back using macAPIs - I think there is a documented API but as previous writers of trash utilities have found it does not work sindresorhus/macos-trash#4
Reverse engineering the .DS_Store could work.

[Byron]

From what I read the sentiment seems to be that using osascript is the way forward here and that there isn't really a choice either. On the bright side, it doesn't seem to affect windows at least not with put-back-like functionality shown off here.

To me it seems once a PoC of purge_all shows that this won't be a problem either, one could see if these osa-scripts can be called from Rust somehow. Maybe calling it using the CLI would be fine, too, but me guessing here seems to indicate it's an area where some work is still needed before we can call this a guaranteed win.

[urwrstkn8mare]

latter part is the reason to use oascript

Sorry but please correct me if I'm wrong. How do we implement Put Back with applescript? The example given only moves it to desktop right?

[jeff-hykin]

Actually I think we're over-complicating the restore. mv "$HOME/.Trash/my_file" "/wherever/blah.txt" seems to work just just as well, it just doesn't make the restored sound which might be seen as a benefit.

If the sound is prefered here's the script for putting it in a target folder:

osascript -e '
tell application "Finder"
    set trashItems to items of trash
    set theFolder to "/Users/jeffhykin/" as POSIX file as alias
    repeat with currentItem in trashItems
        if name of currentItem is "my_file.txt" then
            move currentItem to theFolder
        end if
    end repeat
end tell
'

I don't know of a way of simultaniously renaming and removing from the trash, and I think its possibly not supported by the GUI version of finder and therefor possibly also no way programmatically. However, we could just always move it to a temp folder and then immediately move+rename it right after

[Byron]

This reminds me of a major shortcoming of the examples so far: They don't seem to know where to put files back to. Shouldn't they derive the correct put-back location from information in the trash item itself?

Edit: It does turn out that the current implementation is already using apple-script in one code-path.

[urwrstkn8mare]

@Byron what do you think about some sort of trash manifest in the users home directory or even in .Trash itself? Is that outside the scope of this package? My suggestion is do that with .DS_Store parsing as a fallback. And if the user manually trashes an item without changing the .DS_Store for some MacOS quirk (which does happen) then just allow returning a null value for the source path and maybe give an option in the package to restore a trash item to a specific path. e.g.:

pub fn restore_all<I>(items: I, original_path_dir: Path) -> Result<(), Error>
    where
        I: IntoIterator<Item = TrashItem>,
    {

If not that there is always the option to just allow the user of the package to manually edit TrashItems original path (whether or not it is null).
I don't have time currently to help implement it but just putting it out there.

[Byron]

I think that even though it would be awesome to be able to "fix" trashing on all platforms and make it work uniformly with a single crate, I definitely consider this to be out of scope for this crate which exists to abstract over what's provided on each platform and live with the consequences (should they be unavoidable at least).

Having a platform-independent trash implementation that uses its very own way of keeping trash-metadata, which might easily turn out to be the best implementation across all platforms as it gets to learn from all existing implementations seems like a task for a new crate. Such a crate would be useful for programs that control their own trash or maintain a list of filesystem entries that can be recovered before permanent deletion.

[bestlem]

latter part is the reason to use oascript

Sorry but please correct me if I'm wrong. How do we implement Put Back with applescript? The example given only moves it to desktop right?

Yes sorry I am not that familiar with Applescript and I thought it was the code to trash the file.
That applescript on closer reading does nothing that can't be done with plain POSIX C commands or NSFileManager methods.

However I have seen if you get applescript to tell Finder to do the trash then that does set the Put Back.. Which I think is to use delete filename in Applescript.

Looking at https://github.com/ali-rantakari/trash/blob/master/trash.m they are using AppleEvents not applescript

However looking at the the exported dictionary from Finder.app there is nothing corresponding to PutBack.
So for restore either a move or if all trash is via this program keep your own database or xattr of source and destination

list and purge can be don via C calls - the trash directory can be found from NSFlieManager class in Foundation if you don't want to use AppleScript.

Sorry for the interruption and mis reading of this thread.

[Byron]

With all the information collected here and the references to the trash project, it appears like it's possible to take a stab at the list and purge functionality even though it is still unclear how to do Put Back. All I could gather was that it stores put-back information in in .DS_Store files in a racy fashion, but actually putting a file back would require writing these metadata files which seems like a can of worms.

From what I recall, the macos.rs file already contains examples for both AppleScript and for using the NsFileManager class, which should help. To me it seems fine to use apple script especially for a proof-of-concept, which can eventually be migrated to using the NsFileManager if there is benefit and demand.

[jeff-hykin]

They don't seem to know where to put files back to

Oh I didn't try that yet. I think there might be a way with mdls -name kMDItemWhereFroms but haven't tested it. I'll let you know if I find anything.

[emmalexandria]

They don't seem to know where to put files back to

Oh I didn't try that yet. I think there might be a way with mdls -name kMDItemWhereFroms but haven't tested it. I'll let you know if I find anything.

If I remember correctly, that metadata flag is more for storing the webpages etc that files were downloaded from.

[emmalexandria]

Thanks for the analysis and the writeup, it's much appreciated!

I wonder if there are ways to mimic what the OS is already able to do and get a working implementation that way. After all, the .DS_Store file seems to be broken enough to rather not use it.

Mimicking the OS could work if it solely relies on files on disk that could be read, maybe easily, maybe with some reverse engineering. If MacOS doesn't really seem to care about .DS_Store, why should we? That could be option 4.

Just out of curiosity, what led you to the conclusion that the .DS_Store file is broken/unused?

[Byron]

The user described issues using the .DS_Store file and it seemed unreliable, even when used with the finder.

Calling the file itself broken is certainly incorrect, but the I'd still call the workflow around it broken enough to avoid dealing with it (i.e. no API, reverse engineering needed, Finder itself seems to not handle it correctly).

[bestlem]

I would take the number 1 option from that issue. Put Back using Finder has been reliable for me and I have not seen any reports of issue on the web.

One other point is that trash using AppleScript/AppleEvents does not work if you are not the user with the current desktop.

e.g. if you ssh into a mac you don't have a finder and so the workarounds don't work.

The fix is for Apple to fix their API - but that has been open as a bug for years.

[NathanielJS1541]

Sorry to add more comments to an already long issue, but I just wanted to offer up any help I might be able to provide.

While I'm not an experienced rust or macOS dev, I do have access to macOS hardware (both x86_64 and M2) that I use for iOS development.

I'm happy to run tests or PoC on this hardware if it would be of any help at all. This also means I have access to the macOS SDK if that would be of use.