Copying or moving folder structures to or from Cloud storage

I have no problem moving or copying individual files to or from cloud services. However when I try to copy or move folder structures using Hazel, there are issues as described below. (I can manually drag and drop, or copy/cut and paste folder structures without problems, but Hazel can't always perform the same function).

I am using an iMac with Ventura 13.5.2.
I save files in the cloud (OneDrive).
I use Cryptomator to encrypt sensitive files before they are uploaded to OneDrive.

Cryptomator looks just like another drive on my Mac and is supposed to behave in the same way as any external drive attached to a Mac.

Same thing for OneDrive.

Here is a summary of my testing of the copy and move function Hazel:

So, as you can see, single (or multiple) file copying works in every direction.

But when it comes to copying folders and their contents there are some combinations of copying (or moving) between drives that just don't work.

Here is an example of one of the non-working rules:

Here are the logs of the errors:

Copying from internal Mac drive to Cryptomator
2023-09-19 00:43:01.201 hazelworker[19557] Processing folder Mac Source (forced)
2023-09-19 00:43:03.304 hazelworker[19557] On Mac sub 1: Rule New Rule matched.
2023-09-19 08:25:14.076 hazelworker[22141] Processing folder Mac Source (forced)
2023-09-19 08:25:16.180 hazelworker[22141] Test folder 2 with files: Rule Copy folder structure matched.
2023-09-19 08:25:16.200 hazelworker[22141] Error copying file from /Users/myname/Documents iMac Pro/Newly scanned iMac Pro/Mac Source/Test folder 2 with files to /Volumes/yw9tOuKg-Quh/Cryptomator Desitnation/Test folder 2 with files: Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"
2023-09-19 08:25:16.206 hazelworker[22141] [Error] Copy failed: Could not copy file Test folder 2 with files from folder /Users/myname/Documents iMac Pro/Newly scanned iMac Pro/Mac Source to folder /Volumes/yw9tOuKg-Quh/Cryptomator Desitnation/Test folder 2 with files
2023-09-19 08:25:16.210 hazelworker[22141] Test folder 1 (empty): Rule Copy folder structure matched.
2023-09-19 08:25:16.231 hazelworker[22141] [File Event] File copied: Test folder 1 (empty) copied from folder /Users/myname/Documents iMac Pro/Newly scanned iMac Pro/Mac Source to folder /Volumes/yw9tOuKg-Quh/Cryptomator Desitnation.
2023-09-19 08:25:16.234 hazelworker[22141] Received abort event.
2023-09-19 08:25:16.235 hazelworker[22141] Done processing folder Mac Source

Copying folders from OneDrive to Internal Mac drive
2023-09-19 08:30:23.868 hazelworker[22179] Processing folder Onedrive Source (forced)
2023-09-19 08:30:25.906 hazelworker[22179] Test folder 2 with files: Rule Copy folder structure matched.
2023-09-19 08:30:25.906 hazelworker[22179] Error copying file from /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Source/Test folder 2 with files to /Users/myname/Documents iMac Pro/Newly scanned iMac Pro/Mac Destination/Test folder 2 with files: (null)
2023-09-19 08:30:25.906 hazelworker[22179] [Error] Copy failed: Could not copy file Test folder 2 with files from folder /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Source to folder /Users/myname/Documents iMac Pro/Newly scanned iMac Pro/Mac Destination/Test folder 2 with files
2023-09-19 08:30:25.910 hazelworker[22179] Test folder 1 (empty): Rule Copy folder structure matched.
2023-09-19 08:30:25.910 hazelworker[22179] Error copying file from /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Source/Test folder 1 (empty) to /Users/myname/Documents iMac Pro/Newly scanned iMac Pro/Mac Destination/Test folder 1 (empty): (null)
2023-09-19 08:30:25.910 hazelworker[22179] [Error] Copy failed: Could not copy file Test folder 1 (empty) from folder /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Source to folder /Users/myname/Documents iMac Pro/Newly scanned iMac Pro/Mac Destination/Test folder 1 (empty)
2023-09-19 08:30:25.913 hazelworker[22179] Received abort event.
2023-09-19 08:30:25.914 hazelworker[22179] Done processing folder Onedrive Source

Copying folders from OneDrive to OneDrive
2023-09-19 08:33:16.893 hazelworker[22205] Processing folder Onedrive Source (forced)
2023-09-19 08:33:19.003 hazelworker[22205] Test folder 2 with files: Rule Copy folder structure matched.
2023-09-19 08:33:19.004 hazelworker[22205] Error copying file from /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Source/Test folder 2 with files to /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Destination/Test folder 2 with files: (null)
2023-09-19 08:33:19.004 hazelworker[22205] [Error] Copy failed: Could not copy file Test folder 2 with files from folder /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Source to folder /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Destination/Test folder 2 with files
2023-09-19 08:33:19.007 hazelworker[22205] Test folder 1 (empty): Rule Copy folder structure matched.
2023-09-19 08:33:19.008 hazelworker[22205] Error copying file from /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Source/Test folder 1 (empty) to /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Destination/Test folder 1 (empty): (null)
2023-09-19 08:33:19.008 hazelworker[22205] [Error] Copy failed: Could not copy file Test folder 1 (empty) from folder /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Source to folder /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Destination/Test folder 1 (empty)
2023-09-19 08:33:19.011 hazelworker[22205] Received abort event.
2023-09-19 08:33:19.011 hazelworker[22205] Done processing folder Onedrive Source

Copying folders from OneDrive to Cryptomator
2023-09-19 08:35:34.979 hazelworker[22221] Processing folder Onedrive Source (forced)
2023-09-19 08:35:37.083 hazelworker[22221] Test folder 2 with files: Rule Copy folder structure matched.
2023-09-19 08:35:37.115 hazelworker[22221] Error copying file from /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Source/Test folder 2 with files to /Volumes/yw9tOuKg-Quh/Cryptomator Desitnation/Test folder 2 with files: Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"
2023-09-19 08:35:37.124 hazelworker[22221] [Error] Copy failed: Could not copy file Test folder 2 with files from folder /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Source to folder /Volumes/yw9tOuKg-Quh/Cryptomator Desitnation/Test folder 2 with files
2023-09-19 08:35:37.128 hazelworker[22221] Test folder 1 (empty): Rule Copy folder structure matched.
2023-09-19 08:35:37.155 hazelworker[22221] [File Event] File copied: Test folder 1 (empty) copied from folder /Users/myname/Library/CloudStorage/OneDrive-Personal/Onedrive Source to folder /Volumes/yw9tOuKg-Quh/Cryptomator Desitnation.
2023-09-19 08:35:37.158 hazelworker[22221] Received abort event.
2023-09-19 08:35:37.158 hazelworker[22221] Done processing folder Onedrive Source

Are you sure the files are actually available locally? Frankly, I've found that OneDrive is one of the most troublesome of the cloud drive services out there and even when you tell it to keep files local, it doesn't.

I am absolutley sure the folders and underlying files are all available locally. I just created them locally in the OneDrive folder for the purpose of the test. They would have the picture of the cloud next to them if they were not held locally. See picture below.

Simply put:
I can copy any file to, or from, OneDrive using Hazel.
I can copy any folder to OneDrive using Hazel
Copying any folder from OneDrive using Hazel does not work

incidentally I also tried writing a folder structure to OneDrive from iCloud. That works perfectly. But going in the other direction from OneDrive to iCloud fails to copy the folder structure.

I agree that OneDrive is troublesome, particularly when there is a file with an illegal charcter in the name or a leading or trailing space. Or if there is a non-existing file. Such files can block OneDrive from uploading or downloading files.

I have a Hazel routine which replaces illegal characters, and leading or trailing spaces in file names.

I have not experienced any problems of OneDrive not keeping files local when told to do so, but I can see why this may happen if OneDrive is on pause due to an illegal character in the file name.

I checked that my OneDrive was fully synched and up to date at the time of the tests.

Try turning on debug logging as described here: https://www.noodlesoft.com/kb/hazel-debug-mode/

After that, post updated logs.

Mr Noodle, as reqested the full debug mode log is below.

Problem: Copying a folder or folder structure from OneDrive to anywhere, (including to OneDrive itself) produces an error message, whereas copying single or multiple files does not.

Before starting this test I checked that all files are stored locally and that OneDrive is fully synched ("Your files are synched").

My folder structure on OneDrive looks like this:

My rule looks like this:

When running he rule the folders do not copy.

Here are the complete logs in DEBUG mode from running the rule:

2023-09-23 03:22:04.177 86Z3GCJ4MF.com.noodlesoft.HazelHelper[605] DEBUG: Thread 0x600002bb0200: Run worker for folder: /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder
2023-09-23 03:22:04.205 hazelworker[15747] Running worker (v5.2.2) for folder with identifier: 16777223-295310705.
2023-09-23 03:22:04.206 hazelworker[15747] ###Arch: x86_64
2023-09-23 03:22:04.206 hazelworker[15747] ###main load address: 0x102376000
2023-09-23 03:22:04.206 hazelworker[15747] ###Hazel Core load address: 0x102582000
2023-09-23 03:22:04.206 hazelworker[15747] ###Noodle load address: 0x10288e000
2023-09-23 03:22:04.206 hazelworker[15747] ###CK load address: 0x102474000
2023-09-23 03:22:04.212 hazelworker[15747] DEBUG: Program is licensed.
2023-09-23 03:22:04.245 hazelworker[15747] DEBUG: Error reading file /Users/myname/Library/Application Support/Firefox/prefs.js: Error Domain=NSCocoaErrorDomain Code=260 "The file “prefs.js” couldn’t be opened because there is no such file." UserInfo={NSFilePath=/Users/myname/Library/Application Support/Firefox/prefs.js, NSUnderlyingError=0x600000cec570 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
2023-09-23 03:22:04.263 hazelworker[15747] DEBUG: Could not find entry for default_directory in Chrome preference file.
2023-09-23 03:22:04.274 hazelworker[15747] Processing folder Source folder (forced)
2023-09-23 03:22:04.274 hazelworker[15747] DEBUG: Pausing to wait for things to settle down.
2023-09-23 03:22:06.349 hazelworker[15747] DEBUG: Processing directories: (
"/Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder"
)
2023-09-23 03:22:06.350 86Z3GCJ4MF.com.noodlesoft.HazelHelper[605] DEBUG: Checking events for path /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder, folder Source folder
2023-09-23 03:22:06.350 hazelworker[15747] DEBUG: About to process directory /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder
2023-09-23 03:22:06.352 hazelworker[15747] DEBUG: .DS_Store: File is hidden/invisible. Skipping.
2023-09-23 03:22:06.390 hazelworker[15747] DEBUG: File in source folder.pdf: Did not match any rules.
2023-09-23 03:22:06.392 hazelworker[15747] Source sub1 folder: Rule New Rule matched.
2023-09-23 03:22:06.392 hazelworker[15747] DEBUG: New rule signature. Executing actions.
Old signatures: (
)
New Signature:{typeObject isType: "public.folder"}:{(copy:/Users/myname/Documents iMac Pro/Test to delete/Destination folder,{
keepFolderStructure = 2;
replaceExisting = 0;
throwAwayDupes = 0;
})}
2023-09-23 03:22:06.392 hazelworker[15747] DEBUG: Performing clone operation.
2023-09-23 03:22:06.392 hazelworker[15747] DEBUG: Error cloning /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder/Source sub1 folder to /Users/myname/Documents iMac Pro/Test to delete/Destination folder/Source sub1 folder: Resource deadlock avoided
2023-09-23 03:22:06.392 hazelworker[15747] Error copying file from /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder/Source sub1 folder to /Users/myname/Documents iMac Pro/Test to delete/Destination folder/Source sub1 folder: (null)
2023-09-23 03:22:06.393 hazelworker[15747] [Error] Copy failed: Could not copy file Source sub1 folder from folder /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder to folder /Users/myname/Documents iMac Pro/Test to delete/Destination folder/Source sub1 folder
2023-09-23 03:22:06.393 hazelworker[15747] DEBUG: Tapping error retry sequence
2023-09-23 03:22:06.393 hazelworker[15747] DEBUG: Writing out DB file for /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder to path: /Users/myname/Library/Application Support/Hazel/16777223-295310705.hazeldb
2023-09-23 03:22:06.394 hazelworker[15747] DEBUG: Directory /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder processed in 0.043067 seconds
2023-09-23 03:22:06.394 86Z3GCJ4MF.com.noodlesoft.HazelHelper[605] DEBUG: Checking events for path /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder, folder Source folder
2023-09-23 03:22:06.394 hazelworker[15747] Received abort event.
2023-09-23 03:22:06.394 hazelworker[15747] DEBUG: Sleeping
2023-09-23 03:22:08.470 hazelworker[15747] DEBUG: About to process directory /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder
2023-09-23 03:22:08.470 hazelworker[15747] DEBUG: Directory /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder processed in 0.000414 seconds
2023-09-23 03:22:08.471 86Z3GCJ4MF.com.noodlesoft.HazelHelper[605] DEBUG: Checking events for path /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder, folder Source folder
2023-09-23 03:22:08.471 hazelworker[15747] Received abort event.
2023-09-23 03:22:08.471 hazelworker[15747] DEBUG: Sending metrics to scheduler. Next scheduled run: 4001-01-01 01:00:00.000
2023-09-23 03:22:08.471 hazelworker[15747] Done processing folder Source folder
2023-09-23 03:22:08.471 86Z3GCJ4MF.com.noodlesoft.HazelHelper[605] DEBUG: Received metrics for folder /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder: {
directoryDepth = 0;
requestedSchedulingTime = "4001-01-01 00:00:00 +0000";
triggerPaths = "<NoodlePathSet: 0x600003eafaa0>\n";
unavailablePaths = "{(\n)}";
}
2023-09-23 03:22:08.472 86Z3GCJ4MF.com.noodlesoft.HazelHelper[605] DEBUG: Timer scheduled for folder /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder at 4001-01-01 00:00:00 +0000
2023-09-23 03:22:08.475 86Z3GCJ4MF.com.noodlesoft.HazelHelper[605] DEBUG: Thread 0x600002bb0200: Task removed: [15747]
2023-09-23 03:22:17.196 86Z3GCJ4MF.com.noodlesoft.HazelHelper[605] DEBUG: Thread 0x600002bb0200: Timer fired for folder: /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder
2023-09-23 03:22:17.196 86Z3GCJ4MF.com.noodlesoft.HazelHelper[605] DEBUG: Thread 0x600002bb0200: Run worker for folder: /Users/myname/Library/CloudStorage/OneDrive-Personal/Test to delete/Source folder

The "Resource deadlock avoided" is an issue on OneDrive's end. It comes up if a file isn't synced (even if it says it is) but there are other cases as well. You'll need to contact them to resolve it.