File changes ignored or skipped

[scott.ainsworth]

I am attempting to use a shell script to automatically run jpegoptim (http://www.kokkonen.net/tjko/projects.html) on all files added or changed in a folder. Sometimes the script is triggered, sometimes it is not. I suspect a subtlety in Hazel's change detection heuristic. Here are the clues I have so far:

1. Works: Copy a new JPEG into the folder.
2. Fails: Copy the same JPEG into the folder again, replacing the existing, optimized JPEG.
3. Works: Delete the JPEG, wait at least 10 seconds, and copy the same JPEG again.
4. Fails: Delete the JPEG, wait 2 seconds or less, and copy the same JPEG again (this occasionally works).

It seems like Hazel ignores some changes to existing files (#2) and/or ignores multiple changes within a short (< 2 secs) time window (#4). Assuming this behavior is by design (and not a bug), Is there some why to avoid it and have Hazel recognized all changes (I'll resolve the potential race conditions in the shell script)?

[Mr_Noodle]

Can you post your rule? A lot depends on the conditions you specified.

[scott.ainsworth]

There are two rules for folder "Metadata & Art":

Rule 1: "Optimize Images"
If all of the following conditions are met

• Kind is image
  
• Name does not end with ".tmp"

Do the following to the matched file or folder

• Run shell script (embedded script):
  

  Code: Select all

  log="/Volumes/Data/Metadata & Art/optimize-images.log"
if ( file "$1" | fgrep JPEG >/dev/null ); then  # Only run on JPEGs
    echo jpegoptim -v --preserve --strip-all "$1" >>"$log"
    jpegoptim -v --preserve --strip-all "$1" >>"$log"
fi


  
  
• Display notification with pattern "Optimized JPEG <folder>/<file> (<time>)"

Rule 2: "Run rules on Subfolders"
If all of the following conditions are met

• Kind is folder

Do the following to the matched file or folder

• Run rules on folder contents:

Notes

1. "jpegoptim --preserve" keeps the file's last-modified datetime, which I though might be a problem. However, removing "--preserve" and allowing last-modified to be updated yielded the same results.
   
2. jpegoptim leaves the file unchanged if optimization does not yield a smaller file.
   
3. Running jpegoptim mulitple times on the same file using the same options always yields the same results (in computer science terms it is idempotent).
   
4. Thus, #2 and #3 prevent infinite loops.
   
5. When jpegoptim runs, it creates a temporary file in the same folder as the file being optimized (Thus the "name does not end with '.tmp'" rule).
   
6. I have tested in both the "Metadata & Art" folder and its subfolders, the results are the same.

Thanks for your help!
R/Scott G. Ainsworth

[Mr_Noodle]

The rule as it stands is set up to only run once. If you want it to run anytime there's a change to the file, you need to add something like "Date modified is after date last matched". Give that a shot.

[scott.ainsworth]

Thank you for the suggestion. I see my mental model of operations was off a bit. The file notification events do not trigger actions directly, rather they initiate Hazel's execution of rule sets.

So, I created two new rule sets. One triggered on last modified and on triggered on size change. So there are now 3 rules sets for image optimization, plus the subdirectory rule set. I found I had to place the new rule sets (#1 & #2) before the original rule set (#3)--not really sure why this is since its actions were not triggered. (It looks like a simple match is enough to terminate rule processing.)

Rule set 1: Optimize Images (Last Modified)

IF Kind is image AND name does not end with ".tmp" AND Date Last Modified is after Date Last Matched
THEN Run the bash script AND send a notification.

Rule set 2: Optimize Images (Size)

IF Kind is image AND name does not end with ".tmp" AND Size did change
THEN Run the bash script AND send a notification.

Rule set 3: Optimize Images [unchanged]

IF Kind is image AND name does not end with ".tmp"
THEN Run the bash script AND send a notification.

Rule set 4: Run rules on Subfolders [unchanged]

TESTS

1. Copied an image using Finder (Cmd-C, Cmd-V). Rule set 3 triggered.
2. Modified and saved the image using Pixelmator. Rule set 1 triggered.
3. Deleted the modified image and copied from the original again. No rule sets triggered (of course Date Last Modifed is now before Date Last Matched, but Size changed).
4. Retried #3. Still no rule sets triggered.
5. Hazel then began reprocessing all 1000+ images files (because I changed the rules?); so I waited for the reprocessing to finish and tried steps 1-4 again. This time everything worked.

So, adding the new rules accomplished my goals and there appears to be a "stabilization period" after rules are changed.

Thanks again for your help. I'll keep testing.

R/Scott G. Ainsworth

[Mr_Noodle]

Hard to say without the logs but it may be that Hazel was in a previous processing loop and didn't see some rule changes until the top of the next loop.

If you still run into issues with this, reply back.