Page 1 of 1

File changes ignored or skipped

PostPosted: Sun Jan 08, 2017 3:11 pm
by scott.ainsworth
I am attempting to use a shell script to automatically run jpegoptim (http://www.kokkonen.net/tjko/projects.html) on all files added or changed in a folder. Sometimes the script is triggered, sometimes it is not. I suspect a subtlety in Hazel's change detection heuristic. Here are the clues I have so far:

  1. Works: Copy a new JPEG into the folder.
  2. Fails: Copy the same JPEG into the folder again, replacing the existing, optimized JPEG.
  3. Works: Delete the JPEG, wait at least 10 seconds, and copy the same JPEG again.
  4. Fails: Delete the JPEG, wait 2 seconds or less, and copy the same JPEG again (this occasionally works).
It seems like Hazel ignores some changes to existing files (#2) and/or ignores multiple changes within a short (< 2 secs) time window (#4). Assuming this behavior is by design (and not a bug), Is there some why to avoid it and have Hazel recognized all changes (I'll resolve the potential race conditions in the shell script)?

Re: File changes ignored or skipped

PostPosted: Mon Jan 09, 2017 12:23 pm
by Mr_Noodle
Can you post your rule? A lot depends on the conditions you specified.

Re: File changes ignored or skipped

PostPosted: Mon Jan 09, 2017 2:40 pm
by scott.ainsworth
There are two rules for folder "Metadata & Art":

Rule 1: "Optimize Images"
If all of the following conditions are met
  • Kind is image
  • Name does not end with ".tmp"
Do the following to the matched file or folder
  • Run shell script (embedded script):
    Code: Select all
    log="/Volumes/Data/Metadata & Art/optimize-images.log"
    if ( file "$1" | fgrep JPEG >/dev/null ); then  # Only run on JPEGs
        echo jpegoptim -v --preserve --strip-all "$1" >>"$log"
        jpegoptim -v --preserve --strip-all "$1" >>"$log"
    fi

  • Display notification with pattern "Optimized JPEG <folder>/<file> (<time>)"

Rule 2: "Run rules on Subfolders"
If all of the following conditions are met
  • Kind is folder
Do the following to the matched file or folder
  • Run rules on folder contents:

Notes
  1. "jpegoptim --preserve" keeps the file's last-modified datetime, which I though might be a problem. However, removing "--preserve" and allowing last-modified to be updated yielded the same results.
  2. jpegoptim leaves the file unchanged if optimization does not yield a smaller file.
  3. Running jpegoptim mulitple times on the same file using the same options always yields the same results (in computer science terms it is idempotent).
  4. Thus, #2 and #3 prevent infinite loops.
  5. When jpegoptim runs, it creates a temporary file in the same folder as the file being optimized (Thus the "name does not end with '.tmp'" rule).
  6. I have tested in both the "Metadata & Art" folder and its subfolders, the results are the same.

Thanks for your help!
R/Scott G. Ainsworth

Re: File changes ignored or skipped

PostPosted: Tue Jan 10, 2017 11:47 am
by Mr_Noodle
The rule as it stands is set up to only run once. If you want it to run anytime there's a change to the file, you need to add something like "Date modified is after date last matched". Give that a shot.

Re: File changes ignored or skipped

PostPosted: Tue Jan 10, 2017 5:20 pm
by scott.ainsworth
Thank you for the suggestion. I see my mental model of operations was off a bit. The file notification events do not trigger actions directly, rather they initiate Hazel's execution of rule sets.

So, I created two new rule sets. One triggered on last modified and on triggered on size change. So there are now 3 rules sets for image optimization, plus the subdirectory rule set. I found I had to place the new rule sets (#1 & #2) before the original rule set (#3)--not really sure why this is since its actions were not triggered. (It looks like a simple match is enough to terminate rule processing.)

Rule set 1: Optimize Images (Last Modified)
    IF Kind is image AND name does not end with ".tmp" AND Date Last Modified is after Date Last Matched
    THEN Run the bash script AND send a notification.

Rule set 2: Optimize Images (Size)
    IF Kind is image AND name does not end with ".tmp" AND Size did change
    THEN Run the bash script AND send a notification.

Rule set 3: Optimize Images [unchanged]
    IF Kind is image AND name does not end with ".tmp"
    THEN Run the bash script AND send a notification.

Rule set 4: Run rules on Subfolders [unchanged]

TESTS

  1. Copied an image using Finder (Cmd-C, Cmd-V). Rule set 3 triggered.
  2. Modified and saved the image using Pixelmator. Rule set 1 triggered.
  3. Deleted the modified image and copied from the original again. No rule sets triggered (of course Date Last Modifed is now before Date Last Matched, but Size changed).
  4. Retried #3. Still no rule sets triggered.
  5. Hazel then began reprocessing all 1000+ images files (because I changed the rules?); so I waited for the reprocessing to finish and tried steps 1-4 again. This time everything worked.

So, adding the new rules accomplished my goals and there appears to be a "stabilization period" after rules are changed.

Thanks again for your help. I'll keep testing.

R/Scott G. Ainsworth

Re: File changes ignored or skipped

PostPosted: Wed Jan 11, 2017 2:59 pm
by Mr_Noodle
Hard to say without the logs but it may be that Hazel was in a previous processing loop and didn't see some rule changes until the top of the next loop.

If you still run into issues with this, reply back.