Feedback needed on multiple rules on the same file

For those of you who would like to have the ability to have Hazel match (and execute) multiple rules on a single file, here is your chance to help make that happen. Please provide the feedback below. The more complete it is, the better the picture I can get about how to go about implementing it and the more likely it will get implemented sooner rather than later.

1. Please provide a concrete example of how you would use this. If you cannot provide this, then please do not continue.

2. Hazel currently will re-run rules on a file if the rule it matches changes. So if Hazel matched rule A before and now matches rule B instead, rule B will get run. But if it keeps matching rule A as before, nothing will happen. With this in mind, apply the following scenario to the workflow you described in #1 above:

A file matches rules A, B, and C in that order.

Suppose it no longer matches A, but matches B and C. Do B and C get executed again?
Suppose it no longer matches B? Does A get executed again? How about C?
Suppose it now matches rule D after the others. Do A, B, and C get executed again before D?
Suppose it matches rule Z before rule A. Do A, B and C get executed again after Z?

3. Are you willing to accept numerous side effects of this feature? For example, if rule A moves the file outside the folder, certain pieces of metadata may not make sense anymore (folder level) or just won't work because of implementation details ("is among") for any subsequent rules.

I look forward to your answers.

Hi Mr_Noodle,

thanks for taking this into consideration!

Reading you questions I now understand why this is not an easy task. Please bear with me if I can not just give you a short answer because I think there is another issue here, namely the convention to run a rule only once per file:

1. It raises some questions as it is now:
- What if I change the conditions of a rule already matched? Will it run again if it still matches?
- What if I only change the actions? Will they run again?
- What if I duplicate a rule? Will the identical copy run again? Does this depend on if I put the identical copy above or below the already matched original rule?
- What if I rename, or touch, or tag the already matched file? Will the already matched rule still not get applied again? In other words: when does a file stay "the same" file?

I am sure the answers are documented somewhere, but my point is: this feature is not self explaining at all. It is confusing.

2. As you can see from your questions below: with continued matching, more not really intuitive design decisions will have to be made.

3. Consider what I think is a necessary feature for multi rule matching: a setting in each rule to either "Stop after this rule" or "Continue with next rule". Now, what if a rules has matched, but then I toggle this setting from "Stop after this rule" to "Continue with next rules"? Will Hazel still not re-run the rule, but honour my choice to continue with the following rules? This will get very obscure I am afraid, no matter if the answer is yes or no.

My suggestion is:

- If a rule matches a file, just run it. It should not matter if it did match before or not. I should not matter if any other rules match.

- If a user does not want to run a rule more than once, he/she has to make this explicit. I do this by appending a comment to the file and checking for the absence of the comment in my conditions. Maybe make this easier by offering a default condition/action like this. But often I can easily check in my conditions if the action was applied already, for example if the action did set a label.

- In each rule, add an option to "Stop after this rule". Make the state of the option visible in the list of all folder rules (a little "Stop" sign, maybe).

With this, rules would become easy to design and intuitive. I understand this is a big change, so maybe in a new major version? Or in a new product?

Still, let me answer your questions assuming the "match a rule only once" mechanism must stay:

Here is my scenario:

1. Please provide a concrete example of how you would use this. If you cannot provide this, then please do not continue.

In my Downloads folder, I would like to:

1. Label every new file/folder (created <= 1 day) green
2. Unlabel every non-new file/folder (1 day < created <= 4 weeks)
3. Label every old file/folder (4weeks < created) red
4. Rename every file/folder according to the following scheme (e.g.: "New.File_2011.txt" -> "New File 2011.txt"):
"." -> " "
"_" -> " "
5. Move every file of kind movie to my Movies folder
6. Move every file with extension ".txt" to my Documents folder
7. Apply the rules recursively to any folder

2. Hazel currently will re-run rules on a file if the rule it matches changes. So if Hazel matched rule A before and now matches rule B instead, rule B will get run. But if it keeps matching rule A as before, nothing will happen. With this in mind, apply the following scenario to the workflow you described in #1 above:

A file matches rules A, B, and C in that order.

Suppose it no longer matches A, but matches B and C. Do B and C get executed again?

B and C do not depend on A. So, with the current mechanism, still do not run them again.

Suppose it no longer matches B? Does A get executed again? How about C?

A and C do not depend on B. So, with the current mechanism, still do not run them again.

Suppose it now matches rule D after the others. Do A, B, and C get executed again before D?

A, B and C do not depend on D. So, with the current mechanism, still do not run them again.

Suppose it matches rule Z before rule A. Do A, B and C get executed again after Z?

A, B and C do not depend on Z. So, with the current mechanism, still do not run them again.

3. Are you willing to accept numerous side effects of this feature? For example, if rule A moves the file outside the folder, certain pieces of metadata may not make sense anymore (folder level) or just won't work because of implementation details ("is among") for any subsequent rules.

No, because this would be very confusing. I would expect that if a rule changes a file, then the next rule will see the changed file (or not see it at all, if it was moved away). I understand that this would have an impact on performance, but this may be lessened by an intelligent caching mechanism: cache all file attributes in memory, but make certain actions invalidate certain cached attributes (e.g., changing the name does not invalidate the cached label, but running an external script invalidates all files - or at least this one).

Hope this makes sense, :-)

Olaf

xor2000 wrote:
- What if I change the conditions of a rule already matched? Will it run again if it still matches?

Yes. Because the rule changed.

xor2000 wrote:- What if I only change the actions? Will they run again?

Yes.

xor2000 wrote:- What if I duplicate a rule? Will the identical copy run again? Does this depend on if I put the identical copy above or below the already matched original rule?

It matches using a signature so if the rule is identical, it won't run again.

xor2000 wrote:- What if I rename, or touch, or tag the already matched file? Will the already matched rule still not get applied again? In other words: when does a file stay "the same" file?

If you rename, Hazel will attempt to correlate them but for various reasons depending on the filesystem as such, it may not be able to track it.

Note that it is intuitive based on years of observing behavior from thousands of users. Yes, some cases people want the opposite but that is more the exception than the rule, and again, only applies in certain cases. Hazel cannot read your mind so by default it uses the one which will cause the least grief. The example of a rule which renames or appends something each time: this is still an issue now that I have to deal with in the uncommon case that Hazel can't track the file. This causes tons of confusion and not only that, it results in infinite loops. Requiring the user to always filter out the file would result in tons of issues. Also, re-running the rules every time would be a big resource drain (you may underestimate how often Hazel may end up scanning a folder). There's no need to re-run something when nothing has changed.

As I've stated in the other thread, please stick to your personal use cases and let me interpret from the results. I'm looking to identify the problem, not hear what people consider to be the solution. Also, as stated there, I will not be changing the default to allow multiple matches so a "continue" action will be far more likely than a "stop" one.

xor2000 wrote:Still, let me answer your questions assuming the "match a rule only once" mechanism must stay:

Here is my scenario:

In my Downloads folder, I would like to:

1. Label every new file/folder (created <= 1 day) green
2. Unlabel every non-new file/folder (1 day < created <= 4 weeks)
3. Label every old file/folder (4weeks < created) red
4. Rename every file/folder according to the following scheme (e.g.: "New.File_2011.txt" -> "New File 2011.txt"):
"." -> " "
"_" -> " "
5. Move every file of kind movie to my Movies folder
6. Move every file with extension ".txt" to my Documents folder
7. Apply the rules recursively to any folder

2. Hazel currently will re-run rules on a file if the rule it matches changes. So if Hazel matched rule A before and now matches rule B instead, rule B will get run. But if it keeps matching rule A as before, nothing will happen. With this in mind, apply the following scenario to the workflow you described in #1 above:

A file matches rules A, B, and C in that order.

Suppose it no longer matches A, but matches B and C. Do B and C get executed again?

B and C do not depend on A. So, with the current mechanism, still do not run them again.

Suppose it no longer matches B? Does A get executed again? How about C?

A and C do not depend on B. So, with the current mechanism, still do not run them again.

Suppose it now matches rule D after the others. Do A, B, and C get executed again before D?

A, B and C do not depend on D. So, with the current mechanism, still do not run them again.

Suppose it matches rule Z before rule A. Do A, B and C get executed again after Z?

A, B and C do not depend on Z. So, with the current mechanism, still do not run them again.

No, because this would be very confusing. I would expect that if a rule changes a file, then the next rule will see the changed file (or not see it at all, if it was moved away). I understand that this would have an impact on performance, but this may be lessened by an intelligent caching mechanism: cache all file attributes in memory, but make certain actions invalidate certain cached attributes (e.g., changing the name does not invalidate the cached label, but running an external script invalidates all files - or at least this one).

It will see the changed file. The problem, as I said, is that certain attributes will not make sense. What is the folder level if the file is no longer in the same folder hierarchy? The issues have nothing to do with caching, which is already done. There are certain things which require some data that was recorded inside the folder hierarchy being monitored that were not recorded outside of it. Certain attributes or operators will not work. If you find that too confusing then that will go into my assessment of the feature.

Thanks for your feedback on your use case. It'll be interesting to see if others have similarly independent rules or if there are more dependencies between them.

Mr_Noodle wrote:
xor2000 wrote:I would expect that if a rule changes a file, then the next rule will see the changed file (or not see it at all, if it was moved away).

It will see the changed file. The problem, as I said, is that certain attributes will not make sense. What is the folder level if the file is no longer in the same folder hierarchy?

If the file is no longer in the folder, then I think it would be logical that no subsequent rules would be applied to the file: is it "gone".

That was an initial thought but if someone explicitly says "continue" for a rule that moves the file, that would indicate to me that they would want the file to be processed still.

1. Please provide a concrete example of how you would use this. If you cannot provide this, then please do not continue.

I'll echo here something I posted in another thread yesterday. Disclaimer: I just started evaluating Hazel for my needs so I could be overlooking existing features or their proper usage.

The monitored folder is a high level folder, such as Pictures, which contains hundreds of sub folders. Aperture will periodically and automatically add new files into deeply nested folders. I would like Hazel to automatically find movies placed in these folders and move them to a mirror folder structure in Movies, all the while leaving an alias of the movie files in their original location.

So I think the set of rules for the monitored folder would look like this:
Rule 1:

Rule 2: make local alias

Rule 3: move file

Assumptions:
- The operating system properly updates the alias when the file is moved
- Rules 2 and 3 get executed sequentially so that a file is only moved after its alias has been created

Optionally, a third rule could look for aliases and delete the "-1" added at the end of the file name when the alias is moved in the same folder as the original file.

2. Hazel currently will re-run rules on a file if the rule it matches changes. So if Hazel matched rule A before and now matches rule B instead, rule B will get run. But if it keeps matching rule A as before, nothing will happen. With this in mind, apply the following scenario to the workflow you described in #1 above:

A file matches rules A, B, and C in that order.
Suppose it no longer matches A, but matches B and C. Do B and C get executed again?
Suppose it no longer matches B? Does A get executed again? How about C?
Suppose it now matches rule D after the others. Do A, B, and C get executed again before D?
Suppose it matches rule Z before rule A. Do A, B and C get executed again after Z?

I see how these are important questions for the logical data flow in Hazel. However, I don't think this is relevant to the particular workflow I described above since upon success the target files are gone and will not be processed again.
My general 2 cents: as a newbie user I have an expectation (maybe misguided) that if I change anything to a rule set then when it is applied it is effectively as if it had been created from scratch. So within that naive mindset the answer to your questions would be yes in all cases: I made a new rule set, let Hazel apply it, regardless of past Hazel actions.

3. Are you willing to accept numerous side effects of this feature? For example, if rule A moves the file outside the folder, certain pieces of metadata may not make sense anymore (folder level) or just won't work because of implementation details ("is among") for any subsequent rules.

My answer would be yes.
In the case of a move one could imagine metadata capturing both before and after locations.

Again, I'm hoping I'm not generating noise for lack of proper understanding of Hazel's features and intended workflow. Time to spend more time with the documentation.

Xavier

Actually, I think in your example, you do not need separate rules at all. I think it is all doable as multiple actions within a single rule so your workflow may not be as applicable here.

Mr_Noodle wrote:Actually, I think in your example, you do not need separate rules at all. I think it is all doable as multiple actions within a single rule so your workflow may not be as applicable here.

You are right, possible solutions for my particular example are discussed here http://www.noodlesoft.com/forums/viewtopic.php?f=4&t=1563&p=11580#p6323.

Xavier

Mr_Noodle wrote:For those of you who would like to have the ability to have Hazel match (and execute) multiple rules on a single file, here is your chance to help make that happen. Please provide the feedback below. The more complete it is, the better the picture I can get about how to go about implementing it and the more likely it will get implemented sooner rather than later.

Hi Mr_Noodle,

Happy new year!

Really appreciate you considering this feature, and discussing this with your users! There is so much power in it (like described in http://www.noodlesoft.com/forums/viewtopic.php?f=4&t=1163&p=11511&hilit=multiple#p11486). xor2000 had an excellent suggestion on having a simple option like:

"Run no further rules on this item" (which would be the default), or
"Run next rule on this item"

This would be a great solution because:
A. It wouldn't any break existing rules
B. It would not confuse previous Hazel users as long as the "Run no further rules on this item" is the default option. Of course the phrasing could be optimized if necessary.

Mr_Noodle wrote:1. Please provide a concrete example of how you would use this. If you cannot provide this, then please do not continue.

To answer your questions:

1. See my usage scenario in the separate post at http://www.noodlesoft.com/forums/viewtopic.php?f=4&t=1163&p=11511&hilit=multiple#p11486

2. Hazel currently will re-run rules on a file if the rule it matches changes. So if Hazel matched rule A before and now matches rule B instead, rule B will get run. But if it keeps matching rule A as before, nothing will happen. With this in mind, apply the following scenario to the workflow you described in #1 above:

A file matches rules A, B, and C in that order.

Suppose it no longer matches A, but matches B and C. Do B and C get executed again?

No, it should only go through the same rules once, except if you check a box called "Re-run rules on new matches" (or a similar name). This checkbox should be off as default to avoid crazy loops and to make it more user friendly. However it would add even more flexibility to have this option there for power users.

Suppose it no longer matches B? Does A get executed again? How about C?

If the power user option above is off (the default), it would not execute A or C again. However if it's on, it should execute both A and C.

Suppose it now matches rule D after the others. Do A, B, and C get executed again before D?

A, B, and C ONLY get executed again before D, if the above power user option is on. But as default, no.

Suppose it matches rule Z before rule A. Do A, B and C get executed again after Z?

No, unless the option above is on.

3. Are you willing to accept numerous side effects of this feature? For example, if rule A moves the file outside the folder, certain pieces of metadata may not make sense anymore (folder level) or just won't work because of implementation details ("is among") for any subsequent rules.

Yeah, definitely - that would be expected as I see it. It makes sense and seems logical that Hazel can't do some things, when the prerequisites for doing those actions are gone. I would not consider it a side effect, but a quite natural consequence of this feature.

I look forward to your answers.[/quote]

P.S. On your comment below:

If someone explicitly says "continue" for a rule that moves the file, that would indicate to me that they would want the file to be processed still.

Yes, exactly. It should definitely keep on processing the rules no matter the kind of rules triggered previously (file move, file label, file rename, whatever). That would be the expected/logical action as I see it. The file move action is one of the most useful actions in Hazel. Making it any different and excluding file move actions from the "run multiple rules-feature" would be confusing IMHO and would cripple the feature. Also I see many workflows that won't be possible to build.

Those were my 5 cents. I hope it is useful.

Looking forward to hearing more news on this feature! I'd also be happy to beta test if you need help with that.

Thanks for the input. I'm not sure if I want to make the rules re-running thing an option as I think that complicates things. I'm more concerned with a reasonable behavior in most cases. Can you describe what kinds of loops (or other undesirable behavior) you would see happening when this behavior is turned on?

Mr_Noodle wrote:Thanks for the input. I'm not sure if I want to make the rules re-running thing an option as I think that complicates things. I'm more concerned with a reasonable behavior in most cases. Can you describe what kinds of loops (or other undesirable behavior) you would see happening when this behavior is turned on?

To answer your questions.

1. Based on the way rules are setup in Hazel (each one below another other one) in my mind at least it seems logical that it would stop when it would reach the last rule.
2. If it would continue running through the rules repeatedly, wouldn’t it mean Hazel would reprocess the same item with the same rule again and again, unless the condition that triggered the rule suddenly does not apply anymore?

In for instance a file checking workflow, where Hazel would check if a file doesn’t match the correct video codec, and then add a suffix like “_wrong_codec”, I imagine Hazel would keep adding this suffix again and a again, unless you put in an extra criteria like “If name does not contain _wrong_codec” in the hazel rule. This could result in a loop and a filename that keeps on expanding again and again (unless of course you add the extra criteria above).

I’m thinking you might risk trying to create some rules, where you find out that you simply can’t add an extra condition to avoid the loops you want to avoid. I hope it makes sense?

Maybe you won’t be able to create a scenario where it will be a problem, maybe you will. I don’t know, but I can imagine it might happen, leading to further complications. Personally though I think the more functionality and flexibility the better, so if you could avoid the above situation I would be all for it. What advantages/disadvantages do you see?

Well, the renaming issue is a problem currently. Hazel does try to avoid it but because of some quirks in the filesystem, it doesn't always work.

The notion of re-processing is that currently, if you change a rule, it will set it up so Hazel will run it again. The notion here being that you changed the workflow and want files to do things the new way. Likewise, if you think of the multiple rules as a workflow, any change to anything in that workflow indicates that the workflow as a whole changes.

Ok, that clarifies things. It will probably work great with the reprocessing of rules. Looking forward to any news on this.

Another call for multiple rules being triggered for one file in a folder.

I share my training material with different groups of people for different reasons.

Each group should get its own named rule. Each rule should fire different actions:

#1 Course Attendees Dropbox should be updated with the latest version of the course materials everytime I save a PDF. (This is the reason I purchased Hazel)
...#1a Exception - we don't copy the test handout

#2 Trainer Partners get a copy of most handouts to print (they’ve complained about printing costs so some handouts are skipped).

#3 Other Trainers get a readonly copy of my material via dropbox - this is a whole folder copy with the .git files dropped

Sample File names and rules that should be triggered:

My goal is to have rules that are independent of each other in the same folder.

As you can see each group has slightly different needs. I'm sure I could jam it all into one rule but that won't make sense to a future me that has to figure why I did what, especially when I upgrade to a MacBookAir/Pro with more SSD space.

Does that help clarify the need?
BTW the Image is in my Dropbox - here: https://www.dropbox.com/s/wwoozm8q7a0kw ... _stuff.jpg - I can't get the phpBB to display it.

Hi Mr_Noodle,

I have a scenario that I've created that would likely be applicable to this. I took a cascading folder approach that works nicely. It's a filing system for scanned documents to create notes and name notes in existing Evernote notebooks.

Here's how it currently works:

File gets scanned to a folder called "EvernoteScansForHazel," where it sits for 2 minutes, giving Acrobat time to OCR it.
After the 2 minutes are up, it then gets moved to a folder named "NameSpecificFiles" where it names the files based on content rules and gets moved to 'SortToProperty'
'SortToProperty' is my notebook filing structure (I'm a real estate developer so the bulk of my papers are property related) where it uses more content rules to identify which noteboook the document belongs in, then creates the note it the appropriate notebook. If there isn't a match for notebooks, it simply creates the note in my notebook called ~Inbox
I'm about to add another step to this process, after my NameSpecificFiles, to append the names of the existing with specific tags based on some content matching.

This system/approach actually works really well and is simple for even mere mortals to follow and think through For example, by moving my already named folder to another folder to add the tags, I don't have to work through an 'if/then/else' scenario via some external method like AppleScript. It does have some challenges though. Because I'm moving the files to new folders, adding a new folder/step means that I need to change a specific action in each rule (of which I have multiples applying to each folder).

For example, since I'm going to add another step to add a tag, I have to sort all files through another folder, meaning that I need to go into the rules for the folder before it, and change all of the "move to folder" steps to the new folder's name.

Adding one feature to Hazel in this scenario, would make it easier for this method to be extremely easy to use. If at the bottom of the panel, where it says "Throw Away:", add a "Next Action:", meaning next action on the entire folder. Here you could have the option to Move to another folder (and I'm guessing down the line, others "Next Actions"). Of course, in that next folder, an entirely new set of rules can be applied.

In my scenario, I don't want any of my existing files to be run through a set of rules again, so I can't comment on that.

I hope that helps,

Lynn