Rule to remove color label when removing files from folder?

Is it possible to make a rule that removes the color label of files that are removed from a certain folder, such as the download folder?

Why this would be helpful: It is useful to color files in, for example, the download folder based on filetype and/or when they have been added to the folder. The problem is that when you drag the color labeled files to other folders they remain colored and often that makes no sense. So you have to remove the color label manually if you don't want a mosaic of colored files spread across your filesystem.

There's no way to do that now unless the folder where you drag the file is also a Hazel-monitored folder. While I've considered having some sort of rules you can run on files when they are moved it's very tricky tracking the file properly and in some cases, impossible, without, say having to install some sort of kernel extension to track every file operation, which could slow things down. Given how rarely the feature is used, I'm not sure if it's worth it.

Thanks for the reply. Thought that would be the case. You're probably right that the usability of such a feature is limited. But a shame since it precludes the use of file colour labeling based on how long time a file has been in a folder (unless you accept that they keep the color when moved).