Page 1 of 1

List of duplicate files

PostPosted: Tue Apr 22, 2008 6:33 pm
by PeterSacreas
Hi,


I am working a while now with my MacBook Pro and through time and after copying files from my "Windows past" to the MBP I sitting here with a lot of duplicate files.

Is there a rule to make a list of the duplicate files and the path so I can easily erase?

I can only find the "erase duplicate files" in Hazel.

Many thanks in advance,
Peter

Re: List of duplicate files

PostPosted: Wed Apr 23, 2008 3:32 am
by alastor933
I believe Hazel deals with files one by one, so there's no way you can have her compare 2 files sitting in different folders.

There's a lot of utilities that will do this - 38 matches for 'duplicate' at MacUpdate alone

Re: List of duplicate files

PostPosted: Thu Apr 24, 2008 8:22 pm
by Mr_Noodle
Hazel's function to remove duplicate downloads pertains to files that are downloaded twice. When you download/copy/move a file with the same name into a folder, depending on the program, it will rename it with a number. Hazel looks through these and compares them against the original and if they are indeed the same file (it compares their content) then it will throw them away. It's more of a focused feature than a general "remove all dupes" type function.

As alastor933 already pointed out, there are more specific tools out there that may do what you want.

Re: List of duplicate files

PostPosted: Thu Jun 05, 2008 6:27 pm
by bigmurph
Hazel looks through these and compares them against the original and if they are indeed the same file (it compares their content) then it will throw them away. It's more of a focused feature than a general "remove all dupes" type function.


Can you provide any more detail on how it compares them? I guess by saying that it compares their content, it does something like a MD5 compare to ensure that they are truly unique? I love the app Singular for this reason.

When I first saw this action, I wanted to be sure that if I had scanned some documents that are automatically named the same, but when hazel put them in a folder that they wouldn't be considered duplicates.

Re: List of duplicate files

PostPosted: Fri Jun 06, 2008 7:44 pm
by Mr_Noodle
It compares the files bit by bit. While I could compute md5 sums for every file it seems like it would be wasted effort since very few files actually end up being duplicates so for the few times when they need to be compared, there's nothing more reliable than bit by bit.

Sure, an md5 can be used for a quick up front check but if they are the same, I still feel compelled to do a bit-by-bit comparison. The chance of an md5 collision is pretty small but when the consequence is your file being thrown away, it's not a chance I want to take.