Unpicking and sorting out TBs of Photos/Videos

Hi, given the state of the world I've decided to try do a virtual spring clean of a myriad of my external hard drives from years of family photos.
I've about 10Tb of files, of which I think at least 3Tb are backups.
I've moved everything to a new huge RAID and tried to move everything into a central folder structure and am trying to use Hazel to do the heavy lifting.
Desired outcome is the following structure:
/<externaldrive>/Photos/YEAR/YYYY-MM-DD/
/<externaldrive>/Home Movies/YEAR/YYYY-MM-DD/<DeviceModel>
With a view to importing the photos into CaptureOne/Lightroom per year and the videos into FCPX libraries per year.
I've run a few rules as described below and then a summary of the issues I've faced so far. If anyone has any tips to help this process, I would be immensely grateful!
Photos Strategy
1)Remove empty folders
IF:
Kind is folder
Sub-file/folder Count is 0
DO:
Move to trash
2) Sort photos by date taken
IF:
Kind is Image or Extension is NEF
DO:
Move to folder: SortedPhotos
Sort into subfolder with pattern: Date taken (hazel) YYYY > Date taken (hazel) YYYY-MM-DD
3)Go into subfolders:
IF: Kind is Folder
DO: Run rules on folder contents
Throw away Duplicate files.
Video Strategy
1)Remove empty folders
IF:
Kind is folder
Sub-file/folder Count is 0
DO:
Move to trash
2) Sort videos by date taken
IF:
Kind is Movie
DO:
Move to folder: SortedPhotos
Sort into subfolder with pattern: Date taken (hazel) YYYY > Date taken (hazel) YYYY-MM-DD > device Model
3)Go into subfolders:
IF: Kind is Folder
DO: Run rules on folder contents
Throw away Duplicate files.
1) Many files do not have exif info, as I did a dump down from FLICKR and somehow many of them lost their date taken.
2) Files with the same file name have been sent to the trash during this process, even though they are completely different files
3) Many files with different names, but that are duplicates -1, -2, (1), remain.
To help with 1, I wrote another rule per 'unsorted' files like ones that ended up in 0001-12-31 > 0001, I rewrote the move rule to use "Date Created" which at least helps group similar ones together to manually try and sort.
I tried using tools like Duplicate Detective 2 to remove a bunch of these post-hazel to get rid of the -1 -2 files, but their auto select always wants to pick to keep the -1 or -2 files, and delete the 'original' which irks my OCD somewhat, since they have the same date-time stamp (I've contacted their support on that but guessing reply will be "It doesn't do that, sorry"). I thought the "Throw away Duplicate Files" rule in Hazel would solve this but I can't get it working.
Anyone got hazel rules ninja skills enough to help me combine my above rules together so that in one go, it will:
1) Remove exact duplicates with the same name, or -1, -2, (1)
2) Whilst also sorting into a folder based on date taken, as long as the date taken exists, and if it doesn't, do it based on "Date Created"
please?
Many thanks in advance!
I've about 10Tb of files, of which I think at least 3Tb are backups.
I've moved everything to a new huge RAID and tried to move everything into a central folder structure and am trying to use Hazel to do the heavy lifting.
Desired outcome is the following structure:
/<externaldrive>/Photos/YEAR/YYYY-MM-DD/
/<externaldrive>/Home Movies/YEAR/YYYY-MM-DD/<DeviceModel>
With a view to importing the photos into CaptureOne/Lightroom per year and the videos into FCPX libraries per year.
I've run a few rules as described below and then a summary of the issues I've faced so far. If anyone has any tips to help this process, I would be immensely grateful!
Photos Strategy
1)Remove empty folders
IF:
Kind is folder
Sub-file/folder Count is 0
DO:
Move to trash
2) Sort photos by date taken
IF:
Kind is Image or Extension is NEF
DO:
Move to folder: SortedPhotos
Sort into subfolder with pattern: Date taken (hazel) YYYY > Date taken (hazel) YYYY-MM-DD
3)Go into subfolders:
IF: Kind is Folder
DO: Run rules on folder contents
Throw away Duplicate files.
Video Strategy
1)Remove empty folders
IF:
Kind is folder
Sub-file/folder Count is 0
DO:
Move to trash
2) Sort videos by date taken
IF:
Kind is Movie
DO:
Move to folder: SortedPhotos
Sort into subfolder with pattern: Date taken (hazel) YYYY > Date taken (hazel) YYYY-MM-DD > device Model
3)Go into subfolders:
IF: Kind is Folder
DO: Run rules on folder contents
Throw away Duplicate files.
1) Many files do not have exif info, as I did a dump down from FLICKR and somehow many of them lost their date taken.
2) Files with the same file name have been sent to the trash during this process, even though they are completely different files
3) Many files with different names, but that are duplicates -1, -2, (1), remain.
To help with 1, I wrote another rule per 'unsorted' files like ones that ended up in 0001-12-31 > 0001, I rewrote the move rule to use "Date Created" which at least helps group similar ones together to manually try and sort.
I tried using tools like Duplicate Detective 2 to remove a bunch of these post-hazel to get rid of the -1 -2 files, but their auto select always wants to pick to keep the -1 or -2 files, and delete the 'original' which irks my OCD somewhat, since they have the same date-time stamp (I've contacted their support on that but guessing reply will be "It doesn't do that, sorry"). I thought the "Throw away Duplicate Files" rule in Hazel would solve this but I can't get it working.
Anyone got hazel rules ninja skills enough to help me combine my above rules together so that in one go, it will:
1) Remove exact duplicates with the same name, or -1, -2, (1)
2) Whilst also sorting into a folder based on date taken, as long as the date taken exists, and if it doesn't, do it based on "Date Created"
please?
Many thanks in advance!