7z .log files in Calender Week # archives

[Ultimatemoe]

Hi all

I have this idea of compressing and archiving log files created in the same calender week in one .7z file.

I have a dir that includes more than 100 files with a date in their name like this

Code: Select all

#test_20141207.log

and the name includes the date the file is created. I want to zip the files according to their file name.
So this means every file that ends with 01.log - 07.log should be ziped into one archive.

I already have functioning script working in the command line if I give the file as argument.

So in Hazel:
Rule matches on extenstion being log and file starting with #.

This is what I find in Hazel log

Code: Select all

2014-12-07 14:58:19.650 hazelworker[8825] DEBUG: Action changed file: #test_20140518.log
2014-12-07 14:58:19.655 hazelworker[8825] #test_20140519.log: Rule Weekly 7z matched.
2014-12-07 14:58:19.655 hazelworker[8825] DEBUG: Manual run. Forcing actions to be executed.
2014-12-07 14:58:19.727 hazelworker[8825] DEBUG: == script output ==

== End script output ==


Code: Select all

#!/usr/bin/python
import datetime
import re
import subprocess
import sys
import os

pathwithfile = str(sys.argv[1])
filepath = str(os.path.dirname(pathwithfile)) + "/"
filenameregex = re.compile(r'([^\/]+)$', re.I)

filename = str(filenameregex.search(filewithpath).groups())
dateregex = re.compile(r'(\d{8})(?=\.log)', re.I)

filedate = str(dateregex.search(filename).groups()[0])
syear = int(filedate[0] + filedate[1] + filedate[2] + filedate[3])
smonth = int(filedate[4] + filedate[5])
sday = int(filedate[6] + filedate[7])
# scw is the calender week according to the pieces from the file name
scw = str(datetime.date(syear, smonth, sday).isocalendar()[1])

szip = "7z a -t7z " + filepath +"KW" + str(scw) + "_archive.7z " + filepath + "*" + str(filedate) + "* -mx9 -mmt > /dev/null 2>&1"

subprocess.call(szip, shell=True)

sys.exit(0)


It won't zip the files and I think I catched everything correctly..
I played around with builtin features but couldn't find a solution to zip files in a way like this.

Oh and please excuse my copypasta style of putting this script together.
Some things might not be super efficient I know.

Any ideas? Thank you!

[Mr_Noodle]

Add print statements to the script to check your work along the way. Also, have you tried the script outside of Hazel?

[Ultimatemoe]

Add print statements to the script to check your work along the way. Also, have you tried the script outside of Hazel?

I already have functioning script working in the command line if I give the file as argument.

Yes it does work outside of hazel.
If you go ahead and put it in a .py file, run it with any file that has a name like this "#test_20141207.log" you will end up with a .7z file named KW47_archive.7z. (in this particular case where the file is named "20141207" like this)

I'll try the print statements though.

[Mr_Noodle]

Also, make sure you specify full paths as you can't rely on the environment variables to be set as they are in an interactive shell.

[Ultimatemoe]

Ok I fixed it thanks to your tip with environment variables!

But for future reference and other users here's what I found out.

Code: Select all

== script output ==

This is filewithpath: /Users/moritz/BTSync/Systeme/Pi/test/log/#file_20141206.log

This is filepath: /Users/moritz/BTSync/Systeme/Pi/test/log/

This is filename: ('#file_20141206.log',)

This is filedate: 20141206

This is szip: 7z a -t7z /Users/moritz/BTSync/Systeme/Pi/test/log/KW49_archive.7z /Users/moritz/BTSync/Systeme/Pi/test/log/*20141206* -mx9 -mmt > /dev/null 2>&1

== End script output ==

Printing the variables showed that I messed up the filename variable.
The mistake I made was this:

Code: Select all

filename = str(filenameregex.search(filewithpath).groups())

By accessing the first element of the list/array of the group I fixed it

Code: Select all

filename = str(filenameregex.search(filewithpath).groups()[0])

After that the output in the hazel logs was correct

Code: Select all

This is filename: #test_20141206.log

I then tried to figure out why the hazel deamon couldnt acces the 7z binary.
The reason is: it doesn't know where to find it - like Mr_Noodle said.
I then changed the 7z command from

Code: Select all

szip = "7z a -t7z " + filepath +"KW" + str(scw) + "_archive.7z " + filepath + "*" + str(filedate) + "* -mx9 -mmt > /dev/null 2>&1"

to

Code: Select all

szip = "/usr/local/bin/7z a -t7z " + filepath +"KW" + str(scw) + "_archive.7z " + filepath + "*" + str(filedate) + "* -mx9 -mmt > /dev/null 2>&1"

That was it. I ran the rule and the archives started showing up as I wanted them to.

Thank you for that tip with the environment variables that was key to this!
Maybe someone has a usecase like me and can use this script as a base to start

Here is the full working script

Code: Select all

#!/usr/bin/python
import datetime
import re
import subprocess
import sys
import os

filewithpath = str(sys.argv[1])
print "This is filewithpath: " + filewithpath + "\n"

filepath = str(os.path.dirname(filewithpath)) + "/"
print "This is filepath: " + filepath + "\n"

filenameregex = re.compile(r'([^\/]+)$', re.I)

filename = str(filenameregex.search(filewithpath).groups()[0])
print "This is filename: " + filename + "\n"

regex = re.compile(r'(\d{8})(?=\.log)', re.I)

filedate = str(regex.search(filename).groups()[0])
print "This is filedate: " + filedate + "\n"

syear = int(filedate[0] + filedate[1] + filedate[2] + filedate[3])
smonth = int(filedate[4] + filedate[5])
sday = int(filedate[6] + filedate[7])
scw = str(datetime.date(syear, smonth, sday).isocalendar()[1])

szip = "/usr/local/bin/7z a -t7z " + filepath +"KW" + str(scw) + "_archive.7z " + filepath + "*" + str(filedate) + "* -mx9 -mmt > /dev/null 2>&1"
print "This is szip:" + szip + "\n"

subprocess.call(szip, shell=True)

sys.exit(0)