Home About Articles Resume Contact

Software

iPlaylist Copier

Java Libraries

ItunesUtilities

Scripts

Bash Scripts

File Size File Size Test Free Space Line Counter List Copy MD5 Tool OS X Rotate Logs Rename It Snapshot

PHP Scripts

Write Variable To File WWS Mailer

Javascript Scripts

WWS Background Scaler

Interesting People

Admins

Phil Glockner

Designers

Tonya Browning Catherine Chu Justin Cox Richard Georges Stephen Gray Jonathan Horak Justin Kiehl Andrea Spencer Meathead

Developers

Chris Austin Joost van Dongen Noah Figg Rick Hogge John Quarles Wayne Rittiman Joe Rivera Brent Schneeman David Scott Andy Skelton Pavan Tumati

 

Custom Log File Rotation In OS X

It's not easy to figure out how to configure OS X's internal periodic tasks. I googled around a bit for info on integrating your own log rotation scheme into OS X's existing scheduled tasks, but I could not find any great documentation on the matter, so here's how I solved the problem.

How OS X's log rotation works

log rotation

OS X (10.4) automaticly rotates your log files for you on a weekly basis. It first rotates older logs, then it gzips your current log file up, and then it creates a new log file for your system to continue logging to.

The log files are gzipped into a file like logfile.0.gz where "logfile" is the name of the logfile in question. For example my apache access log would be zipped into access_log.0.gz.

The log files are rotated by incrementing the number in the filename, so logfile.0.gz is renamed to logfile.1.gz before a new logfile.0.gz is made. This incrementing rotation continues for 4 or 5 steps. So eventually you are losing whatever is in logfile.5.gz because it won't be renamed to logfile.6.gz.

log deletion

OS X also DELETES your archived log files on a daily basis. The daily scheduled task looks for log files that are have not been modified in X many days and deletes them. This is problematic because you're losing logs you may not want to lose. For example, on my system I simply edited the weekly log rotation scheme to timestamp each new logfile.gz file into something like logfile.2007-11-15.gz so I wouldn't overwrite older gzipped logs, but the daily deletion task started removing older gzipped logfiles.

Now that we understand how the log rotation is working, it's time for us to dig in and figure out how we can hook into OS X's scheduled tasks and get our own log rotation scheme going.

Integrating with OS X's scheduled tasks

OS X (10.4) uses its proprietary launchd and launchctl programs to run scheduled tasks at selected times. When your system boots up lanunchd reads through the files in your /System/Library/LaunchDaemons directory and discovers which tasks to run at boot and which tasks to run at certain times. We could build our own something.plist file that'd call our own log rotation script and put that file in /System/Library/LaunchDaemons, but for my purposes I wanted to make sure my log rotation script was run as a part of the OS X weekly system task that was already doing log rotation.

OS X's weekly system task is specified in the file /System/Library/LaunchDaemons/com.apple.periodic-weekly.plist. That file looks like this:

The file basically dictates that launchd should run the command /usr/sbin/periodic weekly at 3:15 in the morning on the 6th day of the week.

So, the next question is, what's the periodic program? Well, if we run man periodic we see that there are scripts in the /etc/periodic directory that are run by the program depending on the argument passed in. There are several directories under /etc/periodic such as /etc/periodic/daily and /etc/periodic/weekly. When we run periodic weekly the periodic program looks in the /etc/periodic/weekly directory and executes any scripts it finds there.

On my system, the only file in the weekly directory is a file called 500.weekly. That file contains a few tasks that are run as part of the weekly task, including the log rotation.

There is an interesting section near the bottom of the 500.weekly file:

That code says to run the /etc/weekly.local file if that file exists. That file did not exist on my OS X system, so I created it and edited it to call my own script like so:

Now OS X is going to run it's internal log rotation scheme every week and gzip each of my log files to something like logfile.0.gz, and then it will run /archive/scripts/osxrotatelogs.sh before the weekly task ends.

Using osxrotatelogs.sh on your own machine

For my log rotation scheme I wanted to do two things:

1. Timestamp the log file and gzip the file into something like logfile.2007-11-15 and logfile.2007-11-15.gz. This way my log files have easily sortable information in my filename, and I can unzip a bunch of the logs into one directory without worrying about overwriting previously unzipped logs.
2. Copy the timestamped gzip file to a directory where OS X's daily scheduled task wouldn't delete old log files.

I wrote the osxrotatelogs.sh script to solve both of those problems. You can use the script on your own osx machine if you'd like. Simply download the osxrotatelogs.sh file to some place on your computer, then execute chmod +x osxrotatelogs.sh in your terminal to make the script executable. Then edit your /etc/weekly.local file to call the osxrotatelogs.sh script.