Cron & logrotate: How To Use Cron To Automate Tasks

How to Manage Log Files in Linux/Unix Servers

What is logrotate?

image src: The Geek Stuff

Logrotate is a tool used to manage log files created by system processes. It automatically compresses and removes logs to maximize the convenience of logs and conserve system resources. This is accomplished through automatic rotation, compression, removal, and mailing of log files.

Logrotate is designed to simplify the administration of systems that generate large numbers of log files. It also provides extensive control to the system administrator over how and when log rotation should be processed. Each log file rotation might be handled daily, weekly, monthly, or when it grows too large. Normally, logrotate is run as a daily cron job.

What is Cron?

The software utility cron is a time-based job scheduler in Unix-like computer operating systems, including Linux distributions. Users that set up and maintain software environments use cron to schedule jobs (commands or shell scripts) to run periodically at fixed times, dates, or intervals.

Learn more about cron at https://crontab.guru/

Cron runs in the background and tasks scheduled with cron are referred to as “cron jobs”.

These cron jobs are executed automatically, making cron useful for automating system maintenance-related or administration tasks.

In this article, I’ll demo how I use cron to automate a task. However, I won't be going deep into cron and logrotate syntax because it varies and depends on what you are trying to accomplish. To learn more about logrotate and crop syntax, expression, and configuration settings for rotation of specific logs; just google it.

How and when to use logrotate or cron job to automate tasks for log files management

So recently I needed to purge old backup files for the firewall manager and gateways on multiple backup servers. I wanted to keep only a month’s worth of backup log files for each server that has backups.

Assumptions:

for this part of the article, I’m assuming that you’re familiar with both logrotate and cron job set up or confiigurations

At first, I decided to use logrotate and I set up a logrotate file in /etc/logrotate.d for each server with the following configuration:

my backup logrotate config

So after a couple of weeks, my logrotate did not work as intended. I came to realized that I had a concept and assumption error.

how logrotate works?

Logrotate removes files according to order in a lexically sorted list of rotated log file names, and also by file age (using last modification time of the file). While it’s true that logrotate rotates any files, without a change to the process, logrotate on its own will not rotate properly non .log files.

In my case, I was dealing with log .tgz archives file. The key problem here is that, while logrotate can rotate files, it doesn’t treat the .tgz archives files as one file. But instead, attempts to rotate all of them individually and don’t remove them.

The other issue is that logrotate doesn’t quite handle well rotation of multiple files in the same directory. I was trying to logrotate 3 different files in a directory while trying to ignore other files in the same directory.

Thus, even though the configuration I had in the /etc/logrotate.d is correct but It’s not doing what I assume logrotate should do.

When dealing with non-log files, the best way to remove the old backup file is to script whatever logic operation you’re trying to and set up a cron job to run that script or command.

So, in my case, I need to remove all backup files that are older than 30 days of only 3 specific file names in a directory that has other log files as well. Here is a PowerShell script I wrote to do that:

Make sure that your script is executable by running the command line: chmod u+x /path/YourScript.sh

The next step is to schedule the job you want to run with cron. In my case, I wanted my script to run on the 1st of every month.

Now, a crontab is a special file that holds the schedule of jobs cron will run. However, these are not intended to be edited directly. Instead, it’s recommended that you use the crontab command.

This allows you to edit your user profile’s crontab without changing your privileges with sudo. The crontab command will also let you know if you have syntax errors in the crontab while editing it directly will not.

The following command shows you the contents of your crontab if you want to view before you edit it:

crontab -l

Use the following command to edit your crontab:

crontab -e

This will open up your crontab in your user profile’s default text editor and allow you to edit or add a cron job as shown below:

In my case, I added the following cron job to run my script. It is going to be running on the 1st of every month at 7 AM and will delete files that are older than 30 days.

0 7 1 * * /path/delete_logs.sh > /dev/null 2>&1

I also wanted to run my script but keep it running in the background. To do so, I redirected the script’s output to an empty location, like /dev/null which immediately deletes any data written to it. The cron job also redirects standard error — represented by 2 — to standard output (>&1). Because the standard output is already being redirected to /dev/null, this essentially allows the script to run silently (source: digital ocean).

* Checking this useful site on cron expression generator

You are done🥳!

Cheers!!!