Servers run several processes that write events and errors to log files. These log files are helpful for troubleshooting and auditing purposes. However, without rotation or auto-deletion schedules, these logs can become quite large, consuming valuable disk space. Some applications mange log rotation autonomously, but many other do not. It's prudent to proactively monitor the size of these files and manage them.

Logrotate is a Linux server maintenance utility that periodically rolls, compresses, mails, and deletes system and application log files. This is a convenient utility application, which helps prevent log files from growing unbounded on your system. Logrotate provides a great degree of flexibility in specifying how the rotations should occur and is widely adopted.

How does log rotate work?

Logrotate is run, typically daily, by a scheduled cron script located at /etc/cron.daily/logrotate. On some high transaction systems, Logrotate can alternatively be run hourly by a cron script located at /etc/cron.hourly/logrotate. When this cron script is ran, logrotate reads configuration files to determine which log files it should potentially rotate and how. The primary logrotate file is located at /etc/logrotate.conf.

How do I check if log rotate is installed on my Linux server?

To check if logrotate is installed on your server, simply type logrotate in a terminal window. If logrotate is already installed on your server, you should see a message similar to the one below:

logrotate 3.8.7 - Copyright (C) 1995-2001 Red Hat, Inc.  
This may be freely redistributed under the terms of the GNU Public License

Usage: logrotate [-dfv?] [-d|--debug] [-f|--force] [-m|--mail=command]  
        [-s|--state=statefile] [-v|--verbose] [--version] [-?|--help]
        [--usage] [OPTION...] <configfile>

If Logrotate is not already on your server, install it using the following apt-get commands:

sudo apt-get update  
sudo apt-get install logrotate  

The configuration options and default options for Logrotate should be located in a file at:

/etc/logrotate.conf

Options for specific applications, which replace the default options specified in logrotate.conf are kept in the directory:

/etc/logrotate.d/

Configuration file directives

Some of the more important configuration directives supported by Logrotate in the configuration file are:

  • compress
    • Indicates that Logrotate should compress rolled logs. By default, files are compressed using gzip.
      • delaycompress
        • Delays compression until the next rotation cycle. This may be helpful for application that cannot be instructed to close the current log file and may continue writing to it for some time into the future.
      • compresscmd
        • Specifies which command to use when compressing log files. The default compression algorithm for Logrotate is gzip.
      • uncompresscmd
        • Specifies which command to use when uncompressing log files. The default compression algorithim for Logrotate is gunzip.
  • copy
    • Makes a copy of the log file, but does not make any modifications to the original version. This feature may be useful for creating backups of the log file or taking snapshots at particular points in time.
  • copytruncate
    • Instead of directly moving the original log file to a rotated destination location, this command truncates the source log file to an empty file after creating a copy. This directive could be useful for applications that cannot tolerate closing its log file during rotation.
  • create (mode, owner, group)
    • Immediately following a lot rotation, but before the postrotate script is run, the log file is created with the same name as the log file that was just rotated. All log file create attributes are optional. If no attributes are specified, the file attribute values of the old log file will be applied to the newly created file.
      • mode
        • Specifies the chmod mode for the log file in octal (e.g. 777)
      • owner
        • Specifies the name of the user who will be the owner of the newly created log file
      • group
        • Specifies the group the newly created log file will belong to
  • ifempty
    • Rotates the log file even if it is empty, this is the default behavior of Logrotate. If you do not want empty log files to be rotated, use the notifemtpy directive instead.
  • mail address
    • When a log is rotated out of existence, it is emailed to the provided address. If you do not want logs to be emailed, use the nomail directive instead.
  • maxage count
    • Logs older than count days will be deleted. This check is only performed if the current log file is about to be rotated.
  • maxsize size
    • Log files are rotated when they grow larger than size bytes. If the size threshold is hit before the configured interval rolling options, the file will still be rotated.
  • missingok
    • If a log file is missing, proceed without logging an error message
  • olddir directory
    • Rotated logs are moved into the directory. The directory must be on the same physical device as the log file being rotated.
  • Rotation interval options
    • You can instruct Logrotate how often to roll a particular log file. Possible options include:
      • daily
        • Specifies that the log file should be rotated every day
      • weekly
        • Specifies that the log file should be rotated weekly
      • monthly
        • Specifies that the log file should be rotated each month
      • yearly
        • Specifies that the log file should be rotated annually
  • postrotate
    • Logrotate executes the postrotate script whenever it rolls a log specified in a particular configuration block. Using this command, you could specify additional cleanup routines or even restart processes that require this in order to move to a new log file.

Example application-specific configuration options

Many Linux applications already deploy default Logrotate configuration options that can be customized as needed. If an application does not deploy its own Logrotate configuration options, you can create your own configuration file for that particular application by adding a new configuration file in the directory /etc/logrotate.d/.

Below is an example configuration options file for Uncomplicated Firewall, or ufw, located at /etc/logrotate.d/ufw:

/var/log/ufw.log
{
        rotate 4
        weekly
        missingok
        notifempty
        compress
        delaycompress
        sharedscripts
        postrotate
                invoke-rc.d rsyslog rotate >/dev/null 2>&1 || true
        endscript
}

By looking at this configuration file, we can see that Logrotate is configured to roll the log file located at /var/log/ufw.log weekly if it isn't empty. If the file is missing, Logrotate should continue on without issuing any kind of error. Logrotate should keep the 4 most recently rolled log files for ufw. Additionally, this file specifies that the ufw log should be compressed. However, the compression should be postponed until the next log rotation cycle. sharedscripts specifies that the postrotate script should only be ran once.

The postrotate command is more advanced, and forces rsyslog to reopen the files it is writing logs to so that it logs to the new logfile instead of the old file that was rotated.

How to check the status of Logrotate

By default, Logrotate maintains a state file with status information at /var/lib/logrotate/status This file contains information for the various log files Logrotate is configured to monitor and the last date and time that a particular log file was rotated, below are a few example entries:

"/var/log/syslog" 2016-9-2-6:25:2
"/var/log/nginx/error.log" 2016-8-21-6:25:1
"/var/log/dpkg.log" 2016-9-1-6:25:1
"/var/log/unattended-upgrades/unattended-upgrades.log" 2016-9-1-6:25:1
"/var/log/unattended-upgrades/unattended-upgrades-shutdown.log" 2016-7-24-6:0:0
"/var/log/auth.log" 2016-8-28-6:25:1
"/var/log/apt/term.log" 2016-9-1-6:25:1
"/var/log/apport.log" 2016-9-2-6:0:0
"/var/log/apt/history.log" 2016-9-1-6:25:1
"/var/log/alternatives.log" 2016-9-1-6:25:1
"/var/log/debug" 2016-9-2-6:0:0
"/var/log/mail.log" 2016-9-2-6:0:0
"/var/log/kern.log" 2016-8-28-6:25:1
"/var/log/ufw.log" 2016-8-28-6:25:1

You can periodically monitor this status file to ensure that Logrotate is functioning properly on your server.

Additional flags

If you see that a particular file isn't being rolled on expected intervals, or logroate is not working properly, there are additional flags that can help provide additional information.

  • -v verbose
    • The -v flag turns on verbose logging for Logrotate, which can provide additional insight into what the utility application is doing. This is particularly helpful for determining why a file was or was not rotated.
  • -d debug
    • The -d flag turns on debug mode and also implies that -v should also be enabled. This flag instructs Logrotate to run like it would normally, but without making any actual changes to the log files. This flag can be helpful when testing or verifying the validity of configuration option files.
  • -f force
    • The -f flag forces Logrotate to rotate log files, even if it determines that rotation is not required.

You can run Logroate with any combination of the flags mentioned in this posting. Below is an example of manually running Logroate with verbose logging and forcing it to rotate logs:

logrotate -vf /etc/logrotate.conf