Topic: logrotate doesn't expire old logs

[DanB35]

I was looking for something the other day in /var/log on my SME server, when I realized that log files didn't seem to be expiring as I'd expect.  For example, /var/log/messages goes back three months:

Code: [Select]

[root@e-smith log]# ls -lh messages*
lrwxrwxrwx 1 root root   32 Dec  8 01:12 messages -> /var/log/messages.20171208011214
-rw------- 1 root root  79M Sep  3 03:31 messages.20170901011205
-rw------- 1 root root 146M Sep  8 01:12 messages.20170908011204
-rw------- 1 root root 9.8M Sep 10 03:34 messages.20170908011205
-rw------- 1 root root  82M Sep 14 16:37 messages.20170914163631
-rw------- 1 root root  12M Sep 15 01:12 messages.20170914163632
-rw------- 1 root root  63M Sep 17 03:31 messages.20170915011203
-rw------- 1 root root 109M Sep 22 01:12 messages.20170922011204
-rw------- 1 root root  46M Sep 24 03:27 messages.20170922011205
-rw------- 1 root root 130M Sep 29 01:12 messages.20170929011209
-rw------- 1 root root  20M Sep 29 19:50 messages.20170929011210
-rw------- 1 root root  34M Oct  1 01:16 messages.20170929194508
-rw------- 1 root root 2.4M Oct  1 03:30 messages.20171001011500
-rw------- 1 root root 138M Oct  7 10:54 messages.20171007105231
-rw------- 1 root root  15M Oct  8 01:12 messages.20171007105232
-rw------- 1 root root 2.3M Oct  8 03:38 messages.20171008011234
-rw------- 1 root root 161M Oct 15 01:12 messages.20171015011212
-rw------- 1 root root 1.9M Oct 15 03:35 messages.20171015011213
-rw------- 1 root root 158M Oct 21 07:51 messages.20171021074619
-rw------- 1 root root  39M Oct 22 01:12 messages.20171021074620
-rw------- 1 root root 2.0M Oct 22 03:31 messages.20171022011207
-rw------- 1 root root 208M Oct 29 01:12 messages.20171029011211
-rw------- 1 root root 3.0M Oct 29 03:16 messages.20171029011212
-rw------- 1 root root  79M Nov  1 01:12 messages.20171101011206
-rw------- 1 root root  92M Nov  5 03:37 messages.20171101011207
lrwxrwxrwx 1 root root   32 Nov  1 01:12 messages-20171105 -> /var/log/messages.20171101011207
-rw------- 1 root root  60M Nov  8 01:12 messages.20171108011204
-rw------- 1 root root  93M Nov 12 03:14 messages.20171108011205
lrwxrwxrwx 1 root root   32 Nov  8 01:12 messages-20171112 -> /var/log/messages.20171108011205
-rw------- 1 root root  66M Nov 15 01:12 messages.20171115011203
-rw------- 1 root root  26M Nov 16 05:55 messages.20171115011204
-rw------- 1 root root  15M Nov 16 20:52 messages.20171116055350
-rw------- 1 root root  66M Nov 19 03:36 messages.20171116205149
lrwxrwxrwx 1 root root   32 Nov 16 20:51 messages-20171119 -> /var/log/messages.20171116205149
-rw------- 1 root root 408M Nov 22 01:12 messages.20171122011203
-rw------- 1 root root  85M Nov 26 03:46 messages.20171122011204
lrwxrwxrwx 1 root root   32 Nov 22 01:12 messages-20171126 -> /var/log/messages.20171122011204
-rw------- 1 root root  62M Nov 29 01:12 messages.20171129011204
-rw------- 1 root root  46M Dec  1 01:12 messages.20171129011205
-rw------- 1 root root  43M Dec  3 03:08 messages.20171201011203
-rw------- 1 root root 3.8M Dec  3 03:08 messages-20171203.gz
-rw------- 1 root root 126M Dec  8 01:12 messages.20171208011213
-rw------- 1 root root  60M Dec  9 06:48 messages.20171208011214
[root@e-smith log]#
logrotate.conf looks like it should be expiring old logs after four weeks:

Code: [Select]

[root@e-smith log]# cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4
maxage 28

# create new (empty) log files after rotating old ones
create

# use date as a suffix of the rotated file
dateext

# uncomment this if you want your log files compressed
compress

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp and btmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    create 0664 root utmp
minsize 1M
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0600 root utmp
    rotate 1
}

# system-specific logs may be also be configured here.
[root@e-smith log]#


Code: [Select]

[root@e-smith logrotate.d]# cat syslog
/var/log/cron
/var/log/maillog
/var/log/messages
/var/log/secure
/var/log/spooler
{
    sharedscripts
    postrotate
/bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
    endscript
}
[root@e-smith logrotate.d]#
What gives?  /var/log is about 7 GB right now.  Certainly not the end of the world, but it seems more than a little excessive.

[CharlieBrady]

Please report any possible bug via the bug tracker.

[Jean-Philippe Pialasse]

I can confirm this for the following logs

/var/log/cron
/var/log/maillog
/var/log/messages
/var/log/secure
/var/log/spooler

please open a bug

looks like the last version of logrotate ignore in logrotate.d/* what has been defined in logrotate.conf

[DanB35]

https://bugs.contribs.org/show_bug.cgi?id=10484

[CharlieBrady]

I can confirm this for the following logs

/var/log/cron
/var/log/maillog
/var/log/messages
/var/log/secure
/var/log/spooler

Check /etc/e-smith/events/logrotate/logfiles2timestamp. See also /etc/e-smith/events/logrotate/S75purge-old-logs.

Logfile rotation for those files in managed by 'signal-event logrotate' which runs from /etc/crontab - and not by the logrotate program.

[DanB35]

See also /etc/e-smith/events/logrotate/S75purge-old-logs.

Ah ha.  There's an rsyslog/PurgeLength configuration property that controls how long to retain log files, and defaults to 95 days if it isn't set.  That almost certainly explains what's going on.  But then why have logrotate installed at all?  Just as a red herring?

Just did 'config setprop rsyslog PurgeLength 30 && signal-event console-save'.  I'll see if that had the desired effect tomorrow.

Edit:  Yes, it did.  OK, so we're using a custom log rotation script with undocumented configuration properties, and a frankly strange default to keep logs for just over three months.  That default can be overridden with the property setting noted above.  logrotate is installed, has a config file, and additional configuration in /etc/logrotate.d/, but all of those are superfluous, as logrotate proper is never actually called.  This explains quite well why the logrotate.conf settings aren't having the desired effect.

Is there any way, with SME's custom log rotation script, to compress the rotated logs?

[Jean-Philippe Pialasse]

Ah ha.  There's an rsyslog/PurgeLength configuration property that controls how long to retain log files, and defaults to 95 days if it isn't set.  That almost certainly explains what's going on.  But then why have logrotate installed at all?  Just as a red herring?

Just did 'config setprop rsyslog PurgeLength 30 && signal-event console-save'.  I'll see if that had the desired effect tomorrow.

Edit:  Yes, it did.  OK, so we're using a custom log rotation script with undocumented configuration properties, and a frankly strange default to keep logs for just over three months.  That default can be overridden with the property setting noted above.  logrotate is installed, has a config file, and additional configuration in /etc/logrotate.d/, but all of those are superfluous, as logrotate proper is never actually called.  This explains quite well why the logrotate.conf settings aren't having the desired effect.

indeed not documented : PurgeLength , should open a documentation bug or fill the wiki with this
well it is installed because it is used for other files

Is there any way, with SME's custom log rotation script, to compress the rotated logs?

well this could be an interesting feature, also to allow a max size or a more frequent rotation if needed some could become pretty huge in a 7 days (hundreds of Mo)
or simply turn back to logrotate.

anyway check the bug you allowed to find one !