[plug] How to retain logrotate backups forever

[Nick Bannon]

On Fri, Dec 29, 2023 at 07:15:56PM +0800, Chris McCormick wrote:
> Thanks, I ended up doing this and set it to 100 years of backups.

> On 29/12/23 1:57 pm, Byron Hammond wrote:
> > Can you bump up the rotate number to something massive so it effectively
> > isn't deleted for a very very long time?
> > On Fri, 29 Dec 2023 at 09:22, Chris McCormick <chris at mccormick.cx> wrote:
> >     Does anybody know how to cause logrotate to retain the rotated logs
> >     forever? The man page for `rotate` says this:

Hi Chris, Byron! Yes - I think that probably *is* the right answer to
your question, as asked. It probably means going through a bunch of
files in `find /etc/logrotate.{conf,d}` and changing them to
yearly/ROT_YEARLY/rotate/rotateCount=100; or weekly/ROT_WEEKLY/rotateCount=5200 , etc.

It's been a while since I saw a good mini-HOWTO on setting up a loghost,
and I guess it's telling that none of these mention logrotate or journald:
https://serverfault.com/questions/246785/how-do-i-setup-a-syslog-server-for-my-network
https://web.archive.org/web/20130901082154/http://campin.net/newlogcheck.html
https://web.archive.org/web/20031228143751/http://linuxsecurity.com/feature_stories/remote_logserver-8.html
https://rsyslog.readthedocs.io/en/latest/tutorials/tls_cert_server.html

( Slightly newer, with logrotate and a few comments on rsyslog:
https://web.archive.org/web/20140728100505/http://www.debian-administration.org/articles/278

A properly current one might draw from https://docs.debops.org/ or
https://github.com/Linuxfabrik/lfops ? )

Maybe that's a hint, though? If one has channelled everything important through
rsyslog or syslog-ng , then one can get rid of logrotate completely and archive
directly into /var/log/%FROMHOST%/%$YEAR%/%$MONTH%/%$DAY%/messages.log

https://www.rsyslog.com/doc/master/configuration/modules/omfile.html#dynafile
https://rsyslog.readthedocs.io/en/latest/configuration/properties.html
https://unix.stackexchange.com/questions/251933/rsyslog-receiving-from-multiple-hosts-and-saving-to-different-files

https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.38/administration-guide/36#TOPIC-2026258

For the channeling-everything-through-*log part - reading/writing to
journald looks a lot like reading/writing to single-host Windows Event
logs. For some small environments it's all people want, and there's:

https://www.rsyslog.com/doc/master/configuration/modules/imjournal.html
https://www.rsyslog.com/doc/master/configuration/modules/imuxsock.html
https://www.rsyslog.com/doc/master/configuration/modules/omjournal.html

...and a 9m example on how that can work and what the alternative is:
https://yewtu.be/watch?v=GTS7EuSdFKE (rsyslog journal integration, Rainer Gerhards 2013-06-04)
Integrating syslog Data into the journal (SOHO env)
rsyslog.conf:
/* first, we make sure all the necessary modules are present: */
module(load="imudp") # input module for UDP syslog
module(load="omjournal") # output module for journal
/* then, define the actual server that listens to the
 * router. Note that 514 is the default port for UDP syslog.
 */
input(type="imudp" port="514" ruleset="writeToJournal")
/* inside that ruleset, we just write data to the journal: */
ruleset(name="writeToJournal") {
	action(type="omjournal")
}

People have been using journald to collect logs together, like some of
the things that used to get sent to STDOUT/STDERR, which is a good step
forward...  However, journald on its own doesn't really scale up to big
systems because of things like:
https://github.com/systemd/systemd/issues/9519
via https://unix.stackexchange.com/questions/296453/how-to-configure-per-unit-log-storage-policy-with-journald

I think one of the reasons journald was implemented was to solve
edge-cases like "what if the log writing/reading/rotation is not quite
atomic?" but there's a few solutions to that:
https://superuser.com/questions/291368/log-rotation-of-stdout

There's still a role for things like rsyslog imudp/imfile (or logwatch
with copy/copytruncate/dateext?) to collect things or send them onto
some other aggregator.

Maybe everyone's off building https://github.com/grafana/loki systems?
but... it's not packaged yet: https://bugs.debian.org/1001903

Nick.

-- 
   Nick Bannon   | "I made this letter longer than usual because
nick-sig at rcpt.to | I lack the time to make it shorter." - Pascal