Log files not being rotated on shipped log4j2.xml

Description

As a Repose user, I would like the Repose logs to be rotated so I don't run out of disk space on the server running Repose.

A user alerted us to the /etc/logrotate.d/repose file not being owned by root (see linked ninja task) which prevents logrotate from working. This was surprising as log4j2 should be handling our log rotation, so we should probably deprecate our installing of the repose logrotate.d configuration and remove it in Repose 9. For reference, this is what the file looks like when deployed:

The problem appears to be that the log4j2.xml we ship doesn't rotate the logs.
https://github.com/rackerlabs/repose/blob/master/repose-aggregator/artifacts/src/config/filters/log4j2.xml#L7-L14

The DefaultRolloverStrategy isn't going to do anything when the archived file pattern doesn't contain an %i in it. I verified this on the server I'm currently doing performance testing on. I also saw it mentioned on StackOverflow:
http://stackoverflow.com/questions/24551768/how-does-log4j2-defaultrolloverstrategys-max-attribute-really-work

It looks like there are fancier ways to ensure we're rotating logs out correctly that we should explore.

Also, we should consider gzipping the archived files in the future (Repose 9?). The z* commands (e.g. zless, zgrep, etc.) make it easy to navigate through these files without a problem, and it would mean taking up much less disk space. The only problem would be if anyone's parsing through the logs expecting them to be in plain text. If we do go this route, make sure there doesn't need to be anything else installed on the box for this to work (e.g. making this crash and burn in case they don't have gzip installed).

Useful info:
https://logging.apache.org/log4j/2.x/manual/appenders.html#RollingFileAppender

Acceptance Criteria:

  • Remove logrotate configuration

  • log4j configuration is updated to rotate the logs on a daily basis

  • Logs are kept for a year

    • That way we're providing an example of purging logs that users can update to how long they really want to keep them

  • Upgrade documentation makes it clear that:

    • The default logging config has changed.

    • Everyone should look at their logging config to make sure it fits for their specific API's needs and leverages the new options shown in the default logging config.

    • This suggestion seriously applies to everyone whether or not they use the default config or have their own custom config.

Environment

None

Assignee

Unassigned

Reporter

Mario Lopez

Labels

None

External issue ID

None

CoAssignee

None

Capitalizable

True

Story Points

3

Fix versions

Priority

High
Configure