SLF4J HTTP Logging filter

Contents

Purpose

Out of the box, Repose only provides application logging which is different than access logging. For access logging, the Simple Logging Facade for Java (SLF4J) HTTP Logging filter uses Log4j 2.x to allow logging the information in HTTP requests that are sent to and the responses sent from Repose. It is based on the Apache HTTPD Logging Standard.

If a user-supplied logging configuration file is not found, Repose programmatically sets default log4j properties.  The default properties add a ConsoleAppender to the ROOT logger. The output will be formatted using a PatternLayout set to the pattern "%d %-4r [%t] %-5p %c %x - %m%n".  The default log level is set to DEBUG.

For information regarding the migration from log4j 1.x to log4j 2.x, see the Apache documentation here:

General filter information

Filter name: slf4j-http-logging

Filter configuration: slf4j-http-logging.cfg.xml

Released: version 4.1.5

This filter replaced the HTTP Logging filter which was deprecated in version 4.1.5.

Prerequisites

Required headers: The SLF4J HTTP Logging filter has no required request headers.
Required preceding filters: The SLF4J HTTP Logging filter has no required preceding filters.
Standard filter order:  When you set the order of Repose filters, place the SLF4J HTTP Logging filter first in the sequence.

Basic SLF4J HTTP Logging configuration

Configure the SLF4J HTTP Logging filter by editing the slf4j-http-logging.cfg.xml file.

1. Set Up Repose  

Updated the logging-configuration in the container.cfg.xml.

container.cfg.xml
...
<logging-configuration href="file:///etc/repose/log4j2.xml"/>
...

2. Add the SLF4J HTTP Logging Filter  
Add the  slf4j-http-logging filter to your system-model.cfg.xml within the filters element. Place this filter at the top of the filter sequence.

system-model.cfg.xml
<filters>
	<filter name="slf4j-http-logging" />
	<!-- Place other filters below the slf4j-http-logging filter ->	
</filters>

3. Configure the SLF4J HTTP Logging Filter 

Within the slf4j-http-log element:

  • Configure the id to the desired logger name.
  • Configure either the format attribute or element for what will be logged.
slf4j-http-logging.cfg.xml
...
    <slf4j-http-log id="my-special-log">
        <format>
            <![CDATA[
            { "timestamp": "%t" ... }
            ]]>
        </format>
    </slf4j-http-log>
...

4. Configure Log4J logging level

In the file named in the logging-configuration in the container.cfg.xml (log4j2.xml by default), configure the logger named by the slf4j-http-log element's id attribute in the slf4j-http-logging.cfg.xml file to INFO or below.

log4j2.xml
...
    <Loggers>
        <Root level="warn">
            <AppenderRef ref="RollingFile"/>
        </Root>
        ...
        <Logger name="my-special-log" level="info"/>
    </Loggers>
...

5. Test SLF4J HTTP Logging

  • Run requests.
  • Confirm your configuration by checking the logs in the target file.

Configurable parameters

XML schema definition

Example configuration

The slf4j-http-logging.cfg.xml file contains the following elements and attributes. Add the filter to your Repose deployment through the System Model Configuration.

ElementsAttributes

Required/

Optional

Description
<slf4j-http-log>-RequiredDefines specified logs.
idRequired

Names the SLF4J logger appender name. It can be used in the backend logger configuration (Current log4j 1.2) to direct it's output to an appender. See log4j to learn more.

If the id doesn't match a logger named in the logging configuration, then the default Root logger is used.

formatRequired if the format element is not used.Specifies what format will be used for the logs. See "What is Logged" below for formatting instructions.
<format>-Required if the format attribute is not used.

Specifies what format will be used for the logs. See "What is Logged" below for formatting instructions.

The format attribute and element are mutually exclusive and one is required. If neither or both are present, then the configuration file will fail to validate.


Configuration example

slf4j-http-logging.cfg.xml
<?xml version="1.0" encoding="UTF-8"?>

<slf4j-http-logging xmlns="http://docs.openrepose.org/repose/slf4j-http-logging/v1.0">
        <!-- The id attribute is the named target of the log output,
         it can then be used in log4j backend to determine which appender to go to -->
        <!-- The format includes what will be logged.  The arguments with % are a subset of the apache mod_log_config
             found at http://httpd.apache.org/docs/2.2/mod/mod_log_config.html#formats -->
        <slf4j-http-log
                id="my-special-log"
                format="Response Time=%T seconds\tResponse Time=%D microseconds\tResponse Code Modifiers=%200,201U\tModifier Negation=%!401a\tRemote IP=%a\tLocal IP=%A\tResponse Size(bytes)=%b\tRemote Host=%hRequest Line=&quot;%r&quot;\tRequest Method=%m\tRequest Protocol=%H\tServer Port=%p\tQuery String=%q\tTime Request Received=%t\tStatus=%s\tRemote User=%u\tURL Path Requested=%U\n"/>
        <slf4j-http-log id="my-other-log" format="Remote IP=%a Local IP=%A" />
</slf4j-http-logging>


In the log4j2.xml file, add settings similar to the following:


log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
    <Appenders>
        <Console name="STDOUT">
            <PatternLayout pattern="%d %-4r [%t] %-5p %c - %m%n"/>
        </Console>
        <RollingFile name="RollingFile" fileName="/var/log/repose/current.log"
                     filePattern="/var/log/repose/current-%d{yyyy-MM-dd_HHmmss}.log">
            <PatternLayout pattern="%d %-4r [%t] %-5p %c - %m%n"/>
            <Policies>
                <SizeBasedTriggeringPolicy size="200 MB"/>
            </Policies>
            <DefaultRolloverStrategy max="2"/>
        </RollingFile>
    </Appenders>
    <Loggers>
        <Root level="warn">
             <AppenderRef ref="STDOUT"/>
             <AppenderRef ref="RollingFile"/>
        </Root>
        <Logger name="com.sun.jersey" level="off"/>
        <Logger name="net.sf.ehcache" level="error"/>
        <Logger name="org.apache" level="warn"/>
        <Logger name="org.eclipse.jetty" level="off"/>
        <Logger name="org.openrepose" level="info"/>
        <Logger name="org.springframework" level="warn"/>
        <Logger name="my-special-log" level="info"/>
    </Loggers>
</Configuration>


What is logged

Currently, Repose can be configured to log information from the request URI and headers but not the request message content.The SLF4J HTTP Logging filter can log static text specified in the slf4j-http-log format attribute and specific information about the request/response.  The table below lists the request/response information the user may choose to log:

* This table is a subset of the Apache logging functionality listed at Apache HTTPD Logging Standard. Repose only supports this subset of the Apache logging standard plus the listed non-standard extras.

Format string

Description

\t

Tab

\n

Newline

%%

The percent sign

%a

Remote IP-address

%A

Local IP-address

%b

Size of response in bytes, excluding HTTP headers.  In CLF format, i.e. a '-' rather than a 0 when no bytes are sent.

%B

Size of response in bytes, excluding HTTP headers.  0 is logged when no bytes are sent.

%D

The time taken to serve the request, in microseconds. [As of release 2.3.6]

%gThe internal GUID assigned to the request. [As of release 7.1.2.0]
%HThe request Protocol. [As of release 2.5.0]

%h

Remote host

%{header}i%{header type out-format}i%{header type out-format in-format}i

Request header. Parameters typeout-format and in-format are optional. If not specified, then the header value is output as is.Currently, the only type supported is DATE. Out and in date formats supported are RFC_1123 (HTTP-Date) and ISO_8601.If type is specified, then out-format is required. in-format is optional and defaults to RFC_1123 for DATE types if not specified.Note: Headers are treated in a case insensitive manner.  For example, %{X-Auth-Token}i is equivalent to %{x-auth-token}i.

%MThe Status message if set. [As of release 2.3.6]

%m

The request method.

%{header}o%{header type out-format}o%{header type out-format in-format}o

Response header. Parameters typeout-format and in-format are optional. If not specified, then the header value is output as is.Currently, the only type supported is DATE. Out and in date formats supported are RFC_1123 (HTTP-Date) and ISO_8601.If type is specified, then out-format is required. in-format is optional and defaults to RFC_1123 for DATE types if not specified.An example to output the retry-after header using the ISO_8601 format would be %{retry-after DATE ISO_8601}oNote: Headers are treated in a case insensitive manner.  For example, %{LOCATION}o is equivalent to %{location}o.

%p

The canonical port of the server serving the request.

%q

The query string (prepended with a ? if a query string exists, otherwise an empty string).

%rFirst line of request. [As of release 2.5.0]

%s

Status.  For requests that got internally redirected, this is the status of the \*original\* request \--\- %>s for the last.

%t

Time the request was received. The format for Repose versions 4.1.5 - 7.3.2.0 is: "dd-MM-yyyy-HH:mm:ss.SSS", for versions 8.0.0.0 - current: "yyyy-MM-dd HH:mm:ss".

%{format}tTime the request was received using the specified date format (SimpleDateFormat patterns; format string limited to letters, numbers, dashes, spaces, periods, and colons, e.g. "%{yyyy-MM-dd HH:mm:ss}t"). [As of release 7.1.5.0]

%T

The time taken to serve the request, in seconds. [As of release 2.3.6]

%u

Remote user (from auth; may be bogus if return status (%s) is 401)

%U

The URL path requested, not including any query string.


The user can restrict what gets printed for responses with specific HTTP status codes.  The table below shows examples of this functionality:

Format string

Description

%403,401U

Logs the URL path requested on 403 and 401 responses only.

%\!200,304,302U

Logs the URL path requested for all responses except 200, 304, and 302.

Return codes and conditions

This filter does not return specific response codes. The request will simply pass through to the next filter or to the origin service.


Request headers created

The SLF4J HTTP Logging filter does not create any request headers.

Tips and tricks

Logging To A Separate File

You can configure your log4j2.xml to log the SLF4J HTTP output to a different file to keep your application logging and access logging separate.  Here's a sample log4j2.xml that accomplishes this (which assumes the default slf4j filter configuration):

log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration monitorInterval="15" packages="org.apache.logging.log4j.flume.appender">
    <Appenders>
        <Console name="STDOUT">
            <PatternLayout pattern="%d %-4r [%t] %-5p %c - %m%n"/>
        </Console>

        <RollingFile name="RollingFile" fileName="/var/log/repose/current.log"
                     filePattern="/var/log/repose/current-%d{yyyy-MM-dd_HHmmss}.log">
            <PatternLayout pattern="Trans-Id:%X{traceGuid} - %d %-4r [%t] %-5p %c - %m%n"/>
            <Policies>
                <SizeBasedTriggeringPolicy size="200 MB"/>
            </Policies>
            <DefaultRolloverStrategy max="2"/>
        </RollingFile>

        <!-- Create a new RollingFile configuration element for the separate access log file. -->
        <!-- Optional: The archive file pattern ends with ".gz" to compress these files. Use tools like 'zless' and 'zgrep' to work with them. -->
        <RollingFile name="AccessRollingFile" fileName="/var/log/repose/access.log"
                     filePattern="/var/log/repose/access-%i.log.gz">
            <!-- You only need to configure the message (%m) here. -->
            <PatternLayout pattern="%m"/>
            <Policies>
                <!-- For testing purposes, rollover the log file at 20 KB. -->
                <SizeBasedTriggeringPolicy size="20 KB"/>
            </Policies>
            <!-- Keep the last 10 archive logs. -->
            <DefaultRolloverStrategy max="10"/>
        </RollingFile>

        <File name="PhoneHomeMessages" fileName="/var/log/repose/phone-home.log" append="false">
            <PatternLayout>
                <Pattern>%m</Pattern>
            </PatternLayout>
        </File>
    </Appenders>
    <Loggers>
        <Root level="info">
            <AppenderRef ref="STDOUT"/>
            <AppenderRef ref="RollingFile"/>
        </Root>
        <Logger name="com.sun.jersey" level="off"/>
        <Logger name="net.sf.ehcache" level="error"/>
        <Logger name="org.apache" level="warn"/>
        <Logger name="org.eclipse.jetty" level="off"/>
        <Logger name="org.springframework" level="warn"/>
        <Logger name="intrafilter-logging" level="info"/>
        <Logger name="phone-home-message" level="info" additivity="false">
            <AppenderRef ref="PhoneHomeMessages"/>
        </Logger>

        <!-- Include all of your slf4j-http-log IDs here (e.g. my-special-log). -->
        <!-- Set the "additivity" to false to have your access logs only go to your access log file. Do not set the additivity if you want the logs going to current.log as well. -->
        <Logger name="my-special-log" level="info" additivity="false">
            <appenderRef ref="AccessRollingFile"/>
        </Logger>
        <Logger name="my-other-log" level="info" additivity="false">
            <appenderRef ref="AccessRollingFile"/>
        </Logger>
    </Loggers>
</Configuration>