Conditional Logging in Apache

Apache server records all incoming requests and all requests processed to a log file. This article will show how to conditionally records or prevents log entries into access log file.

The location and content of the access log are controlled by the CustomLog directive. To find Apache access log file location, we could grep httpd.conf file, for example:
#grep CustomLog /usr/local/etc/apache22/httpd.conf

CustomLog /var/log/httpd-accessviaproxy.log combined env=!is-forwarder
CustomLog /var/log/httpd-accessfwd.log fwd env=is-forwarder
CustomLog /var/log/httpd-accessviaproxy.log combined env=!gbrnolog
CustomLog /var/log/httpd-accessfwd.log fwd env=!gbrnolog

The CustomLog directive is used to log requests to the server. A log format is specified, and the logging can optionally be made conditional on request characteristics using environment variables. The syntax is:
CustomLog filepipe formatnickname [env=[!]environment-variable]

The first argument, which specifies the location to which the logs will be written, can take one of the following two types of values:file i.e A filename, relative to the ServerRoot; and pipe i.e The pipe character "", followed by the path to a program to receive the log information on its standard input.

The third argument is optional and controls whether or not to log a particular request based on the presence or absence of a particular variable in the server environment. If the specified environment variable is set for the request (or is not set, in the case of a 'env=!name' clause), then the request will be logged.

For example, the following two sets of directives have exactly the same effect:

# CustomLog with format nickname
LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog logs/access_log common

# CustomLog with explicit format string
CustomLog logs/access_log "%h %l %u %t \"%r\" %>s %b"

In the following example, we separate log file for accessing through a proxy and the other for external access of a web server.


<--IfModule log_config_module>
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" fwd
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
# If you do not specify an ErrorLog directive within a
# container, error messages relating to that virtual host will be
# logged here. If you *do* define an error logfile for a
# container, that host's errors will be logged there and not here.
ErrorLog /var/log/httpd-error.log
SetEnvIf X-Forwarded-For "^.*\..*\..*\..*" is-forwarder

#ignore these elements into logs.
# Here we exclude some files from our log
SetEnvIf Request_URI "(\.png\.gif\.jpg\.ico\.css)$" nologimage
#CustomLog /var/log/httpd-access.log common
CustomLog /var/log/httpd-accessviaproxy.log combined env=!is-forwarder
CustomLog /var/log/httpd-accessfwd.log fwd env=is-forwarder
CustomLog /var/log/httpd-accessviaproxy.log combined env=!nologimage
CustomLog /var/log/httpd-accessfwd.log fwd env=!nologimage
< /ifmodule>

When using conditional logging, the environment variable system sets a variable based on the request. The CustomLog directive accepts an environment condition that will be applied to the CustomLog configured.

If we would like to exclude our own IP address for example localhost, use this:

SetEnvIf Remote_Addr "127\.0\.0\.1" ignoreme

or to mark requests for the robots.txt file

SetEnvIf Request_URI "^/robots\.txt$" ignoreme
CustomLog "logs/access.log" common env=!ignoreme

The format of the access log is highly configurable. Here some of format string for LogFormat:
%a Remote IP-address
%A Local IP-address
%b Size of response in bytes, excluding HTTP headers. In CLF format, i.e. a '-' rather than a 0 when no bytes are sent.
%B Size of response in bytes, excluding HTTP headers.
%h Remote host
%H The request protocol
%l Remote logname (from identd, if supplied). This will return a dash unless mod_ident is present and IdentityCheck is set On.
%r First line of request
%R The handler generating the response (if any).
%s Status. For requests that got internally redirected, this is the status of the *original* request --- %>s for the last.%u Remote user (from auth; may be bogus if return status (%s) is 401)
%U The URL path requested, not including any query string.
%t Time the request was received (standard english format)
%T The time taken to serve the request, in seconds.

By using conditional logging, we could ignore or prevent writing entry at our log file.

For detail:
http://httpd.apache.org/docs/2.2/mod/mod_setenvif.html
http://httpd.apache.org/docs/2.2/mod/mod_log_config.html