syslog-ng and process accounting

In one of my previous posts, I've mentioned that syslog-ng is not for syslog anymore, we aim to support other log formats too, preferably those that have some kind of structure.

In fact syslog-ng is trying to convert all incoming messages (be them unstructured syslog messages, process accounting messages or SNMP traps) into the same, common format:

• name-value pairs
• tag or tags that connects the event to one of the patterndb schemas

This information coming in from different sources can be stored and processed with the same infrastructure. Correllation between SNMP traps and syslog messages or netflow records should be possible.

I probably don't need to mention, that we use patterndb to extract information from syslog messages. But structured information sources contain name-value pairs in the first place, so why not use them natively?

This is what the experimental process accounting feature of syslog-ng demonstrates. With this module, syslog-ng is able to read the process accounting file produced by the Linux kernel directly (this is currently Linux-only, but should be easy to port to other platforms) and produce a set of name-value pairs mimicing the structure of the accounting record.

This is how it works:

• the Linux kernel writes an accounting record to /var/log/account/pacct file (distro dependant though) whenever a process terminates and writes process related information to this record (exit code, execution time, etc)
  
• syslog-ng uses the file() source driver, and periodically polls this file for changes (once per second by default)
• instead of processing this as a plain text file, the "pacctformat" plugin tells syslog-ng to fetch binary records
• the pacctformat plugin then transforms account record members into syslog-ng name-value pairs

Each name-value pair produced by the pacct plugin has a prefix of "pacct", and the members are described in the header file or in acct(5) manual page.

In order to try this feature, you need to tell syslog-ng to compile the "pacctformat" plugin by passing the --enable-pacct command line option to configure.

Also, there's support for the pacct module in the SCL, so in order to fetch process accounting records, you only need a very small configuration file:

@version: 3.2
@include "scl.conf"

source s_pacct {
pacct();
};

log { source(s_pacct); destination(...); };

After that, you only need to enable Linux accounting by issuing an accton command.

That's all.