Skip to main content

syslog-ng 3.2alpha2 released

I've just uploaded syslog-ng 3.2alpha2 to the release directory. The last alpha release didn't compile on all supported platforms and the automatic test-suite was disabled, because it only worked if syslog-ng got installed first.

These obstacles have been overcome and together with some fixes and a couple of new features, 3.2alpha2 is now available. I've also forward ported all bugfixes from syslog-ng 3.1.2.

For those who are starting to experiment with the 3.2 branch, here's the list of new features compared to 3.1. Those who tried 3.2alpha1, the list of changes compared to 3.2alpha1 is at the end of this post.

Since the documentation of syslog-ng is not yet up-to-date with the new features introduced, I've tried to also include URLs for the best known descriptions. The references may not be 100% accurate, but should give anyone interested an idea how to start experimenting.

Also, please note that although this is an alpha release, the bulk of the changes are in the configuration parser, so once your configuration was parsed properly and syslog-ng starts up, an almost unchanged code is processing it. This means that this release should be good enough to start playing with. And feedback about what kind of syslog-ng.conf parsing errors you encounter on real-life configuration files is more than welcome.

Code quality & functionality wise, this could be a beta release, I only expect "procedural" changes, like cleaning up the plugin names, which wouldn't be nice to do in a beta release (though not unheard of :)

New features in 3.2:
  • Plugins: the new architecture replaces the old monolithic one, all syslog-ng functionality is loaded from external plugins when needed. It is possible to write plugins to extend syslog-ng functionality in the following areas: sources, destinations, filter expression, parsers, rewrite ops, message format.
  • The framework for a "syslog-ng configuration library" (aka SCL) a collection of configuration snippets installed along syslog-ng, simplifying the authoring of syslog-ng configuration files.
  • pdbtool match is now able to read a file containing syslog messages and apply patterndb and a filter expression on the contents.
  • pdbtool test is now able to perform pattern testing automatically based on the supplied example log message.
  • Persistent state containing the current file position for file sources is now continously updated during runtime, instead of updating it only at exit, which makes it much more reliable in case syslog-ng doesn't terminate normally.
  • Better syntax error reporting in the configuration file.
  • Support for reusable configuration snippets, similar to macros with parameters, named "blocks".
  • Added a confgen plugin that includes the output of a program into the configuration file, making it possible to generate configuration file snippets dynamically.
  • Support for BSD-style process accounting logs via the pacct() source driver defined in by SCL and the underlying pacctformat plugin.
  • Support for explicit COMMITs in the SQL driver, this speeds up SQL INSERT rate significantly if flush_lines() is non-zero.
  • It is now possible to supply a filter to rewrite expressions and only apply the rewrite rule in case the filter matches.
  • It is now possible to use multiple parser expressions in a single parser object, similar to rewrite rules.
  • Added support for using the include statement from anywhere in the configuration file, instead of only at top-level. Also introduced syslog-ng "global values" that can be defined and the substituted anywhere in the configuration file.

  • Default configuration file supplied as part of SCL.

Incompatible changes:
  • syslog-ng traditionally expected an optional hostname field even when a syslog message is received on a local transport (e.g. /dev/log). However no UNIX version is known to include this field. This caused problems when the application creating the log message has a space in its program name field. This behaviour has been changed for the unix-stream/unix-dgram/pipe drivers if the config version is 3.2 and can be restored by using an explicit 'expect-hostname' flag for the specific source.

Changes since 3.2alpha1:
  • Now compiles on all platforms and the unit/functional tests also run. (tested: AIX, HP-UX, Solaris, FreeBSD, Linux, Tru64)
  • Fixed pdbtool match --debug-pattern output for ESTRING parsers.
  • Fixed a possible memory leak in the lexer, which would accumulate in case SIGHUPs.
  • Fixed Solaris STREAMS device support.
  • Forward ported all bugfixes from syslog-ng OSE 3.0 & 3.1
  • Disable process accounting module by default as it doesn't compile on non-Linux platforms.
  • Added "pdbtool match --file" option to read and parse an existing logfile.
  • Added "pdbtool test" to check the log samples in the patterndb file.
  • Added "dont-create-tables" flag for the SQL destination to inhibit automatic table creation.
  • Added "condition()" support for rewrite expressions, which makes it possible to skip rewrite rules that do not match a filter expression.
  • Added "--module-path" command line option to control where modules are loaded from from the command line.

Happy logging!
Labels:

Comments

Post a Comment

Popular posts from this blog

syslog-ng fun with performance

I like christmas for a number of reasons: in addition to the traditional "meet and have fun with your family", eat lots of delicious food and so on, I like it because this is the season of the year when I have some time to do whatever I feel like. This year I felt like doing some syslog-ng performance analysis. After reading Ulrich Deppert's series about stuff "What every programmer should know about memory" on LWN, I thought I'm more than prepared to improve syslog-ng performance. Before going any further, I'd recommend this reading to any programmer, it's a bit long but every second reading it is worth it. As you need to measure performance in order to improve it, I wrote a tool called "loggen". This program generates messages messages at a user-specifyable rate. Apart from the git repository you can get this tool from the latest syslog-ng snapshots. Loggen supports TCP, UDP and UNIX domain sockets, so really almost everything can be me
17 comments
Read more

syslog-ng roadmap 2.1 & 2.2

We had a meeting on the syslog-ng roadmap today where we decided some important things, and I thought I'd use this channel to tell you about it. The Open Source Edition will see a 2.1 release incorporating all core changes currently in the Premium Edition and additionally the SQL destination driver. We are going to start development on the 2.2 PE features, but some of those will also be incorporated in the open source version: support for the latest work of IETF syslog protocols unique sequence numbering for messages support for parsing message contents Previously syslog-ng followed the odd/even version numbering to denote development/stable releases. I'm going to abandon this numbering now: the next syslog-ng OSE release is going to have a 2.1 version number and will basically come out with tested code changes only. The current feature set in PE were developed in a closed manner and I don't want to repeat this mistake. The features that were decided to be part of the Open
12 comments
Read more

syslog-ng 3.0 and SNMP traps

Last time I've written about how syslog-ng is able to change message contents. I thought it'd be useful to give you a more practical example, instead of a generic description. It is quite common to convert SNMP traps to syslog messages. The easiest implementation is to run snmptrapd and have it create a log message based on the trap. There's a small issue though: snmptrapd uses the UNIX syslog() API, and as such it is not able to propagate the originating host of the SNMP trap to the hostname portion of the syslog message. This means that all traps are logged as messages coming from the host running snmptrapd, and the hostname information is part of the message payload. Of course it'd be much easier to process syslog messages, if this were not the case. A solution would be to patch snmptrapd to send complete syslog frames, but that would require changing snmptrapd source. The alternative is to use the new parse and rewrite features of syslog-ng 3.0. First, you need to f
2 comments
Read more