Monday, March 22, 2010

syslog-ng 3.1 final release

I'm proud to announce that both the Open Source and the Premium editions of syslog-ng 3.1 was published and are available on our website.

This is an important milestone in multiple ways:
  • the new feature/stable release schema is making its debut
  • the patterndb got significant improvements: new parsers, pdbtool, tagging support
  • the ability to change/add RFC5424 style structured data to messages
  • even more supported platforms (Tru64 on alpha, HP-UX 11iv2 on Itanium and older Linux versions)
  • the diverging developments of syslog-ng Open Source Edition, Premium Edition and syslog-ng Store Box was merged into a new base,
Some interesting (ok, for us developers :) statistics follow:

Premium Edition:
  • 586 commits
  • 200 files changed, 23479 insertions(+), 5513 deletions(-)
Open Source Edition:
  • 189 commits
  • 115 files changed, 9020 insertions(+), 3225 deletions(-)
The reason for the big difference is the merger of the currently propriatery log indexer engine used in SSB into the current Premium Edition tree, otherwise the two should be in sync.

The binaries/source packages can be downloaded via the usual URL:

Changelogs for the two releases:

Premium Edition:

Open Source Edition:

And of course the OSE source is also available in our public git repository:

Happy logging!

Saturday, March 06, 2010

plugins branch updated

Since the last post, I could hack a couple of hours on the plugins branch, which now compiles. The plugin framework is capable for supporting a quite important core functionality: all socket like sources/destinations are now found in an external plugin called "afsocket".

The reason I've started with afsocket is to make syslog-ng a bit less dependant on OpenSSL. A couple of distributions didn't include syslog-ng 3.0 in their current releases, because it uses OpenSSL from /usr, while syslog-ng should remain in the root directory.

By separating afsocket from the syslog-ng core, I can compile afsocket with and without TLS support, which can be put into separate packages. Thus syslog-ng can operate without OpenSSL.

And the same plugin framework will enable me to create a wide variety of plugins. My ideas:
  • Plugins for all syslog-ng components (source, destination, filter, rewrite, parser)
  • Python scriptability (a simple correllation engine in Python?)
  • macro transformation functions, for example: $(stripslashes $macro), usable anywhere in templates and stripslashes a plugin that is invoked whenever such an expansion occurs
  • Hooks for transforming the log message as it enters syslog-ng (to fix parsing errors for example),
Do you have other ideas? Please post them as comments or as emails to the mailing list.

Again, this functionality is experimental, and I'm still going to rebase the current code and will probably be integrated to syslog-ng 3.2. I got to release 3.1 final first though. :)

plugins preview

Things have been a little rough last couple of months, that's why I haven't posted here. I'm in a rush right now as well, but I just wanted to let you know that I have started working on modularizing syslog-ng.

It is only a preliminary prototype, and as of now it doesn't compile, but the way it's going to work is already visible: each plugin will have its own plugin and with some trickery the large syslog-ng.conf parser will call out to the plugin parser. The user will recognize such a plugin as an integral part of syslog-ng.

E.g. this is a sample configuration file:

@version: 3.0
@module: dummy


destination d_dummy { dummy(dummy_opt(yes)); };


See the dummy plugin code in my git repository, in the "plugins" branch. Please note that that branch is going to be rebased a couple of times yet, I've released it in the spirit of "release early, release often".

I hope to get some of the recent contributions into plugins, instead of bloating the core syslog-ng code. For example output colorization. I'm also thinking about adding built-in scripting support via Python.