Bazsi's Home Automation blog

I've just pushed a round of updates to the syslog-ng 3.2 repository, featuring some interesting stuff, such as: SQL reorganization: Patrick Hemmer sent in a patch to implement explicit transaction support instead of the previous auto-commit mode used by syslog-ng. I threw in some fixes and refactored the code somewhat. Configuration parser changes: the syntax errors produced by syslog-ng became much more user-friendly: not only the column is displayed, but also the erroneous line is printed and the error location is also highlighted. Additional plugin modules were created: afsql for the SQL destination, and afstreams for Solaris STREAMS devices. Creating a new plugin from core code takes about 15 minutes. I'm quite satisfied. With the addition of these two modules, it is now possible to use syslog-ng without any kind of runtime dependency except libc. The already existing afsocket module (providing tcp/udp sources & destinations) is compiled twice: once with and once withou...

The SQL destination in syslog-ng so far assumed that databases automatically start a new transaction for each INSERT statement that syslog-ng issues. This works fine, however there is a significant overhead of starting new transactions, with sqlite I've measured about 20 times performance increase on my development notebook and my debug build. With explicit-commits: bazsi@bzorp:~/.zwa/install/syslog-ng-ose-3.2$ loggen -x -r 1000000 -I 10 -S log average rate = 9377.28 msg/sec, count=93776, time=10.003, msg size=256, bandwidth=2344.32 kB/sec With per-statement (automatic) commits: bazsi@bzorp:~/.zwa/install/syslog-ng-ose-3.2$ loggen -x -r 1000000 -I 10 -S log average rate = 529.46 msg/sec, count=5299, time=10.083, msg size=256, bandwidth=132.36 kB/sec So this really seem to matter. In order to configure it you can use the following options in an SQL destination: flush_lines/flush_timeout controls how much messages get into the same transaction, similar to what these parameters mean f...

After last the stable syslog-ng 3.1.0 release last week, I've opened the 3.2 branch to receive the new stuff. The first bits are already in the repository: the basic plugin framework and the conversion of the socket related stuff (tcp, udp, unix-dgram, unix-stream, syslog drivers) into a separate plugin. The reason of the afsocket plugin conversion is to help moving the OpenSSL dependency to a separate package in distributions where this dependency cannot be associated with core packages like syslog-ng. But I see a lot of potential behind the plugin framework, and I still have a lot of ideas, just check my last blog post in the topic . Also, there's an even more experimental feature in the "blocks" branch right now, it is still incomplete, the naming and the syntax is still vague. The aim is there to provide syslog-ng.conf C++ template-like functionality in order to make the configuration easier by using pre-configured config snippets. For example: block source tomca...