Bazsi's Home Automation blog

I've just finished preparations for the release of syslog-ng 2.0.3 and now I consider the 2.0.x branch feature complete. As far as I know there's no feature that was present in 1.6.x and missing from the 2.0.x rewrite. The last missing bits were spoof-source and TCP wrapper support, but as those are present in 2.0.3 I consider syslog-ng 2.0 feature complete. I don't plan to add further code that would cause destabilization. Now some plans about the newly opened 2.1 branch: we (as BalaBit) played with an idea of creating a commercial fork of syslog-ng which could increase the amount of resources that I can allocate to developing syslog-ng code. From 2.1 on, two parallel editions of syslog-ng will become available: syslog-ng 2.1 Open Source Edition: the same as syslog-ng 2.0, available under the GPL syslog-ng 2.1 Premium Edition: a commercial fork of syslog-ng, The two releases will be produced from the same source tree with some of the commercial functionality stripped duri...

As a commenter missed spoof-source support from 2.0, I got my act together for an afternoon hacking session and implemented it in the 2.0.x tree. The packet generation for IPv4 packets was straightforward, I could simply "forward-port" it from 1.6.x, after creating the proper place for it in the 2.0.x tree. IPv6 support was a bit more difficult as libnet has a nasty bug with IPv6 and UDP. I was looking at my code for hours, when I tried Google codesearch to check whether I was doing something wrong. Then I've found this . Note the comment above the libnet_build_ipv6() call. I also disabled UDP checksumming and now it works like a charm. The last libnet release was almost 3 years ago, it is not reassuring that its webpage is also down. Too bad, libnet is a fine piece of software, but now as its website is down where can I point syslog-ng users to download libnet from? By the way, I did not announce syslog-ng 2.0.2 here, it was released about a week ago. Nothing fancy, pri...

It's been a while since I last posted here. The reason is simple: work and the lack of time. OK, I know, people have time for what they want, but: * We are launching a nifty new product based on Zorp, but with a much tighter focus. A device capable of recording and replaying SSH sessions for audit purposes. * We have decided to migrate from C to C++ for our management interface, as it seems Gtk+ style OO programming in C really requires a skill, and a lot of developers don't have that. C++ holds their hands somewhat. Thus, I've been refreshing my C++ skills and learning gtkmm while writing a prototype. * and of course the "usual" amount of work. I've tried to provide timely response on the syslog-ng mailing list. I'm sorry if you didn't receive an answer/patch from me in time. By the way I released syslog-ng 2.0.2 yesterday. Nothing really important, some portability fixes, some usertty fixes and an option missed in the 1.6.x -> 2.0.x change. Anot...

I have released syslog-ng 2.0.1, available at the usual places . I have added various missing bits that fell out in the 1.6.x -> 2.0.x change. These include: DNS cache support, overwrite_if_older (used to be called remove_if_older), various fixes, All-in-all 2.0.0 was a successful release, hopefully this one will not be much worse. See the NEWS entry of 2.0.1 for more information.

Just a short note, an interview with me was published on linux.com, more specifically here.

As it turned out the 2.0.0 release was not so bad after all. At least I have not received show-stopper bugreports, which either means that noone is using it, or everything is fine and dandy :). Hopefully it is the latter, rc releases were tested by a few people. In the meanwhile I started adding a few missing bits that were still present in 1.6.x but I never got around to implementing in the 2.0.0 tree. Among them I readded the remove_if_older() option. By the way, I don't really like the name of this option, does anyone have a better idea? If you do, please put it in a comment here or send me an email. (I was thinking about retention_time() but I'm afraid it is more difficult to understand what it would do) The other bit is the new/shiny DNS cache, which also supports persistent entries. This means that syslog-ng can read your /etc/hosts file, resolve IPs that are present there, and use IP addresses for anything else. This removes the dependency on DNS, and should also improve...

You might have already noticed, but I thought I'd write an entry on this blog on this topic: the wait is over I have released syslog-ng 2.0.0. It took a bit longer than I have anticipated, I needed to prepare 4 rc releases, as each had some bug here or there and I really wanted to release a stable 2.0.0 I hopefully succeeded, no breakage reported so far. 2.0.0 was released last saturday, but the announcement went out only on Friday. Some rarely used functionality is still missing though (a prominent example is spoof_source), and I already committed the remove_if_older() option after the release of 2.0.0. I'm going to concentrate on filling the missing bits. I'll try to avoid committing to 1.6.x, migrating to 2.0.x is strongly recommended.

It's been a long while since my last post here, but I was really busy in the past months and after I left for a two weeks vacation to Corsica. I returned on Sunday, started to catch up with work and such but I still was not able to read my syslog-ng mailing list folder, containing almost 100 unread messages. Please be patient, if you have a question open and forgiving in the unfortunate case I'd forget to reply. On the other hand Corsica is a beatiful island, be sure to visit it if you can. Nature is almost untouched at a couple of places, everything is green and a lot of mountains. A pair of hiking shoes is a useful item if you are in Corsica. :) So the island is beatiful we had some minor nuances with waiters, as neither me or my wife speaks French, and this seems to be a sin in the eyes of Corsican waiters. So at the end we came up with cooking for ourselves, lucky us our apartment was nicely equipped with cooking gear. Back to syslog-ng, I'd really like to release 2.0.0...

After some time I needed to do some kernel coding again. To seamlessly support dynamically created interfaces in Zorp, we need something I called "interface groups". Each interface might belong to a single group that basically describes how the interface was created. For instance there's an interface group for each PPP profile, but an interface group can encapsulate interfaces created by PPTPD. It is quite difficult to match dynamic interfaces by their nature: iptables sports wildcard interface name matching with the '+' character but it only works if interface names have some kind of prefix _AND_ if you don't want to differentiate between two groups. If you have two sets of PPP devices (like in the example I described above), then you have no way to create a separate ruleset, unless you reload iptables everytime a new interface is added to the system. Adding to the burden, in Zorp we want to be able to bind a service to these dynamically created interfaces, o...

After my last requests for testing of the latest 1.9.x code base, I have received a couple of bug reports, which were fixed in recent weeks. Since I have received no reports the past two weeks I decided to name the new release as "2.0rc1" to raise awareness of the new codebase. I'm planning to create the new branch for 2.1.x, I have some exciting features in my mind, which I did not want to start before the release of 2.0.0. The old stable series 1.6.x is still supported, but expect less development time to be dedicated in maintaining that release. Build queues for various architectures are not yet up, so only a Debian sarge binary is available for those with binary maintenance contracts.

You might know that there is a standardization effort on the syslog protocol in the IETF. The work has started several years ago and the efforts produced RFC3164, the first documentation of the BSD syslog protocol after being in use for over two decades. This group also produced RFC3195 in 2001, a reliable syslog protocol using the BEEP framework which did not really take of. I personally did not implement this in syslog-ng due to its highly verbose nature and the complexity which BEEP brings in. Couple of months ago an effort started to create a simpler, but still reliable syslog protocol somewhat similar to what syslog-ng has been using for a couple of years now. First some layering was decided, e.g. to define the syslog protocol in a transport independent manner and then define various transports, like legacy UDP and TLS encrypted TCP. After syslog-transport-udp was written by Rainer Gerhards, work has started on the TLS encrypted transport and someone from Huawei (you know the Chin...

This is just a quick note that I'm not yet lost, I'm on holidays for two weeks, as I'm getting married this weekend, or to be more precise 24th June, 2006. I'm returning to work on 3rd July, 2006. As my internet connection is not perfect, I might not be able to respond to e-mail messages timely.

It was just a week or two ago when someone asked me about the planned release date of syslog-ng 2.0.0, the first stable release of the third incarnation of syslog-ng. Probably I did not even respond to the email as I did not know the answer. "When it's ready" is an answer users do not usually perceive very well. It is very difficult to judge when a rewrite of such a critical software package is stable enough for production use: I wrote both functional and unit tests, used syslog-ng on my laptop for over a year now, but as I currently lack a system were non-production code can be uploaded, syslog-ng was drifting slowly in the stabilization process: whenever someone reported a bug, I fixed it. So the release date in the current state is determined by the syslog-ng user community and not me. IF there's certain confidence that a pile of code runs fine, it can be tagged stable and everyone can be happy. If there is no feedback, an optimist might think that everything is go...

Again the question on Solaris message IDs was raised in an email sent to me in private. For those who don't know how a Solaris msgid looks like, look at this example: May 14 18:51:57 inbound2 su: [ID 366847 auth.notice] 'su root' succeeded I was asked to include an MSGNOID macro which excludes this msgid in the final destination. The problem I have with this approach is that it simply does not scale: there are simply too many combinations to cover with various macros, an example using the msgid case: a macro that includes neither the name of the program, nor the msgid a macro that includes program name only a macro that includes msgid but not the program name a macro that includes both the program name and the msgid As you can imagine this quickly becomes a maintenance nightmare even if one finds out a proper name for all of these combinations, especially if you add that other devices have their own extensions to syslog. What I am pondering is to renew my old ...

I have released syslog-ng 1.6.11 which fixes the problems outlined in the previous post. You can find it at the BalaBit website.

Just a quick one, it turned out that syslog-ng 1.6.10 is broken in several ways, first reading messages from /proc/kmsg is broken, and second the time_sleep() feature that was added in 1.6.10 has missed an important chunk from the parser code which made time_sleep() unconfigurable. So a feature that cannot be used and an important problem. :( I'm going to release syslog-ng 1.6.11 soon.

I spent the last week in London, visiting InfoSec Europe. It was a great fun, I liked the exhibition as well as the city itself. I have not been to London before (except for a single-day business trip two years ago, but that does not count), and I liked the city very much. I walked about 40-50km on these three days, I had my legs completely worn out. British people are quite strange I would say. Everything is completely in the reverse: the cars, the direction the trains arrive from, the way the taps need to be opened, I think even the screws must be unmounted in the reverse direction. I hated these non-mixing taps, one tap for cold another for hot water, no way to mix something tepid. Beside this strangeness I liked the atmosphere of the city, I visited all the important places, I even spent two hours in the British Museum, but it was nothing but a scratch on the surface. The exhibition was also interesting, met a couple of interesting persons, like the Watchfire guys who invented HTT...

I have finished IPv6 support for syslog-ng, I'm wondering how this will improve the number of people actually using the new syslog-ng 1.9.x tree. In the implementation I've created separate udp6() and tcp6() source and destination drivers, because this was somewhat easier to implement. I'm expecting some portability trouble, but otherwise the implementation is nice and simple. Some smaller fixes went in recently as well, like: avoid chown/chmod files that do not exist as it clobbered error reporting, added close-on-exec flag to file descriptors to avoid child processes to inherit tcp/udp connection fds, fixed an off-by-one in flush_lines calculation, a possible memory leak, and a fix for non-existing filter references in the internal() message path Apart from IPv6 support these are mainly bugfixes and I'm confident we can have a 2.0.0 real soon now.

I received an email last week asking about IPv6 support in syslog-ng. The question also referred to an IPv6 application page where most of the system logging applications were listed red showing that they lack IPv6 support. I started hacking on it but I thought I would ask you how important you think adding IPv6 support to syslog-ng is? My idea was to add tcp6() and udp6() source and destination drivers, however a lot of applications seem to do IPv6 support with a single listener, e.g. open an AF_INET6 socket and assume that the system shares IPv4 and IPv6 sockets. Which of the two approaches is preferable? The first gives more control the latter seems to be a bit easier to use, it works out of the box.

Just a quick one this time, I have released syslog-ng 1.9.10 available at the usual places . The release contains mainly bugfixes and an implementation of the previously missing netmask() filter and support for bad_hostname() and check_hostname() options.