Tuesday, March 28, 2006

Preparing syslog-ng release

I have started to prepare syslog-ng 1.6.10 for release, the tarball has already been uploaded to the website, but I still have not sent an announcement to the mailing lists. So if you read this here, you might download a still unannounced version :)

Nothing really important in the release, a cleanup in the documentation with several fixes and a migration to DocBook/XML from the SGML favour and a new tunable called time_sleep().

The latter was worked out together with John Morrissey who did some profiling and found that on hosts with a lot of syslog connections syslog-ng might become a bottleneck. The option does nothing but sleep() a defined amount of time which makes syslog-ng to process incoming messages in batches, this way decreasing the number of poll() loop iterations which was listed high (about 67%) in the profiles generated by John.

Setting time_sleep() to about 50ms decreased the CPU load by 80% which is quite significant I'd say.

As Rusty Russell would say I have just received a SIGWIFE, so going to bed now :)


Roberto Nibali said...

I would still like to see the /dev/epoll approach one day since I think this is only temporary plaster. I'd do it if time permitted, but I'm really too busy with SIGWIFE and work myself ;).

Bazsi said...

Sure, /dev/epoll has some merits, however the point is that usually there's no point in hurrying, a lot more events can be processed if we wait a couple of msecs, and the main loop is expensive regardless whether we do it with epoll or plain old poll().

As far as I know epoll only improves the performance of poll() itself, but in our case the code to prepare for poll() call was burning the CPU.