Bazsi's Home Automation blog

As the readers of this blog might know I've been working on persistent disk-buffering and SQL support recently. The configuration interface of the SQL destination became quite close to the description I gave in my last post . I think from the user side it is pretty neat, no need to rely on the mysql client program or to create "buffer files" that are later fed to the database server. You simply define an SQL destination and tables are created automagically with proper (possibly even disk-based) queueing, flow control and error handling. And by using libdbi we immediately have support for 4 or 5 different database servers. The implementation side was somewhat more intrusive however, the client libraries for various database servers use a blocking I/O model and syslog-ng was completely non-blocking until now. I had to create a separate thread for inserting records to tables, and there came some required changes to syslog-ng to support that: various reference counters and ...

Two blog spots in a row on two consecutive days, wow :) Things are happening fast these times. Because of my company's efforts to create a commercial fork of syslog-ng, I have somewhat more time to do syslog-ng hacking. This time I've finished persistent disk-buffering, an often requested feature to be released in the commercial version. It means that if a target server is down, then in addition to the in-memory buffers, syslog-ng is able to store messages in a disk-based queue until the connection is restored. What's more, this queue is persistent and syslog-ng keeps its contents accross restarts. One less reason to keep logs locally. :) Next on my list is native SQL support, combined with this disk-buffer feature to cover times when the database is too slow processing INSERTs. I'm thinking along the lines of: destination d_sql { sql(type(pgsql) host("loghost") user("syslog-ng") password("secret-password") database("logs"...

We have been using GNU arch the last couple of years as a version control system, however Tom Lords' implementation does not scale well, and some of our software packages have 10 thousands of commits. This means that a single commit operation may take _minutes_. It is awful to wait so much time for a single commit, and it really degrades productivity. I was considering Mercurial, Bazaar-NG and git, however this was not an easy decision, as the "modern" version control systems promote the use of branches over anything else, and our current version control model relied on cherry-picking heavily: developer commits the solution for each bug separately to his/her branch QA people pick patches from developer branches and integrate them to a 'test' branch, once the test was successful, release manager picks patches from the 'test' branch and integrates to mainline, if he doesn't find anything odd during review This worked wonderfully in GNU arch, but new VC s...