Two blog spots in a row on two consecutive days, wow :) Things are happening fast these times. Because of my company's efforts to create a commercial fork of syslog-ng, I have somewhat more time to do syslog-ng hacking. This time I've finished persistent disk-buffering, an often requested feature to be released in the commercial version.
It means that if a target server is down, then in addition to the in-memory buffers, syslog-ng is able to store messages in a disk-based queue until the connection is restored. What's more, this queue is persistent and syslog-ng keeps its contents accross restarts.
One less reason to keep logs locally. :)
Next on my list is native SQL support, combined with this disk-buffer feature to cover times when the database is too slow processing INSERTs. I'm thinking along the lines of:
destination d_sql {
sql(type(pgsql) host("loghost") user("syslog-ng") password("secret-password")
database("logs")
table("messages_${HOST}_${R_YEAR}${R_MONTH}${R_DAY}")
columns("date", "host", "program", "pid", "message")
values("$R_DATE", "$HOST", "$PROGRAM", "$PID", "$MSGONLY"));
};
Tables would be created/altered automatically on-demand, just like destination files. Values might refer to builtin macros, or matches within the message (like in sed, e.g. $1 refers to the first match).
The actual implementation might be somewhat different though.
It means that if a target server is down, then in addition to the in-memory buffers, syslog-ng is able to store messages in a disk-based queue until the connection is restored. What's more, this queue is persistent and syslog-ng keeps its contents accross restarts.
One less reason to keep logs locally. :)
Next on my list is native SQL support, combined with this disk-buffer feature to cover times when the database is too slow processing INSERTs. I'm thinking along the lines of:
destination d_sql {
sql(type(pgsql) host("loghost") user("syslog-ng") password("secret-password")
database("logs")
table("messages_${HOST}_${R_YEAR}${R_MONTH}${R_DAY}")
columns("date", "host", "program", "pid", "message")
values("$R_DATE", "$HOST", "$PROGRAM", "$PID", "$MSGONLY"));
};
Tables would be created/altered automatically on-demand, just like destination files. Values might refer to builtin macros, or matches within the message (like in sed, e.g. $1 refers to the first match).
The actual implementation might be somewhat different though.
Comments