WIP - syslogger infrastructure changes

From: Magnus Hagander <magnus(at)hagander(dot)net>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Cc: Joshua Tolley <eggyknap(at)gmail(dot)com>
Subject: WIP - syslogger infrastructure changes
Date: 2009-09-14 19:41:34
Message-ID: 9837222c0909141241u26ea8e50h229c9c0379f33508@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I've been working on a restructure of some of the syslogger
infrastructure with Josh Tolley. Attached is the WIP patch that we
have so far.

First, the patch removes the logging_collector parameter and basically
assumes that logging_collector is always on. With that, it's no longer
necessary to restart your server just to reconfigure the logging, and
it also takes away a confusing parameter (really
"log_destination=stderr, logging_collector=on" is *not* a logical way
to say "log to file"). Instead, it adds a log_destination of "file"
that is the standard log to file. Given that the syslogger is now
always started, those that actually *want* logging to stderr (which I
claim is a low number of people, but that's a different story) will
have it go through the syslogger and then to the stderr of syslogger.
If syslogger is not used at all (say with syslog log target that goes
directly from backend), it'll just sit around doing nothing, and the
cost should be very close to zero.

Second, data transfer from the backends is now in the form of
structured data, and the actual logging message is put together in the
syslogger (today,it's put together in the backend and just sent as a
string to the syslogger). Yes, this means that we will likely send
more data than what's eventually going to be logged, since all fields
don't go out (except with CVS logging, I guess). But you usually don't
send *that* much data in the log. It *does* allow for us to properly
specify multiple log destinations, which we only "almost" can now. For
example, it allows a regular logfile and csvlog at the same time. And
since stderr actually means stderr, you can combine stderr and
logfile.

By putting more available data in the syslogger, this is also intended
to make it easier to make the syslogger functionality more advanced in
the future. This includes the kind of work that Itagaki Takahiro
submitted a patch around recently, and things like being able to write
different things to different logfiles.

As the name says, it's still a WIP, and the code contains both several
FIXME places, and some other non-finished parts so far, but we're
posting it to get comments on the general idea and method for doing
it.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Attachment Content-Type Size
syslogger_infra.patch text/x-diff 46.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Simon Riggs 2009-09-14 19:41:44 Re: Streaming Replication patch for CommitFest 2009-09
Previous Message Andrew Dunstan 2009-09-14 19:37:53 Re: COPY enhancements