logging in high performance systems.

We have a need for logging in systems where it isn't feasible to log
to disk as it negatively impacts performance.

I'd like to be able to creatively solve this problem without modifying
the core, but today I cannot.

So... here's my first whack at solving this with some flexibility.

The first thing I did was add hook points where immediate statement
logging happens "pre_exec" and those that present duration
"post_exec". These should, with optimization turned on, have only a
few instructions of impact when no hooks are registered (we could
hoist the branch outside the function call if that were identified as
an issue).

https://github.com/postwait/postgres/commit/62bb9dfa2d373618f10e46678612720a3a01599a

The second thing I did was write a sample use of those hooks to
implement a completely non-blocking fifo logger. (if it would block,
it drops the log line). The concept is that we could run this without
risk of negative performance impact due to slow log reading (choosing
to drop logs in lieu of pausing). And a simple process could be
written to consume from the fifo. We use this method in other systems
to log many 10s of thousands of log lines per second with negligible
impact on performance.

https://github.com/postwait/postgres/commit/c8f5a346c7b2c3eba9f72ea49077dc72f03a2679

Thoughts? Feedback?

As can be seen, the patch is pretty tiny.

--
Theo Schlossnagle

http://omniti.com/is/theo-schlossnagle