Re: Log rotation

Hi,

Please remind me again why the postmaster cannot close and open the log
file when it receives a SIGHUP (to re-read configuration)? This was
discussed before but I cannot remember if and why this was not possible
or if the arguments are still valid after -l was added.

If this was possible it could even be done after re-reading the
configuration and even use the value of a GUC variable for the log file
name, which would allow us to change the value of -l without the need to
restart (actualy, with a GUC variable set in postgresql.conf one could
set the log without the -l, but would perhaps lose a few initial messages).

regards to all,
Fernando

Fernando Nasser <fnasser(at)redhat(dot)com> writes:
> Please remind me again why the postmaster cannot close and open the log
> file when it receives a SIGHUP (to re-read configuration)?

(a) Because it never opened it in the first place --- the log file is
whatever was passed as stderr.

(b) Because it would not be sufficient to make the postmaster itself
close and reopen the file; every child process would have to do so also.
Doing this in any sort of synchronized fashion seems impossible.

It's much cleaner to have stderr be a pipe to some separate collector
program that can handle log rotation (ie, the Apache solution).

regards, tom lane

Tom Lane wrote:
> Fernando Nasser <fnasser(at)redhat(dot)com> writes:
>
>>Please remind me again why the postmaster cannot close and open the log
>>file when it receives a SIGHUP (to re-read configuration)?
>
>
> (a) Because it never opened it in the first place --- the log file is
> whatever was passed as stderr.
>
> (b) Because it would not be sufficient to make the postmaster itself
> close and reopen the file; every child process would have to do so also.
> Doing this in any sort of synchronized fashion seems impossible.
>

Now I remember. Thanks for reminding me of that.

> It's much cleaner to have stderr be a pipe to some separate collector
> program that can handle log rotation (ie, the Apache solution).
>

We could also create a pipe and start a new process ("logger") and give
it the other end of the pipe and the name of the log file. We could
send it a SIGHUP after we reread the configuration file.

But just doing a pipe on the OS level is way simpler.

I don't really care on how its done, but IMO an enterprise class
database must be able to do log rotation. Logging to Syslog is not an
option (specially with our verbosity) -- users must be able to use flat
files for logging.

I never seem some many customer complaints and bug reports about a
single item like the ones we have received here about logging. I think
this should be the number 1 item in te TODO list.

Thanks again for the clarifications.

Regards,
Fernando

On Fri, Mar 12, 2004 at 09:24:28 -0500,
Fernando Nasser <fnasser(at)redhat(dot)com> wrote:
>
> I don't really care on how its done, but IMO an enterprise class
> database must be able to do log rotation. Logging to Syslog is not an
> option (specially with our verbosity) -- users must be able to use flat
> files for logging.
>
>
> I never seem some many customer complaints and bug reports about a
> single item like the ones we have received here about logging. I think
> this should be the number 1 item in te TODO list.

Are you suggesting the that postgres project develop their own logger
rather than people just using one that has already been developed
by some other group?

Bruno Wolff III <bruno(at)wolff(dot)to> writes:
> Are you suggesting the that postgres project develop their own logger
> rather than people just using one that has already been developed
> by some other group?

The problem from the point of view of Red Hat is to not introduce a
dependency from the Postgres RPM to the Apache RPM ... this is no
problem for people who don't mind hand-customizing their setup, but
it is a problem if you want it to be part of the out-of-the-box setup.

regards, tom lane

Tom Lane wrote:

>Bruno Wolff III <bruno(at)wolff(dot)to> writes:
>
>
>>Are you suggesting the that postgres project develop their own logger
>>rather than people just using one that has already been developed
>>by some other group?
>>
>>
>
>The problem from the point of view of Red Hat is to not introduce a
>dependency from the Postgres RPM to the Apache RPM ... this is no
>problem for people who don't mind hand-customizing their setup, but
>it is a problem if you want it to be part of the out-of-the-box setup.
>
>

Did anything ever come from this thread?
http://archives.postgresql.org/pgsql-hackers/2003-05/msg00603.php
(Heading: "Plan B for log rotation support: borrow Apache code")

cheers

andrew

On Fri, Mar 12, 2004 at 13:17:50 -0500,
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Bruno Wolff III <bruno(at)wolff(dot)to> writes:
> > Are you suggesting the that postgres project develop their own logger
> > rather than people just using one that has already been developed
> > by some other group?
>
> The problem from the point of view of Red Hat is to not introduce a
> dependency from the Postgres RPM to the Apache RPM ... this is no
> problem for people who don't mind hand-customizing their setup, but
> it is a problem if you want it to be part of the out-of-the-box setup.

I can see their problem with making a dependency to all of apache or including
multilog in their distribution. But they probably could include something
that is only a logger either using some project that is only a logger or
splitting out the logger that is bundled with apache. Then it wouldn't
be unreasonable to make a dependency for postgres requiring that logging
rpm. Other services could also make use of this logging package as well.

On Friday 12 March 2004 09:24 am, Fernando Nasser wrote:
> I don't really care on how its done, but IMO an enterprise class
> database must be able to do log rotation. Logging to Syslog is not an
> option (specially with our verbosity) -- users must be able to use flat
> files for logging.

Uh, we have many many many different ways to use syslog. So my other message
on the topic.

> I never seem some many customer complaints and bug reports about a
> single item like the ones we have received here about logging. I think
> this should be the number 1 item in te TODO list.

Uh, upgrading? I'm sure we have more reports about upgrading than logging.

But see my reply to bug 103767 for more.
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC 28772
(828)862-5554
www.pari.edu

Bruno Wolff III wrote:
>
> I can see their problem with making a dependency to all of apache or including
> multilog in their distribution. But they probably could include something
> that is only a logger either using some project that is only a logger or
> splitting out the logger that is bundled with apache. Then it wouldn't
> be unreasonable to make a dependency for postgres requiring that logging
> rpm. Other services could also make use of this logging package as well.
>

Yes that would be nice. I have no idea how difficult it would be to extricate
the logrotate program from Apache. I also don't know if there would be any
license restrictions, would we be able to redistribute it as an independently
package? I don't know the answer.

Regards,
Fernando

Hi Lamar,

Lamar Owen wrote:
> On Friday 12 March 2004 09:24 am, Fernando Nasser wrote:
>
>>I don't really care on how its done, but IMO an enterprise class
>>database must be able to do log rotation. Logging to Syslog is not an
>>option (specially with our verbosity) -- users must be able to use flat
>>files for logging.
>
>
> Uh, we have many many many different ways to use syslog. So my other message
> on the topic.
>

Which other message?

Anyway, Syslog is not an option for us. We need flat files.

>
>>I never seem some many customer complaints and bug reports about a
>>single item like the ones we have received here about logging. I think
>>this should be the number 1 item in te TODO list.
>
>
> Uh, upgrading? I'm sure we have more reports about upgrading than logging.
>

Yeah, but that only comes with a full version upgrade :-)

The logging one keeps popping up as people try and use the server for production.

> But see my reply to bug 103767 for more.

See my reply to your reply ;-)

Best regards,
Fernando

On Fri, Mar 12, 2004 at 15:19:29 -0500,
Fernando Nasser <fnasser(at)redhat(dot)com> wrote:
> Bruno Wolff III wrote:
> >
> >I can see their problem with making a dependency to all of apache or
> >including
> >multilog in their distribution. But they probably could include something
> >that is only a logger either using some project that is only a logger or
> >splitting out the logger that is bundled with apache. Then it wouldn't
> >be unreasonable to make a dependency for postgres requiring that logging
> >rpm. Other services could also make use of this logging package as well.
> >
>
> Yes that would be nice. I have no idea how difficult it would be to
> extricate the logrotate program from Apache. I also don't know if there
> would be any license restrictions, would we be able to redistribute it as
> an independently package? I don't know the answer.

I was suggesting this as something a distro maintainer (such as Redhat)
could do. I think that the postgres developers shouldn't be spending
time doing this. They should be just suggesting some possibilities
in the admin part of the documentation.

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Did anything ever come from this thread?
> http://archives.postgresql.org/pgsql-hackers/2003-05/msg00603.php
> (Heading: "Plan B for log rotation support: borrow Apache code")

Only an entry on my depressingly long personal to-do list :-(

I did take a look at the Apache rotator program, and found that it was
probably more trouble to adopt than it's worth. It seemed to depend on
a lot of configuration and library-routine infrastructure that we don't
share. (No big surprise; I suppose someone trying to pull out a random
bit of our backend code would be at least as unhappy.) I suspect it
would be less trouble, as well as legalistically cleaner, to write our
own from scratch.

Andrew Sullivan offered Afilias' rotator script awhile back also.
I think that works fine if you like a Perl script.

regards, tom lane

On Friday 12 March 2004 03:21 pm, Fernando Nasser wrote:
> Lamar Owen wrote:
> > Uh, we have many many many different ways to use syslog. So my other
> > message on the topic.

> Which other message?

The one I didn't post. :-)

> Anyway, Syslog is not an option for us. We need flat files.

Ok, riddle me this:

If I have PostgreSQL set to log to syslog facility LOCAL0, and a local0.none
on /var/log/messages and local0.* to /var/log/pgsql (assuming only one
postmaster, unfortunately) then you get a flat file.

I can see that in a multipostmaster setting how you might want some
differentiation between postmasters, but ISTM that the tool reading these
logs should be trained in how to separate loglines out. I use a product
called SmoothWall as a firewall/VPN solution, and its log reporting modules
split out loglines in this way, so that I can have the ipsec logs in one
browser page and the l2tp logs elsewhere, and the ppp logs elsewhere, and the
kernel logs elsewhere....

Or you'll have to include some other log rotator.

> > Uh, upgrading? I'm sure we have more reports about upgrading than
> > logging.

> Yeah, but that only comes with a full version upgrade :-)

Which is quite often. And people tend to upgrade as part of the OS upgrade,
which could easily be every other version (in the case of Fedora). Upgrading
from RHAS 2.1 to RHEL3 (I know it's not supported) should prove to be an
interesting one.

> The logging one keeps popping up as people try and use the server for
> production.

Yes, and for the vast majority of cases syslog is enough solution for the job.
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC 28772
(828)862-5554
www.pari.edu

On Saturday 13 March 2004 09:36, Lamar Owen wrote:
> On Friday 12 March 2004 03:21 pm, Fernando Nasser wrote:
> > Lamar Owen wrote:
> > > Uh, we have many many many different ways to use syslog. So my other
> > > message on the topic.
> >
> > Which other message?
>
> The one I didn't post. :-)
>
> > Anyway, Syslog is not an option for us. We need flat files.
>
> Ok, riddle me this:
>
> If I have PostgreSQL set to log to syslog facility LOCAL0, and a
> local0.none on /var/log/messages and local0.* to /var/log/pgsql (assuming
> only one postmaster, unfortunately) then you get a flat file.
>
> I can see that in a multipostmaster setting how you might want some
> differentiation between postmasters, but ISTM that the tool reading these
> logs should be trained in how to separate loglines out. <snip>

Different postmasters = different conf files. Just set your syslog_facility
and/or your syslog_ident differently and it should be pretty easy to seperate
the logs. Actually, now that I have started using syslog fairly regularly, I
find it hard to believe it would be worth the effort to try to recreate a
logging facility as feature complete as syslog when we have syslog available.
ranks right up there with recreate cron, another feature many people think an
enterprise database "has to have". Personally I think in place upgrades are
far, far more important than either of those two, to both end users and to
the development community.

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Bruno Wolff III wrote:
> On Fri, Mar 12, 2004 at 15:19:29 -0500,
> Fernando Nasser <fnasser(at)redhat(dot)com> wrote:
>
>>Bruno Wolff III wrote:
>>
>>>I can see their problem with making a dependency to all of apache or
>>>including
>>>multilog in their distribution. But they probably could include something
>>>that is only a logger either using some project that is only a logger or
>>>splitting out the logger that is bundled with apache. Then it wouldn't
>>>be unreasonable to make a dependency for postgres requiring that logging
>>>rpm. Other services could also make use of this logging package as well.
>>>
>>
>>Yes that would be nice. I have no idea how difficult it would be to
>>extricate the logrotate program from Apache. I also don't know if there
>>would be any license restrictions, would we be able to redistribute it as
>>an independently package? I don't know the answer.
>
>
> I was suggesting this as something a distro maintainer (such as Redhat)
> could do. I think that the postgres developers shouldn't be spending
> time doing this. They should be just suggesting some possibilities
> in the admin part of the documentation.
>

The distro maintainers should only pack proven solutions created by the
community. When they add man power to a project they must do it in the
community process, not in house.

A decent logging mechanism (that allows log rotation) is important to
postgres and should not require one to depend on a distro. They should
be able to get it from pgsql land.

It would be nice if there was a community recommended and community
tested logging solution, even if the instructions to set it up required
one to download and install some apache package.

But adding prodution quality logging to PostgreSQL is still a postgresql
community TODO and should not be delegated to the distros and thus
restricted to a distro only.

Regards,
Fernando

Lamar Owen wrote:
>
>>Anyway, Syslog is not an option for us. We need flat files.
>
>
> Ok, riddle me this:
>
> If I have PostgreSQL set to log to syslog facility LOCAL0, and a local0.none
> on /var/log/messages and local0.* to /var/log/pgsql (assuming only one
> postmaster, unfortunately) then you get a flat file.
>

The problem is that sysloging has more overhead than a plain append to a
file. There are some very strict response time AppServer applications
where we want to keep this things out of the picture.

The other problem is that we have some nice graphical tools for
configuring logging but the /etc/syslog.conf is a very hard one to
automate dur to the pattern matching. We can add some lines there, like
one to get local0 to a specific file, but it will probably be guesswork
and require human inspection anyway.

> I can see that in a multipostmaster setting how you might want some
> differentiation between postmasters, but ISTM that the tool reading these
> logs should be trained in how to separate loglines out. I use a product
> called SmoothWall as a firewall/VPN solution, and its log reporting modules
> split out loglines in this way, so that I can have the ipsec logs in one
> browser page and the l2tp logs elsewhere, and the ppp logs elsewhere, and the
> kernel logs elsewhere....
>

It may be desirable to logrotate them at different times as well, so
they would have to be in different files.

> Or you'll have to include some other log rotator.
>

That is what Tom is proposing.

--
Fernando Nasser
Red Hat Canada Ltd. E-Mail: fnasser(at)redhat(dot)com
2323 Yonge Street, Suite #300
Toronto, Ontario M4P 2C9

> > I can see that in a multipostmaster setting how you might want some
> > differentiation between postmasters, but ISTM that the tool reading these
> > logs should be trained in how to separate loglines out. <snip>
>
> Different postmasters = different conf files. Just set your syslog_facility
> and/or your syslog_ident differently and it should be pretty easy to seperate
> the logs. Actually, now that I have started using syslog fairly regularly, I

Not that I'm volunteering, but I think the biggest issue is many users
simply don't know how to approach the problem. Some docs on using
syslog, cron, etc. with PostgreSQL to accomplish maintenace jobs would
probably be enough.

The only mention we have of syslog is related to the postgresql.conf
entries and their appropriate values.

http://www.postgresql.org/docs/7.4/static/runtime-config.html#RUNTIME-CONFIG-LOGGING-SYSLOG

Robert Treat wrote:
>
> Different postmasters = different conf files. Just set your syslog_facility
> and/or your syslog_ident differently and it should be pretty easy to seperate
> the logs. Actually, now that I have started using syslog fairly regularly, I
> find it hard to believe it would be worth the effort to try to recreate a
> logging facility as feature complete as syslog when we have syslog available.
> ranks right up there with recreate cron, another feature many people think an
> enterprise database "has to have". Personally I think in place upgrades are
> far, far more important than either of those two, to both end users and to
> the development community.
>

The concerns with syslog were related to performance (additional load on
the server). Perhaps these concerns were unfounded.

I am not saying that upgrades are not important. I have been asking for
it and have even tried to come up with a process to keep a continuously
functional pg_upgrade program in synch with the cvs head development,
even adding bits to the catalog for one release to aloow the conversion
to be done. Ask Bruce and Tom, I've mentioned this to them 2 or 3 years ago.

But the log rotation is a relatively small task in comparison, and it is
at least as equaly visible in terms of production users (at least in
the enterprise). I am talking about benefit/effort ratio.

Regards,
Fernando

Rod Taylor wrote:
>>>I can see that in a multipostmaster setting how you might want some
>>>differentiation between postmasters, but ISTM that the tool reading these
>>>logs should be trained in how to separate loglines out. <snip>
>>
>>Different postmasters = different conf files. Just set your syslog_facility
>>and/or your syslog_ident differently and it should be pretty easy to seperate
>>the logs. Actually, now that I have started using syslog fairly regularly, I
>
>
> Not that I'm volunteering, but I think the biggest issue is many users
> simply don't know how to approach the problem. Some docs on using
> syslog, cron, etc. with PostgreSQL to accomplish maintenace jobs would
> probably be enough.
>

It is very easy to setup logging with the Control Center tool. Except
that it does not handle the /etc/syslog.conf bit (you have to add the
entry to get a specific LOCAL to a specific file by hand.

Maybe someone could make some measurements to desmistify the performance
impact of syslog. There is this generalized belief in IS departments
that should minimize it. Perhaps is an old fud that is not anylonger true.

--
Fernando Nasser
Red Hat Canada Ltd. E-Mail: fnasser(at)redhat(dot)com
2323 Yonge Street, Suite #300
Toronto, Ontario M4P 2C9

On Sat, Mar 13, 2004 at 10:36:23AM -0500, Fernando Nasser wrote:
> Lamar Owen wrote:
> >Ok, riddle me this:
> >
> >If I have PostgreSQL set to log to syslog facility LOCAL0, and a
> >local0.none on /var/log/messages and local0.* to /var/log/pgsql (assuming
> >only one postmaster, unfortunately) then you get a flat file.
>
> The problem is that sysloging has more overhead than a plain append to a
> file. There are some very strict response time AppServer applications
> where we want to keep this things out of the picture.

I thought it was an advantage to say "log to that box running syslog over
there and leave my disk alone" - what do you have in mind with "AppServer
applications" ?

> It may be desirable to logrotate them at different times as well, so
> they would have to be in different files.

syslogd with newsyslog, just like any other log file? I must be missing
something.. I don't see why postgresql is different..

Cheers,

Patrick

Patrick Welche wrote:
> On Sat, Mar 13, 2004 at 10:36:23AM -0500, Fernando Nasser wrote:
>
>>Lamar Owen wrote:
>>
>>>Ok, riddle me this:
>>>
>>>If I have PostgreSQL set to log to syslog facility LOCAL0, and a
>>>local0.none on /var/log/messages and local0.* to /var/log/pgsql (assuming
>>>only one postmaster, unfortunately) then you get a flat file.
>>
>>The problem is that sysloging has more overhead than a plain append to a
>>file. There are some very strict response time AppServer applications
>>where we want to keep this things out of the picture.
>
>
> I thought it was an advantage to say "log to that box running syslog over
> there and leave my disk alone" - what do you have in mind with "AppServer
> applications" ?
>

And add more packets to the network, buffering etc.? I don't think so.

There are some applicatons which run in servers with very strict
response times and any syscall, network packet that can be saved counts.

>
>>It may be desirable to logrotate them at different times as well, so
>>they would have to be in different files.
>
>
> syslogd with newsyslog, just like any other log file? I must be missing
> something.. I don't see why postgresql is different..
>

The number of generated messgaes.

Maybe that is an area that can be worked on, i.e. reducing log
verbosity. Is 7.4.x much better than 7.3.x in that respect?

--
Fernando Nasser
Red Hat Canada Ltd. E-Mail: fnasser(at)redhat(dot)com
2323 Yonge Street, Suite #300
Toronto, Ontario M4P 2C9

On Saturday 13 March 2004 10:36 am, Fernando Nasser wrote:
> The problem is that sysloging has more overhead than a plain append to a
> file. There are some very strict response time AppServer applications
> where we want to keep this things out of the picture.

Well, I have a couple of ideas on that. First, a better syslog. SDSC secure
syslog has been available for some time and is BSD licensed, and is
specifically designed for high-performance RFC3195 compliant secure syslog.

Second, if the syslog server is separate, then you might get better
performance as the size of the logs grow, since appending a very large file
is slower than generating an IP datagram.

Third, it seems that you don't have enough profiling data to support a 'syslog
is bad' position. Java is bad too, from a performance standpoint (at this
time, at least, you always get a performance hit of some kind using any
portable code). But if this AppServer is based on the ArsDigita Java
codebase, you have other performance issues already (the Tcl codebase, OTOH,
is very fast, partly because AOLserver is a faster appserver than most Java
appservers).).

> The other problem is that we have some nice graphical tools for
> configuring logging but the /etc/syslog.conf is a very hard one to
> automate dur to the pattern matching. We can add some lines there, like
> one to get local0 to a specific file, but it will probably be guesswork
> and require human inspection anyway.

Control Center looks promising. But I much prefer having a central syslog
server than having each server in a cluster having separate logs.

> It may be desirable to logrotate them at different times as well, so
> they would have to be in different files.

Different facilities can then go to different files. The problem, of course,
is syslog's limited number of facilities.

> > Or you'll have to include some other log rotator.

> That is what Tom is proposing.

I am not opposed to including a small logrotator for stderr logging. I just
think it is redundant when a good highly configurable logging facility
already exists. But, if Red Hat wants to pay Tom to do it... :-)
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC 28772
(828)862-5554
www.pari.edu

On Saturday 13 March 2004 01:00 pm, Fernando Nasser wrote:
> There are some applicatons which run in servers with very strict
> response times and any syscall, network packet that can be saved counts.

Ok, what about pipe overhead? If we're gong to split hairs, let's split all
of them. The design of the pipeline in this logrotator filter will have to
be such that minimal overhead occurs, because, at the real-time level, every
concurrent process also counts.

> The number of generated messgaes.

> Maybe that is an area that can be worked on, i.e. reducing log
> verbosity. Is 7.4.x much better than 7.3.x in that respect?

There are several levels documented in postgresql.conf. Try the terse level
and see what happens.
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC 28772
(828)862-5554
www.pari.edu

Tom Lane wrote:

>Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>
>
>>Did anything ever come from this thread?
>>http://archives.postgresql.org/pgsql-hackers/2003-05/msg00603.php
>>(Heading: "Plan B for log rotation support: borrow Apache code")
>>
>>
>
>Only an entry on my depressingly long personal to-do list :-(
>
>I did take a look at the Apache rotator program, and found that it was
>probably more trouble to adopt than it's worth. It seemed to depend on
>a lot of configuration and library-routine infrastructure that we don't
>share. (No big surprise; I suppose someone trying to pull out a random
>bit of our backend code would be at least as unhappy.) I suspect it
>would be less trouble, as well as legalistically cleaner, to write our
>own from scratch.
>
>Andrew Sullivan offered Afilias' rotator script awhile back also.
>I think that works fine if you like a Perl script.
>
>
>

FWIW, in less than 30 minutes I took the log rotator from apache 1.3.29
(i.e. the latest non-APR version) and imported it into a fresh postgreql
tree. With very little massaging it built happily (see below).

If it will advance matters, I can submit this as a patch filling the
currently empty contrib/apache_logging directory. You could be right
about the legal stuff - worth talking to the apache folks?

cheers

andrew

[andrew(at)alphonso apache_logging]$ make
gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
-Wmissing-declarations rotatelogs.o -L../../src/port
-Wl,-rpath,/home/andrew/tpg/inst//lib -o rotatelogs
[andrew(at)alphonso apache_logging]$ rm rotatelogs
[andrew(at)alphonso apache_logging]$ rm rotatelogs.o
[andrew(at)alphonso apache_logging]$ make
gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
-Wmissing-declarations -I. -I../../src/include -D_GNU_SOURCE -c -o
rotatelogs.o rotatelogs.c -MMD
gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
-Wmissing-declarations rotatelogs.o -L../../src/port
-Wl,-rpath,/home/andrew/tpg/inst//lib -o rotatelogs
[andrew(at)alphonso apache_logging]$ ldd rotatelogs
libc.so.6 => /lib/tls/libc.so.6 (0x0065e000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x0056a000)
[andrew(at)alphonso apache_logging]$ diff -cw
~/apache_1.3.29/src/support/rotatelogs.c rotatelogs.c
*** /home/andrew/apache_1.3.29/src/support/rotatelogs.c Mon Jul 14
14:31:26 2003
--- rotatelogs.c Sat Mar 13 13:47:41 2004
***************
*** 7,16 ****
*/


! #include "ap_config.h"
#include <time.h>
#include <errno.h>
#include <fcntl.h>

#if defined(WIN32) || defined(OS2)
#include <io.h>
--- 7,18 ----
*/


! #include "postgres.h"
#include <time.h>
#include <errno.h>
#include <fcntl.h>
+ #include <unistd.h>
+ #include <sys/types.h>

#if defined(WIN32) || defined(OS2)
#include <io.h>
[andrew(at)alphonso apache_logging]$ cat Makefile
# $PostgreSQL$

subdir = contrib/apache_logging
top_builddir = ../..
include $(top_builddir)/src/Makefile.global

PROGRAM = rotatelogs
OBJS = rotatelogs.o

include $(top_srcdir)/contrib/contrib-global.mk

#LIBS:=$(filter-out -lpgport, $(LIBS))
LIBS:=
[andrew(at)alphonso apache_logging]$

Tom Lane wrote:
> I did take a look at the Apache rotator program, and found that it
> was probably more trouble to adopt than it's worth. It seemed to
> depend on a lot of configuration and library-routine infrastructure
> that we don't share.

Here's a log rotation program that does not share those problems, I
hope:

http://developer.postgresql.org/~petere/pg_logpipe/

I haven't compiled it recently, so beware.

Lamar Owen wrote:
> On Saturday 13 March 2004 10:36 am, Fernando Nasser wrote:
>
>>The problem is that sysloging has more overhead than a plain append to a
>>file. There are some very strict response time AppServer applications
>>where we want to keep this things out of the picture.
>
>
> Well, I have a couple of ideas on that. First, a better syslog. SDSC secure
> syslog has been available for some time and is BSD licensed, and is
> specifically designed for high-performance RFC3195 compliant secure syslog.
>

I hope someone in the OS group is watching it. I will ask.

> Second, if the syslog server is separate, then you might get better
> performance as the size of the logs grow, since appending a very large file
> is slower than generating an IP datagram.
>

It may be. It takes some effort to convince customers to change their
practices though. I will try to set up something like this next time we
run benchmarks and see if there is any noticeable change in the results.

> Third, it seems that you don't have enough profiling data to support a 'syslog
> is bad' position.

That is true. It is from hearsay, from people who run production
environments. It may be a belief based on old experiences though. If
someone had real up-to-date reliable numbers maybe we could use it as an
argument to the users.

> Java is bad too, from a performance standpoint (at this
> time, at least, you always get a performance hit of some kind using any
> portable code).

Actually, this is increasingly less noticeable. With just in time
compilation or pre-compiled code (like with GNU gcj) it is a one time
hit or not even that.

> But if this AppServer is based on the ArsDigita Java
> codebase, you have other performance issues already (the Tcl codebase, OTOH,
> is very fast, partly because AOLserver is a faster appserver than most Java
> appservers).).
>

No, it is based on ObjectWeb JOnAS (Java Open Source Application
Server). The Servelet Container is Tomcat and the database is
PostgreSQL (it also works with Oracle, DB2, MySQL, etc.).

>
>>The other problem is that we have some nice graphical tools for
>>configuring logging but the /etc/syslog.conf is a very hard one to
>>automate dur to the pattern matching. We can add some lines there, like
>>one to get local0 to a specific file, but it will probably be guesswork
>>and require human inspection anyway.
>
>
> Control Center looks promising. But I much prefer having a central syslog
> server than having each server in a cluster having separate logs.
>

There are certain advantages with doing it that way and I guess some
customers will indeed do it. I believe you can very well set up logging
to a centralized syslog with Control Center (if not please open a buf
report).

>
>>It may be desirable to logrotate them at different times as well, so
>>they would have to be in different files.
>
>
> Different facilities can then go to different files. The problem, of course,
> is syslog's limited number of facilities.
>

If not used for other things, maybe 8 would be a good enough number.

>
>>>Or you'll have to include some other log rotator.
>
>
>>That is what Tom is proposing.
>
>
> I am not opposed to including a small logrotator for stderr logging. I just
> think it is redundant when a good highly configurable logging facility
> already exists. But, if Red Hat wants to pay Tom to do it... :-)

Maybe it is a question of preference or what is more convenient depends
on the specific circunstances of the installation. I would prefer to
give the option to the users, if possible.

Cheers,
Fernando

Lamar Owen wrote:
> On Saturday 13 March 2004 01:00 pm, Fernando Nasser wrote:
>
>>There are some applicatons which run in servers with very strict
>>response times and any syscall, network packet that can be saved counts.
>
>
> Ok, what about pipe overhead? If we're gong to split hairs, let's split all
> of them. The design of the pipeline in this logrotator filter will have to
> be such that minimal overhead occurs, because, at the real-time level, every
> concurrent process also counts.
>

Splitting hairs was not my intention :-) But there is always something
to gain or learn from a good debate...

What you say is true. Nobody seems to consider that there si some
overhead in piping as well. Perhaps it is another (unproved?) belief
that it will not be as significant as using a central syslog service.

>
>>The number of generated messgaes.
>
>
>>Maybe that is an area that can be worked on, i.e. reducing log
>>verbosity. Is 7.4.x much better than 7.3.x in that respect?
>
>
> There are several levels documented in postgresql.conf. Try the terse level
> and see what happens.

It used to be that if you lowered it too much you would get fewer
messages but not enough information. There is a tendency that this
improves with time, I would guess.

There are cases where a problem is suspected and then users have to
raise the log level in the hopes that it gives them some clue when the
problem manifests itself (which sometimes, at least on a first instance,
only happens in the production environment as is triggered by some weird
usage pattern).

Best regards,
Fernando

Fernando Nasser <fnasser(at)redhat(dot)com> writes:
> Lamar Owen wrote:
>> Third, it seems that you don't have enough profiling data to support
>> a 'syslog is bad' position.

> That is true. It is from hearsay, from people who run production
> environments. It may be a belief based on old experiences though.

I think it's pretty well established that syslog sucks for high log
volume if you run it in the mode where it fsyncs its log after every
message. But I don't believe we have any data that says it's a problem
even if you avoid that pitfall.

regards, tom lane

On Sunday 14 March 2004 12:19, Tom Lane wrote:
> Fernando Nasser <fnasser(at)redhat(dot)com> writes:
> > Lamar Owen wrote:
> >> Third, it seems that you don't have enough profiling data to support
> >> a 'syslog is bad' position.
> >
> > That is true. It is from hearsay, from people who run production
> > environments. It may be a belief based on old experiences though.
>
> I think it's pretty well established that syslog sucks for high log
> volume if you run it in the mode where it fsyncs its log after every
> message. But I don't believe we have any data that says it's a problem
> even if you avoid that pitfall.
>

Yep. The other basic thing I recomend is putting syslog/log output on a
different disk than where your data goes. Maybe not proactical for everyone,
but not a big deal for "enterprise" level users (which was the op's target
audience)

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Tom Lane wrote:
> Fernando Nasser <fnasser(at)redhat(dot)com> writes:
> > Lamar Owen wrote:
> >> Third, it seems that you don't have enough profiling data to support
> >> a 'syslog is bad' position.
>
> > That is true. It is from hearsay, from people who run production
> > environments. It may be a belief based on old experiences though.
>
> I think it's pretty well established that syslog sucks for high log
> volume if you run it in the mode where it fsyncs its log after every
> message. But I don't believe we have any data that says it's a problem
> even if you avoid that pitfall.

What versions of syslog fsync, and where is the syslog.conf option. I
don't see it on FreeBSD or Linux.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> What versions of syslog fsync, and where is the syslog.conf option. I
> don't see it on FreeBSD or Linux.

It's a per-output-file option. My Linux manpage for syslogd quoth

The - may only be used to prefix a filename if you want to omit
sync'ing the file after every write to it.

I believe this notation is inherited from BSD. I don't see anything
about it in the HPUX man page for syslogd, though.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > What versions of syslog fsync, and where is the syslog.conf option. I
> > don't see it on FreeBSD or Linux.
>
> It's a per-output-file option. My Linux manpage for syslogd quoth
>
> The - may only be used to prefix a filename if you want to omit
> sync'ing the file after every write to it.
>
> I believe this notation is inherited from BSD. I don't see anything
> about it in the HPUX man page for syslogd, though.

I see no mention of that flag in Free/NetBSD, or bsdi. I do see a
mention in Red Hat 9.

Looking at my syslog source code, the only lines that get fsync'ed are
lines from /dev/klog (kernel log messages). I think non-kernel messages
use the /var/run/log socket.

syslogd manual page says:

Syslogd reads messages from the LOCAL domain socket /var/run/log, from an
Internet domain socket specified in /etc/services, and from the special
device /dev/klog (to read kernel messages). Messages received on the In-
ternet and LOCAL domain sockets may be NULL terminated and may include a
single trailing newline, any other non-printable characters are encoded
into a visible representation by strvisx(3).

I wonder if this fsync for PostgreSQL messages is some change made to
Linux syslog.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> I wonder if this fsync for PostgreSQL messages is some change made to
> Linux syslog.

You're missing the point: by default syslog fsyncs *all* messages.
You can turn this off on a per-output-file basis by putting "-" on the
desired lines of the syslog config file. It has nothing whatever to do
with Postgres.

The fact that you don't see any fsync calls doesn't prove anything. How
about O_SYNC or O_DSYNC flags when opening the output files? That would
be the natural way to implement it given the spec.

strace'ing syslogd on my HPUX box shows that it doesn't issue explicit
fsync calls either, but I don't know of any way to tell whether it's got
the log files opened O_SYNC.

BTW the Linux man page says specifically that their syslogd tries to be
compatible with BSD.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > I wonder if this fsync for PostgreSQL messages is some change made to
> > Linux syslog.
>
> You're missing the point: by default syslog fsyncs *all* messages.
> You can turn this off on a per-output-file basis by putting "-" on the
> desired lines of the syslog config file. It has nothing whatever to do
> with Postgres.
>
> The fact that you don't see any fsync calls doesn't prove anything. How
> about O_SYNC or O_DSYNC flags when opening the output files? That would
> be the natural way to implement it given the spec.

Well, doing a grep for sync, I see:

(2) grep -i -4 SYNC syslogd.c

/* Flags to logmsg(). */

#define IGN_CONS 0x001 /* don't print on console */
#define SYNC_FILE 0x002 /* do fsync on file after printing */
#define ADDDATE 0x004 /* add a date to the message */
#define MARK 0x008 /* this message is a mark */

/* Types of actions */
--
char *lp;

lp = msg + msglen;
for (p = msg; p < lp; p = q + 1) {
flags = SYNC_FILE | ADDDATE; /* fsync file after write */
if (parse_pri(&p, &pri)) {
if (pri & ~(LOG_FACMASK|LOG_PRIMASK))
pri = DEFSPRI;
} else {
--
f->f_name));
}
} else {
f->f_warning = 0;
if (flags & SYNC_FILE)
(void)fsync(f->f_file);
}
break;

case F_USER:
--
p += snprintf(p, sizeof(buf) + buf - p, ": %s",
strerror(errno));
dprintf(("%s\n", buf));
logmsg(LOG_SYSLOG|LOG_ERR, NULL, buf, p - buf, LocalHostName,
ADDDATE|SYNC_FILE);
}

Which basically shows one fsync, no O_SYNC's, and setting of the flag
only for klog reads.

> strace'ing syslogd on my HPUX box shows that it doesn't issue explicit
> fsync calls either, but I don't know of any way to tell whether it's got
> the log files opened O_SYNC.
>
> BTW the Linux man page says specifically that their syslogd tries to be
> compatible with BSD.

Yes, I saw that too, but maybe not in this area. It is all in one file,
so here is the FreeBSD version:

http://www.freebsd.org/cgi/cvsweb.cgi/src/usr.sbin/syslogd/syslogd.c?rev=1.59.2.28&content-type=text/x-cvsweb-markup

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

I wrote:
>> strace'ing syslogd on my HPUX box shows that it doesn't issue explicit
>> fsync calls either, but I don't know of any way to tell whether it's got
>> the log files opened O_SYNC.

It occurred to me to SIGHUP syslogd and watch it reopen its log files
with strace. No O_SYNC.

So it seems fairly likely that the fsync-by-default business is indeed
a Linux-ism not shared by other Unixen.

regards, tom lane

Fernando Nasser wrote:

> Lamar Owen wrote:
>
>>
>>
>> I am not opposed to including a small logrotator for stderr logging.
>> I just think it is redundant when a good highly configurable logging
>> facility already exists. But, if Red Hat wants to pay Tom to do
>> it... :-)
>
>
> Maybe it is a question of preference or what is more convenient
> depends on the specific circunstances of the installation. I would
> prefer to give the option to the users, if possible.
>

This strikes me as the crux of the matter. Using syslog in certain
configurations may well be the best solution, but *we* should not be
enforcing that choice, IMNSHO.

cheers

andrew

Bruce Momjian wrote:

>Which basically shows one fsync, no O_SYNC's, and setting of the flag
>only for klog reads.
>
>

Which sysklogd do you look at? The version from RedHat 9 contains this
block:

> /*
> * Crack a configuration file line
> */
>
> void cfline(line, f)
> char *line;
> register struct filed *f;
> {
> register char *p;
> [snip]
> if (*p == '-')
> {
> syncfile = 0;
> p++;
> } else
> syncfile = 1;
> [snip]
> if (syncfile)
> f->f_flags |= SYNC_FILE;

And the the fsync depends on SYNC_FILE. As documented in man syslog.conf:

> You may prefix each entry with the minus ``-'' sign to omit
> syncing the
> file after every logging. Note that you might lose
> information if the
> system crashes right behind a write attempt. Nevertheless
> this might
> give you back some performance, especially if you run programs
> that use
> logging in a very verbose manner.

It's sysklogd-1.4.1rh, I'm not sure what part of it are Redhat specific.

--
Manfred

Manfred Spraul wrote:
> Bruce Momjian wrote:
>
> >Which basically shows one fsync, no O_SYNC's, and setting of the flag
> >only for klog reads.
> >
> >
>
> Which sysklogd do you look at? The version from RedHat 9 contains this
> block:

I looked on NetBSD, FreeBSD, and BSD/OS.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> Manfred Spraul wrote:
>
>>Bruce Momjian wrote:
>>
>>
>>>Which basically shows one fsync, no O_SYNC's, and setting of the flag
>>>only for klog reads.
>>>
>>>
>>
>>Which sysklogd do you look at? The version from RedHat 9 contains this
>>block:
>
>
> I looked on NetBSD, FreeBSD, and BSD/OS.
>

Bruce, I've asked around in Red Hat and that was the answer. I confess
I did not understand if "upstream" means the Linux community or the
syslogd source...

Anyway, do you have recent FreeBSD or NetBSD sources?

Regards,
Fernando

---------------------------------------------------------------------------
Fernando Nasser (fnasser(at)redhat(dot)com) said:

>> It seems that our syslogd fsyncs on each write, while the NetBSD,
>> FreeBSD, and BSD/OS ones don't.

Standard upstream behavior, been that way pretty much forever.
As for any reason why, it's probably tradition at this point.

As mentioned, you can configure it away for any log that you
don't want it.

Fernando Nasser wrote:
> Bruce Momjian wrote:
> > Manfred Spraul wrote:
> >
> >>Bruce Momjian wrote:
> >>
> >>
> >>>Which basically shows one fsync, no O_SYNC's, and setting of the flag
> >>>only for klog reads.
> >>>
> >>>
> >>
> >>Which sysklogd do you look at? The version from RedHat 9 contains this
> >>block:
> >
> >
> > I looked on NetBSD, FreeBSD, and BSD/OS.
> >
>
> Bruce, I've asked around in Red Hat and that was the answer. I confess
> I did not understand if "upstream" means the Linux community or the
> syslogd source...
>
> Anyway, do you have recent FreeBSD or NetBSD sources?

I just jumped on the web sites of the projects and looked at the CVS
head for usr.sbin/syslogd/syslogd.c.

NetBSD is here:

http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.sbin/syslogd/

FreeBSD is here:

http://www.freebsd.org/cgi/cvsweb.cgi/src/usr.sbin/syslogd/

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> Fernando Nasser wrote:
>>Anyway, do you have recent FreeBSD or NetBSD sources?
>
>
> I just jumped on the web sites of the projects and looked at the CVS
> head for usr.sbin/syslogd/syslogd.c.
>
> NetBSD is here:
>
> http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.sbin/syslogd/
>
> FreeBSD is here:
>
> http://www.freebsd.org/cgi/cvsweb.cgi/src/usr.sbin/syslogd/
>

It seems these are different implementations of syslogd from the one that comes
with Red Hat Linux (for instance) that has the '-' parameter and fsync by
default since 2000 and it was not added by Red hat (it was in the original sources).

W.r.t. syslogd Alan Cox said:

"Its configurable in Linux. The advantage of fsync each time is that data
is not lost from the logs on poweroff, the tradeoff is speed of logging,
although since syslog isnt intended for mass data dumping it isnt normally
a problem

The use of "-" for non fsyncing is also recommended for laptops to improve
disk sleep patterns."

On Sat, Mar 13, 2004 at 10:45:35AM -0500, Rod Taylor wrote:
> Not that I'm volunteering, but I think the biggest issue is many users
> simply don't know how to approach the problem. Some docs on using
> syslog, cron, etc. with PostgreSQL to accomplish maintenace jobs would
> probably be enough.

There is the basic problem that if you don't have root on your host,
setting up syslog is one more thing you have to get the rootly ones
to do. Some of them are, uh, challenged in that area.

A

--
Andrew Sullivan | ajs(at)crankycanuck(dot)ca

Andrew Sullivan wrote:
> On Sat, Mar 13, 2004 at 10:45:35AM -0500, Rod Taylor wrote:
>
>>Not that I'm volunteering, but I think the biggest issue is many users
>>simply don't know how to approach the problem. Some docs on using
>>syslog, cron, etc. with PostgreSQL to accomplish maintenace jobs would
>>probably be enough.
>
>
> There is the basic problem that if you don't have root on your host,
> setting up syslog is one more thing you have to get the rootly ones
> to do. Some of them are, uh, challenged in that area.

What a polite way to say that they have zero business logging in as root ;)

>
> A
>

--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd(at)commandprompt(dot)com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL

On Sunday 14 March 2004 1:00 pm, Tom Lane wrote:
...
> So it seems fairly likely that the fsync-by-default business is
> indeed a Linux-ism not shared by other Unixen.

Excerpt from the Postfix 2.0.8 README_FILES/LINUX_README file in case
it proves interesting:
-----------------
LINUX syslogd uses synchronous writes by default. Because of this,
syslogd can actually use more system resources than Postfix. To
avoid such madness, disable synchronous mail logfile writes by
editing /etc/syslog.conf and by prepending a - to the logfile name:

mail.* -/var/log/mail.log

Send a "kill -HUP" to the syslogd to make the change effective.
-----------------

Cheers,
Steve

This thread seems to have died without a conclusion. AFAICS, we have 5
options:

. the apache program - see below
pro: robust, portable, extremely well tested, no effort to import
con: possible license issues, limited features
. Peter Eisentraut's program
pro: portable, better featured, no license issues
con: code state uncertain, less well tested
. the Afilias script
pro: well featured, no license issues
con: not portable (relies on perl), testing status uncertain
. something new
pro: no license issues, can make as featured as desired, portable
con: lots of effort, untested
. nothing
pro: no effort, no license issues :-)
con: feature is desired

The issues seem to have been thrashed out ad nauseam. Surely we can put
*something* in contrib for this? After all, nobody has to use it if they
don't want to.

cheers

andrew

I wrote:

>
>
> Tom Lane wrote:
>
>> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>>
>>
>>> Did anything ever come from this thread?
>>> http://archives.postgresql.org/pgsql-hackers/2003-05/msg00603.php
>>> (Heading: "Plan B for log rotation support: borrow Apache code")
>>>
>>
>>
>> Only an entry on my depressingly long personal to-do list :-(
>>
>> I did take a look at the Apache rotator program, and found that it was
>> probably more trouble to adopt than it's worth. It seemed to depend on
>> a lot of configuration and library-routine infrastructure that we don't
>> share. (No big surprise; I suppose someone trying to pull out a random
>> bit of our backend code would be at least as unhappy.) I suspect it
>> would be less trouble, as well as legalistically cleaner, to write our
>> own from scratch.
>>
>> Andrew Sullivan offered Afilias' rotator script awhile back also.
>> I think that works fine if you like a Perl script.
>>
>>
>>
>
> FWIW, in less than 30 minutes I took the log rotator from apache
> 1.3.29 (i.e. the latest non-APR version) and imported it into a fresh
> postgreql tree. With very little massaging it built happily (see below).
>
> If it will advance matters, I can submit this as a patch filling the
> currently empty contrib/apache_logging directory. You could be right
> about the legal stuff - worth talking to the apache folks?
>
>
>
>

On Wed, Mar 24, 2004 at 09:55:34AM -0500, Andrew Dunstan wrote:
> This thread seems to have died without a conclusion. AFAICS, we have 5
> options:
>
> . the apache program - see below
> pro: robust, portable, extremely well tested, no effort to import
> con: possible license issues, limited features

One of the folks who works with me at Afilias, Andrew Hammond, is
trying to clean up a patch he made to include the apache log rotator
"natively". We've been looking at it because I'm worried about the
I/O our current approach imposes, and we need to have something to
compare it with. I think he told me that he'll have a patch ready to
offer in a day or two. Of course, it would depend on actually having
the rotator program sitting around somewhere.

A

--
Andrew Sullivan | ajs(at)crankycanuck(dot)ca
The fact that technology doesn't work is no bar to success in the marketplace.
--Philip Greenspun

On Wed, 24 Mar 2004, Andrew Dunstan wrote:

> . Peter Eisentraut's program
> pro: portable, better featured, no license issues
> con: code state uncertain, less well tested

Where is Peter's code available, I'd like to try it out.

scott.marlowe wrote:

>On Wed, 24 Mar 2004, Andrew Dunstan wrote:
>
>
>
>>. Peter Eisentraut's program
>> pro: portable, better featured, no license issues
>> con: code state uncertain, less well tested
>>
>>
>
>Where is Peter's code available, I'd like to try it out.
>
>

http://developer.postgresql.org/~petere/pg_logpipe/

cheers

andrew

Andrew Dunstan wrote:
>
>
> This thread seems to have died without a conclusion. AFAICS, we have 5
> options:
>
> . the apache program - see below
> pro: robust, portable, extremely well tested, no effort to import
> con: possible license issues, limited features
> . Peter Eisentraut's program
> pro: portable, better featured, no license issues
> con: code state uncertain, less well tested
> . the Afilias script
> pro: well featured, no license issues
> con: not portable (relies on perl), testing status uncertain
> . something new
> pro: no license issues, can make as featured as desired, portable
> con: lots of effort, untested
> . nothing
> pro: no effort, no license issues :-)
> con: feature is desired
>
> The issues seem to have been thrashed out ad nauseam. Surely we can put
> *something* in contrib for this? After all, nobody has to use it if they
> don't want to.

Shouldn't all of these just be on gborg?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Wednesday 24 March 2004 12:31, Bruce Momjian wrote:
> Andrew Dunstan wrote:
> > This thread seems to have died without a conclusion. AFAICS, we have 5
> > options:
> >
> > . the apache program - see below
> > pro: robust, portable, extremely well tested, no effort to import
> > con: possible license issues, limited features
> > . Peter Eisentraut's program
> > pro: portable, better featured, no license issues
> > con: code state uncertain, less well tested
> > . the Afilias script
> > pro: well featured, no license issues
> > con: not portable (relies on perl), testing status uncertain
> > . something new
> > pro: no license issues, can make as featured as desired, portable
> > con: lots of effort, untested
> > . nothing
> > pro: no effort, no license issues :-)
> > con: feature is desired
> >
> > The issues seem to have been thrashed out ad nauseam. Surely we can put
> > *something* in contrib for this? After all, nobody has to use it if they
> > don't want to.
>
> Shouldn't all of these just be on gborg?

Some could simply be an article/guide on techdocs and/or a chapter in the
docs... "how to use postgresql with apache logrotate" for example

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert Treat wrote:
> On Wednesday 24 March 2004 12:31, Bruce Momjian wrote:
> > Andrew Dunstan wrote:
> > > This thread seems to have died without a conclusion. AFAICS, we have 5
> > > options:
> > >
> > > . the apache program - see below
> > > pro: robust, portable, extremely well tested, no effort to import
> > > con: possible license issues, limited features
> > > . Peter Eisentraut's program
> > > pro: portable, better featured, no license issues
> > > con: code state uncertain, less well tested
> > > . the Afilias script
> > > pro: well featured, no license issues
> > > con: not portable (relies on perl), testing status uncertain
> > > . something new
> > > pro: no license issues, can make as featured as desired, portable
> > > con: lots of effort, untested
> > > . nothing
> > > pro: no effort, no license issues :-)
> > > con: feature is desired
> > >
> > > The issues seem to have been thrashed out ad nauseam. Surely we can put
> > > *something* in contrib for this? After all, nobody has to use it if they
> > > don't want to.
> >
> > Shouldn't all of these just be on gborg?
>
> Some could simply be an article/guide on techdocs and/or a chapter in the
> docs... "how to use postgresql with apache logrotate" for example

We already have an example of using logroate in the docs, right?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073