Lists: | pgsql-hackers |
---|
From: | Euler Taveira de Oliveira <euler(at)timbira(dot)com> |
---|---|
To: | Pgsql Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | improving log management |
Date: | 2010-01-14 03:00:53 |
Message-ID: | 4B4E88E5.6030603@timbira.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers |
Hi,
We support several methods for logging server messages. The "native" methods
(stderr, csvlogs) has poor management. We can't compress logs, send them to
another location/server, or just remove old ones. Another problem is that we
can't remove (automatically) old logs based on the number of existing log
files (of course we can do it using log_truncate_on_rotation and a
log_filename that matches the number of files -- but it *isn't* flexible). The
other log management softwares have a way to do that so why our logger doesn't
have such capability?
I want to propose a new command to be execute after the log file is rotated. A
GUC parameter log_after_rotation_command that takes a (set of) command(s) that
will be executed after a log file is rotated. It should be set only by
superuser. For example:
# compress and store the log file in another location
log_after_rotation_command = 'gzip %f && mv %p.gz /a/mylogs'
Depending on the comments, I will clean up the patch and send it later today.
--
Euler Taveira de Oliveira
http://www.timbira.com/
From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | Euler Taveira de Oliveira <euler(at)timbira(dot)com> |
Cc: | Pgsql Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: improving log management |
Date: | 2010-01-14 03:37:58 |
Message-ID: | 13621.1263440278@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers |
Euler Taveira de Oliveira <euler(at)timbira(dot)com> writes:
> I want to propose a new command to be execute after the log file is
> rotated.
(1) Windows compatibility?
(2) What happens if the command takes a significant amount of time to
execute? We can't afford to have the log collector blocked.
(3) What do you intend those %p and %f symbols to signify?
regards, tom lane
From: | Takahiro Itagaki <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp> |
---|---|
To: | Euler Taveira de Oliveira <euler(at)timbira(dot)com> |
Cc: | Pgsql Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: improving log management |
Date: | 2010-01-14 03:57:39 |
Message-ID: | 20100114125739.9A1E.52131E4D@oss.ntt.co.jp |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers |
Euler Taveira de Oliveira <euler(at)timbira(dot)com> wrote:
> other log management softwares have a way to do that so why our logger doesn't
> have such capability?
>
> I want to propose a new command to be execute after the log file is rotated. A
> GUC parameter log_after_rotation_command that takes a (set of) command(s) that
> will be executed after a log file is rotated.
If you have better loggers already, why don't you use them? In another word,
should we cooperate with them instead of re-inventing alternative loggers?
We have "Logging Brainstorm" topic in out wiki. It might help you.
http://wiki.postgresql.org/wiki/Logging_Brainstorm
Regards,
---
Takahiro Itagaki
NTT Open Source Software Center
From: | Euler Taveira de Oliveira <euler(at)timbira(dot)com> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | Pgsql Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: improving log management |
Date: | 2010-01-14 12:29:19 |
Message-ID: | 4B4F0E1F.6030905@timbira.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers |
Tom Lane escreveu:
> (1) Windows compatibility?
>
Yes.
> (2) What happens if the command takes a significant amount of time to
> execute? We can't afford to have the log collector blocked.
>
You're right. We can't have that command in the same process because DBAs
could have high values for log_rotation_* parameters or resources could become
unavailable. :( One idea is to have another process (logger worker) to execute
it. Also we need to store the logfiles that were already rotated.
> (3) What do you intend those %p and %f symbols to signify?
>
%p and %f are relative log path and log filename, respectively.
--
Euler Taveira de Oliveira
http://www.timbira.com/
From: | Euler Taveira de Oliveira <euler(at)timbira(dot)com> |
---|---|
To: | Takahiro Itagaki <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp> |
Cc: | Pgsql Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: improving log management |
Date: | 2010-01-14 12:42:53 |
Message-ID: | 4B4F114D.5090801@timbira.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers |
Takahiro Itagaki escreveu:
> If you have better loggers already, why don't you use them? In another word,
> should we cooperate with them instead of re-inventing alternative loggers?
>
We can use but they don't features that our logger has. If we have a logger,
let's improve it.
> We have "Logging Brainstorm" topic in out wiki. It might help you.
> http://wiki.postgresql.org/wiki/Logging_Brainstorm
>
Interesting. Don't know about that. Why some of those ideas aren't in our TODO?
--
Euler Taveira de Oliveira
http://www.timbira.com/