Re: [TODO] Track number of files ready to be archived in pg_stat_archiver

From: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
To: "Brightwell, Adam" <adam(dot)brightwell(at)crunchydatasolutions(dot)com>
Cc: Julien Rouhaud <rjuju123(at)gmail(dot)com>, Julien Rouhaud <julien(dot)rouhaud(at)dalibo(dot)com>, Gilles Darold <gilles(dot)darold(at)dalibo(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [TODO] Track number of files ready to be archived in pg_stat_archiver
Date: 2014-11-18 07:36:37
Message-ID: CAB7nPqTYK+Dj7oTTsx32tQNYHmGEFbEP4CKd9ZFOL3+AzEQsow@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Oct 22, 2014 at 12:50 AM, Brightwell, Adam
<adam(dot)brightwell(at)crunchydatasolutions(dot)com> wrote:
> Though, I would think that the general desire would be to keep the patch
> relevant ONLY to the necessary changes. I would not qualify making those
> types of changes as relevant, IMHO. I do think this is potential for
> cleanup, however, I would suspect that would be best done in a separate
> patch. But again, I'd defer to a committer whether such changes are even
> necessary/acceptable.

I have been looking at this patch, and I think that it is a mistake to
count the .ready files present in archive_status when calling
pg_stat_get_archiver(). If there are many files waiting to be
archived, this penalizes the run time of this function, and the
application behind relying on those results, not to mention that
actually the loop used to count the .ready files is a copy of what is
in pgarch.c. Hence I think that we should simply count them in
pgarch_readyXlog, and then return a value back to
pgarch_ArchiverCopyLoop, value that could be decremented by 1 each
time a file is successfully archived to keep the stats as precise as
possible, and let the information know useful information when
archiver process is within a single loop process of
pgarch_ArchiverCopyLoop. This way, we just need to extend
PgStat_MsgArchiver with a new counter to track this number.

The attached patch, based on v2 sent previously, does so. Thoughts?
--
Michael

Attachment Content-Type Size
pg_stat_archiver_ready_count-v3.patch text/x-patch 12.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Davis 2014-11-18 07:39:14 Re: 9.5: Better memory accounting, towards memory-bounded HashAgg
Previous Message Michael Paquier 2014-11-18 06:20:50 Re: [TODO] Track number of files ready to be archived in pg_stat_archiver