Re: Hot Standby tuning for btree_xlog_vacuum()

Simple tuning of btree_xlog_vacuum() using an idea I had a while back,
just never implemented. XXX comments removed.

Allows us to avoid reading in blocks during VACUUM replay that are only
required for correctness of index scans.

Objections to commit?

--
Simon Riggs www.2ndQuadrant.com

Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
> Objections to commit?

This is not the time to be hacking stuff like this. You haven't even
demonstrated that there's a significant performance issue here.

regards, tom lane

On Apr 29, 2010, at 3:20 PM, Tom Lane wrote:
> Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
>> Objections to commit?
>
> This is not the time to be hacking stuff like this. You haven't even
> demonstrated that there's a significant performance issue here.

I tend to agree that this point of the cycle isn't a good one to be making changes, but your performance statement confuses me. If a fairly small patch means we can avoid un-necessary reads why shouldn't we avoid them?
--
Jim C. Nasby, Database Architect jim(at)nasby(dot)net
512.569.9461 (cell) http://jim.nasby.net

Jim Nasby <decibel(at)decibel(dot)org> writes:
> On Apr 29, 2010, at 3:20 PM, Tom Lane wrote:
>> This is not the time to be hacking stuff like this. You haven't even
>> demonstrated that there's a significant performance issue here.

> I tend to agree that this point of the cycle isn't a good one to be making changes, but your performance statement confuses me. If a fairly small patch means we can avoid un-necessary reads why shouldn't we avoid them?

Well, by "time of the cycle" I meant "the day before beta1". I'm not
necessarily averse to making such a change at some point when it would
get more than no testing before hitting our long-suffering beta testers.
But I'd still want to see some evidence that there's a significant
performance improvement to be had.

regards, tom lane

On Mon, 2010-05-17 at 16:10 -0400, Tom Lane wrote:
> Jim Nasby <decibel(at)decibel(dot)org> writes:
> > On Apr 29, 2010, at 3:20 PM, Tom Lane wrote:
> >> This is not the time to be hacking stuff like this. You haven't even
> >> demonstrated that there's a significant performance issue here.
>
> > I tend to agree that this point of the cycle isn't a good one to be making changes, but your performance statement confuses me. If a fairly small patch means we can avoid un-necessary reads why shouldn't we avoid them?
>
> Well, by "time of the cycle" I meant "the day before beta1". I'm not
> necessarily averse to making such a change at some point when it would
> get more than no testing before hitting our long-suffering beta testers.
> But I'd still want to see some evidence that there's a significant
> performance improvement to be had.

That patch only applies to one record type. However, since we've used
Greg's design of spidering out to each heap record that can usually mean
150-200 random I/Os per btree delete. That will take some time, perhaps
1s per WAL record of this type on a very large I/O bound table. That's
enough to give me cause for concern without performance measurements.

To derive such measurements we'd need to instrument each record type,
which we don't do right now either.

It might be easier to have a look at the patch and see if you think its
worth the fuss of measuring it.

I don't think this is the patch that will correct the potential/
partially observed context switching issue, but we have yet to recreate
that in lab conditions.

--
Simon Riggs www.2ndQuadrant.com

On Thu, Apr 29, 2010 at 4:12 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> Simple tuning of btree_xlog_vacuum() using an idea I had a while back,
> just never implemented. XXX comments removed.
>
> Allows us to avoid reading in blocks during VACUUM replay that are only
> required for correctness of index scans.

Review:

1. The block comment in XLogConfirmBufferIsUnpinned appears to be
copied from somewhere else, and doesn't really seem appropriate for a
new function since it refers to "the original coding of this routine".
I think you could just delete the parenthesized portion of the
comment.

2. This bit from ConfirmBufferIsUnpinned looks odd to me.

+ /*
+ * Found it. Now, pin/unpin the buffer to prove it's unpinned.
+ */
+ if (PinBuffer(buf, NULL))
+ UnpinBuffer(buf, false);

I don't think pinning and unpinning the buffer is sufficient to
provide that it isn't otherwise pinned. If the buffer isn't in shared
buffers at all, it seems clear that no one can have it pinned. But if
it's present in shared buffers, it seems like you still need
LockBufferForCleanup().

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

Just wanted to say thanks for the review, since I haven't yet managed to
fix and commit this. I expect to later this month.

On Mon, 2010-09-27 at 23:06 -0400, Robert Haas wrote:
> On Thu, Apr 29, 2010 at 4:12 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> > Simple tuning of btree_xlog_vacuum() using an idea I had a while back,
> > just never implemented. XXX comments removed.
> >
> > Allows us to avoid reading in blocks during VACUUM replay that are only
> > required for correctness of index scans.
>
> Review:
>
> 1. The block comment in XLogConfirmBufferIsUnpinned appears to be
> copied from somewhere else, and doesn't really seem appropriate for a
> new function since it refers to "the original coding of this routine".
> I think you could just delete the parenthesized portion of the
> comment.
>
> 2. This bit from ConfirmBufferIsUnpinned looks odd to me.
>
> + /*
> + * Found it. Now, pin/unpin the buffer to prove it's unpinned.
> + */
> + if (PinBuffer(buf, NULL))
> + UnpinBuffer(buf, false);
>
> I don't think pinning and unpinning the buffer is sufficient to
> provide that it isn't otherwise pinned. If the buffer isn't in shared
> buffers at all, it seems clear that no one can have it pinned. But if
> it's present in shared buffers, it seems like you still need
> LockBufferForCleanup().

--
Simon Riggs http://www.2ndQuadrant.com/books/
PostgreSQL Development, 24x7 Support, Training and Services