Re: Simulating Clog Contention

On Mon, Jan 30, 2012 at 12:26 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> I was thinking the opposite.  That -i should only print progress
>> indication when -d is given.  Or at least knock an order of magnitude
>> or two off of how often it does so.
>
> I'd be in all in favor of having -i emit progress reports 10x less
> often; even on a laptop, the current reports are very chatty.  But I
> think 100x less often might be taking it too far.

Trivial patch for that attached.

> Either way, if we're going to have an option for inserts, they should
> produce the same progress reports that COPY does - though possibly
> more often, since I'm guessing it's likely to be way slower.

I looked at this a little more and I'm coming around to the view that
Heikki expressed originally: I think this too much of a one-tricky
pony to justify including it in pgbench. It's an interesting hack for
testing, but the thing is that, to really make it do anything
interesting, you've got to not only use INSERT instead of COPY and
create the indexes before loading the table, BUT ALSO frob the
existing code to prevent the WAL bypass from doing its thing. I
suppose we could have a separate option for all THREE of those
behaviors, rather than just the two I mentioned in my previous email,
but that seems over the top. So if we're going to have this at all,
we might as well just call it --artificially-inflate-clog-pain and be
done with it.

But I think that's kind of a narrow special case that isn't really
worth catering for. Our CLOG contention right now is not so bad that
we need to focus a major development effort on making it less, and
even if we do want to do that that there's no real evidence that a
half-hour pgbench run isn't sufficient to demonstrate the problem
perfectly adequately with the code we have right now. After a few
minutes the frequency of hitting previously-updated rows is high
enough to measure the problem anyway. In the process of developing
the various performance improvements we've committed for 9.2, I and
others have developed various test cases - Heikki has one he uses for
his xlog scaling patch, for example. We can't commit all of those as
pgbench options, or we'll go nuts. Ideally it would be nice if
pgbench was flexible enough to handle these kinds of uses cases via
configuration rather than by hard-coding them.

Given all the above, I'm inclined to conclude that this is just
another special-purpose test case which we should use for testing and
that's it, as Simon already proposed upthread. However, I do think
that we should go ahead and make -n work in initialization mode,
because I've wanted that a few times. So, patch for that attached,
too.

(Note that no actual benchmarking is happening right now with regard
to the CLOG history patch because, as previously noted, the most
recent version does not compile. This probably doesn't matter for
that patch hugely anyway, since that mechanism as currently designed
does not kick in until a million transactions have been processed, and
by that time you'll have quite a spread of XIDs in pgbench_acounts
anyway. I don't believe there are any other remaining patches in this
CommitFest to which the test case would be applicable; please let me
know if I am wrong.)

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company