Re: Cost of AtEOXact_Buffers in --enable-cassert

Hi,

I do test (and even run) some stuff running with cassert enabled. For many
things the reliability and performance is ok enough and its useful, especially
if you have your own c functions and such.
Imho thats useful as it makes catching some bugs more likely...

The most prohibitively expensive part is the AtEOXact_Buffers check of running
through all buffers and checking their pin count. And it makes $app's
regression tests take thrice their time...

Would somebody object agains putting those in an extra define so that those can
be disabled in pg_config_manual? Or even disable it by default entirely...

Andres

Andres Freund <andres(at)anarazel(dot)de> writes:
> The most prohibitively expensive part is the AtEOXact_Buffers check of running
> through all buffers and checking their pin count. And it makes $app's
> regression tests take thrice their time...

> Would somebody object agains putting those in an extra define so that those can
> be disabled in pg_config_manual? Or even disable it by default entirely...

Not a chance for the latter; this is an important sanity check that
catches real coding mistakes with some frequency.

I'd be willing to consider a "half assert" mode that turns off some of
the most expensive checks, but AtEOXact_Buffers is hardly the only thing
that ought to be in that list. The CLOBBER_FREED_MEMORY and memory
context checking stuff is pretty durn expensive too.

regards, tom lane

On Friday 06 August 2010 20:23:15 Tom Lane wrote:
> Andres Freund <andres(at)anarazel(dot)de> writes:
> > The most prohibitively expensive part is the AtEOXact_Buffers check of
> > running through all buffers and checking their pin count. And it makes
> > $app's regression tests take thrice their time...
> >
> > Would somebody object agains putting those in an extra define so that
> > those can be disabled in pg_config_manual? Or even disable it by default
> > entirely...
> Not a chance for the latter; this is an important sanity check that
> catches real coding mistakes with some frequency.
Ok.

> I'd be willing to consider a "half assert" mode that turns off some of
> the most expensive checks, but AtEOXact_Buffers is hardly the only thing
> that ought to be in that list. The CLOBBER_FREED_MEMORY and memory
> context checking stuff is pretty durn expensive too.
I personally have seen that catching way more bugs than the AtEOXact_Buffers
check, but that might be because I have found mostly bugs in random c
functions, not in pg stuff ;-)

I will wait a bit and wait for more suggestions about expensive checks and/or
other comments and will provide a patch for such a mode.

Andres

Andres Freund wrote:
> The most prohibitively expensive part is the AtEOXact_Buffers check of running
> through all buffers and checking their pin count. And it makes $app's
> regression tests take thrice their time...
>

Have you tried reducing shared_buffers from the default the system found
by probing to make this overhead smaller?

--
Greg Smith 2ndQuadrant US Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com www.2ndQuadrant.us

On Wed, Aug 11, 2010 at 12:51:36AM -0400, Greg Smith wrote:
> Andres Freund wrote:
> >The most prohibitively expensive part is the AtEOXact_Buffers
> >check of running through all buffers and checking their pin count.
> >And it makes $app's regression tests take thrice their time...
> Have you tried reducing shared_buffers from the default the system
> found by probing to make this overhead smaller?
Yes. Its getting slower just as you make them bigger. Which is not
surprising...
Using a smaller value than the default is painfull again as well
though...

Andres

Andres Freund <andres(at)anarazel(dot)de> writes:
> On Friday 06 August 2010 20:23:15 Tom Lane wrote:
>> I'd be willing to consider a "half assert" mode that turns off some of
>> the most expensive checks, but AtEOXact_Buffers is hardly the only thing
>> that ought to be in that list. The CLOBBER_FREED_MEMORY and memory
>> context checking stuff is pretty durn expensive too.

> I personally have seen that catching way more bugs than the AtEOXact_Buffers
> check, but that might be because I have found mostly bugs in random c
> functions, not in pg stuff ;-)

Nobody else seems to have commented, but maybe what this suggests is
we need to be able to individually disable a few of the most expensive
checks. I'm not sure what a reasonable API is for that ... not sure
that I like the thought of a GUC for each one.

regards, tom lane

On Sun, Aug 15, 2010 at 03:41:09PM -0400, Tom Lane wrote:
> Andres Freund <andres(at)anarazel(dot)de> writes:
> > On Friday 06 August 2010 20:23:15 Tom Lane wrote:
> >> I'd be willing to consider a "half assert" mode that turns off some of
> >> the most expensive checks, but AtEOXact_Buffers is hardly the only thing
> >> that ought to be in that list. The CLOBBER_FREED_MEMORY and memory
> >> context checking stuff is pretty durn expensive too.
>
> > I personally have seen that catching way more bugs than the AtEOXact_Buffers
> > check, but that might be because I have found mostly bugs in random c
> > functions, not in pg stuff ;-)
>
> Nobody else seems to have commented, but maybe what this suggests is
> we need to be able to individually disable a few of the most expensive
> checks. I'm not sure what a reasonable API is for that ... not sure
> that I like the thought of a GUC for each one.
I personally would be happy enough with a
CFLAGS="-DDISABLE_MEMCTX_ASSERT" alike solution (as long as I dont
have to edit source files for that, thats fine with me).
Changing it at server startup would be more convenient, sure.

Andres

Tom Lane wrote:
> Nobody else seems to have commented, but maybe what this suggests is
> we need to be able to individually disable a few of the most expensive
> checks. I'm not sure what a reasonable API is for that ... not sure
> that I like the thought of a GUC for each one.
>

I'd really like to be able to do more long-running performance tests
with the rest of the assertions on, to help catch bugs in higher level
code. There's maybe three major categories of these people might want
to turn off, right? If you consider wal_debug as an example of
something similar that's already there, the footprint of supporting that is:

-15 lines in config.sgml
-12 lines in guc.c
-4 lines in xlog.c

Plus all the places that have the #IFDEF around them to only enable if
this is on, which is the same regardless of the UI to toggle it. So
there'd be ~35 lines of new code per option to add them in the same way,
as GUCs you can view but not set, and that aren't included in the
postgresql.conf.sample and such.

Right now --enable-cassert => debug_assertions makes it easy on the user
side to figure out whether they have the expensive debugging stuff
turned on from a UI everybody knows--type a psql command. I'm biased
toward just doing the minor cut/paste bloat to do something similar for
the most expensive performance bits too. Then, as this escapes into the
wild, we can continue to sort out performance reports that easily. As
I've ran into a few times now, not everybody even has pg_config
installed, because it's usually sitting in a devel package instead of
the main server one.

If it's compiler option only, no GUC, and you have to use pg_config to
figure out if you did it, that's completely acceptable too. I don't
have a strong opinion here, just a preference. No arguments from me if
the decision is "that's too much code to add for something so marginal".

--
Greg Smith 2ndQuadrant US Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com www.2ndQuadrant.us

On Mon, Aug 16, 2010 at 4:00 AM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> Tom Lane wrote:
>>
>> Nobody else seems to have commented, but maybe what this suggests is
>> we need to be able to individually disable a few of the most expensive
>> checks.  I'm not sure what a reasonable API is for that ... not sure
>> that I like the thought of a GUC for each one.
>>
>
> I'd really like to be able to do more long-running performance tests with
> the rest of the assertions on, to help catch bugs in higher level code.
>  There's maybe three major categories of these people might want to turn
> off, right?  If you consider wal_debug as an example of something similar
> that's already there, the footprint of supporting that is:
>
> -15 lines in config.sgml
> -12 lines in guc.c
> -4 lines in xlog.c
>
> Plus all the places that have the #IFDEF around them to only enable if this
> is on, which is the same regardless of the UI to toggle it.  So there'd be
> ~35 lines of new code per option to add them in the same way, as GUCs you
> can view but not set, and that aren't included in the postgresql.conf.sample
> and such.
>
> Right now --enable-cassert => debug_assertions makes it easy on the user
> side to figure out whether they have the expensive debugging stuff turned on
> from a UI everybody knows--type a psql command.  I'm biased toward just
> doing the minor cut/paste bloat to do something similar for the most
> expensive performance bits too.  Then, as this escapes into the wild, we can
> continue to sort out performance reports that easily.  As I've ran into a
> few times now, not everybody even has pg_config installed, because it's
> usually sitting in a devel package instead of the main server one.
>
> If it's compiler option only, no GUC, and you have to use pg_config to
> figure out if you did it, that's completely acceptable too.  I don't have a
> strong opinion here, just a preference.  No arguments from me if the
> decision is "that's too much code to add for something so marginal".

What if we just added one GUC whose remit was to disable some of the
things that enable-cassert does, with a comma-separated list of values
specifying which ones? We could turn it into a bit-field under the
covers.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company