Decoding of (nearly) empty transactions and regression tests

From: Andres Freund <andres(at)2ndquadrant(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Decoding of (nearly) empty transactions and regression tests
Date: 2014-06-29 14:25:11
Message-ID: 20140629142511.GA26930@awork2.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

As shown by the CLOBBER_CACHE_ALWAYS animal test_decoding's tests fail
if they take very long. The failures aren't bugs, but diffs like:
BEGIN
+ COMMIT
+ BEGIN
table public.tr_sub: INSERT: id[integer]:1 path[text]:'1-top-#1'
The added transaction is an analyze started by autovacuum.

So, I've been wondering for a while to get rid of this, but I haven't
come up with something I like.

To recap, currently the rules for visibly decoding a transaction
(i.e. calling the output plugin callbacks) are:
1) it has been WAL logged (duh)
2) it happened in the database we're decoding
3) it contains something. 'Something' currently means that a logged
table has changed, or the transaction contains invalidations.

Note that just because a transaction contains 'something', these changes
aren't necessarily shown as we don't decode system table changes. I
think that behaviour makes sense because otherwise something like CREATE
TABLE without further changes wouldn't show up in the change stream.

Solution I)
Change ReorderBufferCommit() to not call the begin()/commit() callbacks
if no change() callback has been called. Technically that's trivial, but
I don't think that'd end up being a better behaviour.

Solution II)
Disable autovacuum/analyze for the tables in the regression tests. We
test vacuum explicitly, so we wouldn't loose too much test coverage.

Solution III)
Mark transactions that happen purely internally as such, using a
xl_xact_commit->xinfo flag. Not particularly pretty and the most
invasive of the solutions.

I'm favoring II) so far... Other opinions?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2014-06-29 14:30:14 Re: Doing better at HINTing an appropriate column within errorMissingColumn()
Previous Message Tom Lane 2014-06-29 14:24:22 Re: PostgreSQL for VAX on NetBSD/OpenBSD