Re: Logical decoding & exported base snapshot

On 2012-12-11 22:20:18 -0500, Tom Lane wrote:
> Joachim Wieland <joe(at)mcknight(dot)de> writes:
> > On Tue, Dec 11, 2012 at 6:52 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> >> One problem I see is that while exporting a snapshot solves the
> >> visibility issues of the table's contents it does not protect against
> >> schema changes. I am not sure whether thats a problem.
> >>
> >> If somebody runs a CLUSTER or something like that, the table's contents
> >> will be preserved including MVCC semantics. That's fine.
> >> The more problematic cases I see are TRUNCATE, DROP and ALTER
> >> TABLE.
>
> > This is why the pg_dump master process executes a
> > lock table <table> in access share mode
> > for every table, so your commands would all block.
>
> A lock doesn't protect against schema changes made before the lock was
> taken. The reason that the described scenario is problematic is that
> pg_dump is going to be expected to work against a snapshot made before
> it gets a chance to take those table locks. Thus, there's a window
> where DDL is dangerous, and will invalidate the dump --- perhaps without
> any warning.

Exactly.

> Now, we have this problem today, in that pg_dump has to read pg_class
> before it can take table locks so some window exists already.

Yea. And if somebody else already has conflicting locks its pretty damn
easy to hit as show in my answer to Joachim... I am pretty sure there
are lots of nasty silent variants.

> What's
> bothering me about what Andres describes is that the window for trouble
> seems to be getting much bigger.

Me too. If it only were clearly visible errors I wouldn't be bothered
too much, but ...

This morning I wondered whether we couldn't protect against that by
acquiring share locks on the catalog rows pg_dump reads, that would
result in "could not serialize access due to concurrent update" type of
errors which would be easy enough discernible/translateable.
While pretty damn ugly that should take care of most of those issues,
shouldn't it?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services