Re: [PATCH 10/16] Introduce the concept that wal has a 'origin' node

On Wednesday, June 20, 2012 05:34:42 PM Kevin Grittner wrote:
> Simon Riggs <simon(at)2ndQuadrant(dot)com> wrote:
> > This is not transaction metadata, it is WAL record metadata
> > required for multi-master replication, see later point.

> > We need to add information to every WAL record that is used as the
> > source for generating LCRs.
> If the origin ID of a transaction doesn't count as transaction
> metadata (i.e., data about the transaction), what does? It may be a
> metadata element about which you have special concerns, but it is
> transaction metadata. You don't plan on supporting individual WAL
> records within a transaction containing different values for origin
> ID, do you? If not, why is it something to store in every WAL
> record rather than once per transaction? That's not intended to be
> a rhetorical question.
Its definitely possible to store it per transaction (see the discussion around
http://archives.postgresql.org/message-
id/201206201605(dot)43634(dot)andres(at)2ndquadrant(dot)com) it just makes the filtering via
the originating node a considerably more complex thing. With our proposal you
can do it without any complexity involved, on a low level. Storing it per
transaction means you can only stream out the data to other nodes *after*
fully reassembling the transaction. Thats a pitty, especially if we go for a
design where the decoding happens in a proxy instance.

Other metadata will not be needed on such a low level.

I also have to admit that I am very hesitant to start developing some generic
"transaction metadata" framework atm. That seems to be a good way to spend a
good part of time in discussion and disagreeing. Imo thats something for
later.

> I think it's because you're still thinking
> of the WAL stream as *the medium* for logical replication data
> rather than *the source* of logical replication data.
I don't think thats true. See the above referenced subthread for reasons why I
think the origin id is important.

Andres
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services