Re: Logical decoding & exported base snapshot

On 2012-12-13 11:02:06 -0500, Steve Singer wrote:
> On 12-12-12 06:20 AM, Andres Freund wrote:
> >>Possible solutions:
> >>1) INIT_LOGICAL_REPLICATION waits for an answer from the client that
> >>confirms that logical replication initialization is finished. Before
> >>that the walsender connection cannot be used for anything else.
> >>
> >>2) we remove the snapshot as soon as any other commend is received, this
> >>way the replication connection stays usable, e.g. to issue a
> >>START_LOGICAL_REPLICATION in parallel to the initial data dump. In that
> >>case the snapshot would have to be imported *before* the next command
> >>was received as SET TRANSACTION SNAPSHOT requires the source transaction
> >>to be still open.
> >>Option 2 sounds more flexible. Is it more difficult to implement?
> >No, I don't think so. It's a bit more intrusive in that it requires
> >knowledge about logical replication in more parts of walsender, but it
> >should be ok.
> >
> >Note btw, that my description of 1) was easy to misunderstand. The
> >"that" in "Before that the walsender connection cannot be used for
> >anything else." is the answer from the client, not the usage of the
> >exported snapshot. Once the snapshot has been iimported into other
> >session(s) the source doesn't need to be alive anymore.
> >Does that explanation change anything?
>
> I think I understood you were saying the walsender connection can't be used
> for anything else (ie streaming WAL) until the exported snapshot has been
> imported. I think your clarification is still consistent with this?

Yes, thats correct.

> WIth option 2 I can still get the option 1 behaviour by not sending the next
> command to the walsender until I am done importing the snapshot. However if
> I want to start processing WAL before the snapshot has been imported I can
> do that with option 2.
>
> I am not sure I would need to do that, I'm just saying having the option is
> more flexible.

True.

Still not sure whats better, but since youre slightly leaning towards 2)
I am going to implement that.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services