Re: Logical Decoding follows timelines

Currently, it doesn't.

This patch is a WIP version of doing that, but only currently attempts
to do this in the WALSender.

Objective is to allow cascaded logical replication.

Very WIP, but here for comments.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 12/15/2014 08:54 PM, Simon Riggs wrote:
> Currently, it doesn't.
>
> This patch is a WIP version of doing that, but only currently attempts
> to do this in the WALSender.
>
> Objective is to allow cascaded logical replication.
>
> Very WIP, but here for comments.

With the patch, XLogSendLogical uses the same logic to calculate
SendRqstPtr that XLogSendPhysical does. It would be good to refactor
that into a common function, rather than copy-paste.

SendRqstPtr isn't actually used for anything in XLogSendLogical.

- Heikki

On 16 December 2014 at 14:25, Heikki Linnakangas
<hlinnakangas(at)vmware(dot)com> wrote:
> On 12/15/2014 08:54 PM, Simon Riggs wrote:
>>
>> Currently, it doesn't.
>>
>> This patch is a WIP version of doing that, but only currently attempts
>> to do this in the WALSender.
>>
>> Objective is to allow cascaded logical replication.
>>
>> Very WIP, but here for comments.
>
>
> With the patch, XLogSendLogical uses the same logic to calculate SendRqstPtr
> that XLogSendPhysical does. It would be good to refactor that into a common
> function, rather than copy-paste.

Some of the logic is similar, but not all.

> SendRqstPtr isn't actually used for anything in XLogSendLogical.

It exists to allow the call which resets TLI.

I'll see if I can make it exactly identical; I didn't think so when I
first looked, will look again.

Thanks

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 16 December 2014 at 21:17, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:

>>> This patch is a WIP version of doing that, but only currently attempts

>> With the patch, XLogSendLogical uses the same logic to calculate SendRqstPtr
>> that XLogSendPhysical does. It would be good to refactor that into a common
>> function, rather than copy-paste.
>
> Some of the logic is similar, but not all.
>
>> SendRqstPtr isn't actually used for anything in XLogSendLogical.
>
> It exists to allow the call which resets TLI.
>
> I'll see if I can make it exactly identical; I didn't think so when I
> first looked, will look again.

Yes, that works. New version attached

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 12/17/2014 10:35 AM, Simon Riggs wrote:
> On 16 December 2014 at 21:17, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>
>>>> This patch is a WIP version of doing that, but only currently attempts
>
>>> With the patch, XLogSendLogical uses the same logic to calculate SendRqstPtr
>>> that XLogSendPhysical does. It would be good to refactor that into a common
>>> function, rather than copy-paste.
>>
>> Some of the logic is similar, but not all.
>>
>>> SendRqstPtr isn't actually used for anything in XLogSendLogical.
>>
>> It exists to allow the call which resets TLI.
>>
>> I'll see if I can make it exactly identical; I didn't think so when I
>> first looked, will look again.
>
> Yes, that works. New version attached

Some comments, mostly on readability (not all of these were this patch's
fault):

> /*
> * Check that the timeline the client requested for exists, and
> * the requested start location is on that timeline.
> */
> (void) ReadSendTimeLine(cmd->timeline);
>
> /*
> * Found the requested timeline in the history. Check that
> * requested startpoint is on that timeline in our history.
> *
> * This is quite loose on purpose. We only check that we didn't
> * fork off the requested timeline before the switchpoint. We
> * don't check that we switched *to* it before the requested
> * starting point. This is because the client can legitimately
> * request to start replication from the beginning of the WAL
> * segment that contains switchpoint, but on the new timeline, so
> * that it doesn't end up with a partial segment. If you ask for a
> * too old starting point, you'll get an error later when we fail
> * to find the requested WAL segment in pg_xlog.
> *
> * XXX: we could be more strict here and only allow a startpoint
> * that's older than the switchpoint, if it's still in the same
> * WAL segment.
> */

The first comment implies that the ReadSendTimeLine call checks that the
requested start location is on the timeline, but that's actually done by
the code that follows the second comment. I would merge these two
comments, and move the ReadSendTimeLine call below the merged comment.

> @@ -577,8 +571,8 @@ StartReplication(StartReplicationCmd *cmd)
> * that's older than the switchpoint, if it's still in the same
> * WAL segment.
> */
> - if (!XLogRecPtrIsInvalid(switchpoint) &&
> - switchpoint < cmd->startpoint)
> + if (!XLogRecPtrIsInvalid(sendTimeLineValidUpto) &&
> + sendTimeLineValidUpto < cmd->startpoint)
> {
> ereport(ERROR,
> (errmsg("requested starting point %X/%X on timeline %u is not in this server's history",

IMHO using the local 'switchpoint' variable was more clear.

> @@ -941,6 +936,8 @@ StartLogicalReplication(StartReplicationCmd *cmd)
> * Force a disconnect, so that the decoding code doesn't need to care
> * about an eventual switch from running in recovery, to running in a
> * normal environment. Client code is expected to handle reconnects.
> + * This covers the race condition where we are promoted half way
> + * through starting up.
> */
> if (am_cascading_walsender && !RecoveryInProgress())
> {

We could exit recovery immediately after this check. Why is this check
needed?

> /*
> + * Find the timeline for the start location, or throw an error.
> + *
> + * Logical replication relies upon replication slots. Each slot has a
> + * single timeline history baked into it, so this should be easy.
> + */

I don't understand what "baked in" means here.

> + /*
> + * Get the SendRqstPtr and follow any timeline changes.
> + */
> + SendRqstPtr = GetLatestRequestPtr();

The old comment used to say "Figure out how far we can safely send the
WAL". I think that was much more clear. It's not clear what following
timeline changes means here, and the fact that it "gets the SendRqstPtr"
is obvious from the code.

> +
> +static XLogRecPtr
> +GetLatestRequestPtr(void)

This function desperately needs comment to explain what it does. I don't
much like its name either.

> +static TimeLineID
> +ReadSendTimeLine(TimeLineID tli)

Ditto. This function is also missing a "return".

I think it would slightly more intuitive if this function didn't set the
global variables directly, but simply returned the returned values to
the caller.

- Heikki

On 2015-01-03 12:07:29 +0200, Heikki Linnakangas wrote:
> >@@ -941,6 +936,8 @@ StartLogicalReplication(StartReplicationCmd *cmd)
> > * Force a disconnect, so that the decoding code doesn't need to care
> > * about an eventual switch from running in recovery, to running in a
> > * normal environment. Client code is expected to handle reconnects.
> >+ * This covers the race condition where we are promoted half way
> >+ * through starting up.
> > */
> > if (am_cascading_walsender && !RecoveryInProgress())
> > {
>
> We could exit recovery immediately after this check. Why is this check
> needed?

I probably wrote that ched and I don't think it really is needed. I
think that's a remnant of what the physical pendant used to do.

I think this needs slightly more abstraction because the infrastructure
is local to walsender.c - but logical decoding is also possible via
SQL. I'm not yet sure how that should look like. It'd be awesome if in
the course of that we could get rid of the nearly duplicated XLogRead()
:(

Simon, have you checked that this actually correctly follows timelines?
Afaics the patch as is won't allow to start logical decoding on a standby.

To allow logical decoding from clients I (apperently) wrote the the
following comment:
/* ----
* TODO: We got to change that someday soon...
*
* There's basically three things missing to allow this:
* 1) We need to be able to correctly and quickly identify the timeline a
* LSN belongs to
* 2) We need to force hot_standby_feedback to be enabled at all times so
* the primary cannot remove rows we need.
* 3) support dropping replication slots referring to a database, in
* dbase_redo. There can't be any active ones due to HS recovery
* conflicts, so that should be relatively easy.
* ----
*/
if (RecoveryInProgress())
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("logical decoding cannot be used while in recovery")));

You're implementing 1) here. 3) doesn't look very challenging.

But 2) imo is rather more interesting/complex. I guess we'd have to
force that streaming replication is used, that a physical replication
slot is used and that hot_standby_feedback is enabled.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Wed, Dec 17, 2014 at 5:35 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:

> On 16 December 2014 at 21:17, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>
> >>> This patch is a WIP version of doing that, but only currently attempts
>
> >> With the patch, XLogSendLogical uses the same logic to calculate
> SendRqstPtr
> >> that XLogSendPhysical does. It would be good to refactor that into a
> common
> >> function, rather than copy-paste.
> >
> > Some of the logic is similar, but not all.
> >
> >> SendRqstPtr isn't actually used for anything in XLogSendLogical.
> >
> > It exists to allow the call which resets TLI.
> >
> > I'll see if I can make it exactly identical; I didn't think so when I
> > first looked, will look again.
>
> Yes, that works. New version attached
>

Moved patch to CF 2015-02 to not lose track of it, also because it does not
seem it received a proper review.
--
Michael

On Fri, Feb 13, 2015 at 4:57 PM, Michael Paquier wrote:
> Moved patch to CF 2015-02 to not lose track of it, also because it does not
> seem it received a proper review.

This patch does not apply anymore, so attached is a rebased version.
The comments mentioned here have not been addressed:
http://www.postgresql.org/message-id/54A7BF61.9080708@vmware.com
Also, what kind of tests have been done? Logical decoding cannot be
used while a node is in recovery.
Regards,
--
Michael

On 21 April 2015 at 05:49, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
wrote:

> On Fri, Feb 13, 2015 at 4:57 PM, Michael Paquier wrote:
> > Moved patch to CF 2015-02 to not lose track of it, also because it does
> not
> > seem it received a proper review.
>
> This patch does not apply anymore, so attached is a rebased version.
> The comments mentioned here have not been addressed:
> http://www.postgresql.org/message-id/54A7BF61.9080708@vmware.com
> Also, what kind of tests have been done? Logical decoding cannot be
> used while a node is in recovery.
>

Returned with Feedback, I think. I have a new approach to be coded for next
release.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/>
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services