Re: pgsql: Introduce replication slots.

From: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
To: Andres Freund <andres(at)2ndquadrant(dot)com>
Cc: Robert Haas <rhaas(at)postgresql(dot)org>, pgsql-committers <pgsql-committers(at)postgresql(dot)org>
Subject: Re: pgsql: Introduce replication slots.
Date: 2014-02-02 15:04:34
Message-ID: CAHGQGwEuBryFA5q_FiDRP2921Ljh0aSNWLnLfZRGF8=tV_7qVg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

On Sun, Feb 2, 2014 at 11:06 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2014-02-02 14:44:17 +0900, Fujii Masao wrote:
>> On Sat, Feb 1, 2014 at 6:37 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> > On 2014-02-01 18:33:08 +0900, Fujii Masao wrote:
>> >> On Sat, Feb 1, 2014 at 6:09 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> >> > On 2014-02-01 16:47:47 +0900, Fujii Masao wrote:
>> >> >> This patch changed basebackup.c so that it skips pg_replslot. It's OK
>> >> >> to skip all files in that directory, but an empty pg_replslot must be
>> >> >> included in the backup. Otherwise we cannot start PostgreSQL from
>> >> >> the backup taken via pg_basebackup. Attached patch fixes this problem.
>> >> >
>> >> > That's a pretty fair point. Not sure how that could escape my
>> >> > notice. The patch does look sane to me.
>> >> >
>> >> > I wonder if we additionally should add code to recreate pg_replslot on
>> >> > startup, similar to pg_xlog?
>> >>
>> >> Similar to pg_xlog/archive_status, not pg_xlog? That might be an option.
>> >> But I'm not inclined to do that for now. The fact that the essential
>> >> directory like pg_replslot doesn't exist indicates the symptom of
>> >> something strange. And, changing that way might prevent us from
>> >> detecting such symptom.
>> >
>> > The reason I am wondering is that it makes a fair bit of sense to
>> > exclude it in open-coded base backups as well, and excluding the
>> > entire directory might be the easiest way there. But I guess people
>> > manage for pg_xlog/, so it's really not something that would reduce pain
>> > measurably.
>>
>> On second thought, we should always delete all files in pg_replslot
>> when starting recovery from the backup?
>
> Are you suggesting to always delete them when in standby_mode? If so,
> no, that'd be bad, we intentially *do* want to support situations in
> which we stream from the standby, i.e. cascading types of setups.
>
> If you mean doing so when initially starting with a backup label,
> hm. Maybe. That would make it impossible to keep replication slots when
> moving to a new server with a short downtime, which seems a bit
> annoying.

Yeah, I was thinking the latter.

>> The users who are
>> using their own backup script instead of pg_basebackup might
>> include pg_replslot files in the backup. Currently, in this case,
>> the replication slots which were created before would be available
>> even after the recovery. Is this OK? If not, all files in pg_replslot
>> should be removed at the beginning of the PITR. OTOH, if that's OK,
>> I think that pg_basebackup should not skip pg_replslot files.
>> Thought?
>
> Robert raised this previously in
> http://archives.postgresql.org/message-id/CA%2BTgmoar6BLb%2B7BQUYEmkmdFSE1f8khCZCDP-aCojOrESiNLBg%40mail.gmail.com :
>> - Exclude pg_replslot from base backups. This might need more thought
>> and documentation; people who use the filesystem method to perform
>> backups might need to be advised to remove this directory in some
>> cases also, or people who use pg_basebackup might want to keep it in
>> some cases (not sure).
>
> I can see usecases for removing and keeping them. Removing them has the
> big advantage that the user won't be surprised by a slot's existance
> which prevents resources (WAL, xmin horizon/VACUUM) from being
> reclaimed.

Agreed.

Regards,

--
Fujii Masao

In response to

Browse pgsql-committers by date

  From Date Subject
Next Message Fujii Masao 2014-02-02 15:43:08 pgsql: Add primary_slotname to recovery.conf.sample.
Previous Message Andres Freund 2014-02-02 14:06:35 Re: pgsql: Introduce replication slots.