Re: Streaming base backups

On Mon, Jan 17, 2011 at 11:18, Dimitri Fontaine <dimitri(at)2ndquadrant(dot)fr> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> With pg_basebackup, you can set up streaming replication in what's
>> basically a single command (run the base backup, copy i na
>> recovery.conf file). In my first version I even had a switch that
>> would create the recovery.conf file for you - should we bring that
>> back?
>
> +1.  Well, make it optional maybe?

It has always been optional. Basically it just creates a recovery.conf file with
primary_conninfo=<whatever pg_streamrecv was using>
standby_mode=on

>> It does require you to set a "reasonable" wal_keep_segments, though,
>> but that's really all you need to do on the master side.
>
> Until we get integrated WAL streaming while the base backup is ongoing.
> We don't know when that is (9.1 or future), but that's what we're aiming
> to now, right?

Yeah, it does sound like a plan. But to still allow both - streaming
it in parallell will eat two connections, and I'm sure some people
might consider that a higher cost.

>>>> What Fujii-san unsuccessfully proposed was to have the master restore
>>>> segments from the archive and stream them to clients, on request.  It
>>>> was deemed better to have the slave obtain them from the archive
>>>> directly.
>>>
>>> Did Fuji-san agreed on the conclusion?
>>
>> I can see the point of the mastering being able to do this, but it
>> seems like a pretty narrow usecase, really. I think we invented
>> wal_keep_segments partially to solve this problem in a neater way?
>
> Well I still think that the easier setup we can offer here is to ship
> with integrated libpq based archive and restore commands.  Those could
> be bin/pg_walsender and bin/pg_walreceiver.  They would have some
> switches to make them suitable for running in subprocesses of either the
> base backup utility or the default libpq based archive daemon.

Not sure why they'd run as an archive command and not like now as a
replication client - but let's keep that out of this thread and in a
new one :)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/