Re: [PATCH] Make pg_basebackup configure and start standby

Hi,

attached is a patch that does $SUBJECT.

It's a usability enhancement, to take a backup, write
a minimalistic recovery.conf and start the streaming
standby in one go.

Comments?

Best regards,
Zoltán Böszörményi

--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/

On Sun, Jul 1, 2012 at 8:02 PM, Boszormenyi Zoltan <zb(at)cybertec(dot)at> wrote:
> Hi,
>
> attached is a patch that does $SUBJECT.
>
> It's a usability enhancement, to take a backup, write
> a minimalistic recovery.conf and start the streaming
> standby in one go.
>
> Comments?

Could you add the patch to the next CommitFest?

If the backup is taken from the standby server, the standby's recovery.conf
is included in the backup. What happens in this case?

Regards,

--
Fujii Masao

Hi,

2012-07-01 17:38 keltezéssel, Fujii Masao írta:
> On Sun, Jul 1, 2012 at 8:02 PM, Boszormenyi Zoltan <zb(at)cybertec(dot)at> wrote:
>> Hi,
>>
>> attached is a patch that does $SUBJECT.
>>
>> It's a usability enhancement, to take a backup, write
>> a minimalistic recovery.conf and start the streaming
>> standby in one go.
>>
>> Comments?
> Could you add the patch to the next CommitFest?

I will.

> If the backup is taken from the standby server, the standby's recovery.conf
> is included in the backup. What happens in this case?

As documented, the command line parameters of pg_basebackup
will be used for recovery.conf. So, the new standby will replicate
the previous one. Cascading replication works since 9.2.

Best regards,
Zoltán Böszörményi

--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/

On Sun, Jul 1, 2012 at 1:02 PM, Boszormenyi Zoltan <zb(at)cybertec(dot)at> wrote:
> Hi,
>
> attached is a patch that does $SUBJECT.
>
> It's a usability enhancement, to take a backup, write
> a minimalistic recovery.conf and start the streaming
> standby in one go.

I like the writing of recovery.conf. In fact, I had it in my code at
one very early point and took it out in order to get a clean patch
ready :)

But I think that part is lacking in functionality: AFAICT it's
hardcoded to only handle host, port, user and password. What about
other connection parameters, likely passed to pg_basebackup through
the environment in that case? isn't that quite likely to break the
server later?

Maybe the proper way around that is to provide the ability for
pg_basebackup to take a full connection string, just like we allow
psql to do?

I'm not sure we should go the way of providing the "start slave".
Given thta how you want to start the slave differs so much on
platforms. The most glaring example is on windows you really need to
*start the service* rather than use pg_ctl. Sure, you can document
your way around that, but I'm not sure the functionality added is
really worth it. What about all the other potential connection
parameters?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Mon, Jul 2, 2012 at 12:42 AM, Boszormenyi Zoltan <zb(at)cybertec(dot)at> wrote:
> Hi,
>
> 2012-07-01 17:38 keltezéssel, Fujii Masao írta:
>
>> On Sun, Jul 1, 2012 at 8:02 PM, Boszormenyi Zoltan <zb(at)cybertec(dot)at> wrote:
>>>
>>> Hi,
>>>
>>> attached is a patch that does $SUBJECT.
>>>
>>> It's a usability enhancement, to take a backup, write
>>> a minimalistic recovery.conf and start the streaming
>>> standby in one go.
>>>
>>> Comments?
>>
>> Could you add the patch to the next CommitFest?
>
>
> I will.
>
>
>> If the backup is taken from the standby server, the standby's
>> recovery.conf
>> is included in the backup. What happens in this case?
>
>
> As documented, the command line parameters of pg_basebackup
> will be used for recovery.conf. So, the new standby will replicate
> the previous one. Cascading replication works since 9.2.

So pg_basebackup overwrites the recovery.conf which was backed up
from the standby with the recovery.conf which was created by using
the command line parameters of pg_basebackup?

Regards,

--
Fujii Masao

On Mon, Jul 2, 2012 at 12:44 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Sun, Jul 1, 2012 at 1:02 PM, Boszormenyi Zoltan <zb(at)cybertec(dot)at> wrote:
>> Hi,
>>
>> attached is a patch that does $SUBJECT.
>>
>> It's a usability enhancement, to take a backup, write
>> a minimalistic recovery.conf and start the streaming
>> standby in one go.
>
> I like the writing of recovery.conf.

Agreed.

> In fact, I had it in my code at
> one very early point and took it out in order to get a clean patch
> ready :)
>
> But I think that part is lacking in functionality: AFAICT it's
> hardcoded to only handle host, port, user and password. What about
> other connection parameters, likely passed to pg_basebackup through
> the environment in that case? isn't that quite likely to break the
> server later?

What about something like PQconninfo which returns the connection
string value established at connection?

> Maybe the proper way around that is to provide the ability for
> pg_basebackup to take a full connection string, just like we allow
> psql to do?

+1

> I'm not sure we should go the way of providing the "start slave".
> Given thta how you want to start the slave differs so much on
> platforms. The most glaring example is on windows you really need to
> *start the service* rather than use pg_ctl. Sure, you can document
> your way around that, but I'm not sure the functionality added is
> really worth it.

Agreed.

Regards,

--
Fujii Masao

On Jul 1, 2012, at 5:44 PM, Magnus Hagander wrote:

> On Sun, Jul 1, 2012 at 1:02 PM, Boszormenyi Zoltan <zb(at)cybertec(dot)at> wrote:
>> Hi,
>>
>> attached is a patch that does $SUBJECT.
>>
>> It's a usability enhancement, to take a backup, write
>> a minimalistic recovery.conf and start the streaming
>> standby in one go.
>
> I like the writing of recovery.conf. In fact, I had it in my code at
> one very early point and took it out in order to get a clean patch
> ready :)
>
> But I think that part is lacking in functionality: AFAICT it's
> hardcoded to only handle host, port, user and password. What about
> other connection parameters, likely passed to pg_basebackup through
> the environment in that case? isn't that quite likely to break the
> server later?
>

one option would be to check the environments and take them if needed.
however, i am not sure if this is a good idea either - just thing of PGPASSWORD or so. do we really want to take it and write it to a file straight away? i guess there are arguments for both ideas.

still, i guess your argument is a reasonable one.

> Maybe the proper way around that is to provide the ability for
> pg_basebackup to take a full connection string, just like we allow
> psql to do?
>

this would make things redundant. i am quite sure some users might not get the distinction straight away.

>
>
> I'm not sure we should go the way of providing the "start slave".
> Given thta how you want to start the slave differs so much on
> platforms. The most glaring example is on windows you really need to
> *start the service* rather than use pg_ctl. Sure, you can document
> your way around that, but I'm not sure the functionality added is
> really worth it. What about all the other potential connection
> parameters.

regards,

hans

--
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de

On mån, 2012-07-02 at 01:10 +0900, Fujii Masao wrote:
> > But I think that part is lacking in functionality: AFAICT it's
> > hardcoded to only handle host, port, user and password. What about
> > other connection parameters, likely passed to pg_basebackup through
> > the environment in that case? isn't that quite likely to break the
> > server later?
>
> What about something like PQconninfo which returns the connection
> string value established at connection?
>
> > Maybe the proper way around that is to provide the ability for
> > pg_basebackup to take a full connection string, just like we allow
> > psql to do?
>
> +1
>
I think both of these would be necessary to make this work smoothly.

You also need to take into account situations like when pg_basebackup
found a server with the help of PG* environment variables that might no
longer be set when the server tries to start recovery. So you need
something like PQconninfo.

On Tue, Jul 3, 2012 at 9:47 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On mån, 2012-07-02 at 01:10 +0900, Fujii Masao wrote:
>> > But I think that part is lacking in functionality: AFAICT it's
>> > hardcoded to only handle host, port, user and password. What about
>> > other connection parameters, likely passed to pg_basebackup through
>> > the environment in that case? isn't that quite likely to break the
>> > server later?
>>
>> What about something like PQconninfo which returns the connection
>> string value established at connection?
>>
>> > Maybe the proper way around that is to provide the ability for
>> > pg_basebackup to take a full connection string, just like we allow
>> > psql to do?
>>
>> +1
>>
> I think both of these would be necessary to make this work smoothly.
>
> You also need to take into account situations like when pg_basebackup
> found a server with the help of PG* environment variables that might no
> longer be set when the server tries to start recovery. So you need
> something like PQconninfo.

Zoltan,

are you planning to work on the things discussed in this thread? I
notice the patch is sitting with "waiting on author" in the CF app -
so the second question is that if you are doing that, do you think it
will be done within the scope of the current CF?

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Hi,

2012-10-03 10:25 keltezéssel, Magnus Hagander írta:
> On Tue, Jul 3, 2012 at 9:47 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
>> On mån, 2012-07-02 at 01:10 +0900, Fujii Masao wrote:
>>>> But I think that part is lacking in functionality: AFAICT it's
>>>> hardcoded to only handle host, port, user and password. What about
>>>> other connection parameters, likely passed to pg_basebackup through
>>>> the environment in that case? isn't that quite likely to break the
>>>> server later?
>>> What about something like PQconninfo which returns the connection
>>> string value established at connection?
>>>
>>>> Maybe the proper way around that is to provide the ability for
>>>> pg_basebackup to take a full connection string, just like we allow
>>>> psql to do?
>>> +1
>>>
>> I think both of these would be necessary to make this work smoothly.
>>
>> You also need to take into account situations like when pg_basebackup
>> found a server with the help of PG* environment variables that might no
>> longer be set when the server tries to start recovery. So you need
>> something like PQconninfo.
> Zoltan,
>
> are you planning to work on the things discussed in this thread? I
> notice the patch is sitting with "waiting on author" in the CF app -
> so the second question is that if you are doing that, do you think it
> will be done within the scope of the current CF?

yes, I am on it so it can be done in this CF.

Best regards,
Zoltán Böszörményi

--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/

On Wed, Oct 3, 2012 at 10:37 AM, Boszormenyi Zoltan <zb(at)cybertec(dot)at> wrote:
> Hi,
>
> 2012-10-03 10:25 keltezéssel, Magnus Hagander írta:
>>
>> On Tue, Jul 3, 2012 at 9:47 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
>>>
>>> On mĺn, 2012-07-02 at 01:10 +0900, Fujii Masao wrote:
>>>>>
>>>>> But I think that part is lacking in functionality: AFAICT it's
>>>>> hardcoded to only handle host, port, user and password. What about
>>>>> other connection parameters, likely passed to pg_basebackup through
>>>>> the environment in that case? isn't that quite likely to break the
>>>>> server later?
>>>>
>>>> What about something like PQconninfo which returns the connection
>>>> string value established at connection?
>>>>
>>>>> Maybe the proper way around that is to provide the ability for
>>>>> pg_basebackup to take a full connection string, just like we allow
>>>>> psql to do?
>>>>
>>>> +1
>>>>
>>> I think both of these would be necessary to make this work smoothly.
>>>
>>> You also need to take into account situations like when pg_basebackup
>>> found a server with the help of PG* environment variables that might no
>>> longer be set when the server tries to start recovery. So you need
>>> something like PQconninfo.
>>
>> Zoltan,
>>
>> are you planning to work on the things discussed in this thread? I
>> notice the patch is sitting with "waiting on author" in the CF app -
>> so the second question is that if you are doing that, do you think it
>> will be done within the scope of the current CF?
>
>
> yes, I am on it so it can be done in this CF.

Great, thanks!

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

2012-07-01 18:01 keltezéssel, Fujii Masao írta:
> On Mon, Jul 2, 2012 at 12:42 AM, Boszormenyi Zoltan <zb(at)cybertec(dot)at> wrote:
>> Hi,
>>
>> 2012-07-01 17:38 keltezéssel, Fujii Masao írta:
>>
>>> On Sun, Jul 1, 2012 at 8:02 PM, Boszormenyi Zoltan <zb(at)cybertec(dot)at> wrote:
>>>> Hi,
>>>>
>>>> attached is a patch that does $SUBJECT.
>>>>
>>>> It's a usability enhancement, to take a backup, write
>>>> a minimalistic recovery.conf and start the streaming
>>>> standby in one go.
>>>>
>>>> Comments?
>>> Could you add the patch to the next CommitFest?
>>
>> I will.
>>
>>
>>> If the backup is taken from the standby server, the standby's
>>> recovery.conf
>>> is included in the backup. What happens in this case?
>>
>> As documented, the command line parameters of pg_basebackup
>> will be used for recovery.conf. So, the new standby will replicate
>> the previous one. Cascading replication works since 9.2.
> So pg_basebackup overwrites the recovery.conf which was backed up
> from the standby with the recovery.conf which was created by using
> the command line parameters of pg_basebackup?

Only if requested.

Best regards,
Zoltán Böszörményi

--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/