Re: [GENERAL] pg_upgrade -u

A minor detail in 9.2.4, but I noticed that the pg_upgrade flag for
superuser, -u, does not get carried to a -U flag on the vacuumdb commands
written to analyze_new_cluster.sh.

On Wed, May 8, 2013 at 05:05:05PM -0400, Ray Stell wrote:
> A minor detail in 9.2.4, but I noticed that the pg_upgrade flag for
> superuser, -u, does not get carried to a -U flag on the vacuumdb commands
> written to analyze_new_cluster.sh.

OK, let me look at this issue.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On Wed, May 8, 2013 at 08:52:40PM -0400, Bruce Momjian wrote:
> On Wed, May 8, 2013 at 05:05:05PM -0400, Ray Stell wrote:
> > A minor detail in 9.2.4, but I noticed that the pg_upgrade flag for
> > superuser, -u, does not get carried to a -U flag on the vacuumdb commands
> > written to analyze_new_cluster.sh.
>
> OK, let me look at this issue.

I have thought about this and there are potentially several options
specified to pg_upgrade that could be passed into scripts:

-O, --new-options=OPTIONS new cluster options to pass to the server
-P, --new-port=NEWPORT new cluster port number (default 50432)
-u, --user=NAME cluster superuser (default "root")

However, if we pass these items into the scripts, we then force these
values to be used, even if the user wants to use a different value. It
is a balance between supplying defaults vs. requiring the user to supply
or change the values used during the ugprade.

At this point, I have favored _not_ supplying defaults in the script.
Do you have an alternative argument in favor of supplying defaults?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On May 21, 2013, at 2:41 PM, Bruce Momjian wrote:

> On Wed, May 8, 2013 at 08:52:40PM -0400, Bruce Momjian wrote:
>> On Wed, May 8, 2013 at 05:05:05PM -0400, Ray Stell wrote:
>>> A minor detail in 9.2.4, but I noticed that the pg_upgrade flag for
>>> superuser, -u, does not get carried to a -U flag on the vacuumdb commands
>>> written to analyze_new_cluster.sh.
>>
>> OK, let me look at this issue.
>
> I have thought about this and there are potentially several options
> specified to pg_upgrade that could be passed into scripts:
>
> -O, --new-options=OPTIONS new cluster options to pass to the server
> -P, --new-port=NEWPORT new cluster port number (default 50432)
> -u, --user=NAME cluster superuser (default "root")
>
> However, if we pass these items into the scripts, we then force these
> values to be used, even if the user wants to use a different value. It
> is a balance between supplying defaults vs. requiring the user to supply
> or change the values used during the ugprade.
>
> At this point, I have favored _not_ supplying defaults in the script.
> Do you have an alternative argument in favor of supplying defaults?

Well, the story really began when I ran initdb with a -U arg. vacuumdb takes the -U also, but pg_upgrade does not.

It seems like if I have to supply a -u in order to get pg_upgrade to function in the case where there is no default superuser in the cluster, then an associated vacuumdb command requires a -U arg.

Perhaps just documenting the behavior is all that is needed, but -U is everywhere and I think that's a good thing.

On Wed, May 22, 2013 at 03:05:57PM -0400, Ray Stell wrote:
> > However, if we pass these items into the scripts, we then force
> > these values to be used, even if the user wants to use a different
> > value. It is a balance between supplying defaults vs. requiring the
> > user to supply or change the values used during the ugprade.
> >
> > At this point, I have favored _not_ supplying defaults in the
> > script. Do you have an alternative argument in favor of supplying
> > defaults?
>
>
>
> Well, the story really began when I ran initdb with a -U arg. vacuumdb
> takes the -U also, but pg_upgrade does not.
>
> It seems like if I have to supply a -u in order to get pg_upgrade
> to function in the case where there is no default superuser in the
> cluster, then an associated vacuumdb command requires a -U arg.
>
> Perhaps just documenting the behavior is all that is needed, but -U is
> everywhere and I think that's a good thing.

[ moved to hacker ]

Wow, I never realized other tools used -U for user, instead of -u.
Should I change pg_upgrade to use -U for 9.4? I can keep supporting -u
as an undocumented option.

I have applied the attached patch to document that you might need to set
connection parameters for vacuumdb.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On 05/28/2013 07:55 PM, Bruce Momjian wrote:

>> Perhaps just documenting the behavior is all that is needed, but -U is
>> everywhere and I think that's a good thing.
>
> [ moved to hacker ]
>
> Wow, I never realized other tools used -U for user, instead of -u.
> Should I change pg_upgrade to use -U for 9.4? I can keep supporting -u
> as an undocumented option.

Yes, -U makes the most sense as that is what is used with the other
tools. I think you should just drop -u, this isn't something people are
doing everyday (like psql). The backward compatibility argument is
pretty slim.

JD

On May 28, 2013, at 10:55 PM, Bruce Momjian wrote:

> On Wed, May 22, 2013 at 03:05:57PM -0400, Ray Stell wrote:
>>> However, if we pass these items into the scripts, we then force
>>> these values to be used, even if the user wants to use a different
>>> value. It is a balance between supplying defaults vs. requiring the
>>> user to supply or change the values used during the ugprade.
>>>
>>> At this point, I have favored _not_ supplying defaults in the
>>> script. Do you have an alternative argument in favor of supplying
>>> defaults?
>>
>>
>>
>> Well, the story really began when I ran initdb with a -U arg. vacuumdb
>> takes the -U also, but pg_upgrade does not.
>>
>> It seems like if I have to supply a -u in order to get pg_upgrade
>> to function in the case where there is no default superuser in the
>> cluster, then an associated vacuumdb command requires a -U arg.
>>
>> Perhaps just documenting the behavior is all that is needed, but -U is
>> everywhere and I think that's a good thing.
>
> [ moved to hacker ]
>
> Wow, I never realized other tools used -U for user, instead of -u.
> Should I change pg_upgrade to use -U for 9.4? I can keep supporting -u
> as an undocumented option.

That would make for consistency, but not change the broken behavior. Comments on this below.

> I have applied the attached patch to document that you might need to set
> connection parameters for vacuumdb.

That will work as this is not a big deal, but have to admit, I didn't understand the logic you posted in your question. If the src cluster has a alternate superuser (from initdb -U), and the the pg_upgrade command has to be supplied a -u/-U in order to work with the src cluster, why would you assume somehow the output cluster has changed to the default superuser when you build the vacuum script on the other side of pg_upgrade? Is that even possible to accomplish? Your statement about "forcing the values" throws me, as it seems to me the user is requesting the variation. I have not dug into pg_upgrade's guts, so I may just be uninformed.

On 5/21/13 2:41 PM, Bruce Momjian wrote:
> I have thought about this and there are potentially several options
> specified to pg_upgrade that could be passed into scripts:
>
> -O, --new-options=OPTIONS new cluster options to pass to the server
> -P, --new-port=NEWPORT new cluster port number (default 50432)
> -u, --user=NAME cluster superuser (default "root")
>
> However, if we pass these items into the scripts, we then force these
> values to be used, even if the user wants to use a different value. It
> is a balance between supplying defaults vs. requiring the user to supply
> or change the values used during the ugprade.
>
> At this point, I have favored _not_ supplying defaults in the script.
> Do you have an alternative argument in favor of supplying defaults?

You could put environment variable assignments at the top of the script,
so they are easy to change or remove. But it seems to me the values
should be in there somehow.

On 5/28/13 10:55 PM, Bruce Momjian wrote:
> Wow, I never realized other tools used -U for user, instead of -u.
> Should I change pg_upgrade to use -U for 9.4? I can keep supporting -u
> as an undocumented option.

It seems to me that that option shouldn't be necessary anyway.
pg_upgrade should somehow be able to find out by itself what the
superuser of the old cluster was.

On Wed, May 29, 2013 at 08:59:42AM -0400, Ray Stell wrote:
> > [ moved to hacker ]
> >
> > Wow, I never realized other tools used -U for user, instead of -u.
> > Should I change pg_upgrade to use -U for 9.4? I can keep supporting
> > -u as an undocumented option.
>
> That would make for consistency, but not change the broken behavior.
> Comments on this below.
>
>
> > I have applied the attached patch to document that you might need to
> > set connection parameters for vacuumdb.
>
> That will work as this is not a big deal, but have to admit, I didn't
> understand the logic you posted in your question. If the src cluster
> has a alternate superuser (from initdb -U), and the the pg_upgrade
> command has to be supplied a -u/-U in order to work with the src
> cluster, why would you assume somehow the output cluster has changed
> to the default superuser when you build the vacuum script on the
> other side of pg_upgrade? Is that even possible to accomplish? Your
> statement about "forcing the values" throws me, as it seems to me the
> user is requesting the variation. I have not dug into pg_upgrade's
> guts, so I may just be uninformed.

OK, so let's look again at what you can supply to pg_upgrade for
new cluster connections:

-O, --new-options=OPTIONS new cluster options to pass to the server
-P, --new-port=NEWPORT new cluster port number (default 50432)
-u, --user=NAME cluster superuser (default "postgres")

So, probably all of these _could_ be propagated to created scripts, like
analyze_new_cluster.sh. However, if we do that, we hardwire these
values into the script, meaning you can't override them with environment
variables like PGUSER. The question is whether hard-wiring these helps
more than it hurts, and which ones should be hard-wired.

One new idea would be to add "$@" to the vacuumdb script parameters so
users could supply their own "-U myuser" options to the script. That is
easy to do for Unix, which I assume all support "$@", but perhaps hard
for Windows.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On Wed, May 29, 2013 at 09:42:12AM -0400, Peter Eisentraut wrote:
> On 5/21/13 2:41 PM, Bruce Momjian wrote:
> > I have thought about this and there are potentially several options
> > specified to pg_upgrade that could be passed into scripts:
> >
> > -O, --new-options=OPTIONS new cluster options to pass to the server
> > -P, --new-port=NEWPORT new cluster port number (default 50432)
> > -u, --user=NAME cluster superuser (default "root")
> >
> > However, if we pass these items into the scripts, we then force these
> > values to be used, even if the user wants to use a different value. It
> > is a balance between supplying defaults vs. requiring the user to supply
> > or change the values used during the ugprade.
> >
> > At this point, I have favored _not_ supplying defaults in the script.
> > Do you have an alternative argument in favor of supplying defaults?
>
> You could put environment variable assignments at the top of the script,
> so they are easy to change or remove. But it seems to me the values
> should be in there somehow.

Yes, that was another option, though I am worried that I am going to
start getting script-specific, no? Can I just do this:

PGUSER="postgres"
export PGUSER

vacuumedb -U "$PGUSER"

Is that portable? I know I can't do:

export PGUSER="postgres"

as older shells can't process that. What about -P? Do I pass that too?
I now realize we can't pass -O.

This is all easy to do for 9.4 if we can just agree on it.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On May 29, 2013, at 11:07 AM, Bruce Momjian wrote:

> On Wed, May 29, 2013 at 08:59:42AM -0400, Ray Stell wrote:
>>> [ moved to hacker ]
>>> The question is whether hard-wiring these helps more than it hurts, and which ones should be hard-wired.

I seems to me that superuser is exactly that special case and that if an alternate superuser is hardwired in the src cluster then -u/-U and that specific value will be required on both sides of pg_upgrade, no variability is needed and perhaps not possible. You're point is well taken for port.

On Tue, May 28, 2013 at 09:08:03PM -0700, Joshua D. Drake wrote:
>
> On 05/28/2013 07:55 PM, Bruce Momjian wrote:
>
> >>Perhaps just documenting the behavior is all that is needed, but -U is
> >>everywhere and I think that's a good thing.
> >
> >[ moved to hacker ]
> >
> >Wow, I never realized other tools used -U for user, instead of -u.
> >Should I change pg_upgrade to use -U for 9.4? I can keep supporting -u
> >as an undocumented option.
>
> Yes, -U makes the most sense as that is what is used with the other
> tools. I think you should just drop -u, this isn't something people
> are doing everyday (like psql). The backward compatibility argument
> is pretty slim.

Done for 9.4. I also simplified some of the option values in --help and
the docs.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On Wed, May 29, 2013 at 09:44:00AM -0400, Peter Eisentraut wrote:
> On 5/28/13 10:55 PM, Bruce Momjian wrote:
> > Wow, I never realized other tools used -U for user, instead of -u.
> > Should I change pg_upgrade to use -U for 9.4? I can keep supporting -u
> > as an undocumented option.
>
> It seems to me that that option shouldn't be necessary anyway.
> pg_upgrade should somehow be able to find out by itself what the
> superuser of the old cluster was.

Uh, any idea how to do that?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On Thu, May 30, 2013 at 08:42:21AM -0400, Ray Stell wrote:
>
> On May 29, 2013, at 11:07 AM, Bruce Momjian wrote:
>
> > On Wed, May 29, 2013 at 08:59:42AM -0400, Ray Stell wrote:
> >>> [ moved to hacker ] The question is whether hard-wiring these
> >>> helps more than it hurts, and which ones should be hard-wired.
>
> I seems to me that superuser is exactly that special case and that if
> an alternate superuser is hardwired in the src cluster then -u/-U and
> that specific value will be required on both sides of pg_upgrade, no
> variability is needed and perhaps not possible. You're point is well
> taken for port.

You have convinced me. The attached, applied patch for PG 9.4 passes
the command-line-supplied username into the created analyze script.

I have also attached a sample output analyze script.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On 6/28/13 6:06 PM, Bruce Momjian wrote:
> On Wed, May 29, 2013 at 09:44:00AM -0400, Peter Eisentraut wrote:
>> On 5/28/13 10:55 PM, Bruce Momjian wrote:
>>> Wow, I never realized other tools used -U for user, instead of -u.
>>> Should I change pg_upgrade to use -U for 9.4? I can keep supporting -u
>>> as an undocumented option.
>>
>> It seems to me that that option shouldn't be necessary anyway.
>> pg_upgrade should somehow be able to find out by itself what the
>> superuser of the old cluster was.
>
> Uh, any idea how to do that?

select rolname from pg_authid where oid = 10;

On Fri, Jun 28, 2013 at 09:15:31PM -0400, Peter Eisentraut wrote:
> On 6/28/13 6:06 PM, Bruce Momjian wrote:
> > On Wed, May 29, 2013 at 09:44:00AM -0400, Peter Eisentraut wrote:
> >> On 5/28/13 10:55 PM, Bruce Momjian wrote:
> >>> Wow, I never realized other tools used -U for user, instead of -u.
> >>> Should I change pg_upgrade to use -U for 9.4? I can keep supporting -u
> >>> as an undocumented option.
> >>
> >> It seems to me that that option shouldn't be necessary anyway.
> >> pg_upgrade should somehow be able to find out by itself what the
> >> superuser of the old cluster was.
> >
> > Uh, any idea how to do that?
>
> select rolname from pg_authid where oid = 10;

Uh, how do I know what username to connect as to issue that query?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> On 6/28/13 6:06 PM, Bruce Momjian wrote:
>> On Wed, May 29, 2013 at 09:44:00AM -0400, Peter Eisentraut wrote:
>>> pg_upgrade should somehow be able to find out by itself what the
>>> superuser of the old cluster was.

>> Uh, any idea how to do that?

> select rolname from pg_authid where oid = 10;

But that only works if you've already been able to connect, no?

regards, tom lane

On 6/28/13 9:43 PM, Bruce Momjian wrote:
> On Fri, Jun 28, 2013 at 09:15:31PM -0400, Peter Eisentraut wrote:
>> On 6/28/13 6:06 PM, Bruce Momjian wrote:
>>> On Wed, May 29, 2013 at 09:44:00AM -0400, Peter Eisentraut wrote:
>>>> On 5/28/13 10:55 PM, Bruce Momjian wrote:
>>>>> Wow, I never realized other tools used -U for user, instead of -u.
>>>>> Should I change pg_upgrade to use -U for 9.4? I can keep supporting -u
>>>>> as an undocumented option.
>>>>
>>>> It seems to me that that option shouldn't be necessary anyway.
>>>> pg_upgrade should somehow be able to find out by itself what the
>>>> superuser of the old cluster was.
>>>
>>> Uh, any idea how to do that?
>>
>> select rolname from pg_authid where oid = 10;
>
> Uh, how do I know what username to connect as to issue that query?

single-user mode?

On Fri, Jun 28, 2013 at 11:59:58PM -0400, Peter Eisentraut wrote:
> On 6/28/13 9:43 PM, Bruce Momjian wrote:
> > On Fri, Jun 28, 2013 at 09:15:31PM -0400, Peter Eisentraut wrote:
> >> On 6/28/13 6:06 PM, Bruce Momjian wrote:
> >>> On Wed, May 29, 2013 at 09:44:00AM -0400, Peter Eisentraut wrote:
> >>>> On 5/28/13 10:55 PM, Bruce Momjian wrote:
> >>>>> Wow, I never realized other tools used -U for user, instead of -u.
> >>>>> Should I change pg_upgrade to use -U for 9.4? I can keep supporting -u
> >>>>> as an undocumented option.
> >>>>
> >>>> It seems to me that that option shouldn't be necessary anyway.
> >>>> pg_upgrade should somehow be able to find out by itself what the
> >>>> superuser of the old cluster was.
> >>>
> >>> Uh, any idea how to do that?
> >>
> >> select rolname from pg_authid where oid = 10;
> >
> > Uh, how do I know what username to connect as to issue that query?
>
> single-user mode?

Yes, but single-user mode is a whole new set of problems.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +