Re: pg_dumpall reccomendation in release notes

Bruce, Tom:

Can we change this text in the template release notes?

A dump/restore using
pg_dumpall<http://www.postgresql.org/docs/9.3/static/app-pg-dumpall.html>,
or use of
pg_upgrade<http://www.postgresql.org/docs/9.3/static/pgupgrade.html>, is
required for those wishing to migrate data from any previous release.

Again, here we're recommending pg_dumpall with its many limitations, and
not mentioning pg_dump/pg_restore at all. This has caused several
people to ask me if pg_dump is not supported for upgrading anymore. Fix?

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

Josh Berkus <josh(at)agliodbs(dot)com> writes:
> Can we change this text in the template release notes?

> A dump/restore using
> pg_dumpall<http://www.postgresql.org/docs/9.3/static/app-pg-dumpall.html>,
> or use of
> pg_upgrade<http://www.postgresql.org/docs/9.3/static/pgupgrade.html>, is
> required for those wishing to migrate data from any previous release.

> Again, here we're recommending pg_dumpall with its many limitations, and
> not mentioning pg_dump/pg_restore at all. This has caused several
> people to ask me if pg_dump is not supported for upgrading anymore. Fix?

There's a very good reason for not recommending pg_dump in this context:
it won't dump everything. Yeah, if you know what you're doing, you might
use per-database pg_dump runs plus pg_dumpall -g to catch the roles etc,
but we're not going to try to wedge all that info into one or two
sentences of release-note boilerplate. If you can get that right then
you don't need the release notes to remind you anyway. If you aren't
likely to get it right then the release notes would do you no service
by suggesting it.

I'm not sure what "many limitations" you think pg_dumpall has that pg_dump
doesn't.

I do think that it might be time to reword this to recommend pg_upgrade
first, though. ISTM that the current wording dates from when pg_upgrade
could charitably be described as experimental.

regards, tom lane

On 02/25/2014 03:41 PM, Tom Lane wrote:
> Josh Berkus <josh(at)agliodbs(dot)com> writes:
>> Can we change this text in the template release notes?
>
>> A dump/restore using
>> pg_dumpall<http://www.postgresql.org/docs/9.3/static/app-pg-dumpall.html>,
>> or use of
>> pg_upgrade<http://www.postgresql.org/docs/9.3/static/pgupgrade.html>, is
>> required for those wishing to migrate data from any previous release.
>
>> Again, here we're recommending pg_dumpall with its many limitations, and
>> not mentioning pg_dump/pg_restore at all. This has caused several
>> people to ask me if pg_dump is not supported for upgrading anymore. Fix?
>
> There's a very good reason for not recommending pg_dump in this context:
> it won't dump everything. Yeah, if you know what you're doing, you might
> use per-database pg_dump runs plus pg_dumpall -g to catch the roles etc,
> but we're not going to try to wedge all that info into one or two
> sentences of release-note boilerplate. If you can get that right then
> you don't need the release notes to remind you anyway. If you aren't
> likely to get it right then the release notes would do you no service
> by suggesting it.

Right. But the fact that we don't mention it *at all* has caused some
users to ask me if regular pg_dump doesn't work for upgrades anymore.
Which reminds me, I need to get the doc patch for the upgrade page in
this week.

It does make me wonder if we should direct users to the upgrade page
though, instead of the individual command pages.

> I'm not sure what "many limitations" you think pg_dumpall has that pg_dump
> doesn't.

Lack of parallel dump and restore is the biggest one.

> I do think that it might be time to reword this to recommend pg_upgrade
> first, though. ISTM that the current wording dates from when pg_upgrade
> could charitably be described as experimental.

Yah.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

Josh Berkus <josh(at)agliodbs(dot)com> writes:
> On 02/25/2014 03:41 PM, Tom Lane wrote:
>> There's a very good reason for not recommending pg_dump in this context:
>> it won't dump everything. Yeah, if you know what you're doing, you might
>> use per-database pg_dump runs plus pg_dumpall -g to catch the roles etc,
>> but we're not going to try to wedge all that info into one or two
>> sentences of release-note boilerplate. If you can get that right then
>> you don't need the release notes to remind you anyway. If you aren't
>> likely to get it right then the release notes would do you no service
>> by suggesting it.

> Right. But the fact that we don't mention it *at all* has caused some
> users to ask me if regular pg_dump doesn't work for upgrades anymore.

TBH, anybody who's asking that kind of question probably falls in the
category of "wouldn't get it right". I've heard enough bitching from
novices who thought that pg_dump would be enough to get everything out
of their now-gone database that I have no desire to encourage use of
bare pg_dump here.

(Whether we shouldn't redesign the functionality of these programs
is a different discussion. The release notes have to reflect what is,
though, not what might ideally be.)

> It does make me wonder if we should direct users to the upgrade page
> though, instead of the individual command pages.

If we had a page discussing the pros and cons of different upgrade
methods, yeah, I'd be in favor of reducing the release-note text to a
pointer to that page. I don't see such a page in a quick skim of the
fine manual's table of contents though?

regards, tom lane

On 02/25/2014 03:59 PM, Tom Lane wrote:
> If we had a page discussing the pros and cons of different upgrade
> methods, yeah, I'd be in favor of reducing the release-note text to a
> pointer to that page. I don't see such a page in a quick skim of the
> fine manual's table of contents though?

I owe an update of
http://www.postgresql.org/docs/9.3/static/upgrading.html; I think I can
easily include a discussion of the various options there.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On Tue, Feb 25, 2014 at 06:41:26PM -0500, Tom Lane wrote:
> I'm not sure what "many limitations" you think pg_dumpall has that pg_dump
> doesn't.
>
> I do think that it might be time to reword this to recommend pg_upgrade
> first, though. ISTM that the current wording dates from when pg_upgrade
> could charitably be described as experimental.

Wow, so pg_upgrade takes the lead! And from Tom too! :-)

I agree with Tom that mentioning pg_dump/restore is going to lead to
global object data loss, and throwing the users to a URL with no
explaination isn't going to help either. What we could do is to
restructure the existing text and add a link to the upgrade URL for more
details.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

On 02/25/2014 04:42 PM, Bruce Momjian wrote:
> On Tue, Feb 25, 2014 at 06:41:26PM -0500, Tom Lane wrote:
>> I'm not sure what "many limitations" you think pg_dumpall has that pg_dump
>> doesn't.
>>
>> I do think that it might be time to reword this to recommend pg_upgrade
>> first, though. ISTM that the current wording dates from when pg_upgrade
>> could charitably be described as experimental.
>
> Wow, so pg_upgrade takes the lead! And from Tom too! :-)
>
> I agree with Tom that mentioning pg_dump/restore is going to lead to
> global object data loss, and throwing the users to a URL with no
> explaination isn't going to help either. What we could do is to
> restructure the existing text and add a link to the upgrade URL for more
> details.

What I was suggesting was something like:

"Users upgrading from earlier versions will need to go through the
entire upgrade procedure, as described on our upgrade page: <link>"

The problem is that anything we say about "how to upgrade" in one short
sentence is going to confuse some people. BTW, the reason I got that
question about pg_dump was that 9.2's release notes say "pg_dump" and
9.3's say "pg_dumpall", causing users to think there's been some kind of
change.

Of course, this means I need to fix the upgrade page, and I need to
write backported versions of that fix for at least 9.1 and 9.2.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On Tue, Feb 25, 2014 at 05:05:09PM -0800, Josh Berkus wrote:
> On 02/25/2014 04:42 PM, Bruce Momjian wrote:
> > On Tue, Feb 25, 2014 at 06:41:26PM -0500, Tom Lane wrote:
> >> I'm not sure what "many limitations" you think pg_dumpall has that pg_dump
> >> doesn't.
> >>
> >> I do think that it might be time to reword this to recommend pg_upgrade
> >> first, though. ISTM that the current wording dates from when pg_upgrade
> >> could charitably be described as experimental.
> >
> > Wow, so pg_upgrade takes the lead! And from Tom too! :-)
> >
> > I agree with Tom that mentioning pg_dump/restore is going to lead to
> > global object data loss, and throwing the users to a URL with no
> > explanation isn't going to help either. What we could do is to
> > restructure the existing text and add a link to the upgrade URL for more
> > details.
>
> What I was suggesting was something like:
>
> "Users upgrading from earlier versions will need to go through the
> entire upgrade procedure, as described on our upgrade page: <link>"
>
> The problem is that anything we say about "how to upgrade" in one short
> sentence is going to confuse some people. BTW, the reason I got that
> question about pg_dump was that 9.2's release notes say "pg_dump" and
> 9.3's say "pg_dumpall", causing users to think there's been some kind of
> change.
>
> Of course, this means I need to fix the upgrade page, and I need to
> write backported versions of that fix for at least 9.1 and 9.2.

I have developed the attached patch to address the issues raised above:

o non-text output of pg_dump is mentioned
o mentions of using OID for keys is removed
o the necessity of pg_dumpall --globals-only is mentioned
o using pg_dump parallel mode rather than pg_dumpall for upgrades is mentioned
o pg_upgrade is mentioned more prominently for upgrades
o replication upgrades are in their own section

I don't think we want to mention pg_upgrade as the _primary_
major-version upgrade method. While the pg_dump upgrade section is
long, it is mostly about starting/stoping the server, moving
directories, etc, the same things you have to do for pg_upgrade, so I
just mentioned that int the pg_upgrade section. Other ideas?

I plan to apply this to head and 9.4.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

On Thu, Aug 21, 2014 at 12:18:46PM -0400, Bruce Momjian wrote:
> I have developed the attached patch to address the issues raised above:
>
> o non-text output of pg_dump is mentioned
> o mentions of using OID for keys is removed
> o the necessity of pg_dumpall --globals-only is mentioned
> o using pg_dump parallel mode rather than pg_dumpall for upgrades is mentioned
> o pg_upgrade is mentioned more prominently for upgrades
> o replication upgrades are in their own section
>
> I don't think we want to mention pg_upgrade as the _primary_
> major-version upgrade method. While the pg_dump upgrade section is
> long, it is mostly about starting/stoping the server, moving
> directories, etc, the same things you have to do for pg_upgrade, so I
> just mentioned that int the pg_upgrade section. Other ideas?
>
> I plan to apply this to head and 9.4.

Updated patch attached and applied.

Any other suggestions? Please let me know.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +