A few user-level questions on Streaming Replication and pg_upgrade

Hi,

Here are a few questions that were asked by a customer, who are trying
to assess the pros and cons of using Postgres and its SR feature. I would
like to get an opinion of someone more involved with the community than me.

.) Will Postgres support Streaming Replication from 9.0.x to 9.1.x; i.e.
across major releases.

I am pretty sure the answer is "no, it won't", but just double-checking
with the community.

.) Is Streaming Replication supported across minor releases, in reverse
direction; e.g. 9.0.3 to 9.0.1

I think the answer is "it depends", since it would depend upon whether
any SR related bug has been fixed in the 'greater' of the minor releases.

I am assuming that smaller minor release to bigger minor release will
always be supported (e.g. 9.0.1 to 9.0.3)

.) How reliable is `pg_upgrade -c` (dry run) currently; that is, how
accurate is pg_upgrade at predicting any potential problem with the eventual
in-place upgrade.

I'd say it is as reliable as it gets since this is the official tool
supported by the project, and it should not contain any known bugs. One has
to use the latest and greatest 'minor' version of the tool for the major
release they are upgrading to, though.

I'd also like to mention a piece of information that may be surprising
to some. Per Tom at a PGCon dinner, Postgres project does not promise
continued guarantee of in-place upgrades across future major releases.
Although the project will try hard to avoid having to make any changes that
may affect in-place upgrade capability, but if a case can be made that a
feature would give a significant improvement at the cost of compromising
this capability, then the in-place upgrade capability may be forgone for
that release.

Thanks in advance,
--
Gurjeet Singh
EnterpriseDB Corporation
The Enterprise PostgreSQL Company

Gurjeet Singh wrote:
> Hi,
>
> Here are a few questions that were asked by a customer, who are trying
> to assess the pros and cons of using Postgres and its SR feature. I would
> like to get an opinion of someone more involved with the community than me.
>
> .) Will Postgres support Streaming Replication from 9.0.x to 9.1.x; i.e.
> across major releases.
>
> I am pretty sure the answer is "no, it won't", but just double-checking
> with the community.

[ CC to general removed --- emailing only hackers; cross-posting is
frowned upon. ]

Right.

> .) Is Streaming Replication supported across minor releases, in reverse
> direction; e.g. 9.0.3 to 9.0.1
>
> I think the answer is "it depends", since it would depend upon whether
> any SR related bug has been fixed in the 'greater' of the minor releases.
>
> I am assuming that smaller minor release to bigger minor release will
> always be supported (e.g. 9.0.1 to 9.0.3)

Yes. We could mention in the minor release notes if we break streaming
replication for a minor release --- or someone will tell us when we do.

> .) How reliable is `pg_upgrade -c` (dry run) currently; that is, how
> accurate is pg_upgrade at predicting any potential problem with the eventual
> in-place upgrade.
>
> I'd say it is as reliable as it gets since this is the official tool
> supported by the project, and it should not contain any known bugs. One has
> to use the latest and greatest 'minor' version of the tool for the major
> release they are upgrading to, though.

Well, we make no guarantees about the software at all, so it is hard to
make any guarantee about pg_upgrade either.

> I'd also like to mention a piece of information that may be surprising
> to some. Per Tom at a PGCon dinner, Postgres project does not promise
> continued guarantee of in-place upgrades across future major releases.
> Although the project will try hard to avoid having to make any changes that
> may affect in-place upgrade capability, but if a case can be made that a
> feature would give a significant improvement at the cost of compromising
> this capability, then the in-place upgrade capability may be forgone for
> that release.

Doesn't surprise me --- I know a time will come when we must change the
data format enough to break pg_upgrade's ability to perform major
upgrades. It is not 'if', but 'when'.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On Tue, Jul 19, 2011 at 8:45 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:

> Gurjeet Singh wrote:
>
> [ CC to general removed --- emailing only hackers; cross-posting is
> frowned upon. ]
>

I thought these questions were of interest to the general public too.

>
> > .) Is Streaming Replication supported across minor releases, in reverse
> > direction; e.g. 9.0.3 to 9.0.1
> >
> > I think the answer is "it depends", since it would depend upon
> whether
> > any SR related bug has been fixed in the 'greater' of the minor releases.
> >
> > I am assuming that smaller minor release to bigger minor release will
> > always be supported (e.g. 9.0.1 to 9.0.3)
>
> Yes.

I am assuming that's a "yes" to both the directions: older -> newer , and
newer -> older minor releases.

> We could mention in the minor release notes if we break streaming
> replication for a minor release --- or someone will tell us when we do.
>

I am pretty sure the Postgres community would notify its user base via
release notes.

> > .) How reliable is `pg_upgrade -c` (dry run) currently; that is, how
> > accurate is pg_upgrade at predicting any potential problem with the
> eventual
> > in-place upgrade.
> >
> > I'd say it is as reliable as it gets since this is the official tool
> > supported by the project, and it should not contain any known bugs. One
> has
> > to use the latest and greatest 'minor' version of the tool for the major
> > release they are upgrading to, though.
>
> Well, we make no guarantees about the software at all, so it is hard to
> make any guarantee about pg_upgrade either.
>

:) Given the BSD-style license, that's a fair point.

Thanks,
--
Gurjeet Singh
EnterpriseDB Corporation
The Enterprise PostgreSQL Company

Gurjeet Singh wrote:
> On Tue, Jul 19, 2011 at 8:45 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>
> > Gurjeet Singh wrote:
> >
> > [ CC to general removed --- emailing only hackers; cross-posting is
> > frowned upon. ]
> >
>
> I thought these questions were of interest to the general public too.

What I usually do is to discuss on hackers and post a summary of
information 'general' would find interesting.

> > > .) Is Streaming Replication supported across minor releases, in reverse
> > > direction; e.g. 9.0.3 to 9.0.1
> > >
> > > I think the answer is "it depends", since it would depend upon
> > whether
> > > any SR related bug has been fixed in the 'greater' of the minor releases.
> > >
> > > I am assuming that smaller minor release to bigger minor release will
> > > always be supported (e.g. 9.0.1 to 9.0.3)
> >
> > Yes.
>
>
> I am assuming that's a "yes" to both the directions: older -> newer , and
> newer -> older minor releases.

Yes, I believe both directions would work unless we mentioned it the
release notes, in which cases it might not work, or might work older to
newer but newer to older.

> > We could mention in the minor release notes if we break streaming
> > replication for a minor release --- or someone will tell us when we do.
> >
>
> I am pretty sure the Postgres community would notify its user base via
> release notes.

We will if we know it, but it is possible to have rare cases where we
don't find out until after the minor release.

> > > .) How reliable is `pg_upgrade -c` (dry run) currently; that is, how
> > > accurate is pg_upgrade at predicting any potential problem with the
> > eventual
> > > in-place upgrade.
> > >
> > > I'd say it is as reliable as it gets since this is the official tool
> > > supported by the project, and it should not contain any known bugs. One
> > has
> > > to use the latest and greatest 'minor' version of the tool for the major
> > > release they are upgrading to, though.
> >
> > Well, we make no guarantees about the software at all, so it is hard to
> > make any guarantee about pg_upgrade either.
> >
>
> :) Given the BSD-style license, that's a fair point.

I just found out, thanks to EnterpriseDB testing, that pg_upgrade -l
doesn't work on Windows. I will post the fix in an hour for all
released versions of pg_upgrade. You will find it entertaining that -l
works in check mode but not in actual upgrade mode.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On Wed, Jul 20, 2011 at 2:53 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> I am assuming that's a "yes" to both the directions: older -> newer , and
>> newer -> older minor releases.
>
> Yes, I believe both directions would work unless we mentioned it the
> release notes, in which cases it might not work, or might work older to
> newer but newer to older.

I don't see how this would get broken in a minor release. We'd have
to change the WAL format or the tuple format, which is definitely not
minor release material.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas wrote:
> On Wed, Jul 20, 2011 at 2:53 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> >> I am assuming that's a "yes" to both the directions: older -> newer , and
> >> newer -> older minor releases.
> >
> > Yes, I believe both directions would work unless we mentioned it the
> > release notes, in which cases it might not work, or might work older to
> > newer but newer to older.
>
> I don't see how this would get broken in a minor release. We'd have
> to change the WAL format or the tuple format, which is definitely not
> minor release material.

If there was a bug in the xlog stream content we might not be able to
fix it without breaking compatibility. Rare, but possible.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On Wed, Jul 20, 2011 at 3:29 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Robert Haas wrote:
>> On Wed, Jul 20, 2011 at 2:53 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> >> I am assuming that's a "yes" to both the directions: older -> newer , and
>> >> newer -> older minor releases.
>> >
>> > Yes, I believe both directions would work unless we mentioned it the
>> > release notes, in which cases it might not work, or might work older to
>> > newer but newer to older.
>>
>> I don't see how this would get broken in a minor release.  We'd have
>> to change the WAL format or the tuple format, which is definitely not
>> minor release material.
>
> If there was a bug in the xlog stream content we might not be able to
> fix it without breaking compatibility.  Rare, but possible.

OK, fair enough. But I think the likelihood is extremely low.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas wrote:
> On Wed, Jul 20, 2011 at 3:29 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > Robert Haas wrote:
> >> On Wed, Jul 20, 2011 at 2:53 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> >> >> I am assuming that's a "yes" to both the directions: older -> newer , and
> >> >> newer -> older minor releases.
> >> >
> >> > Yes, I believe both directions would work unless we mentioned it the
> >> > release notes, in which cases it might not work, or might work older to
> >> > newer but newer to older.
> >>
> >> I don't see how this would get broken in a minor release. ?We'd have
> >> to change the WAL format or the tuple format, which is definitely not
> >> minor release material.
> >
> > If there was a bug in the xlog stream content we might not be able to
> > fix it without breaking compatibility. ?Rare, but possible.
>
> OK, fair enough. But I think the likelihood is extremely low.

It would require a case where we couldn't distinguish a valid from an
invalid WAL entry, or the streaming used by the old server was
sufficiently broken that we didn't want to support it.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +