Re: alpha3 release schedule?

Do people want more time to play with hot standby? Otherwise alpha3
should go out on Monday or Tuesday.

>Do people want more time to play with hot standby? Otherwise alpha3
>should go out on Monday or Tuesday.
>

Well, I want to know whether the problem I refered to
in http://archives.postgresql.org/pgsql-hackers/2009-12/msg01641.php
is must-fix or not.

This problem is a corollary of the deadlock problem. This is less catstrophic
but more likely to happen.

If you leave this problem, for example, any long-running transactions,
holding any cursors in whatever tables, have a possibility of freezing
whole recovery work in HotStandby node until the transaction commit.

regards,

--
Hiroyuki YAMADA
Kokolink Corporation
yamada(at)kokolink(dot)net

On Sat, Dec 19, 2009 at 7:20 AM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> Do people want more time to play with hot standby?  Otherwise alpha3
> should go out on Monday or Tuesday.

I think we should try to wrap it promptly. It's true that Hot Standby
almost certainly has bugs and/or annoying limitations, as one would
expect with a feature of this magnitude, but I think we'll get a
better idea what they are and which ones are the most important by
getting something out there for people to test. AIUI, the reason why
Simon has been busting ass to get this committed is precisely so that
it could go into alpha3 and get more testing, and speaking in my
capacity as a guy who is anal about the schedule, I couldn't be
happier about that! Postponing alpha3 would seem to defeat the purpose
of all that hard work.

...Robert

Hiroyuki Yamada <yamada(at)kokolink(dot)net> writes:
> Well, I want to know whether the problem I refered to
> in http://archives.postgresql.org/pgsql-hackers/2009-12/msg01641.php
> is must-fix or not.

> This problem is a corollary of the deadlock problem. This is less catstrophic
> but more likely to happen.

> If you leave this problem, for example, any long-running transactions,
> holding any cursors in whatever tables, have a possibility of freezing
> whole recovery work in HotStandby node until the transaction commit.

Seems like something we should fix ASAP, but I do not see why it need
hold up an alpha release. Alpha releases are expected to have bugs,
and this one doesn't look like it would stop people from finding
other bugs.

regards, tom lane

Tom Lane wrote:
> Hiroyuki Yamada <yamada(at)kokolink(dot)net> writes:
>> Well, I want to know whether the problem I refered to
>> in http://archives.postgresql.org/pgsql-hackers/2009-12/msg01641.php
>> is must-fix or not.
>
>> This problem is a corollary of the deadlock problem. This is less catstrophic
>> but more likely to happen.
>
>> If you leave this problem, for example, any long-running transactions,
>> holding any cursors in whatever tables, have a possibility of freezing
>> whole recovery work in HotStandby node until the transaction commit.
>
> Seems like something we should fix ASAP, but I do not see why it need
> hold up an alpha release. Alpha releases are expected to have bugs,
> and this one doesn't look like it would stop people from finding
> other bugs.

yeah afaik alpha tarballs are a forma of a checkpoint at the end of a
commitfest to get people a reasonable testing target. Every feature (not
only HS) deserves getting serious testing so I vote for getting alpha3
out as soon as possible.

Stefan

>Hiroyuki Yamada <yamada(at)kokolink(dot)net> writes:
>> Well, I want to know whether the problem I refered to
>> in http://archives.postgresql.org/pgsql-hackers/2009-12/msg01641.php
>> is must-fix or not.
>
>> This problem is a corollary of the deadlock problem. This is less catstrophic
>> but more likely to happen.
>
>> If you leave this problem, for example, any long-running transactions,
>> holding any cursors in whatever tables, have a possibility of freezing
>> whole recovery work in HotStandby node until the transaction commit.
>
>Seems like something we should fix ASAP, but I do not see why it need
>hold up an alpha release. Alpha releases are expected to have bugs,
>and this one doesn't look like it would stop people from finding
>other bugs.
>

At the beginning of this commit fest, Heikki said in
http://archives.postgresql.org/pgsql-hackers/2009-11/msg00914.php

>Of course there should be several phases! We've *already* punted a lot
>of stuff from this first increment we're currently working on. The
>criteria for getting this first phase committed is: could we release
>with no further changes?

And other patches seem to be checked with similar criteria, as long as
I read mails in this list. So I wanted to know whether the problem is
must-fix, and if it is, why the criteria has been changed during the
commit fest.

Anyway, thanks for answering my question.

regards,

--
Hiroyuki YAMADA
Kokolink Corporation
yamada(at)kokolink(dot)net

On Sat, 2009-12-19 at 18:12 +0100, Stefan Kaltenbrunner wrote:
> > Seems like something we should fix ASAP, but I do not see why it
> need
> > hold up an alpha release. Alpha releases are expected to have bugs,
> > and this one doesn't look like it would stop people from finding
> > other bugs.
>
> yeah afaik alpha tarballs are a forma of a checkpoint at the end of a
> commitfest to get people a reasonable testing target. Every feature
> (not
> only HS) deserves getting serious testing so I vote for getting
> alpha3
> out as soon as possible.
>
>

+1 for both.
--
Devrim GÜNDÜZ, RHCE
Command Prompt - http://www.CommandPrompt.com
devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org Twitter: http://twitter.com/devrimgunduz

Hiroyuki Yamada wrote:
>> Hiroyuki Yamada <yamada(at)kokolink(dot)net> writes:
>>> Well, I want to know whether the problem I refered to
>>> in http://archives.postgresql.org/pgsql-hackers/2009-12/msg01641.php
>>> is must-fix or not.
>>> This problem is a corollary of the deadlock problem. This is less catstrophic
>>> but more likely to happen.
>>> If you leave this problem, for example, any long-running transactions,
>>> holding any cursors in whatever tables, have a possibility of freezing
>>> whole recovery work in HotStandby node until the transaction commit.
>> Seems like something we should fix ASAP, but I do not see why it need
>> hold up an alpha release. Alpha releases are expected to have bugs,
>> and this one doesn't look like it would stop people from finding
>> other bugs.
>
> At the beginning of this commit fest, Heikki said in
> http://archives.postgresql.org/pgsql-hackers/2009-11/msg00914.php
>
>> Of course there should be several phases! We've *already* punted a lot
>> of stuff from this first increment we're currently working on. The
>> criteria for getting this first phase committed is: could we release
>> with no further changes?
>
> And other patches seem to be checked with similar criteria, as long as
> I read mails in this list. So I wanted to know whether the problem is
> must-fix, and if it is, why the criteria has been changed during the
> commit fest.

Well, that was the criteria I used to decide whether to commit or not.
Not everyone agreed to begin with, and the reason I used that criteria
was a selfish one: I didn't want to be forced to fix loose ends after
the commitfest myself. The big reason for that was that I didn't know
how much time I would have for that. I have no complaints about Simon's
commit. Knowing that I'm not on the hook to close the loose ends, I'm
very happy that it's finally in. (That doesn't mean that I'll stop
paying attention to this patch; I will do as much as I have time to.)

Regarding the bugs you found, I put them on the TODO list at
https://wiki.postgresql.org/wiki/Hot_Standby_TODO, under the must-fix
category. I think they need to be fixed before final release, but
there's no need to delay the alpha release for them.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

>Well, that was the criteria I used to decide whether to commit or not.
>Not everyone agreed to begin with, and the reason I used that criteria
>was a selfish one: I didn't want to be forced to fix loose ends after
>the commitfest myself. The big reason for that was that I didn't know
>how much time I would have for that. I have no complaints about Simon's
>commit. Knowing that I'm not on the hook to close the loose ends, I'm
>very happy that it's finally in. (That doesn't mean that I'll stop
>paying attention to this patch; I will do as much as I have time to.)
>
>Regarding the bugs you found, I put them on the TODO list at
>https://wiki.postgresql.org/wiki/Hot_Standby_TODO, under the must-fix
>category. I think they need to be fixed before final release, but
>there's no need to delay the alpha release for them.
>

I never think it's selfish. But I see. Thanks for your kind reply.

regards,

--
Hiroyuki YAMADA
Kokolink Corporation
yamada(at)kokolink(dot)net

On Sat, 2009-12-19 at 20:59 +0200, Heikki Linnakangas wrote:

> Well, that was the criteria I used to decide whether to commit or not.
> Not everyone agreed to begin with, and the reason I used that criteria
> was a selfish one: I didn't want to be forced to fix loose ends after
> the commitfest myself. The big reason for that was that I didn't know
> how much time I would have for that. I have no complaints about Simon's
> commit. Knowing that I'm not on the hook to close the loose ends, I'm
> very happy that it's finally in. (That doesn't mean that I'll stop
> paying attention to this patch; I will do as much as I have time to.)
>
> Regarding the bugs you found, I put them on the TODO list at
> https://wiki.postgresql.org/wiki/Hot_Standby_TODO, under the must-fix
> category. I think they need to be fixed before final release, but
> there's no need to delay the alpha release for them.

Hmmm, well, if you are still paying attention you'll know that neither
of those issues are bugs in the code that was committed. One of them has
been fixed and the deadlock problem has a workaround applied.

That workaround, err... works, but I accept its not ideal. But then a
few things are not ideal and it does seem unlikely that every one of
them will have a perfect fix in the next two months.

I'll change the TODO page.

--
Simon Riggs www.2ndQuadrant.com

On Sat, 2009-12-19 at 14:20 +0200, Peter Eisentraut wrote:

> Do people want more time to play with hot standby? Otherwise alpha3
> should go out on Monday or Tuesday.

No thanks. There were no known bugs in the code I committed, excepting
the need to address VACUUM FULL. That will take longer than two days and
isn't sufficient reason to halt Alpha3, IMHO.

If others wish to revoke, then maybe we should consider a specific Alpha
version just for Hot Standby.

--
Simon Riggs www.2ndQuadrant.com

On Sat, 2009-12-19 at 20:59 +0200, Heikki Linnakangas wrote:
> I put them on the TODO list at
> https://wiki.postgresql.org/wiki/Hot_Standby_TODO, under the must-fix
> category.

I notice you also re-arranged other items on there, specifically the
notion that starting from a shutdown checkpoint is somehow important.
It's definitely not any kind of bug.

We've discussed this on-list and I've requested that you justify this.
So far, nothing you've said on that issue has been at all convincing for
me or others. The topic is already mentioned on the HS todo, since if
one person requests something we should track that, just in case others
eventually agree. But having said that, it clearly isn't a priority, so
rearranging the item like that was not appropriate, unless you were
thinking of doing it yourself, though that wasn't marked.

--
Simon Riggs www.2ndQuadrant.com

On Sat, 2009-12-19 at 23:22 +0900, Hiroyuki Yamada wrote:
> >Do people want more time to play with hot standby? Otherwise alpha3
> >should go out on Monday or Tuesday.
> >
>
> Well, I want to know whether the problem I refered to
> in http://archives.postgresql.org/pgsql-hackers/2009-12/msg01641.php
> is must-fix or not.
>
> This problem is a corollary of the deadlock problem. This is less catstrophic
> but more likely to happen.
>
> If you leave this problem, for example, any long-running transactions,
> holding any cursors in whatever tables, have a possibility of freezing
> whole recovery work in HotStandby node until the transaction commit.

You seem very insistent on bringing up problems just before release.
Almost as if you have a reason to back some other technology other than
this one.

The problem you mention here has been documented and very accessible for
months and not a single person mentioned it up to now. What's more, the
equivalent problem happens in the latest production version of Postgres
- users can delay VACUUM endlessly in just the same way, yet I've not
seen this raised as an issue in many years of using Postgres. Similarly,
there are some ways that Postgres can deadlock that it need not, yet
those negative behaviours are accepted and nobody is rushing to fix
them, nor demanding that they should be. Few things are theoretically
perfect on their first release.

--
Simon Riggs www.2ndQuadrant.com

On Sun, Dec 20, 2009 at 3:42 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> On Sat, 2009-12-19 at 20:59 +0200, Heikki Linnakangas wrote:
>> I put them on the TODO list at
>> https://wiki.postgresql.org/wiki/Hot_Standby_TODO, under the must-fix
>> category.
>
> I notice you also re-arranged other items on there, specifically the
> notion that starting from a shutdown checkpoint is somehow important.
> It's definitely not any kind of bug.
>
> We've discussed this on-list and I've requested that you justify this.
> So far, nothing you've said on that issue has been at all convincing for
> me or others. The topic is already mentioned on the HS todo, since if
> one person requests something we should track that, just in case others
> eventually agree. But having said that, it clearly isn't a priority, so
> rearranging the item like that was not appropriate, unless you were
> thinking of doing it yourself, though that wasn't marked.

This doesn't match my recollection of the previous discussion on this
topic. I am not sure that I'd call it a bug, but I'd definitely like
to see it fixed, and I think I mentioned that previously, though I
don't have the email in front ATM. I am also not aware that anyone
other than yourself has opined that we should not worry about fixing
it, although I might be wrong about that too. At any rate, "clearly
not a priority" seems like an overstatement relative to my memory of
that conversation.

...Robert

Simon Riggs wrote:
> On Sat, 2009-12-19 at 20:59 +0200, Heikki Linnakangas wrote:
>> I put them on the TODO list at
>> https://wiki.postgresql.org/wiki/Hot_Standby_TODO, under the must-fix
>> category.
>
> I notice you also re-arranged other items on there, specifically the
> notion that starting from a shutdown checkpoint is somehow important.

I didn't rearrange anything. I added that item because it was missing.

Yes, it is important in my opinion.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

>The problem you mention here has been documented and very accessible for
>months and not a single person mentioned it up to now. What's more, the
>equivalent problem happens in the latest production version of Postgres
>- users can delay VACUUM endlessly in just the same way, yet I've not
>seen this raised as an issue in many years of using Postgres. Similarly,
>there are some ways that Postgres can deadlock that it need not, yet
>those negative behaviours are accepted and nobody is rushing to fix
>them, nor demanding that they should be. Few things are theoretically
>perfect on their first release.
>

Sorry for annoying you, at the very first.

Well, this is certainly a well-known problem, but the cursor example
(or deadlock example) reveals that the problem is more severe than
it was considered before, I guess.

Following comments in backup.sgml(which are now replaced by the deadlock example)

> Waits for buffer cleanup locks do not currently result in query
> cancellation. Long waits are uncommon, though can happen in some cases
> with long running nested loop joins.

...refered only to the example where startup process should wait
until the end of one query. And long waits are assumed to be uncommon.

The cursor example shows, however, the waits can be as long as one
transaction, and occur in usual use case. FYI, I wrote a typical freeze
scenario in the mail posted in the original deadlock example thread.

Then the startup process may have to wait until the end of transaction,
and we can not expect when the pin-holder transaction ends.

Also, you mentioned the VACCUM case of the production version, but following
two problems have different impacts.

* One VACUUM process freezes until the end of a certain transaction.
* Startup process(and whole recovery work) freezes until the end of
a certain transaction.

The startup process is the last process to freeze. So I guess
this problem may become must-fix.

Anyway, the patch are committed and alpha 3 are to be released.
Do you think this problem is must-fix for the final release ?

regards,

--
Hiroyuki YAMADA
Kokolink Corporation
yamada(at)kokolink(dot)net

On Mon, 2009-12-21 at 18:42 +0900, Hiroyuki Yamada wrote:

> Do you think this problem is must-fix for the final release ?

We should be clear that this is a behaviour I told you about, not a
shock discovery by yourself. There is no permanent freeze, just a wait,
from which the Startup process wakes up at the appropriate time. There
is no crash or hang as is usually implied by the word freeze.

It remains to be seen whether this is a priority for usability
enhancement in this release. There are other issues as well and it is
doubtful that every user will be fully happy with the functionality in
this release. I will work on things in the order in which I understand
them to be important for the majority, given my time and budget
constraints and the resolvability of the issues.

When you report bugs, I say thanks. When you start agitating about
already-documented restrictions and I see which other software you
promote, I think you may have other motives. Regrettably that reduces
the weight I give your claims, in relation to other potential users.

If you genuinely care about this topic then I hope and expect that you
would start thinking about improvements, or even writing some.

I am already in touch with many potential users and will be engaging
more widely to understand users's reactions from the Alpha release.

--
Simon Riggs www.2ndQuadrant.com

On Sun, 2009-12-20 at 19:11 -0500, Robert Haas wrote:
> On Sun, Dec 20, 2009 at 3:42 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> > On Sat, 2009-12-19 at 20:59 +0200, Heikki Linnakangas wrote:
> >> I put them on the TODO list at
> >> https://wiki.postgresql.org/wiki/Hot_Standby_TODO, under the must-fix
> >> category.
> >
> > I notice you also re-arranged other items on there, specifically the
> > notion that starting from a shutdown checkpoint is somehow important.
> > It's definitely not any kind of bug.
> >
> > We've discussed this on-list and I've requested that you justify this.
> > So far, nothing you've said on that issue has been at all convincing for
> > me or others. The topic is already mentioned on the HS todo, since if
> > one person requests something we should track that, just in case others
> > eventually agree. But having said that, it clearly isn't a priority, so
> > rearranging the item like that was not appropriate, unless you were
> > thinking of doing it yourself, though that wasn't marked.
>
> This doesn't match my recollection of the previous discussion on this
> topic. I am not sure that I'd call it a bug, but I'd definitely like
> to see it fixed, and I think I mentioned that previously, though I
> don't have the email in front ATM. I am also not aware that anyone
> other than yourself has opined that we should not worry about fixing
> it, although I might be wrong about that too. At any rate, "clearly
> not a priority" seems like an overstatement relative to my memory of
> that conversation.

Please check the thread then. Nobody but me has "opined that we should
not worry about fixing it", but then nobody else other than Heikki has
suggested it is even a feature worthy of inclusion, ever. One person
agreed with my position, nobody has spoken in favour of Heikki's
position. However, I had already included the feature on the todo; it
was further down the todo before a second copy was added, second copy
now removed.

If you are saying being able to start Hot Standby from a shutdown
checkpoint is an important feature for you, then say so, and why.

Please also be careful that you don't mix this up with other
improvements, nor say "they all need fixing". This isn't a general
discussion on those points. There are other important things.

--
Simon Riggs www.2ndQuadrant.com

On 22.12.09 9:34 , Simon Riggs wrote:
> If you are saying being able to start Hot Standby from a shutdown
> checkpoint is an important feature for you, then say so, and why.

I think it's not so much an important feature but more the removal of a
footgun.

Image a reporting database where all transactions but a few daily bulk
imports are read-only. To spread the load, you do your bulk loads on the
master, but run the reporting queries against a read-only HS slave. Now
you take the master down for maintenance. Since all clients but the bulk
loader use the slave already, and since the bulk loads can be deferred
until after the maintenance window closes again, you don't actually do a
fail-over.

Now you're already pointing at your foot with the gun. All it takes to
ruin your day is *some* reason for the slave to restart. Maybe due to a
junior DBA's typo, or maybe due to a bug in postgres. Anway, once the
slave is down, it won't come up until you manage to get the master up
and running again. And this limitation is pretty surprising, since one
would assume that if the slave survives a *crash* of the master, it'd
certainly survive a simple *shutdown*.

best regards,
Florian Pflug

On Tue, Dec 22, 2009 at 8:34 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> If you are saying being able to start Hot Standby from a shutdown
> checkpoint is an important feature for you, then say so, and why.

Can you explain the consequences of missing this? It sounds to me like
if I lose my master and it happened to be while it was shut down for
whatever reason then I'll be stuck and won't be able to use my
standby. If that's true it seems like it's a major problem. Or does it
just mean I would have to follow a different procedure when failing
over?

I'm not sure if it's relevant but one thing to realize is that a lot
of MySQL people are used to doing failovers to do regular maintenance
tasks like creating indexes or making schema changes. Besides,a lot of
sites build in regular failovers to ensure that their failover
procedure works. In both cases they usually want to do a clean shut
down of the master to ensure they don't lose any transactions during
the failover.

--
greg

On Tue, 2009-12-22 at 12:32 +0100, Florian Pflug wrote:
> On 22.12.09 9:34 , Simon Riggs wrote:
> > If you are saying being able to start Hot Standby from a shutdown
> > checkpoint is an important feature for you, then say so, and why.
>
> I think it's not so much an important feature but more the removal of a
> footgun.
>
> Image a reporting database where all transactions but a few daily bulk
> imports are read-only. To spread the load, you do your bulk loads on the
> master, but run the reporting queries against a read-only HS slave. Now
> you take the master down for maintenance. Since all clients but the bulk
> loader use the slave already, and since the bulk loads can be deferred
> until after the maintenance window closes again, you don't actually do a
> fail-over.
>
> Now you're already pointing at your foot with the gun. All it takes to
> ruin your day is *some* reason for the slave to restart. Maybe due to a
> junior DBA's typo, or maybe due to a bug in postgres. Anway, once the
> slave is down, it won't come up until you manage to get the master up
> and running again. And this limitation is pretty surprising, since one
> would assume that if the slave survives a *crash* of the master, it'd
> certainly survive a simple *shutdown*.

Well, you either wait for master to come up again and restart, or you
flip into normal mode and keep running queries from there. You aren't
prevented from using the server, except by your own refusal to failover.

That's not enough for me to raise the priority for this feature.

But it was already on the list and remains there now. If someone does
add this, it will require careful thought about how to avoid introducing
further subtle ways to break HS, all of which will need testing and
re-testing to avoid regression.

So I'm not personally going to be working on it, for this release and
likely the next also, nor will I encourage others to do so, for anyone
looking to assist. There are more important things for us to do, IMHO.

--
Simon Riggs www.2ndQuadrant.com

On Tue, 2009-12-22 at 11:41 +0000, Greg Stark wrote:
> On Tue, Dec 22, 2009 at 8:34 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> > If you are saying being able to start Hot Standby from a shutdown
> > checkpoint is an important feature for you, then say so, and why.
>
> Can you explain the consequences of missing this? It sounds to me like
> if I lose my master and it happened to be while it was shut down for
> whatever reason then I'll be stuck and won't be able to use my
> standby. If that's true it seems like it's a major problem. Or does it
> just mean I would have to follow a different procedure when failing
> over?

Failover isn't prevented in this case.

If we were going to spend time on anything it would be to make failover
and switchback easier so that people aren't afraid of it. I've spent a
few weeks trying to remove the shutdown checkpoint, but no luck so far.

Switchback optimization is probably something for next release now,
unless you're looking for a project?

--
Simon Riggs www.2ndQuadrant.com

Simon Riggs wrote:
> If someone does
> add this, it will require careful thought about how to avoid introducing
> further subtle ways to break HS, all of which will need testing and
> re-testing to avoid regression.

Well, I *did* add that, but you removed it...

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Tue, 2009-12-22 at 16:09 +0200, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> > If someone does
> > add this, it will require careful thought about how to avoid introducing
> > further subtle ways to break HS, all of which will need testing and
> > re-testing to avoid regression.
>
> Well, I *did* add that, but you removed it...

It was already on there when you added the second one. It is still there
now, even after I removed the duplicate entry.

By "add" I meant to write the feature, test and then support it
afterwards, not to re-discuss editing the Wiki.

--
Simon Riggs www.2ndQuadrant.com

On 22.12.09 13:21 , Simon Riggs wrote:
> On Tue, 2009-12-22 at 12:32 +0100, Florian Pflug wrote:
>> Image a reporting database where all transactions but a few daily
>> bulk imports are read-only. To spread the load, you do your bulk
>> loads on the master, but run the reporting queries against a
>> read-only HS slave. Now you take the master down for maintenance.
>> Since all clients but the bulk loader use the slave already, and
>> since the bulk loads can be deferred until after the maintenance
>> window closes again, you don't actually do a fail-over.
>>
>> Now you're already pointing at your foot with the gun. All it
>> takes to ruin your day is *some* reason for the slave to restart.
>> Maybe due to a junior DBA's typo, or maybe due to a bug in
>> postgres. Anway, once the slave is down, it won't come up until you
>> manage to get the master up and running again. And this limitation
>> is pretty surprising, since one would assume that if the slave
>> survives a *crash* of the master, it'd certainly survive a simple
>> *shutdown*.
>
> Well, you either wait for master to come up again and restart, or you
> flip into normal mode and keep running queries from there. You aren't
> prevented from using the server, except by your own refusal to
> failover.

Very true. However, that "refusal" as you put it might actually be the
most sensible thing to do in a lot of setups. Not everyone needs extreme
up-time guarantees, and for those people setting up, testing and
*continuously* exercising fail-over is just not worth the effort.
Especially since fail-over with asynchronous replication is tricky to
get right if you want to avoid data loss.

So I still believe that there are very real use-cases for HS where this
limitation can be quite a PITA.

But you are of course free to work on whatever you feel like, and
probably need to satisfy your client's needs first. So I'm in no way
implying that this issue is a must-fix issue, or that you're in any way
obliged to take care of it. I merely wanted to make the point that there
*are* valid use-cases where this behavior is not ideal.

best regards,
Florian Pflug

On Tue, Dec 22, 2009 at 3:32 PM, Florian Pflug <fgp(dot)phlo(dot)org(at)gmail(dot)com> wrote:
>> Well, you either wait for master to come up again and restart, or you
>> flip into normal mode and keep running queries from there. You aren't
>> prevented from using the server, except by your own refusal to
>> failover.
>
> Very true. However, that "refusal" as you put it might actually be the
> most sensible thing to do in a lot of setups. Not everyone needs extreme
> up-time guarantees, and for those people setting up, testing and
> *continuously* exercising fail-over is just not worth the effort.
> Especially since fail-over with asynchronous replication is tricky to
> get right if you want to avoid data loss.

To say nothing that the replica might not be a suitable master at all.
It could be running on inferior hardware or be on a separate network
perhaps too slow to reach from production services.

HA is not the only use case for HS or even the main one in my experience

--
greg

On Tue, 2009-12-22 at 16:32 +0100, Florian Pflug wrote:

> But you are of course free to work on whatever you feel like, and
> probably need to satisfy your client's needs first.

Alluding to me as whimsical or mercenary isn't likely to change my mind.

IMHO this isn't one of the more important features, for the majority, in
this release. I do intend to check that.

If there are people that believe otherwise, knock yourselves out.

--
Simon Riggs www.2ndQuadrant.com

On Tue, 2009-12-22 at 15:38 +0000, Greg Stark wrote:
> On Tue, Dec 22, 2009 at 3:32 PM, Florian Pflug <fgp(dot)phlo(dot)org(at)gmail(dot)com> wrote:
> >> Well, you either wait for master to come up again and restart, or you
> >> flip into normal mode and keep running queries from there. You aren't
> >> prevented from using the server, except by your own refusal to
> >> failover.
> >
> > Very true. However, that "refusal" as you put it might actually be the
> > most sensible thing to do in a lot of setups. Not everyone needs extreme
> > up-time guarantees, and for those people setting up, testing and
> > *continuously* exercising fail-over is just not worth the effort.
> > Especially since fail-over with asynchronous replication is tricky to
> > get right if you want to avoid data loss.
>
> To say nothing that the replica might not be a suitable master at all.
> It could be running on inferior hardware or be on a separate network
> perhaps too slow to reach from production services.
>
> HA is not the only use case for HS or even the main one in my experience

I can invent scenarios in which all the outstanding issues give
problems. What I have to do is balance which of those is more likely and
which have useful workarounds. This is about priority and in particular,
my priority. IMHO my time would be misplaced to work upon this issue,
though I will check that other users feel that way also.

--
Simon Riggs www.2ndQuadrant.com

Simon Riggs wrote:
> By "add" I meant to write the feature, test and then support it
> afterwards, not to re-discuss editing the Wiki.

That's exactly what I meant too. I *did* write the feature, but you
removed it before committing.

I can extract the removed parts from the git repository and send you as
a new patch for review, if you'd like.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Tue, 2009-12-22 at 18:17 +0200, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> > By "add" I meant to write the feature, test and then support it
> > afterwards, not to re-discuss editing the Wiki.
>
> That's exactly what I meant too. I *did* write the feature, but you
> removed it before committing.

I removed it because you showed it wouldn't work. If you want to fix
that problem, test, commit and support it, go right ahead.

--
Simon Riggs www.2ndQuadrant.com

Simon Riggs wrote:
> On Tue, 2009-12-22 at 18:17 +0200, Heikki Linnakangas wrote:
>> Simon Riggs wrote:
>>> By "add" I meant to write the feature, test and then support it
>>> afterwards, not to re-discuss editing the Wiki.
>> That's exactly what I meant too. I *did* write the feature, but you
>> removed it before committing.
>
> I removed it because you showed it wouldn't work.

I did?

I believe this is the discussion that lead to you removing it (6th of
December, thread "Hot Standby, recent changes"):

Simon Riggs wrote:
> > On Sun, 2009-12-06 at 12:32 +0200, Heikki Linnakangas wrote:
>> > 4. Need to handle the case where master is started up with
>> > wal_standby_info=true, shut down, and restarted with
>> > wal_standby_info=false, while the standby server runs continuously. And
>> > the code in StartupXLog() to initialize recovery snapshot from a
>> > shutdown checkpoint needs to check that too.
>
> I don't really understand the use case for shutting down the server and
> then using it as a HS base backup. Why would anyone do that? Why would
> they have their server down for potentially hours, when they can take
> the backup while the server is up? If the server is idle, it can be
> up-and-idle just as easily as down-and-idle, in which case we wouldn't
> need to support this at all. Adding yards of code for this capability
> isn't important to me. I'd rather strip the whole lot out than keep
> fiddling with a low priority area. Please justify this as a real world
> solution before we continue trying to support it.

The issue I mentioned had nothing to do with starting from a shutdown
checkpoint - it's still a problem if you keep the standby running
through the restart cycle in the master) - but maybe you thought it was?
Or was there something else?

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Tue, 2009-12-22 at 18:40 +0200, Heikki Linnakangas wrote:

> The issue I mentioned had nothing to do with starting from a shutdown
> checkpoint - it's still a problem if you keep the standby running
> through the restart cycle in the master) - but maybe you thought it was?
> Or was there something else?

Strangely enough that exact same problem already happens with
archive_mode, and we see a fix coming for that soon also.

That fix takes the same approach as HS already takes. HS will flip out
when it sees the next record (checkpoint). The only way out is to
re-take base backup, just the same.

Even after that fix is applied, HS will still work as well as
archive-mode, so if anything HS is ahead of other functionality.

Fixing obscure cases where people actively try to get past configuration
options is not a priority. I'm not sure why you see it as important,
especially when you've argued we don't even need the parameter in the
first place.

You've been perfectly happy for *years* with the situation that recovery
would fail if max_prepared_transactions was not correctly. You're not
going to tell me you never noticed? Why is avoidance of obvious
misconfiguration of HS such a heavy priority when nothing else ever was?

I'm going to concentrate on fixing important issues. I'd rather you
helped with those.

--
Simon Riggs www.2ndQuadrant.com

Simon Riggs wrote:
> You've been perfectly happy for *years* with the situation that recovery
> would fail if max_prepared_transactions was not correctly. You're not
> going to tell me you never noticed? Why is avoidance of obvious
> misconfiguration of HS such a heavy priority when nothing else ever was?

That's not a priority, and I never said it was.

It almost sounds like we're in a violant agreement: this issue of
flipping wal_standby_info in the master has nothing to do with the
removal of the capability to start standby from a shutdown checkpoint.

So what *was* the reason? Was there something wrong with it? If not,
please put it back.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Simon Riggs wrote:
> On Mon, 2009-12-21 at 18:42 +0900, Hiroyuki Yamada wrote:
>
> > Do you think this problem is must-fix for the final release ?
>
> We should be clear that this is a behaviour I told you about, not a
> shock discovery by yourself. There is no permanent freeze, just a wait,
> from which the Startup process wakes up at the appropriate time. There
> is no crash or hang as is usually implied by the word freeze.
>
> It remains to be seen whether this is a priority for usability
> enhancement in this release. There are other issues as well and it is
> doubtful that every user will be fully happy with the functionality in
> this release. I will work on things in the order in which I understand
> them to be important for the majority, given my time and budget
> constraints and the resolvability of the issues.
>
> When you report bugs, I say thanks. When you start agitating about
> already-documented restrictions and I see which other software you
> promote, I think you may have other motives. Regrettably that reduces
> the weight I give your claims, in relation to other potential users.

Simon, where did this come from? "Other software?"

I think Simon's comments are way off base here and only serve to
increase tension in this discussion.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Tue, 2009-12-22 at 19:30 +0200, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> > You've been perfectly happy for *years* with the situation that recovery
> > would fail if max_prepared_transactions was not correctly. You're not
> > going to tell me you never noticed? Why is avoidance of obvious
> > misconfiguration of HS such a heavy priority when nothing else ever was?
>
> That's not a priority, and I never said it was.
>
> It almost sounds like we're in a violant agreement: this issue of
> flipping wal_standby_info in the master has nothing to do with the
> removal of the capability to start standby from a shutdown checkpoint.

I removed the capability to start at shutdown checkpoints because you
said it would cause this bug. That gives two choices: fix the bug,
remove the feature. I don't think it is a priority to support that
feature, so I removed it in favour of other work.

I will work on issues in priority order and this was already on the list
and remains so. I don't have endless time, so realistically, given its
current priority it is unlikely to be addressed in this release.

--
Simon Riggs www.2ndQuadrant.com

On 22.12.09 16:45 , Simon Riggs wrote:
> On Tue, 2009-12-22 at 16:32 +0100, Florian Pflug wrote:
>> But you are of course free to work on whatever you feel like, and
>> probably need to satisfy your client's needs first.
>
> Alluding to me as whimsical or mercenary isn't likely to change my
> mind.

Simon, you *completely* miss-understood my last paragraph!

I never intended to call you whimsical or mercenary, and I honestly
don't believe I did. The only thing I "alluded" to you was seeing HS
mostly as a solution for HA setups, whereas I felt that I has quite a
few use-cases beside that. Plus that your view of what the important
use-cases are is influenced by the projects you usually work on, and
that it's perfectly reasonable for your priorities to reflect that view.

None of this was meant as an insult of any kind.

best regards,
Florian Pflug

On Tue, 2009-12-22 at 19:53 +0100, Florian Pflug wrote:

> None of this was meant as an insult of any kind.

Then I apologise completely.

I've clearly been working too hard and will retire for some rest (even
though that is not listed as a task on the Wiki).

--
Simon Riggs www.2ndQuadrant.com

On Dec 22, 2009, at 11:02 AM, Simon Riggs wrote:

> I've clearly been working too hard and will retire for some rest (even
> though that is not listed as a task on the Wiki).

Someone add it!

David

On Tue, Dec 22, 2009 at 11:04:29AM -0800, David Wheeler wrote:
> On Dec 22, 2009, at 11:02 AM, Simon Riggs wrote:
>
> > I've clearly been working too hard and will retire for some rest (even
> > though that is not listed as a task on the Wiki).
>
> Someone add it!

Done! :)

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate