Re: tsearch V2 (Was: Re: Two weeks to feature freeze)

We have less than two weeks to feature freeze. Win32 is still in an
uncompleted state, and I haven't been able to return to it recently.
Jan is working on getting exec() working, and hopefully someone can help
me on signals. If I get those two done, I think I can tweek Win32 in
minor ways during beta.

I talked to Patrick about PITR, and with JR now back involved, he might
get it done.

Basically, we might get them both in, or it might be a disaster that we
delayed beta for one month.

I am heading to MIT now and will try to get all the outstanding patches
in within the next few days. There are only a few left, mostly ones
that appeared while I was applying the last patch backlog.

I have also been asked to complete my O'Reilly slides by the end of next
week, so I will have little time for Win32.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

> We have less than two weeks to feature freeze. Win32 is still in an
> uncompleted state, and I haven't been able to return to it recently.
> Jan is working on getting exec() working, and hopefully someone can help
> me on signals. If I get those two done, I think I can tweek Win32 in
> minor ways during beta.
>
> I talked to Patrick about PITR, and with JR now back involved, he might
> get it done.
>
> Basically, we might get them both in, or it might be a disaster that we
> delayed beta for one month.

What about the nested transaction stuff?

Do we have any "killer" features added to 7.4 that we can shout about?
There's usually been one or two in the past...?

Chris

On Thu, 2003-06-19 at 01:27, Christopher Kings-Lynne wrote:
> > We have less than two weeks to feature freeze. Win32 is still in an
> > uncompleted state, and I haven't been able to return to it recently.
> > Jan is working on getting exec() working, and hopefully someone can help
> > me on signals. If I get those two done, I think I can tweek Win32 in
> > minor ways during beta.
> >
> > I talked to Patrick about PITR, and with JR now back involved, he might
> > get it done.
> >
> > Basically, we might get them both in, or it might be a disaster that we
> > delayed beta for one month.
>
> What about the nested transaction stuff?
>
> Do we have any "killer" features added to 7.4 that we can shout about?
> There's usually been one or two in the past...?

A quick glance at the TODO list shows a number of speed improvements in
specific areas (IN, GROUP BY, Subselects in views), ARRAY improvements,
some utility command improvements / additions, and a significant
protocol update.

The protocol update may not be flashy, but it is a large step forward in
presenting a clean experience for developers using PostgreSQL (reduces
chance of rare, unexpected, and difficult to find logic errors).

If nothing else, it makes for an excellent cleanup release that rounds
off some of the sharp corners (tab completion for schema elements in
psql, schema dump in psql, fixed cluster support, transactional
truncate, alter sequence, new regex code for fast MultiByte, etc).

--
Rod Taylor <rbt(at)rbt(dot)ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

On Wed, 2003-06-18 at 21:27, Christopher Kings-Lynne wrote:
> Do we have any "killer" features added to 7.4 that we can shout about?
> There's usually been one or two in the past...?

Isn't the index growth problem solved in this release? I think that is
a killer feature that solves a big problem for alot of people.

On Wed, Jun 18, 2003 at 09:59:17PM -0400, Matthew T. O'Connor wrote:
> On Wed, 2003-06-18 at 21:27, Christopher Kings-Lynne wrote:
> > Do we have any "killer" features added to 7.4 that we can shout about?
> > There's usually been one or two in the past...?
>
> Isn't the index growth problem solved in this release? I think that is
> a killer feature that solves a big problem for alot of people.

Yes. That qualifies for "killer features" at least to some big users
here.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Granting software the freedom to evolve guarantees only different results,
not better ones." (Zygo Blaxell)

On Thu, Jun 19, 2003 at 09:27:22AM +0800, Christopher Kings-Lynne wrote:
> > We have less than two weeks to feature freeze.
>
> What about the nested transaction stuff?

I don't know if it will be completed before feature freeze... we can and
will try, of course. Sadly, like most other people, I have lots of
other things and can't give it the time I wish.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Porque francamente, si para saber manejarse a uno mismo hubiera que
rendir examen... ¿Quién es el machito que tendría carnet?" (Mafalda)

"Christopher Kings-Lynne" <chriskl(at)familyhealth(dot)com(dot)au> writes:
> What about the nested transaction stuff?

With all due respect to Alvaro et al, I can't imagine that that will
make it into 7.4. (I have no confidence that PITR or Win32 native port
will make it either...)

> Do we have any "killer" features added to 7.4 that we can shout about?

We have a lot of pretty good stuff. You're not happy that the
performance of IN (subselect) has been fixed? That btree index bloat is
fixed (at least in large part, it remains to be seen whether the field
performance is all that we need...)?

In my opinion the project is not at a state where whizzy new Features
with a capital F are going to jump out of the woodwork. We are making
good advances in performance, reliability, SQL spec compliance, and
stuff like that, but fancy-sounding bullet points are hard to come by.

I can tell you that Red Hat's CCM group (the former Ars Digita) is
waiting with bated breath for 7.4, because it fixes a number of problems
(IN-subselect being one) that prevent 7.3 from being a serious
competitor to Oracle for their platform. 7.4 is a killer release for
them, and has been since about February, and they're getting tired of
waiting. I think a lot of other people are in the same situation,
even though they may not know it ;-)

We can't slip this puppy any more --- it's time to wrap her up and
push her out.

regards, tom lane

On Thu, 19 Jun 2003, Christopher Kings-Lynne wrote:

> > We have less than two weeks to feature freeze. Win32 is still in an
> > uncompleted state, and I haven't been able to return to it recently.
> > Jan is working on getting exec() working, and hopefully someone can help
> > me on signals. If I get those two done, I think I can tweek Win32 in
> > minor ways during beta.
> >
> > I talked to Patrick about PITR, and with JR now back involved, he might
> > get it done.
> >
> > Basically, we might get them both in, or it might be a disaster that we
> > delayed beta for one month.
>
> What about the nested transaction stuff?
>
> Do we have any "killer" features added to 7.4 that we can shout about?
> There's usually been one or two in the past...?

I'm not sure if contrib/tsearch is a "killer" feature, but we hope
to submit completely new version of tsearch V2 before July 1.
Actually, we have stable code already used in some projects but
currently lacking documentation. Several people are working on tutorial,
reference guide. The problem is that Bruce seems is very overloaded and
for sure he'll have many patches close to July 1. Is it possible
to get rights to commit our changes ?

>
> Chris
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faqs/FAQ.html
>

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

Maybe a better strategy would be to get a release out soon but not wait 6
months for another release which would contain the Win32 port and the PITR
stuff (assuming those aren't done in time for this release).

Just a thought.

andrew

Tom Lane wrote:
> Robert Treat <xzilla(at)users(dot)sourceforge(dot)net> writes:
>> Well, I suppose that history has shown that waiting on specific
>> features causes trouble with postgresql development, but I don't see
>> why a release can't be based around waiting for feature x as long as
>> feature x is being actively worked on by trusted developers who have
>> an endgame in sight.
>
> We have been led down that garden path before, and it's been a losing
> proposition every time.
>

On Thursday 19 June 2003 03:27, Christopher Kings-Lynne wrote:
> Do we have any "killer" features added to 7.4 that we can shout about?

We should not forget the availability of PostgreSQL companion products, like
pgAdmin3 and PhpPgAdmin3. These two GUIs should be ready for release during
July, although I am not in the shoes of the project leaders, Dave and
Christopher and cannot speak of it officially.

They are not part of the PostgreSQL distribution, but well could be.

Providing a reliable bundle including PostgreSQL, a graphical GUI (pgAdmin3)
and a web administration interface (PhpPgAdmin3) is a big news. This will
convince entry users, who normally turn to MySQL, that PostgreSQL is the best
choice. We are not downgrading features but upgrading user needs...

When PostgreSQL win32 port is ready, all platforms and entry user needs will
be covered. Isn't it a big news? Just my 2 cents...

Best regards,
Jean-Michel POURE

On Wed, 2003-06-18 at 23:07, Tom Lane wrote:
> "Christopher Kings-Lynne" <chriskl(at)familyhealth(dot)com(dot)au> writes:
> > What about the nested transaction stuff?
>
> With all due respect to Alvaro et al, I can't imagine that that will
> make it into 7.4. (I have no confidence that PITR or Win32 native port
> will make it either...)
>

Heres hoping for win32, that is a killer feature for so many people and
we're so close to it...

> > Do we have any "killer" features added to 7.4 that we can shout about?
>
> We have a lot of pretty good stuff. You're not happy that the
> performance of IN (subselect) has been fixed? That btree index bloat is
> fixed (at least in large part, it remains to be seen whether the field
> performance is all that we need...)?
>

I think the auto vacuum work will be pretty big, and I personally think
statement level triggers are pretty important too. (Which reminds me I
really need to start banging on those a bit more.)

> In my opinion the project is not at a state where whizzy new Features
> with a capital F are going to jump out of the woodwork. We are making
> good advances in performance, reliability, SQL spec compliance, and
> stuff like that, but fancy-sounding bullet points are hard to come by.
>

You mean like that other database that just recently added transaction
support ;-)

I do see a number of capital F features that haven't been done yet,
win32, replication, nested transactions... imho those features could
each warrant a development cycle on their own.

> I can tell you that Red Hat's CCM group (the former Ars Digita) is
> waiting with bated breath for 7.4, because it fixes a number of problems
> (IN-subselect being one) that prevent 7.3 from being a serious
> competitor to Oracle for their platform. 7.4 is a killer release for
> them, and has been since about February, and they're getting tired of
> waiting. I think a lot of other people are in the same situation,
> even though they may not know it ;-)
>
> We can't slip this puppy any more --- it's time to wrap her up and
> push her out.
>

Well, I suppose that history has shown that waiting on specific features
causes trouble with postgresql development, but I don't see why a
release can't be based around waiting for feature x as long as feature x
is being actively worked on by trusted developers who have an endgame in
sight.

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert Treat <xzilla(at)users(dot)sourceforge(dot)net> writes:
> Well, I suppose that history has shown that waiting on specific features
> causes trouble with postgresql development, but I don't see why a
> release can't be based around waiting for feature x as long as feature x
> is being actively worked on by trusted developers who have an endgame in
> sight.

We have been led down that garden path before, and it's been a losing
proposition every time.

regards, tom lane

On Thu, 2003-06-19 at 06:12, Andrew Dunstan wrote:
> Maybe a better strategy would be to get a release out soon but not wait 6
> months for another release which would contain the Win32 port and the PITR
> stuff (assuming those aren't done in time for this release).

Thats what Justin was saying about this one, that it should be released
early due to win32 being complete.

I would expect, even once it compiles and runs (it doesn't yet) that it
will need some good testing before being released in any form. Platform
changes of this significance tend to introduce lots of new and
unexpected issues.

Tracking down several users for windows testing is a good idea. Rushing
it out in an unknown state to get the testing isn't such a good idea in
my mind.

--
Rod Taylor <rbt(at)rbt(dot)ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

Tom wrote:
> > Do we have any "killer" features added to 7.4 that we can shout about?
>
> We have a lot of pretty good stuff. You're not happy that the
> performance of IN (subselect) has been fixed? That btree index bloat is
> fixed...

For warehousing & reporting, "Add hash for evaluating GROUP BY aggregates"
can be a killer feature.

> 7.4 is a killer release for them, and has been since about February,
> and they're getting tired of waiting. I think a lot of other people
> are in the same situation, even though they may not know it ;-)

A nightly reporting process that starts here at midnight each night
takes about 12 hours on 7.3 and about 9 hours on 7.4alpha; possibly
thanks to HashAggregates. While this may not sound like much, it
means Marketing could see results when they arive at work, instead
of waiting for afternoon.

The perceived improvement of "ready before work" vs. "wait three hours"
is a killer feature for this system.

Ron

On Thu, 19 Jun 2003, Robert Treat wrote:

> Well, I suppose that history has shown that waiting on specific features
> causes trouble with postgresql development, but I don't see why a
> release can't be based around waiting for feature x as long as feature x
> is being actively worked on by trusted developers who have an endgame in
> sight.

Everyone has an 'endgame in sight', at least when they ask for a release
to be postponed ... but then their date keeps slipping, etc ...

The thing is, if win32 is 'that close to being finished', then as soon as
v7.4 is out, that code should be ready to throw in ... and the same for
every other features that could 'postpone a release' ...

I'd rather see the dev cycle shortened by a month, then extended ...

> We have a lot of pretty good stuff. You're not happy that the
> performance of IN (subselect) has been fixed?

All our code uses workaround now :)

> That btree index bloat is
> fixed (at least in large part, it remains to be seen whether the field
> performance is all that we need...)?

Yes, that's a good feature.

> We can't slip this puppy any more --- it's time to wrap her up and
> push her out.

OK, let's do it - we've got full support for it in phpPgAdmin already :)

Chris

On Thu, 19 Jun 2003, Andrew Dunstan wrote:

>
> Maybe a better strategy would be to get a release out soon but not wait
> 6 months for another release which would contain the Win32 port and the
> PITR stuff (assuming those aren't done in time for this release).
>
> Just a thought.

And definitely in agreement here ... I'd rather see a shortened dev cycle
prompted by a big feature being added, then delaying a release because "oh
oh, I need another few weeks" that draws out when something unexpected
happens :(

> > andrew
>
> Tom Lane wrote:
> > Robert Treat <xzilla(at)users(dot)sourceforge(dot)net> writes:
> >> Well, I suppose that history has shown that waiting on specific
> >> features causes trouble with postgresql development, but I don't see
> >> why a release can't be based around waiting for feature x as long as
> >> feature x is being actively worked on by trusted developers who have
> >> an endgame in sight.
> >
> > We have been led down that garden path before, and it's been a losing
> > proposition every time.
> >
>
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo(at)postgresql(dot)org
>

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy(at)hub(dot)org secondary: scrappy(at){freebsd|postgresql}.org

On Thu, 19 Jun 2003, Oleg Bartunov wrote:

> I'm not sure if contrib/tsearch is a "killer" feature, but we hope to
> submit completely new version of tsearch V2 before July 1. Actually, we
> have stable code already used in some projects but currently lacking
> documentation. Several people are working on tutorial, reference guide.
> The problem is that Bruce seems is very overloaded and for sure he'll
> have many patches close to July 1. Is it possible to get rights to
> commit our changes ?

Is there a strong reason why tsearch isn't in gborg?

On Thu, 19 Jun 2003, The Hermit Hacker wrote:

> On Thu, 19 Jun 2003, Oleg Bartunov wrote:
>
> > I'm not sure if contrib/tsearch is a "killer" feature, but we hope to
> > submit completely new version of tsearch V2 before July 1. Actually, we
> > have stable code already used in some projects but currently lacking
> > documentation. Several people are working on tutorial, reference guide.
> > The problem is that Bruce seems is very overloaded and for sure he'll
> > have many patches close to July 1. Is it possible to get rights to
> > commit our changes ?
>
> Is there a strong reason why tsearch isn't in gborg?
>

How gborg could help us submitting changes to pgsql CVS ?

> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

On Fri, 20 Jun 2003, Oleg Bartunov wrote:

> > Is there a strong reason why tsearch isn't in gborg?
> >
>
> How gborg could help us submitting changes to pgsql CVS ?

It wouldn't ... is there a reason why tsearch needs to be in the pgsql CVS
any more then, say, ODBC drivers, or the tcl interface, or the python
interface, or ... ?

On Fri, 20 Jun 2003, The Hermit Hacker wrote:

> On Fri, 20 Jun 2003, Oleg Bartunov wrote:
>
> > > Is there a strong reason why tsearch isn't in gborg?
> > >
> >
> > How gborg could help us submitting changes to pgsql CVS ?
>
> It wouldn't ... is there a reason why tsearch needs to be in the pgsql CVS
> any more then, say, ODBC drivers, or the tcl interface, or the python
> interface, or ... ?

because tsearch lives under pgsql source tree, in contrib directory though.
I'd never asked for any rights (I have enough beside postgresq :),
but practice of development life is so, that Bruce has no time to apply
them in reasonable time. Last patch waited about 1 month.
I'd like to
see tsearch in contrib directory because we have direct benefit from that -
rather often Tom change contrib sources to sync with his changes in
main tree. This is the only real reason why I want to stay under postgres
tree. I was afraid, and Bruce actually wrote he will be busy,
we couldn't submit our new version.

Freebsd has separate rights for core and ports.

>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
> (send "unregister YourEmailAddressHere" to majordomo(at)postgresql(dot)org)
>

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

> On Fri, 20 Jun 2003, The Hermit Hacker wrote:
> Is there a strong reason why tsearch isn't in gborg?

I think text search is a pretty important facility that should
eventually be part of the core distribution. It's more likely to get
there from contrib than from gborg ...

regards, tom lane

On Fri, 20 Jun 2003, Tom Lane wrote:

> > On Fri, 20 Jun 2003, The Hermit Hacker wrote:
> > Is there a strong reason why tsearch isn't in gborg?
>
> I think text search is a pretty important facility that should
> eventually be part of the core distribution. It's more likely to get
> there from contrib than from gborg ...

Why part of the core distribution, and not just left as a loadable module,
like it is now?

> Why part of the core distribution, and not just left as a loadable module,
> like it is now?

The day I can go 'CREATE FULLTEXT INDEX ...' just like MySQL can I will be a
very happy chappy...

Chris

The Hermit Hacker kirjutas R, 20.06.2003 kell 08:28:
> On Fri, 20 Jun 2003, Tom Lane wrote:
>
> > > On Fri, 20 Jun 2003, The Hermit Hacker wrote:
> > > Is there a strong reason why tsearch isn't in gborg?
> >
> > I think text search is a pretty important facility that should
> > eventually be part of the core distribution. It's more likely to get
> > there from contrib than from gborg ...
>
> Why part of the core distribution, and not just left as a loadable module,
> like it is now?

I remember Tom saying that builtin functions calls are a lot faster than
loadable C functions.

If that can be fixed, then it *could* stay loadable.

Also, having built-in full text indexing is very desirable. And I don't
see any even nearly as good competing fulltext indexing modules
anywhere.

If we had to move something *out* of core in order to get tsearch in,
then I personally would not mind if all geometry types go to gborg, but
I'm sure there are some users who would mind ;)

---------------
Hannu

On Thu, 19 Jun 2003, The Hermit Hacker wrote:

> On Thu, 19 Jun 2003, Andrew Dunstan wrote:
>
> >
> > Maybe a better strategy would be to get a release out soon but not wait
> > 6 months for another release which would contain the Win32 port and the
> > PITR stuff (assuming those aren't done in time for this release).
> >
> > Just a thought.
>
> And definitely in agreement here ... I'd rather see a shortened dev cycle
> prompted by a big feature being added, then delaying a release because "oh
> oh, I need another few weeks" that draws out when something unexpected
> happens :(
>
>...

I'm not sure why another delay is being considered. There's been a delay of
a week because of the server problems hasn't there and wasn't the original
delay only acceptable on the basis that that was that and there wasn't going to
be another extension?

--
Nigel Andrews

On Fri, 20 Jun 2003, Christopher Kings-Lynne wrote:

> > Why part of the core distribution, and not just left as a loadable module,
> > like it is now?
>
> The day I can go 'CREATE FULLTEXT INDEX ...' just like MySQL can I will be a
> very happy chappy...
>

with new tserach v2 we're pretty close to that day. we need more testing,
more suggestions and more documentation. There are several issues remains,
mostly with core GiST not tsearch. The most important is concurrency support !
We've several times planned to work on it, but real life is rather complex
thing :(

> Chris
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend
>

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

The Hermit Hacker wrote:

> On Thu, 19 Jun 2003, Andrew Dunstan wrote:
>
>
>>Maybe a better strategy would be to get a release out soon but not wait
>>6 months for another release which would contain the Win32 port and the
>>PITR stuff (assuming those aren't done in time for this release).
>>
>>Just a thought.
>
>
> And definitely in agreement here ... I'd rather see a shortened dev cycle
> prompted by a big feature being added, then delaying a release because "oh
> oh, I need another few weeks" that draws out when something unexpected
> happens :(

Yep, this makes sense. Looks like it'll be PostgreSQL 7.4 being all the
present improvements, but without PITR and Win32. Then, in a few months
(hopefully less than 3) we'll have PostgreSQL 8.0, with both of those
major features in it (and whatever other enhancements have been added).

The only thing that makes me wince is that we have a protocol change at
PostgreSQL 7.4 release instead of 8.0. It kind of doesn't sound right,
having a protocol change in the "7 series", when we have an "8 series"
coming up soon after.

Oh well, so it's not perfect...

;-)

Regards and best wishes,

Justin Clift

<snip>

On Fri, 2003-06-20 at 04:41, Nigel J. Andrews wrote:
>
> On Thu, 19 Jun 2003, The Hermit Hacker wrote:
>
> > On Thu, 19 Jun 2003, Andrew Dunstan wrote:
> >
> > >
> > > Maybe a better strategy would be to get a release out soon but not wait
> > > 6 months for another release which would contain the Win32 port and the
> > > PITR stuff (assuming those aren't done in time for this release).
> > >
> > > Just a thought.
> >
> > And definitely in agreement here ... I'd rather see a shortened dev cycle
> > prompted by a big feature being added, then delaying a release because "oh
> > oh, I need another few weeks" that draws out when something unexpected
> > happens :(
> >
> >...
>
> I'm not sure why another delay is being considered. There's been a delay of
> a week because of the server problems hasn't there and wasn't the original
> delay only acceptable on the basis that that was that and there wasn't going to
> be another extension?
>

There really isn't for this release. Any talk of delay is just a thought
on general policy for future releases.

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

On Fri, 2003-06-20 at 06:59, Justin Clift wrote:
> The Hermit Hacker wrote:
> > On Thu, 19 Jun 2003, Andrew Dunstan wrote:
> Yep, this makes sense. Looks like it'll be PostgreSQL 7.4 being all the
> present improvements, but without PITR and Win32. Then, in a few months
> (hopefully less than 3) we'll have PostgreSQL 8.0, with both of those
> major features in it (and whatever other enhancements have been added).
>
> The only thing that makes me wince is that we have a protocol change at
> PostgreSQL 7.4 release instead of 8.0. It kind of doesn't sound right,
> having a protocol change in the "7 series", when we have an "8 series"
> coming up soon after.
>
> Oh well, so it's not perfect...
>

...which is why I'd advocate making this release an 8.0 regardless of
win32 or pitr. I know it's old school to actually base versioning on
technical criteria instead of buzzwords, but there's no reason we have
to follow the corporate mold. Still, I'd rather not get into that debate
yet since I don't want to let the win32 guys off the hook yet!

win32 for the next release! go guys go!

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert Treat <xzilla(at)users(dot)sourceforge(dot)net> writes:
> On Fri, 2003-06-20 at 06:59, Justin Clift wrote:
>> The only thing that makes me wince is that we have a protocol change at
>> PostgreSQL 7.4 release instead of 8.0.

> ...which is why I'd advocate making this release an 8.0 regardless of
> win32 or pitr.

<shrug> ... The backend will still talk to old clients, and libpq will
still talk to old backends, so I don't think the protocol change is
really going to cause a flag day for anyone. On a technical level it's
probably not an adequate reason to call this release 8.0.

On the other hand, I dislike the notion that we would call a release 8.0
simply because it now has a native Windows port. (And if there is a
short release cycle after this one, that might be about the only big new
thing there.) Considering that we aren't going to be recommending the
Windows port for production work, we should not let the release
numbering give the impression we think it's the greatest Postgres
feature ever to come down the pike.

I'm happy to keep calling 'em 7.* for the foreseeable future, myself.

regards, tom lane

And, actually, for some reason I hadn't thought of the tsearch as being
another 'INDEX' type ... I crawl back over and be quiet now :)

Oleg, as far as commits are concerned, I have no problems with extending
the privileges to one of your guys for this, just email me seperately who,
and I'll get it setup ...

On Fri, 20 Jun 2003, Oleg Bartunov wrote:

> On Fri, 20 Jun 2003, Christopher Kings-Lynne wrote:
>
> > > Why part of the core distribution, and not just left as a loadable module,
> > > like it is now?
> >
> > The day I can go 'CREATE FULLTEXT INDEX ...' just like MySQL can I will be a
> > very happy chappy...
> >
>
> with new tserach v2 we're pretty close to that day. we need more testing,
> more suggestions and more documentation. There are several issues remains,
> mostly with core GiST not tsearch. The most important is concurrency support !
> We've several times planned to work on it, but real life is rather complex
> thing :(
>
> > Chris
> >
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 8: explain analyze is your friend
> >
>
> Regards,
> Oleg
> _____________________________________________________________
> Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
> Sternberg Astronomical Institute, Moscow University (Russia)
> Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
> phone: +007(095)939-16-83, +007(095)939-23-83
>

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy(at)hub(dot)org secondary: scrappy(at){freebsd|postgresql}.org

On Fri, 2003-06-20 at 10:42, Tom Lane wrote:
> Robert Treat <xzilla(at)users(dot)sourceforge(dot)net> writes:
> > On Fri, 2003-06-20 at 06:59, Justin Clift wrote:
> >> The only thing that makes me wince is that we have a protocol change at
> >> PostgreSQL 7.4 release instead of 8.0.
>
> > ...which is why I'd advocate making this release an 8.0 regardless of
> > win32 or pitr.
>
> <shrug> ... The backend will still talk to old clients, and libpq will
> still talk to old backends, so I don't think the protocol change is
> really going to cause a flag day for anyone. On a technical level it's
> probably not an adequate reason to call this release 8.0.
>

Can you give me an example of a technical change that would warrant a
major version bump?

> On the other hand, I dislike the notion that we would call a release 8.0
> simply because it now has a native Windows port. (And if there is a
> short release cycle after this one, that might be about the only big new
> thing there.) Considering that we aren't going to be recommending the
> Windows port for production work, we should not let the release
> numbering give the impression we think it's the greatest Postgres
> feature ever to come down the pike.
>

yep.

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert Treat <xzilla(at)users(dot)sourceforge(dot)net> writes:
> On Fri, 2003-06-20 at 10:42, Tom Lane wrote:
>> <shrug> ... The backend will still talk to old clients, and libpq will
>> still talk to old backends, so I don't think the protocol change is
>> really going to cause a flag day for anyone. On a technical level it's
>> probably not an adequate reason to call this release 8.0.

> Can you give me an example of a technical change that would warrant a
> major version bump?

Well, if we hadn't gotten the work done to make libpq still able to talk
to older backends, then we'd have had enough of a compatibility issue
that I think calling it 8.0 would have been a reasonable thing to do.

If you want a feature-with-a-capital-F reason for going to 8.0, there is
only one candidate Feature in my personal view, and that's a built-in
replication solution. That doesn't seem to be getting any nearer :-(

regards, tom lane

The Hermit Hacker <scrappy(at)postgresql(dot)org> writes:

> On Thu, 19 Jun 2003, Robert Treat wrote:
>
>> Well, I suppose that history has shown that waiting on specific features
>> causes trouble with postgresql development, but I don't see why a
>> release can't be based around waiting for feature x as long as feature x
>> is being actively worked on by trusted developers who have an endgame in
>> sight.
>
> Everyone has an 'endgame in sight', at least when they ask for a
> release to be postponed ... but then their date keeps slipping, etc
> ...
>
> The thing is, if win32 is 'that close to being finished', then as
> soon as v7.4 is out, that code should be ready to throw in ... and
> the same for every other features that could 'postpone a release'
> ...
>
> I'd rather see the dev cycle shortened by a month, then extended ...

Why couldn't you just release the win32 version of 7.4 when it was
finished. If it takes an extra month then that just gives you guys
the chance to circulate *two* press releases. The Native Win32 port
is likely to make a big enough splash all by itself.

Jason

Rod Taylor wrote:
> > Do we have any "killer" features added to 7.4 that we can shout about?
> > There's usually been one or two in the past...?
>
> A quick glance at the TODO list shows a number of speed improvements in
> specific areas (IN, GROUP BY, Subselects in views), ARRAY improvements,
> some utility command improvements / additions, and a significant
> protocol update.
>
> The protocol update may not be flashy, but it is a large step forward in
> presenting a clean experience for developers using PostgreSQL (reduces
> chance of rare, unexpected, and difficult to find logic errors).
>
> If nothing else, it makes for an excellent cleanup release that rounds
> off some of the sharp corners (tab completion for schema elements in
> psql, schema dump in psql, fixed cluster support, transactional
> truncate, alter sequence, new regex code for fast MultiByte, etc).

The problem with cleanup releases is that most of our recent releases
have been of that type. Each release is a good step forward, but I was
hoping for a set of killer features for this release.

Tom said that our low-hanging fruit is gone and only hard items are
left. This is certainly true. What is hard to accept is that those big
items take _weeks_ of focused development, and we just don't have enough
full-time developers who can spend that amount of time to do them. The
sad truth is that there is alway something _else_ to do, rather than
block out weeks to code a complex feature. And these are usually
features that can't be done incrementally, but require a huge input of
time before there is any payback.

I tried with Win32, and spent a few weeks getting us closer, but my
other work of housecleaning (email/patches/cleanup), and marketing
(speaking and tutorial preparation) just make it impossible to spend the
time needed to complete a big item. And people were rightly upset that
the patches weren't getting applied or cleanup done in a timely manner.

It is depressing.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Tom Lane wrote:
> "Christopher Kings-Lynne" <chriskl(at)familyhealth(dot)com(dot)au> writes:
> > What about the nested transaction stuff?
>
> With all due respect to Alvaro et al, I can't imagine that that will
> make it into 7.4. (I have no confidence that PITR or Win32 native port
> will make it either...)
>
> > Do we have any "killer" features added to 7.4 that we can shout about?
>
> We have a lot of pretty good stuff. You're not happy that the
> performance of IN (subselect) has been fixed? That btree index bloat is
> fixed (at least in large part, it remains to be seen whether the field
> performance is all that we need...)?
>
> In my opinion the project is not at a state where whizzy new Features
> with a capital F are going to jump out of the woodwork. We are making
> good advances in performance, reliability, SQL spec compliance, and
> stuff like that, but fancy-sounding bullet points are hard to come by.

What does bother me is that we weren't getting any closer on those
_hard_ items. At least with this release, we will be _closer_ on Win32
and PITR.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Oleg Bartunov wrote:
> > Do we have any "killer" features added to 7.4 that we can shout about?
> > There's usually been one or two in the past...?
>
> I'm not sure if contrib/tsearch is a "killer" feature, but we hope
> to submit completely new version of tsearch V2 before July 1.
> Actually, we have stable code already used in some projects but
> currently lacking documentation. Several people are working on tutorial,
> reference guide. The problem is that Bruce seems is very overloaded and
> for sure he'll have many patches close to July 1. Is it possible
> to get rights to commit our changes ?

I am sorry there has been such a delay in patches. I will try go
improve that, or someone else can apply them.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Andrew Dunstan wrote:
>
> Maybe a better strategy would be to get a release out soon but not wait 6
> months for another release which would contain the Win32 port and the PITR
> stuff (assuming those aren't done in time for this release).

What concerns me is that we thought that after 7.3, and didn't do much
work on either until we got near 7.4 release. I wonder if this is just
going to be a pattern, where these items are so large, we can't get any
motivation to focus on them until we get near the final release. I
guess if each final release gets us a little closer, eventually we will
get there, but this process is not ideal.

The big puzzle is how do you get people (including myself) motivated to
work on a feature that takes a _huge_ amount of work to see any payoff?
I would like to know. Anyone?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Jason Earl wrote:
> > I'd rather see the dev cycle shortened by a month, then extended ...
>
> Why couldn't you just release the win32 version of 7.4 when it was
> finished. If it takes an extra month then that just gives you guys
> the chance to circulate *two* press releases. The Native Win32 port
> is likely to make a big enough splash all by itself.

I am working to try to get fork/exec and signals in before the feature
freeze so a Win32 patch to 7.4 is possible.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Sat, 21 Jun 2003, Bruce Momjian wrote:

> What does bother me is that we weren't getting any closer on those
> _hard_ items. At least with this release, we will be _closer_ on Win32
> and PITR.

Maybe our problem is such a ... hatred of #ifdef? Maybe its time to go
back a bit to our roots ... get the 'experimental features' in with #ifdef
so that others have a chance to look at and work on it, and once ready for
prime time, pull the #ifdef's out ... ?

The Hermit Hacker wrote:
> On Sat, 21 Jun 2003, Bruce Momjian wrote:
>
> > What does bother me is that we weren't getting any closer on those
> > _hard_ items. At least with this release, we will be _closer_ on Win32
> > and PITR.
>
> Maybe our problem is such a ... hatred of #ifdef? Maybe its time to go
> back a bit to our roots ... get the 'experimental features' in with #ifdef
> so that others have a chance to look at and work on it, and once ready for
> prime time, pull the #ifdef's out ... ?

That's a tough call. I do worry about readability. We have made Win32
changes, and they aren't ifdefs, and we still have a running system, and
I think we can do that for PITR too. I think the big issue, which may be
your point, is to get incremental work into CVS as soon as possible so
we continue to take small steps.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Tom said that our low-hanging fruit is gone and only hard items are
> left. This is certainly true. What is hard to accept is that those big
> items take _weeks_ of focused development, and we just don't have enough
> full-time developers who can spend that amount of time to do them. The
> sad truth is that there is alway something _else_ to do, rather than
> block out weeks to code a complex feature. And these are usually
> features that can't be done incrementally, but require a huge input of
> time before there is any payback.

I spent weeks doing hash aggregates, weeks doing IN-subselect
optimization, and am in the middle of many weeks on FE/BE protocol
improvement. I am sorry that you don't see these as killer features
... but they are all things that we desperately needed to do.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom said that our low-hanging fruit is gone and only hard items are
> > left. This is certainly true. What is hard to accept is that those big
> > items take _weeks_ of focused development, and we just don't have enough
> > full-time developers who can spend that amount of time to do them. The
> > sad truth is that there is alway something _else_ to do, rather than
> > block out weeks to code a complex feature. And these are usually
> > features that can't be done incrementally, but require a huge input of
> > time before there is any payback.
>
> I spent weeks doing hash aggregates, weeks doing IN-subselect
> optimization, and am in the middle of many weeks on FE/BE protocol
> improvement. I am sorry that you don't see these as killer features
> ... but they are all things that we desperately needed to do.
>

Yes, I know they are _very_ needed, but they don't increase
functionality the way Win32 or PITR would do.

Please don't feel I am minimizing these features. If I had to choose, I
would choose those features over Win32 or PITR. It is just that I
wanted all of them. :-(

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

----- Original Message -----
From: "Bruce Momjian" <pgman(at)candle(dot)pha(dot)pa(dot)us>

> Tom said that our low-hanging fruit is gone and only hard
items are
> left. This is certainly true. What is hard to accept is that
those big
> items take _weeks_ of focused development, and we just don't
have enough
> full-time developers who can spend that amount of time to do
them. The
> sad truth is that there is alway something _else_ to do,
rather than
> block out weeks to code a complex feature. And these are
usually
> features that can't be done incrementally, but require a huge
input of
> time before there is any payback.
>
> I tried with Win32, and spent a few weeks getting us closer,
but my
> other work of housecleaning (email/patches/cleanup), and
marketing
> (speaking and tutorial preparation) just make it impossible to
spend the
> time needed to complete a big item. And people were rightly
upset that
> the patches weren't getting applied or cleanup done in a
timely manner.
>
> It is depressing.

I was disappointed that Satoshi Nagayasu's two-phase commit
patches seemed to be implicitly rejected by lack of an
enthusiastic response by any of the core members. Distributed
query (not replication) would have been a very nice feature.
It's what separates, in part, Oracle Enterprise Edition from the
Standard Edition, and it appeared someone (Satoshi Nagayasu) was
more than willing to get the ball rolling. But the flight path
bothered some I guess so we got nothin'

Mike Mascari
mascarm(at)mascari(dot)com

Tom Lane wrote:

>
>I spent weeks doing hash aggregates, weeks doing IN-subselect
>optimization, and am in the middle of many weeks on FE/BE protocol
>improvement. I am sorry that you don't see these as killer features
>... but they are all things that we desperately needed to do.
>
>
>
For me, the 7.4 enhancements are essential, the join optimizations make
the difference between "app works" and "app doesn't work", because
queries (that can't be changed) that previously ran for ages for
non-obvious reasons now speed up to <<1 second. The planner reached a
new level of maturity.

I'd recommend continuing enhancement work on pgsql, and if a majority
feels that features are so killing the version is bumped up one major.
I wouldn't see a yet another platform as a reason for this, rather
something that vastly extends the field of operation (2PC was mentioned,
maybe PITR).

Regards,
Andreas

Mike Mascari wrote:
> I was disappointed that Satoshi Nagayasu's two-phase commit
> patches seemed to be implicitly rejected by lack of an
> enthusiastic response by any of the core members. Distributed
> query (not replication) would have been a very nice feature.
> It's what separates, in part, Oracle Enterprise Edition from the
> Standard Edition, and it appeared someone (Satoshi Nagayasu) was
> more than willing to get the ball rolling. But the flight path
> bothered some I guess so we got nothin'

I sure want two-phase commit. I don't remember it as being rejected,
and we certainly need it, independent of replication.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Sun, 2003-06-22 at 07:44, Bruce Momjian wrote:
> Mike Mascari wrote:
> > I was disappointed that Satoshi Nagayasu's two-phase commit
> > patches seemed to be implicitly rejected by lack of an
> > enthusiastic response by any of the core members. Distributed

They weren't ready to be committed at the time, nor are they now.

The hardest parts are still to come (resume, forget, etc.).

I believe he is still working on the third phase:

http://snaga.org/pgsql/

--
Rod Taylor <rbt(at)rbt(dot)ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

On Sat, 21 Jun 2003, Bruce Momjian wrote:

> That's a tough call. I do worry about readability. We have made Win32
> changes, and they aren't ifdefs, and we still have a running system, and
> I think we can do that for PITR too. I think the big issue, which may be
> your point, is to get incremental work into CVS as soon as possible so
> we continue to take small steps.

Exactly ... its gotten to the point that the changes needed are large, so
ppl are waiting until its all finished before submitting it ...

On Sun, 22 Jun 2003, Bruce Momjian wrote:

> Tom Lane wrote:
> >
> > I spent weeks doing hash aggregates, weeks doing IN-subselect
> > optimization, and am in the middle of many weeks on FE/BE protocol
> > improvement. I am sorry that you don't see these as killer features
> > ... but they are all things that we desperately needed to do.
> >
>
> Yes, I know they are _very_ needed, but they don't increase
> functionality the way Win32 or PITR would do.

They don't increase functionality for whom? When someone is comparing
PostgreSQL to Oracle, as an example, for consideration in a project, I
would think that speed would be one thing that they would consider key
'functionality' in that comparison ... no?

I'll never use a Win32 port ... so Tom's work on optimizing queries is
more important to me then a Win32 port is ... 'functionality' is
completely in 'the eye of the beholder' ...

On Sun, 22 Jun 2003, Bruce Momjian wrote:

> Mike Mascari wrote:
> > I was disappointed that Satoshi Nagayasu's two-phase commit
> > patches seemed to be implicitly rejected by lack of an
> > enthusiastic response by any of the core members. Distributed
> > query (not replication) would have been a very nice feature.
> > It's what separates, in part, Oracle Enterprise Edition from the
> > Standard Edition, and it appeared someone (Satoshi Nagayasu) was
> > more than willing to get the ball rolling. But the flight path
> > bothered some I guess so we got nothin'
>
> I sure want two-phase commit. I don't remember it as being rejected,
> and we certainly need it, independent of replication.

I don't recall the patch itself :(

Mike, do you recall the date(s) for this? Reasons for rejections?

The Hermit Hacker wrote:
> On Sun, 22 Jun 2003, Bruce Momjian wrote:
>
>>Mike Mascari wrote:
>>
>>>I was disappointed that Satoshi Nagayasu's two-phase commit
>>>patches seemed to be implicitly rejected by lack of an
>>>enthusiastic response by any of the core members. Distributed
>>>query (not replication) would have been a very nice feature.
>>>It's what separates, in part, Oracle Enterprise Edition from the
>>>Standard Edition, and it appeared someone (Satoshi Nagayasu) was
>>>more than willing to get the ball rolling. But the flight path
>>>bothered some I guess so we got nothin'
>>
>>I sure want two-phase commit. I don't remember it as being rejected,
>>and we certainly need it, independent of replication.
>
> I don't recall the patch itself :(
>
> Mike, do you recall the date(s) for this? Reasons for rejections?

I choose my words poorly. A discussion arose regarding the 7.4
protocol changes. I suggested looking forward to allow for a 2PC
implementation. Satoshi Nagayasu remarked about the work done on 2PC
and posted a link to patches:

http://snaga.org/pgsql/pgsql-20021025.tgz

The thread was here:

http://archives.postgresql.org/pgsql-hackers/2002-11/msg00143.php

Various people critiqued the work that had been done - protocol change
instead of a purely statement-driven implementation, the use of 2PC
for sync. replication, etc. And that was the last (and first, IIRC)
post from Satoshi Nagayasu. I was worried that PostgreSQL lost the
opportunity to have a 2PC implementation, because no one followed up,
allowing it to die on the vine.

I have learned from Rod Taylor that lack of posts on hackers doesn't
mean lack of work:

"They weren't ready to be committed at the time, nor are they now.
The hardest parts are still to come (resume, forget, etc.).
I believe he is still working on the third phase:

http://snaga.org/pgsql/

-- Rod Taylor <rbt(at)rbt(dot)ca>"

So I stand corrected.

Mike Mascari
mascarm(at)mascari(dot)com

The Hermit Hacker wrote:
>
> And, actually, for some reason I hadn't thought of the tsearch as being
> another 'INDEX' type ... I crawl back over and be quiet now :)
>
> Oleg, as far as commits are concerned, I have no problems with extending
> the privileges to one of your guys for this, just email me seperately who,
> and I'll get it setup ...

That would help me too.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> I sure want two-phase commit. I don't remember it as being rejected,
> and we certainly need it, independent of replication.

Is 2PC a real-world solution to any real-world problem? I have never
seen a satisfactory explanation of what you do when you've reported that
you're ready to commit and no confirmation ever comes back. Sooner or
later you must violate the protocol in one direction or the other (ie,
commit without confirmation or roll back in violation of your promise
of being able to commit).

I think it's a cool-sounding phrase that does not actually work in
practice.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > I sure want two-phase commit. I don't remember it as being rejected,
> > and we certainly need it, independent of replication.
>
> Is 2PC a real-world solution to any real-world problem? I have never
> seen a satisfactory explanation of what you do when you've reported that
> you're ready to commit and no confirmation ever comes back. Sooner or
> later you must violate the protocol in one direction or the other (ie,
> commit without confirmation or roll back in violation of your promise
> of being able to commit).
>
> I think it's a cool-sounding phrase that does not actually work in
> practice.

I think 2PC can be used to build more complex features, like using dblink
for cross-db modification queries.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Sun, 22 Jun 2003, Bruce Momjian wrote:

> Tom Lane wrote:
> > Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > > I sure want two-phase commit. I don't remember it as being rejected,
> > > and we certainly need it, independent of replication.
> >
> > Is 2PC a real-world solution to any real-world problem? I have never
> > seen a satisfactory explanation of what you do when you've reported that
> > you're ready to commit and no confirmation ever comes back. Sooner or
> > later you must violate the protocol in one direction or the other (ie,
> > commit without confirmation or roll back in violation of your promise
> > of being able to commit).
> >
> > I think it's a cool-sounding phrase that does not actually work in
> > practice.
>
> I think 2PC can be used to build more complex features, like using
> dblink for cross-db modification queries.

That was my understanding too ... as soon as you try and go distributed,
you need some method of ensuring that the data is constant across them all
...

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Tom Lane wrote:
>> I think it's a cool-sounding phrase that does not actually work in
>> practice.

> I think 2PC can be used to build more complex features,

Only if it works to begin with. If it's unreliable, what are you gonna
build on it?

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom Lane wrote:
> >> I think it's a cool-sounding phrase that does not actually work in
> >> practice.
>
> > I think 2PC can be used to build more complex features,
>
> Only if it works to begin with. If it's unreliable, what are you gonna
> build on it?

The question was whether 2PC is useful. The question wasn't if an
unreliable 2PC was useful.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> The question was whether 2PC is useful. The question wasn't if an
> unreliable 2PC was useful.

My question is whether there is such a thing as reliable 2PC. I sure
don't see how you build that.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > The question was whether 2PC is useful. The question wasn't if an
> > unreliable 2PC was useful.
>
> My question is whether there is such a thing as reliable 2PC. I sure
> don't see how you build that.

Other databases use 2PC --- are you saying none of them are reliable?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
>> I sure want two-phase commit. I don't remember it as being rejected,
>> and we certainly need it, independent of replication.
>
> Is 2PC a real-world solution to any real-world problem? I have never
> seen a satisfactory explanation of what you do when you've reported that
> you're ready to commit and no confirmation ever comes back. Sooner or
> later you must violate the protocol in one direction or the other (ie,
> commit without confirmation or roll back in violation of your promise
> of being able to commit).
>
> I think it's a cool-sounding phrase that does not actually work in
> practice.

The other problem I was missing being addressed is what happens if one
promised "I can commit" and crashes? Not exactly at the time he crashes,
but more at the time he restarts? Doesn't he have to restart into
exactly that state of "I can commit", with all locks in place and yet
being able to rollback and then again ask "and what now"? I would be
surprised if said patch does that ... very *positively* surprised!

Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Other databases use 2PC --- are you saying none of them are reliable?

Perhaps they're just smarter than I am. My question stands: what do
you do when the controller doesn't respond after you promise to commit?
Without a believable answer to that, I have no confidence in the idea.

regards, tom lane

>>>>> "Bruce" == Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:

Bruce> Tom Lane wrote:
>> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes: > The question
>> was whether 2PC is useful. The question wasn't if an >
>> unreliable 2PC was useful.
>>
>> My question is whether there is such a thing as reliable 2PC.
>> I sure don't see how you build that.

Bruce> Other databases use 2PC --- are you saying none of them are
Bruce> reliable?

And they use them for both federated read/write (what you refer to as
distributed access through dblink) and for clustered configurations.

I'm not sure if I understand Tom's beef - I think he is concerned
about what happens if a subordinate does not respond to a prepare
message. I would assume that the co-ordinator would not let the commit
go through until it has received confirmations from every
subordinate. The application's commit will remain blocked against the
co-ordinator when this takes place.

That said, I agree that 2PC (and variants) is rather heavy weight when
in widely distributed configurations.

(Although I guess in practice, many people use Presumed Abort and not
vanilla 2PC as PA results in fewer log flushes for read-only
transactions.)

--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh

Jan Wieck <JanWieck(at)Yahoo(dot)com> writes:
> The other problem I was missing being addressed is what happens if one
> promised "I can commit" and crashes? Not exactly at the time he crashes,
> but more at the time he restarts? Doesn't he have to restart into
> exactly that state of "I can commit", with all locks in place

Yes, I think he does --- which adds a whole 'nother layer of complexity
and performance penalty to the thing, because all those held locks etc
have to be recorded on disk before you promise to commit.

That part is soluble in theory though, ie, I believe that it can be
done (not efficiently, but it can be done). I don't see what to do
about the no-commit-ack problem.

regards, tom lane

Sailesh Krishnamurthy <sailesh(at)cs(dot)berkeley(dot)edu> writes:
> I'm not sure if I understand Tom's beef - I think he is concerned
> about what happens if a subordinate does not respond to a prepare
> message. I would assume that the co-ordinator would not let the commit
> go through until it has received confirmations from every
> subordinate.

No. I want to know what the subordinate does when it's promised to
commit and the co-ordinator never responds. AFAICS the subordinate
is screwed --- it can't commit, and it can't abort, and it can't expect
to make progress indefinitely on other work while it's holding locks
for the not-quite-committed transaction.

regards, tom lane

> No. I want to know what the subordinate does when it's promised to
> commit and the co-ordinator never responds. AFAICS the subordinate
> is screwed --- it can't commit, and it can't abort, and it can't expect
> to make progress indefinitely on other work while it's holding locks
> for the not-quite-committed transaction.

It takes itself offline and you need to resync it later??

Chris

On Sun, 22 Jun 2003, Bruce Momjian wrote:

> Tom Lane wrote:
> > Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > > Tom Lane wrote:
> > >> I think it's a cool-sounding phrase that does not actually work in
> > >> practice.
> >
> > > I think 2PC can be used to build more complex features,
> >
> > Only if it works to begin with. If it's unreliable, what are you gonna
> > build on it?
>
> The question was whether 2PC is useful. The question wasn't if an
> unreliable 2PC was useful.

I have to back Bruce up on this one ... is there a reason why 2PC couldn't
be made reliable? I'm guessin that Oracle supports 2PC ... ? If so, is
it unreliable?

On Mon, 23 Jun 2003, Sailesh Krishnamurthy wrote:

> I'm not sure if I understand Tom's beef - I think he is concerned about
> what happens if a subordinate does not respond to a prepare message. I
> would assume that the co-ordinator would not let the commit go through
> until it has received confirmations from every subordinate. The
> application's commit will remain blocked against the co-ordinator when
> this takes place.

Wouldn't 2PC have some sort of 'heartbeat' between the co-ordinator and
subordinate? Like, if you had multiple subordinates and one crashed, the
co-ordinator would have to know that and be able to continue on, no?

On Mon, 23 Jun 2003, Christopher Kings-Lynne wrote:

> > No. I want to know what the subordinate does when it's promised to
> > commit and the co-ordinator never responds. AFAICS the subordinate
> > is screwed --- it can't commit, and it can't abort, and it can't expect
> > to make progress indefinitely on other work while it's holding locks
> > for the not-quite-committed transaction.
>
> It takes itself offline and you need to resync it later??

Hrmmm, I see Tom's point (I think!) ... but what if, for instance, the
co-ordinator crashes? From the subordinates point of view, it has the
complete transaction to commit, but the co-ordinator has gone down without
telling it to do so ...

The Hermit Hacker <scrappy(at)postgresql(dot)org> writes:
> Hrmmm, I see Tom's point (I think!) ... but what if, for instance, the
> co-ordinator crashes?

Or you just lose the network connection for awhile. The worst case
scenario I think is where the co-ordinator got everyone's promise to
commit, and told some of the subordinates to commit, but your own
response gets lost due to network failure. Now what? If you time
out and decide to abort, you're inconsistent with the other
subordinates. On the other hand, you can't commit after a timeout
either, because that loses in the other scenario (where the coordinator
didn't decide to commit). Basically, the subordinate must be willing
to hold its breath *forever*. I don't see how this can work.

regards, tom lane

>>>>> "Tom" == Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

Tom> Sailesh Krishnamurthy <sailesh(at)cs(dot)berkeley(dot)edu> writes:
>> I'm not sure if I understand Tom's beef - I think he is
>> concerned about what happens if a subordinate does not respond
>> to a prepare message. I would assume that the co-ordinator
>> would not let the commit go through until it has received
>> confirmations from every subordinate.

Tom> No. I want to know what the subordinate does when it's
Tom> promised to commit and the co-ordinator never responds.
Tom> AFAICS the subordinate is screwed --- it can't commit, and it
Tom> can't abort, and it can't expect to make progress
Tom> indefinitely on other work while it's holding locks for the
Tom> not-quite-committed transaction.

Okay I understand what you mean now.

AFAIK the general way things happen is that each site has a "recovery
procedure" that kicks in after a crash. If the co-ordinator crashes
(which could be before or after it sends out COMMIT messages to some
of the subordinates), its recovery manager will bring the system up,
read the log and ready information about all uncommitted transactions
in virtual storage.

If a Xact is in the PREPARE stage it will periodically send a message
to the co-ordinator asking about what happened to the transaction in
question. Once the co-ordinator has come back online it can respond to
the query.

Of course in the case of a co-ordinator going out of action totally
and remaining unconnected this is not a viable solution.

If you're making the case that 2PC is not viable on very wide area
networks with intermitted connectivity, I agree whole-heartedly.

That said, 2PC (and its children, PA and PC) have their place, and are
indeed used in many systems.

For instance, say you are rigging up a message queueing infrastructure
(like MQ-series) to your database (say with NOTIFY), you'd at least
like to have the db act as a co-ordinator with the MQ.

Or the parallel cluster example I gave earlier. Clustered linux boxes
are definitely here although no open-source DBMS offers a parallel
solution.

--
Pip-pip
Sailesh
http://www.cs.berkeley.edu/~sailesh

Tom Lane wrote:

> The Hermit Hacker <scrappy(at)postgresql(dot)org> writes:
>
>>Hrmmm, I see Tom's point (I think!) ... but what if, for instance, the
>>co-ordinator crashes?
>
> Or you just lose the network connection for awhile. The worst case
> scenario I think is where the co-ordinator got everyone's promise to
> commit, and told some of the subordinates to commit, but your own
> response gets lost due to network failure. Now what? If you time
> out and decide to abort, you're inconsistent with the other
> subordinates. On the other hand, you can't commit after a timeout
> either, because that loses in the other scenario (where the coordinator
> didn't decide to commit). Basically, the subordinate must be willing
> to hold its breath *forever*.

Yep. And if the cohort crashes while waiting for the coordinator to
come back on-line, if I understand the world correctly, it must be
capable of committing the database changes associated with the
COMMIT-VOTE response it supplied to the coordinator's PREPARE. It
seems this would require REDO? And yet there are thousands of
installed distributed databases running enterprises every day.

A paper on a "A New Presumed Commit Optimization for Two Phase Commit"
describes the cohort as:

"If a prepared cohort does not receive a transaction outcome message
promptly, or crashes without remembering the outcome, the cohort asks
the coordinator for the outcome. It keeps on asking until it gets an
answer. (This is the blocking aspect of 2PC.)"

I'd just like to point out that:

1) The XA interface defines a 2PC protocol library which allows
transaction managers, such as BEAS Tuxedo (and Oracle, for that
matter) to use the database in a distributed transaction. Lack of an
XA interface for PostgreSQL prohibits its use in major enterprise
applications. BEAS Tuxedo can talk to PostgreSQL, but won't allow it
to participate in a distributed tx.

2) The users of distributed databases will/should/can know that a
cohort will block waiting for the coordinator. We're not talking
asynchronous multi-master replication of 4 databases distributed over
low-speed communication lines across the country. We're talking about
the sales dept. database having a few linked tables to the accounting
dept. database, where inserts into the one result in inserts into the
other.

Mike Mascari
mascarm(at)mascari(dot)com

I wrote:

> Tom Lane wrote:
>
>>Basically, the subordinate must be willing to hold its breath *forever*.
>
>
> Yep. And if the cohort crashes while waiting for the coordinator to
> come back on-line, if I understand the world correctly, it must be
> capable of committing the database changes associated with the
> COMMIT-VOTE response it supplied to the coordinator's PREPARE. It
> seems this would require REDO? And yet there are thousands of
> installed distributed databases running enterprises every day.

Please ignore the REDO remark. It's late where I am...

Mike Mascari
mascarm(at)mascari(dot)com

Scott MArlowe wrote:
> On Sat, 21 Jun 2003, Bruce Momjian wrote:
>
>> The big puzzle is how do you get people (including myself) motivated
>> to work on a feature that takes a _huge_ amount of work to see any
>> payoff? I would like to know. Anyone?
>
> Pizza? :-)

Unfortunately it's off my diet :-(

Seriously, I think an increased willingness to share the work around a bit
would be beneficial. I know that I volunteered to work on the w32 port at a
time when I was, as they say, "between jobs". The response was not
encouraging. Now I am working again and don't have much time available.

I know you can't simply divide tasks with infinite granularity ("nine women
can't make a bay in a month"). Some tasks require one or a few people to get
done and that's all that can be done. But if she/he/they can't get it done,
it's time to send out a call for help, IMNSHO.

Not meaning to criticize - the core team does a great job. I, too, have a
tendency to overcommit and under-delegate. It's very understandable.

andrew

On Sat, 21 Jun 2003, Bruce Momjian wrote:

> Andrew Dunstan wrote:
> >
> > Maybe a better strategy would be to get a release out soon but not wait 6
> > months for another release which would contain the Win32 port and the PITR
> > stuff (assuming those aren't done in time for this release).
>
> What concerns me is that we thought that after 7.3, and didn't do much
> work on either until we got near 7.4 release. I wonder if this is just
> going to be a pattern, where these items are so large, we can't get any
> motivation to focus on them until we get near the final release. I
> guess if each final release gets us a little closer, eventually we will
> get there, but this process is not ideal.
>
> The big puzzle is how do you get people (including myself) motivated to
> work on a feature that takes a _huge_ amount of work to see any payoff?
> I would like to know. Anyone?

Pizza? :-)

On Sat, 2003-06-21 at 22:55, Bruce Momjian wrote:
> Andrew Dunstan wrote:
> >
> > Maybe a better strategy would be to get a release out soon but not wait 6
> > months for another release which would contain the Win32 port and the PITR
> > stuff (assuming those aren't done in time for this release).
>
> What concerns me is that we thought that after 7.3, and didn't do much
> work on either until we got near 7.4 release. I wonder if this is just
> going to be a pattern, where these items are so large, we can't get any
> motivation to focus on them until we get near the final release. I
> guess if each final release gets us a little closer, eventually we will
> get there, but this process is not ideal.
>
> The big puzzle is how do you get people (including myself) motivated to
> work on a feature that takes a _huge_ amount of work to see any payoff?
> I would like to know. Anyone?
>

Here's a sure to be wildly unpopular suggestion:

Make the deciding factor for the next release support of "foo" (foo can
be win32, pitr, replication, 2PC, whatever...). This would put ample
pressure on the developers of "foo" to get it done since the whole
release rides on their shoulders. It also might encourage others to help
out more since they can't get something new until "foo" is completed.
This would also prioritize said developers time on the large project
rather than other non-prioritized tasks, and it also helps to ensure a
"killer feature" for those who want such things,

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert Treat <xzilla(at)users(dot)sourceforge(dot)net> writes:
> Here's a sure to be wildly unpopular suggestion:

> Make the deciding factor for the next release support of "foo" (foo can
> be win32, pitr, replication, 2PC, whatever...).

We've done that before (see WAL in 7.1), with unhappy results. The
fundamental problem with it is that towards the end of the cycle,
other developers don't know how to schedule their time, because they
don't know when feature freeze is really going to be. People end up
twiddling their thumbs while the schedule slips a few days at a time.

The target-date-based approach we've taken in the last couple of
releases seems much more productive.

regards, tom lane

On Monday 23 June 2003 10:43 am, Tom Lane wrote:
> Robert Treat <xzilla(at)users(dot)sourceforge(dot)net> writes:
> > Here's a sure to be wildly unpopular suggestion:
> >
> > Make the deciding factor for the next release support of "foo" (foo can
> > be win32, pitr, replication, 2PC, whatever...).
>
> We've done that before (see WAL in 7.1), with unhappy results.

well, I did say it would be wildly unpopular ;-)

> The
> fundamental problem with it is that towards the end of the cycle,
> other developers don't know how to schedule their time, because they
> don't know when feature freeze is really going to be. People end up
> twiddling their thumbs while the schedule slips a few days at a time.
>

Ok, what if feature freeze comes 1 month after completion of "foo" feature.
This way the release is still feature dependent, but people arn't sitting
around day by day waiting for foo, and they also don't have to worry about
getting caught in the middle of something when foo gets done. (i'm kind of
picking 1 month arbitraily, this could be two weeks if that works better).

> The target-date-based approach we've taken in the last couple of
> releases seems much more productive.
>

productive on a small scale; for sure. productive for large scale features...
well, that's why it's being discussed.

Robert Treat

On Mon, 23 Jun 2003, Robert Treat wrote:

> > The target-date-based approach we've taken in the last couple of
> > releases seems much more productive.
> >
>
> productive on a small scale; for sure. productive for large scale
> features... well, that's why it's being discussed.

'K, but if we extend the dev cycle in order to get 'foo' in, how is that
better then having 'foo' continue to be developed thru the release and
committed in the next cycle?

If it takes foo 6 months to develop, I'd rather have the release happen
after 4 months as per normal (or close to it) and have 'foo' brought in
part way into dev cycle 2 ... at least then, if 'foo' ends up taking 7
months, we aren't delaying even further ...

Its not like our dev cycles have 'idle periods' where nothing is happening
and we're waiting for a feature to come along ... there is *alot* of
changes going on that affect alot of ppl that don't really care about
feature 'foo' coming along ...

Thank's Robert,

that was probably what Bruce needs to call me every other hour now ...

Jan

Robert Treat wrote:
> On Sat, 2003-06-21 at 22:55, Bruce Momjian wrote:
>> Andrew Dunstan wrote:
>> >
>> > Maybe a better strategy would be to get a release out soon but not wait 6
>> > months for another release which would contain the Win32 port and the PITR
>> > stuff (assuming those aren't done in time for this release).
>>
>> What concerns me is that we thought that after 7.3, and didn't do much
>> work on either until we got near 7.4 release. I wonder if this is just
>> going to be a pattern, where these items are so large, we can't get any
>> motivation to focus on them until we get near the final release. I
>> guess if each final release gets us a little closer, eventually we will
>> get there, but this process is not ideal.
>>
>> The big puzzle is how do you get people (including myself) motivated to
>> work on a feature that takes a _huge_ amount of work to see any payoff?
>> I would like to know. Anyone?
>>
>
> Here's a sure to be wildly unpopular suggestion:
>
> Make the deciding factor for the next release support of "foo" (foo can
> be win32, pitr, replication, 2PC, whatever...). This would put ample
> pressure on the developers of "foo" to get it done since the whole
> release rides on their shoulders. It also might encourage others to help
> out more since they can't get something new until "foo" is completed.
> This would also prioritize said developers time on the large project
> rather than other non-prioritized tasks, and it also helps to ensure a
> "killer feature" for those who want such things,
>
> Robert Treat

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #

On Mon, 2003-06-23 at 21:36, The Hermit Hacker wrote:
> On Mon, 23 Jun 2003, Robert Treat wrote:
>
> > > The target-date-based approach we've taken in the last couple of
> > > releases seems much more productive.
> > >
> >
> > productive on a small scale; for sure. productive for large scale
> > features... well, that's why it's being discussed.
>
> 'K, but if we extend the dev cycle in order to get 'foo' in, how is that
> better then having 'foo' continue to be developed thru the release and
> committed in the next cycle?
>

I think it makes it easier to code 'foo' since you're not coding against
(quite as much of) a moving target. I could also help developers stay
focused on particular projects since they are the "hot potato" for a
given release. It could also help people plan better since they would
know that foo is coming in the next release.

> If it takes foo 6 months to develop, I'd rather have the release happen
> after 4 months as per normal (or close to it) and have 'foo' brought in
> part way into dev cycle 2 ... at least then, if 'foo' ends up taking 7
> months, we aren't delaying even further ...
>

i'm sure everyone who doesn't want foo would agree with that position.
The counter though is those folks who did want foo but now have to wait
4-6 months for the next release since foo took a month longer to develop
than was initially planned.

> Its not like our dev cycles have 'idle periods' where nothing is happening
> and we're waiting for a feature to come along ... there is *alot* of
> changes going on that affect alot of ppl that don't really care about
> feature 'foo' coming along ...
>

this doesn't really change anything for those folks, since the only
rational is "every 6 months we do a release." ie. there are *alot* of
changes going on that affect alot of ppl that don't really care about
waiting n more months...

the whole discussion is based on how do we get big projects done when no
one is motivated to work on 'foo' until there faced with a deadline;
this idea puts the pressure on 'foo' developers from the get go. i'm not
saying this a guaranteed way to solve that problem but i think it is a
possible solution. i'm sure there will be big projects most people don't
care about (win32) and big projects most people would like (replication)
so the amount of like or dislike of the idea would be relative in
practice, the key question is would this actually motivate folks more to
get big projects finished faster? since there are only a handful of
folks who fall into that category they can decide for themselves, and if
it wouldn't motivate them, then the question can be asked again, what
would?

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Robert Treat wrote:
> the whole discussion is based on how do we get big projects done when no
> one is motivated to work on 'foo' until there faced with a deadline;
> this idea puts the pressure on 'foo' developers from the get go. i'm not
> saying this a guaranteed way to solve that problem but i think it is a
> possible solution. i'm sure there will be big projects most people don't
> care about (win32) and big projects most people would like (replication)
> so the amount of like or dislike of the idea would be relative in
> practice, the key question is would this actually motivate folks more to
> get big projects finished faster? since there are only a handful of
> folks who fall into that category they can decide for themselves, and if
> it wouldn't motivate them, then the question can be asked again, what
> would?

I can confirm that there are several people working on Win32/PITR right
now, maybe four, that wouldn't if we hadn't set the beta freeze at July
1 --- so such pressure is a real motivator --- though certainly this
isn't the way we want to motivate developers.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Tue, 24 Jun 2003, Robert Treat wrote:

> On Mon, 2003-06-23 at 21:36, The Hermit Hacker wrote:
> > On Mon, 23 Jun 2003, Robert Treat wrote:
> >
> > > > The target-date-based approach we've taken in the last couple of
> > > > releases seems much more productive.
> > > >
> > >
> > > productive on a small scale; for sure. productive for large scale
> > > features... well, that's why it's being discussed.
> >
> > 'K, but if we extend the dev cycle in order to get 'foo' in, how is that
> > better then having 'foo' continue to be developed thru the release and
> > committed in the next cycle?
> >
>
> I think it makes it easier to code 'foo' since you're not coding against
> (quite as much of) a moving target.

I *soooooo* disagree with this one ... the only way that postgresql is
going to stop being a moving target so that someone can code 'foo' is if
everyone else goes to sleep until that happens ...

> It could also help people plan better since they would know that foo is
> coming in the next release.

'K, that helps the end users, but that doesn't do anything for the person
developing 'foo' ...

> i'm sure everyone who doesn't want foo would agree with that position.
> The counter though is those folks who did want foo but now have to wait
> 4-6 months for the next release since foo took a month longer to develop
> than was initially planned.

Ya, but, based on past experience (hell, this release has been a good
example) ... you delay the release because developer of 'foo' figures "oh,
it will be ready in another month", and then something comes up that
causes that "commitment" not to happen, so we delay it "yet another
month"? And I'm not saying that the second delay was due to
mis-estimating the time needed ... suddenly getting really busy on a
contract, a day job, a death in the family, etc ... you cannot predict
what could cause a delay, or how long such a delay would take ...

> the whole discussion is based on how do we get big projects done when no
> one is motivated to work on 'foo' until there faced with a deadline;
> this idea puts the pressure on 'foo' developers from the get go. i'm not
> saying this a guaranteed way to solve that problem but i think it is a
> possible solution. i'm sure there will be big projects most people don't
> care about (win32) and big projects most people would like (replication)
> so the amount of like or dislike of the idea would be relative in
> practice, the key question is would this actually motivate folks more to
> get big projects finished faster? since there are only a handful of
> folks who fall into that category they can decide for themselves, and if
> it wouldn't motivate them, then the question can be asked again, what
> would?

First, we already seen that such a 'pressure' doesn't matter, especially
if when push comes to shove, they know we'll postpone to accommodate them
...

Second ... I don't believe the problem *is* the release cycles ... I think
the problem that needs a solution is how to "open up" these large projects
so that the deadline(s) don't fall on ones person's shoulders to get done
.. how do we encourage/promote "work groups" for the large projects?