Getting a move on for 8.2 beta

(I've already bounced this off the core committee, but it's time for
wider discussion.)

September is upon us and it doesn't seem like we are ready to ship
a beta. I think it's time to start making some hard choices.

In the main code I see these outstanding features/patches:

* bitmap indexes
* updatable views
* GUC variable reload + refactoring (previously applied and reverted)
* plpython improvements (needs review by someone who knows plpython)
* plpgsql improvements for returning record types
* patch to build on VC
* make libpq default client_encoding setting from locale?

In contrib we've got:

* new ISBN/etc module
* hstore (finally proposed for inclusion)
* new sslinfo module
* pgstattuple changes
* removing the deadwood

These are not all the open issues, for sure, but these are what I think
have to be resolved before we can go beta. Everything else is in the
category of "bug fixes", and none of it seems like a showstopper (in
fact a lot of the small stuff on my todo list is bugs reported against
8.1, so those certainly are not beta-stoppers).

My feeling is that we ought to bounce bitmap indexes and updatable views
as not being ready, accept all the contrib stuff, and try to get the
other items done in time for a beta at, say, the end of next week.

I'm not thrilled about postponing those two large items till 8.3, but
we are a month past feature freeze and we do not have credible patches
in hand for either. The alternative is to be willing to slip the
schedule until whenever they are ready, and we have learned the hard way
that that's not good project management.

Comments?

regards, tom lane

> My feeling is that we ought to bounce bitmap indexes and updatable views
> as not being ready, accept all the contrib stuff, and try to get the
> other items done in time for a beta at, say, the end of next week.
>
> I'm not thrilled about postponing those two large items till 8.3, but
> we are a month past feature freeze and we do not have credible patches
> in hand for either.

I know there is a lot of "backend" work that has been done for 8.2 but
the two features you are suggesting we bounce, are probably the most
visible of marketable features we will have for this release. Especially
updateable views.

> The alternative is to be willing to slip the
> schedule until whenever they are ready, and we have learned the hard way
> that that's not good project management.

Well I don't like the idea either, but if it isn't done, it isn't done.
How often do we hear the complaint about software that is shipped before
it is done?

Outside of feature freeze, we really haven't had AFAICS a "solid"
release schedule. Is it going to be push Beta for a month? Does that
push us till the end of the year?

I don't know which is the better solution, but you asked for comments :)

Sincerely,

Joshua D. Drake

>
> Comments?
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq
>

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Tom Lane wrote:

> September is upon us and it doesn't seem like we are ready to ship
> a beta. I think it's time to start making some hard choices.
>
> In the main code I see these outstanding features/patches:
>
> * bitmap indexes
> * updatable views

IMHO these two should be bounced. They are big features that still seem
to need a lot of work and will probably take long before they are ready
for inclusion. Interested parties should continue working on them with
an eye to be included *at the start* of the 8.3 development cycle.

> * GUC variable reload + refactoring (previously applied and reverted)
> * plpython improvements (needs review by someone who knows plpython)
> * plpgsql improvements for returning record types
> * patch to build on VC
> * make libpq default client_encoding setting from locale?

> In contrib we've got:
>
> * new ISBN/etc module
> * hstore (finally proposed for inclusion)
> * new sslinfo module
> * pgstattuple changes
> * removing the deadwood

These all seem to be manageable within reasonable time.

I can help with the contrib stuff.

> My feeling is that we ought to bounce bitmap indexes and updatable views
> as not being ready, accept all the contrib stuff, and try to get the
> other items done in time for a beta at, say, the end of next week.

+1.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Tom,

> September is upon us and it doesn't seem like we are ready to ship
> a beta. I think it's time to start making some hard choices.
>
> In the main code I see these outstanding features/patches:
>
> * bitmap indexes
> * updatable views
> * GUC variable reload + refactoring (previously applied and reverted)
> * plpython improvements (needs review by someone who knows plpython)
> * plpgsql improvements for returning record types
> * patch to build on VC

What's VC?

> * make libpq default client_encoding setting from locale?
>
> In contrib we've got:
>
> * new ISBN/etc module
> * hstore (finally proposed for inclusion)
> * new sslinfo module
> * pgstattuple changes
> * removing the deadwood

This is just a delete. I've fixed the pgFoundry permissions issues and
will be loading the last CVS snapshot today. At that point, Bruce can
delete stuff.

Can you be a little clearer on which things are non-working patches, and
which things are pending due to lack of review? I'm all for bouncing
stuff that's not ready to go, but bouncing stuff because we haven't
recruited enough code reviewers is just bad practice.

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Josh Berkus <josh(at)agliodbs(dot)com> writes:
> Can you be a little clearer on which things are non-working patches, and
> which things are pending due to lack of review?

The only things I currently want to reject as not having provided a
timely patch are bitmap indexes and updatable views. The other stuff
I mentioned is in need of review, but I don't currently have a reason
to think it can't get in.

> I'm all for bouncing
> stuff that's not ready to go, but bouncing stuff because we haven't
> recruited enough code reviewers is just bad practice.

Well, it's taken us the full month to get this far through the queue, so
I'd sure like to have more people on board reviewing next time. Alvaro
helped but I miss Neil Conway, and some other people have helped in the
past. However --- the present situation has nothing to do with lack of
reviewers, it's lack of time to finish the patches.

regards, tom lane

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> My feeling is that we ought to bounce bitmap indexes and updatable views
> as not being ready, accept all the contrib stuff, and try to get the
> other items done in time for a beta at, say, the end of next week.

I agree with bouncing these to get the beta out, though I hate having
to wait another year for them. If things change in some way and we have
the opportunity to fit one in this time, I'd rather see updatable views
than bitmap indexes, personally.

Overall, I really think 8.2 is going to be an excellent release. I wish
autovacuum could have been enabled by default and I'd just like to ask,
now and I'll try to remember again once 8.2 is out, please let's turn it
on by default for 8.3 (and early on so we get some good testing of it).

Thanks,

Stephen

"Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
>> My feeling is that we ought to bounce bitmap indexes and updatable views
>> as not being ready, accept all the contrib stuff, and try to get the
>> other items done in time for a beta at, say, the end of next week.

> I know there is a lot of "backend" work that has been done for 8.2 but
> the two features you are suggesting we bounce, are probably the most
> visible of marketable features we will have for this release. Especially
> updateable views.

Josh said nearly the same in core's discussion, but come now. The
updateable-view patch (in its current form) does not offer *one single
thing* you can't do today, indeed for many PG releases past; it just
saves writing out some tedious rules. If that's the most marketable
feature in 8.2 we're already in trouble. I don't think it is anyway.
Looking at the CVS logs I see

INSERT/UPDATE/DELETE RETURNING
VALUES lists
CREATE INDEX CONCURRENTLY
pl/pgsql debugger (OK, not in core, but available)
restartable WAL recovery
constraint exclusion works on UPDATE/DELETE queries
multiple-argument aggregate functions
SQL2003-standard statistical aggregates
COPY (SELECT ...) TO ...
pg_dump selectivity options
customizable timezone names
GIN indexes
FILLFACTOR for indexes and tables

and that's just back to the beginning of July, so it probably represents
about a quarter of the work that's gone into 8.2, but I got tired of
reading the logs at that point.

We have this discussion every single release cycle: "hey, if we wait X
amount more time then we can have cool features Y and Z". But there's
always another X, Y, and Z. More importantly, this view ignores the
very real disadvantages of not getting already-finished cool features
A through W out to our users sooner.

regards, tom lane

>
> Josh said nearly the same in core's discussion, but come now. The
> updateable-view patch (in its current form) does not offer *one single
> thing* you can't do today, indeed for many PG releases past; it just
> saves writing out some tedious rules.

Sure, I can write the rules, you can write the rules but we can do
rollup queries using unions too, but we still don't have rollup :)

Perception is everything and we don't have Updateable views until we
don't have to write those rules (yes, its stupid)...

> If that's the most marketable
> feature in 8.2 we're already in trouble. I don't think it is anyway.
> Looking at the CVS logs I see
>
> INSERT/UPDATE/DELETE RETURNING
> VALUES lists
> CREATE INDEX CONCURRENTLY
> constraint exclusion works on UPDATE/DELETE queries
> multiple-argument aggregate functions
> SQL2003-standard statistical aggregates
> COPY (SELECT ...) TO ...
> pg_dump selectivity options
> customizable timezone names
> GIN indexes
> FILLFACTOR for indexes and tables

Let's take multiple-argument aggregate functions for example... The
marketable portion of that is limited. 85% of the people we are
marketing too don't even know what that means. Heck, I haven't even run
into the pervious limitation :)

Versus SQL2003-standard statistical aggregates. That wins hands down
just because of the SQL2003-standard part of the string :)

COPY (SELECT...) TO, is only marketable to current PostgreSQL users.
Those who are not users don't know the limitation and it is likely not
something they would look for when evaluating PostgreSQL.

For my current customers the things that are the most marketable (note
current customers) is probably:

* CREATE INDEX CONCURRENTLY (this is a huge in production boon)

* GIN indexes (after explanation)

* constraint exclusion works on UPDATE/DELETE queries (We are using
alot of tp now, so this is great)

* Bitmap Indexes

It does not mean all those features are useful, they definitely are. I
am just trying to look at it from at:

WHIZ* BANG* POW* perspective.

What makes the analyst (and I know this isn't the communities problem)
say, "Holy crap batman, we need to be talking to more people about this
database". I think those would be Updateable views, Bitmap Indexes and
the SQL2003-standard statistical aggregates.

Sincerely,

Joshua D. Drake

>
> and that's just back to the beginning of July, so it probably represents
> about a quarter of the work that's gone into 8.2, but I got tired of
> reading the logs at that point.
>
> We have this discussion every single release cycle: "hey, if we wait X
> amount more time then we can have cool features Y and Z". But there's
> always another X, Y, and Z. More importantly, this view ignores the
> very real disadvantages of not getting already-finished cool features
> A through W out to our users sooner.
>
> regards, tom lane
>

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Josh Berkus wrote:
> Tom,
>
> > September is upon us and it doesn't seem like we are ready to ship
> > a beta. I think it's time to start making some hard choices.
> >
> > In the main code I see these outstanding features/patches:
> >
> > * bitmap indexes
> > * updatable views
> > * GUC variable reload + refactoring (previously applied and reverted)
> > * plpython improvements (needs review by someone who knows plpython)
> > * plpgsql improvements for returning record types
> > * patch to build on VC
>
> What's VC?

MS Visual C++

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Tom Lane wrote:

> > I'm all for bouncing
> > stuff that's not ready to go, but bouncing stuff because we haven't
> > recruited enough code reviewers is just bad practice.
>
> Well, it's taken us the full month to get this far through the queue, so
> I'd sure like to have more people on board reviewing next time. Alvaro
> helped but I miss Neil Conway, and some other people have helped in the
> past. However --- the present situation has nothing to do with lack of
> reviewers, it's lack of time to finish the patches.

Also, for next time it would be good to have the reviewers got through
the patches *before* feature freeze, so that developers get early
feedback. (Which in turn means more time to finish the patches).

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Bruce,

> > What's VC?
>
> MS Visual C++

Given that this could lead to us recruiting more developers out of our
Windows user base, I'd prioritize it.

Overall, I think this whole process is pointing up that there's a problem
with our historic Feature Freeze+1 month|beta|RCs cycle. Given the number
of developers and companies involved and the increasing size of the code,
we need to develop a new schedule that allows us to get draft patches (or
at least full specifications) eariler than the month before feature
freeze. In short,the issue is that Feature Freeze isn't just Feature
Freeze, it's "final patch submission".

If you look at the two "incomplete" patches, and the "misfired" one
(Bitmaps, Updatable Views, and WITH RECURSIVE) all of them are patches
where the submitter had been working on them months ago, and might have
made the release (or let us know they weren't on schedule) if we'd held
them to an earlier deadline. As I recall, it was the same situation last
year: getting large patches dropped on us out of the blue the week before
feature freeze, which then didnt' make it in.

Therefore I propose that we adopt the following schedule for 8.3, assuming
an September 1 Beta date:

June 1: Specification Freeze: specifications for all new features due
July 1: Feature Freeze: Draft patches and any minor tweaks due
August 1: Final (completed, mostly debugged) patches due
September 1: First Beta

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Joshua D. Drake wrote:

> It does not mean all those features are useful, they definitely are. I
> am just trying to look at it from at:
>
> WHIZ* BANG* POW* perspective.

Holy crap, Batman! This database can do

INSERT INTO foo VALUES (1,1, 'so long'), (42, 2, 'and thanks'),
(142857, 3, 'for all the fish')

now! We should be talking to more people about that!

That will make some MySQL users happy at least a bit :-)

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Tom Lane wrote:
> Josh Berkus <josh(at)agliodbs(dot)com> writes:
> > Can you be a little clearer on which things are non-working patches, and
> > which things are pending due to lack of review?
>
> The only things I currently want to reject as not having provided a
> timely patch are bitmap indexes and updatable views. The other stuff
> I mentioned is in need of review, but I don't currently have a reason
> to think it can't get in.
>
> > I'm all for bouncing
> > stuff that's not ready to go, but bouncing stuff because we haven't
> > recruited enough code reviewers is just bad practice.
>
> Well, it's taken us the full month to get this far through the queue, so
> I'd sure like to have more people on board reviewing next time. Alvaro
> helped but I miss Neil Conway, and some other people have helped in the
> past. However --- the present situation has nothing to do with lack of
> reviewers, it's lack of time to finish the patches.

I did try to get us additional help in reviewing. Neil was unavailable,
and Alvaro could only give part of his time.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Joshua D. Drake wrote:
> Let's take multiple-argument aggregate functions for example... The
> marketable portion of that is limited. 85% of the people we are
> marketing too don't even know what that means. Heck, I haven't even
> run into the pervious limitation :)

As far as market analysis goes, I'd guess that 100% of the people who
already use PostgreSQL by definition don't need any new features and
would rather see incremental improvements across the board, which is
what this or any release would give them.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Josh Berkus wrote:
> Bruce,
>
> > > What's VC?
> >
> > MS Visual C++
>
> Given that this could lead to us recruiting more developers out of our
> Windows user base, I'd prioritize it.
>
> Overall, I think this whole process is pointing up that there's a problem
> with our historic Feature Freeze+1 month|beta|RCs cycle. Given the number
> of developers and companies involved and the increasing size of the code,
> we need to develop a new schedule that allows us to get draft patches (or
> at least full specifications) eariler than the month before feature
> freeze. In short,the issue is that Feature Freeze isn't just Feature
> Freeze, it's "final patch submission".
>
> If you look at the two "incomplete" patches, and the "misfired" one
> (Bitmaps, Updatable Views, and WITH RECURSIVE) all of them are patches
> where the submitter had been working on them months ago, and might have
> made the release (or let us know they weren't on schedule) if we'd held
> them to an earlier deadline. As I recall, it was the same situation last
> year: getting large patches dropped on us out of the blue the week before
> feature freeze, which then didnt' make it in.
>
> Therefore I propose that we adopt the following schedule for 8.3, assuming
> an September 1 Beta date:
>
> June 1: Specification Freeze: specifications for all new features due
> July 1: Feature Freeze: Draft patches and any minor tweaks due
> August 1: Final (completed, mostly debugged) patches due
> September 1: First Beta

No rejiggering is going to get people to complete things they didn't
complete under the old system. The plan you list above is what we did
for this release.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Tom Lane wrote:
> Well, it's taken us the full month to get this far through the queue,
> so I'd sure like to have more people on board reviewing next time.
> Alvaro helped but I miss Neil Conway, and some other people have
> helped in the past. However --- the present situation has nothing to
> do with lack of reviewers, it's lack of time to finish the patches.

Not that I've had particularly plenty of time to give, but what's
concerning me is that while we're moving toward an increasingly rigid
process, we don't really have the process support to go with it. How
would anyone even know what patches are pending review? Something to
think about ...

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Bruce Momjian <bruce(at)momjian(dot)us> writes:
> Tom Lane wrote:
>> Well, it's taken us the full month to get this far through the queue, so
>> I'd sure like to have more people on board reviewing next time. Alvaro
>> helped but I miss Neil Conway, and some other people have helped in the
>> past. However --- the present situation has nothing to do with lack of
>> reviewers, it's lack of time to finish the patches.

> I did try to get us additional help in reviewing. Neil was unavailable,
> and Alvaro could only give part of his time.

It strikes me that setting feature freeze in midsummer might not be the
best strategy for having manpower available to review --- people tend to
be on vacation in August. Maybe the answer is just to move the dates a
bit one way or the other.

regards, tom lane

> Therefore I propose that we adopt the following schedule for 8.3, assuming
> an September 1 Beta date:
>
> June 1: Specification Freeze: specifications for all new features due

I would suggest the above be:

June 1: Specification Freeze: specifications for all new features due &
approved.

The last thing we want it to spend all of june arguing about how
something should be implemented and thus no real coding gets done.

> July 1: Feature Freeze: Draft patches and any minor tweaks due
> August 1: Final (completed, mostly debugged) patches due
> September 1: First Beta

other then that w00t!

Joshua D. Drake

>

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Josh Berkus <josh(at)agliodbs(dot)com> writes:
> Bruce,
>>> What's VC?
>>
>> MS Visual C++

> Given that this could lead to us recruiting more developers out of our
> Windows user base, I'd prioritize it.

Yeah, that's why I listed it as a major item. I haven't had a chance to
look at the patch, but if it's not too ugly I would like to get it in
this time rather than next --- that could lead directly to having more
people available next time.

> If you look at the two "incomplete" patches, and the "misfired" one
> (Bitmaps, Updatable Views, and WITH RECURSIVE) all of them are patches
> where the submitter had been working on them months ago, and might have
> made the release (or let us know they weren't on schedule) if we'd held
> them to an earlier deadline.

Perhaps, but I'm not sure what we can or should do about it. Moving
deadlines up will either create a dead zone where we all sit around
twiddling our thumbs, or people will keep on coding till the last minute
anyway.

I think having a few patches that don't make the deadline isn't a bad
thing: it means we didn't have people sitting idle. It's not like the
work will go to waste --- those things can still get in in the next
devel cycle.

regards, tom lane

Alvaro Herrera wrote:
> Joshua D. Drake wrote:
>
>> It does not mean all those features are useful, they definitely are. I
>> am just trying to look at it from at:
>>
>> WHIZ* BANG* POW* perspective.
>
> Holy crap, Batman! This database can do
>
> INSERT INTO foo VALUES (1,1, 'so long'), (42, 2, 'and thanks'),
> (142857, 3, 'for all the fish')
>
> now! We should be talking to more people about that!
>
>
> That will make some MySQL users happy at least a bit :-)

To be fair, it will make some of our customers happy to.. they can now
get rid of the weird union hack they had to do instead :)

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

On Fri, 1 Sep 2006, Tom Lane wrote:

> My feeling is that we ought to bounce bitmap indexes and updatable views
> as not being ready, accept all the contrib stuff, and try to get the
> other items done in time for a beta at, say, the end of next week.

For what it's worth, Jie and I hope to have finished the bitmap streaming
this weekend which is the only outstanding issue with the code as it
currently stands. I now realise (nothing like hindsight) that we should
have posted a patch before we looked at streaming bitmaps because they've
proved more fiddly to implement than I thought :-(. At that point, we'd
more or less addressed the issues I'd raised when I posted at feature
freeze time.

Thanks,

Gavin

Bruce, Tom, all:

> No rejiggering is going to get people to complete things they didn't
> complete under the old system.

It'll help the new people. A lot of people -- if not most -- submitting
their first major patch to PostgreSQL dramatically underestimate the
amount of fix-up that's going to be required, and assume that there won't
be a spec discussion, which there often is. By getting them to submit a
little at a time, *earlier*, we can avoid doing those things at the last
minute.

Alternately, we can just make sure that first-time patchers have mentors
who check progress well before feature freeze.

> The plan you list above is what we did
> for this release.

No, it's not. There's a bunch of patches which we had nothing on -- not
spec, not design draft, not anything -- until we got them on July 20th.
Our current system is to have only one deadline, at which point you're
expected to have 85% of the patch done and up to PostgreSQL standards.
That's quite a bit of "jumping in with both feet" for a newbie.

> I did try to get us additional help in reviewing. Neil was unavailable,
> and Alvaro could only give part of his time

Asking two people is not exactly an all-out effort to get reviewers.

> It strikes me that setting feature freeze in midsummer might not be the
> best strategy for having manpower available to review --- people tend to
> be on vacation in August. Maybe the answer is just to move the dates a
> bit one way or the other.

We've discussed that issue before, yes. Since we're proposing a new
roadmap process for 8.3, and will likely be dealing with a lot of major
patches, maybe that's the release to delay?

However, as PR maven I do want to point out that doing the final release in
December would be a bad idea. Hard to get news coverage. Also, we'd have
the same issue with people being gone.

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Gavin Sherry <swm(at)linuxworld(dot)com(dot)au> writes:
> On Fri, 1 Sep 2006, Tom Lane wrote:
>> My feeling is that we ought to bounce bitmap indexes and updatable views
>> as not being ready, accept all the contrib stuff, and try to get the
>> other items done in time for a beta at, say, the end of next week.

> For what it's worth, Jie and I hope to have finished the bitmap streaming
> this weekend which is the only outstanding issue with the code as it
> currently stands.

AFAIR, you told me "it'll be done this weekend" last week. And the week
before.

In any case, it's going to be a large complicated patch, and if I spend
all next week reviewing it then I won't make any progress on the other
items on my to-do list. At some point we have to say "this has slipped
too far, we have to hold it over for 8.3".

regards, tom lane

Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

>> If you look at the two "incomplete" patches, and the "misfired" one
>> (Bitmaps, Updatable Views, and WITH RECURSIVE) all of them are patches
>> where the submitter had been working on them months ago, and might have
>> made the release (or let us know they weren't on schedule) if we'd held
>> them to an earlier deadline.
>
> Perhaps, but I'm not sure what we can or should do about it. Moving
> deadlines up will either create a dead zone where we all sit around
> twiddling our thumbs, or people will keep on coding till the last minute
> anyway.

I don't think that's what would happen at all. In fact what I think would
happen is that it would free up people like you to work on stuff you're
interested in instead of reviewing patches.

As an extreme example consider the new Linux release cycle. They have a
non-freeze period of a couple weeks, followed by months of frozen time. Users
who want to try out new features on different hardware can do so and often
turn up problems developers reviewing the patches missed. Developers spend the
months of frozen time working on new patches which, if they're ready on time,
all go into the queue in the first few weeks of a new release. If they miss
the window they'll make the next one.

As a result they don't get things like what we saw with 8.0 where major new
features like PITR, nested transactions and so on all hit the tree in the
final days before the freeze. Instead they go in early and if they turn out to
be immature they get pulled long before the actual release.

I would love to see the bitmap indexes and updatable views get merged into the
tree just weeks after the release comes out rather than disappear again until
just before the next release. If I were a user depending on these new features
it would give me a hell of a lot more confidence if people could say they've
been in the CVS version for months and nobody's had problems with them than to
be told they were reviewed by one person three weeks ago and nobody else has
had much of a chance to use it.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com

Josh Berkus <josh(at)agliodbs(dot)com> writes:
> No, it's not. There's a bunch of patches which we had nothing on -- not
> spec, not design draft, not anything -- until we got them on July 20th.
> Our current system is to have only one deadline,

Well, no, it's not. We have told people till we're blue in the face
"post early, post often". Now I will plead guilty to not always having
spent as much time giving feedback on draft patches as I should've, but
the process is pretty clear. As I see it the main problem is people
undertaking patches off in corners somewhere rather than discussing
their work on the mailing lists while they do it.

If you want more process in this we could institute rules like "feature
patches will be rejected out of hand if there wasn't previously a spec
proposal posted to pghackers". I don't think this is really a good
idea, but I'm not sure what else to do.

regards, tom lane

Tom Lane wrote:
> Well, no, it's not. We have told people till we're blue in the face
> "post early, post often". Now I will plead guilty to not always
> having spent as much time giving feedback on draft patches as I
> should've, but the process is pretty clear. As I see it the main
> problem is people undertaking patches off in corners somewhere rather
> than discussing their work on the mailing lists while they do it.

Again, process support. If all we can offer people is to post
multi-megabyte patches to the mailing list every month, that totally
doesn't help. We'd need ways to track the progress on these things:
what was the specification for that patch, where was the discussion on
it, what has changed in the patch since the last time, since the time
before last time, what is left to be done, who has worked on it, etc.
Figuring out the answer to those questions from a mailing list archive
is tedious to the point that no one wants to do it.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Tom Lane wrote:
> > Well, it's taken us the full month to get this far through the queue,
> > so I'd sure like to have more people on board reviewing next time.
> > Alvaro helped but I miss Neil Conway, and some other people have
> > helped in the past. However --- the present situation has nothing to
> > do with lack of reviewers, it's lack of time to finish the patches.
>
> Not that I've had particularly plenty of time to give, but what's
> concerning me is that while we're moving toward an increasingly rigid
> process, we don't really have the process support to go with it. How
> would anyone even know what patches are pending review? Something to
> think about ...

They are in the patches queue?

http://momjian.postgresql.org/cgi-bin/pgpatches

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Josh Berkus wrote:
> Bruce, Tom, all:
>
> > No rejiggering is going to get people to complete things they didn't
> > complete under the old system.
>
> It'll help the new people. A lot of people -- if not most -- submitting
> their first major patch to PostgreSQL dramatically underestimate the
> amount of fix-up that's going to be required, and assume that there won't
> be a spec discussion, which there often is. By getting them to submit a
> little at a time, *earlier*, we can avoid doing those things at the last
> minute.

That is in the developer's FAQ.

> Alternately, we can just make sure that first-time patchers have mentors
> who check progress well before feature freeze.
>
> > The plan you list above is what we did
> > for this release.
>
> No, it's not. There's a bunch of patches which we had nothing on -- not
> spec, not design draft, not anything -- until we got them on July 20th.
> Our current system is to have only one deadline, at which point you're
> expected to have 85% of the patch done and up to PostgreSQL standards.
> That's quite a bit of "jumping in with both feet" for a newbie.

Right. The developer's FAQ says they should follow a process. Making
another process doesn't mean they will follow that either.

>
> > I did try to get us additional help in reviewing. Neil was unavailable,
> > and Alvaro could only give part of his time
>
> Asking two people is not exactly an all-out effort to get reviewers.

Well, not sure what else I can do. Those are the people who used to
help out a lot.

> > It strikes me that setting feature freeze in midsummer might not be the
> > best strategy for having manpower available to review --- people tend to
> > be on vacation in August. Maybe the answer is just to move the dates a
> > bit one way or the other.
>
> We've discussed that issue before, yes. Since we're proposing a new
> roadmap process for 8.3, and will likely be dealing with a lot of major
> patches, maybe that's the release to delay?

Moving it away from summer might help, yea.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Tom Lane wrote:
> Josh Berkus <josh(at)agliodbs(dot)com> writes:
> > No, it's not. There's a bunch of patches which we had nothing on -- not
> > spec, not design draft, not anything -- until we got them on July 20th.
> > Our current system is to have only one deadline,
>
> Well, no, it's not. We have told people till we're blue in the face
> "post early, post often". Now I will plead guilty to not always having
> spent as much time giving feedback on draft patches as I should've, but
> the process is pretty clear. As I see it the main problem is people
> undertaking patches off in corners somewhere rather than discussing
> their work on the mailing lists while they do it.
>
> If you want more process in this we could institute rules like "feature
> patches will be rejected out of hand if there wasn't previously a spec
> proposal posted to pghackers". I don't think this is really a good
> idea, but I'm not sure what else to do.

Well stated.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Peter Eisentraut wrote:
> Tom Lane wrote:
> > Well, no, it's not. We have told people till we're blue in the face
> > "post early, post often". Now I will plead guilty to not always
> > having spent as much time giving feedback on draft patches as I
> > should've, but the process is pretty clear. As I see it the main
> > problem is people undertaking patches off in corners somewhere rather
> > than discussing their work on the mailing lists while they do it.
>
> Again, process support. If all we can offer people is to post
> multi-megabyte patches to the mailing list every month, that totally
> doesn't help. We'd need ways to track the progress on these things:
> what was the specification for that patch, where was the discussion on
> it, what has changed in the patch since the last time, since the time
> before last time, what is left to be done, who has worked on it, etc.
> Figuring out the answer to those questions from a mailing list archive
> is tedious to the point that no one wants to do it.

Uh, Tom has been tracking Gavin on the bitmap patch every week for
weeks, and I pummelled EnterpriseDB/Jonah over the recursive query
patch. Neither effort was very fruitful, but tracking wasn't what made
them fail. I am not saying tracking is wrong, but rather tracking would
not have helped make these things happen faster.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> Peter Eisentraut wrote:
>> Tom Lane wrote:
>>> Well, it's taken us the full month to get this far through the queue,
>>> so I'd sure like to have more people on board reviewing next time.
>>> Alvaro helped but I miss Neil Conway, and some other people have
>>> helped in the past. However --- the present situation has nothing to
>>> do with lack of reviewers, it's lack of time to finish the patches.
>> Not that I've had particularly plenty of time to give, but what's
>> concerning me is that while we're moving toward an increasingly rigid
>> process, we don't really have the process support to go with it. How
>> would anyone even know what patches are pending review? Something to
>> think about ...
>
> They are in the patches queue?
>
> http://momjian.postgresql.org/cgi-bin/pgpatches

Side note::

Perhaps this would be better served from the main website under the
developer and community sections.

Joshua D. Drake

>

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Joshua D. Drake wrote:
> Bruce Momjian wrote:
> > Peter Eisentraut wrote:
> >> Tom Lane wrote:
> >>> Well, it's taken us the full month to get this far through the queue,
> >>> so I'd sure like to have more people on board reviewing next time.
> >>> Alvaro helped but I miss Neil Conway, and some other people have
> >>> helped in the past. However --- the present situation has nothing to
> >>> do with lack of reviewers, it's lack of time to finish the patches.
> >> Not that I've had particularly plenty of time to give, but what's
> >> concerning me is that while we're moving toward an increasingly rigid
> >> process, we don't really have the process support to go with it. How
> >> would anyone even know what patches are pending review? Something to
> >> think about ...
> >
> > They are in the patches queue?
> >
> > http://momjian.postgresql.org/cgi-bin/pgpatches
>
> Side note::
>
> Perhaps this would be better served from the main website under the
> developer and community sections.

It pulls my a mailbox file I use, and it does instant updates as soon as
I change it. It is a URL. Why do people care where it is?

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

>>> They are in the patches queue?
>>>
>>> http://momjian.postgresql.org/cgi-bin/pgpatches
>> Side note::
>>
>> Perhaps this would be better served from the main website under the
>> developer and community sections.
>
> It pulls my a mailbox file I use, and it does instant updates as soon as
> I change it. It is a URL. Why do people care where it is?

I would think that it would be easier to find :)

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Bruce Momjian wrote:
> Uh, Tom has been tracking Gavin on the bitmap patch every week for
> weeks, and I pummelled EnterpriseDB/Jonah over the recursive query
> patch. Neither effort was very fruitful, but tracking wasn't what made
> them fail. I am not saying tracking is wrong, but rather tracking would
> not have helped make these things happen faster.
>
>

It would make the process more transparent, which is something several
people have expressed a desire for.

cheers

andrew

Gregory Stark <stark(at)enterprisedb(dot)com> writes:
> As an extreme example consider the new Linux release cycle. They have a
> non-freeze period of a couple weeks, followed by months of frozen time. Users
> who want to try out new features on different hardware can do so and often
> turn up problems developers reviewing the patches missed. Developers spend the
> months of frozen time working on new patches which, if they're ready on time,
> all go into the queue in the first few weeks of a new release. If they miss
> the window they'll make the next one.

That doesn't seem particularly appealing for our purposes. What it
sounds like is that all the development happens somewhere outside the
main tree, and every so often everybody tries to land a bunch of large
patches at once. That's going to be a mess if the patches interact at
all, and it certainly isn't going to address my primary concern of
encouraging a publicly-visible development process instead of having
people hiding in a corner until they can drop a large patch on us.

> I would love to see the bitmap indexes and updatable views get merged
> into the tree just weeks after the release comes out rather than
> disappear again until just before the next release.

Here I agree - partly. With some continuing effort these patches could
be in fine shape to apply by the time we branch CVS for 8.3 development.
However, our traditional approach to beta period is that it's supposed
to be a "quiet time" where people focus on testing, debugging, and
documentation. Shepherding people who are doing major development
during that period is likely to be a serious distraction from making the
release ready and high-quality.

regards, tom lane

Bruce Momjian wrote:
> It pulls my a mailbox file I use, and it does instant updates as soon
> as I change it. It is a URL. Why do people care where it is?

The complaint has been that not enough people help with reviewing
patches. That would indicate that the problem is not location but
scalability. If everything has to go through your private mailbox,
then it's not a very obvious process to outsiders.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Joshua D. Drake wrote:
>
> >>> They are in the patches queue?
> >>>
> >>> http://momjian.postgresql.org/cgi-bin/pgpatches
> >> Side note::
> >>
> >> Perhaps this would be better served from the main website under the
> >> developer and community sections.
> >
> > It pulls my a mailbox file I use, and it does instant updates as soon as
> > I change it. It is a URL. Why do people care where it is?
>
> I would think that it would be easier to find :)

We have a link from the developer's page:

http://www.postgresql.org/developer/roadmap

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > It pulls my a mailbox file I use, and it does instant updates as soon
> > as I change it. It is a URL. Why do people care where it is?
>
> The complaint has been that not enough people help with reviewing
> patches. That would indicate that the problem is not location but
> scalability. If everything has to go through your private mailbox,
> then it's not a very obvious process to outsiders.

Well, you can grab items from there and apply them. I will remove them
from the mailbox when I see the commit. You have another system that
will not be significantly more work? Any you can apply them right from
the email lists so they never get to the queue.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> Peter Eisentraut wrote:
> > Bruce Momjian wrote:
> > > It pulls my a mailbox file I use, and it does instant updates as soon
> > > as I change it. It is a URL. Why do people care where it is?
> >
> > The complaint has been that not enough people help with reviewing
> > patches. That would indicate that the problem is not location but
> > scalability. If everything has to go through your private mailbox,
> > then it's not a very obvious process to outsiders.
>
> Well, you can grab items from there and apply them. I will remove them
> from the mailbox when I see the commit. You have another system that
> will not be significantly more work? Any you can apply them right from
> the email lists so they never get to the queue.

Another novel idea: have a "bugtracker" of sorts where selected people
can add "feature requests" and append patches to them. There, people
could add "fake" feature requests describing the stuff they are working
on, and publish the patches they have for them.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Bruce Momjian wrote:
> Uh, Tom has been tracking Gavin on the bitmap patch every week for
> weeks, and I pummelled EnterpriseDB/Jonah over the recursive query
> patch.

Great, but where is this documented, so others know about this?

> Neither effort was very fruitful, but tracking wasn't what
> made them fail. I am not saying tracking is wrong, but rather
> tracking would not have helped make these things happen faster.

The fallacy here is assuming that all these things should be
single-person tasks. As long as we only have one coder and
one "manager", we don't need much process support, but then we're
pretty nearly at the point we're now, where two or three people review
patches while the rest just sits around and wonders what this feature
freeze thing is supposed to be about.

I can tell you plenty of stories about the updatable views patch. One
month after feature freeze, we notice that we didn't even have an
accepted design specification. I'm sure it was posted sometime, but
how do we find it now? People complain unjustly that the patch was
posted at the last minute, but in fact updated patches and information
have been posted regularly for more than one year. But it's impossible
to tie these things together unless you are mailing list crawling
software with artificial intelligence capabilities. And during the
last two weeks, no make that six months, Bernd has spent half his time
analyzing and reverting breakage that well-meaning reviewers had
injected into his patch, with the other half possibly spent keeping the
patch up to date with the moving development tree.

There is, of course, no silver bullet. But more successful involvement
of people who are not in the inner circle needs more support in many
ways.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Bruce Momjian wrote:
> Well, you can grab items from there and apply them. I will remove
> them from the mailbox when I see the commit. You have another
> system that will not be significantly more work? Any you can apply
> them right from the email lists so they never get to the queue.

What the current system lacks is documentation, which makes the process
behind those lists known to others, and the ability to interact with
the system. If I can't see any feedback in the system for the actions
I do, then the system is not very useful to me.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Alvaro Herrera wrote:
> I've been bitten by having stuff rejected because there was no
> immediate use for them; so I had to maintain them in a private tree,
> where they were subsequently broken by someone else's parallel
> development.

A better version control system would surely seem to help here and in
other instances.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > Uh, Tom has been tracking Gavin on the bitmap patch every week for
> > weeks, and I pummelled EnterpriseDB/Jonah over the recursive query
> > patch.
>
> Great, but where is this documented, so others know about this?

I see no value in documenting it.

> > Neither effort was very fruitful, but tracking wasn't what
> > made them fail. I am not saying tracking is wrong, but rather
> > tracking would not have helped make these things happen faster.
>
> The fallacy here is assuming that all these things should be
> single-person tasks. As long as we only have one coder and
> one "manager", we don't need much process support, but then we're
> pretty nearly at the point we're now, where two or three people review
> patches while the rest just sits around and wonders what this feature
> freeze thing is supposed to be about.
>
> I can tell you plenty of stories about the updatable views patch. One
> month after feature freeze, we notice that we didn't even have an
> accepted design specification. I'm sure it was posted sometime, but
> how do we find it now? People complain unjustly that the patch was
> posted at the last minute, but in fact updated patches and information
> have been posted regularly for more than one year. But it's impossible
> to tie these things together unless you are mailing list crawling
> software with artificial intelligence capabilities. And during the
> last two weeks, no make that six months, Bernd has spent half his time
> analyzing and reverting breakage that well-meaning reviewers had
> injected into his patch, with the other half possibly spent keeping the
> patch up to date with the moving development tree.
>
> There is, of course, no silver bullet. But more successful involvement
> of people who are not in the inner circle needs more support in many
> ways.

I do things only if others do not. If committers applied patches as
they came in, the patch queue would be empty, and if others tracked open
issues, I wouldn't have to.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Alvaro Herrera wrote:
> Another novel idea: have a "bugtracker" of sorts where selected
> people can add "feature requests" and append patches to them. There,
> people could add "fake" feature requests describing the stuff they
> are working on, and publish the patches they have for them.

Certainly, much of this discussion is trying to yell "tracking
software". Track patches, track comments, track open items, and so on.
This doesn't have to be the same as a bug tracker, though.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > Well, you can grab items from there and apply them. I will remove
> > them from the mailbox when I see the commit. You have another
> > system that will not be significantly more work? Any you can apply
> > them right from the email lists so they never get to the queue.
>
> What the current system lacks is documentation, which makes the process
> behind those lists known to others, and the ability to interact with
> the system. If I can't see any feedback in the system for the actions
> I do, then the system is not very useful to me.

What do you want to happen? Just apply the patch? You want to log into
a system to say you applied it? If people are not doing applications
when they have no overhead, I doubt they will do it when there are
additional steps.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Peter Eisentraut wrote:
> Alvaro Herrera wrote:
> > Another novel idea: have a "bugtracker" of sorts where selected
> > people can add "feature requests" and append patches to them. There,
> > people could add "fake" feature requests describing the stuff they
> > are working on, and publish the patches they have for them.
>
> Certainly, much of this discussion is trying to yell "tracking
> software". Track patches, track comments, track open items, and so on.
> This doesn't have to be the same as a bug tracker, though.

OK, why do we use email then? Move it all into a tracker? If not, who
is going to move info from email to the tracking system?

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On 9/1/06, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> I pummelled Jonah over the recursive query patch.

He did. Trust me on this... think I still have some bruises too :)

> Neither effort was very fruitful, but tracking wasn't what made
> them fail. I am not saying tracking is wrong, but rather tracking would
> not have helped make these things happen faster.

This is correct. I just had too much stuff to do and there wasn't
enough time to get the hierarchical query patch worthy of someone
spending their time on a review.

IMHO, tracking occasionally is alright. However, as a developer,
being "pummelled" does sometimes get irritating; even to the point
that I will sometimes discontinue working on something because it's
too much of a hassle for me to spend my free-time on. Though, this
was not the case for hierarchical queries at all; that really was due
to a lack of time.

There's got to be a "happy medium" in which we can keep our status
updated without it becoming an irritation. Has anyone looked at
something like dotProject? It may be something we could use for
development. Of course, there's lots of other tools... but it would
be nice if we had a central location for each task's status so that we
don't have to resort to searching email and/or archives.

--
Jonah H. Harris, Software Architect | phone: 732.331.1300
EnterpriseDB Corporation | fax: 732.331.1301
33 Wood Ave S, 2nd Floor | jharris(at)enterprisedb(dot)com
Iselin, New Jersey 08830 | http://www.enterprisedb.com/

Bruce Momjian wrote:
> I see no value in documenting it.

The value in documenting it is that the general awareness about the
progress of development is raised, so more people know where and how to
get involved, as opposed to only three people knowing and the rest
finding out one month after feature freeze that the patch isn't ready.

> I do things only if others do not. If committers applied patches as
> they came in, the patch queue would be empty, and if others tracked
> open issues, I wouldn't have to.

Fair enough. Perhaps we should find others with ideas about better
tracking and the willingness to implement them.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:

> Tom Lane wrote:
> > Well, no, it's not. We have told people till we're blue in the face
> > "post early, post often". Now I will plead guilty to not always
> > having spent as much time giving feedback on draft patches as I
> > should've, but the process is pretty clear. As I see it the main
> > problem is people undertaking patches off in corners somewhere rather
> > than discussing their work on the mailing lists while they do it.
>
> Again, process support. If all we can offer people is to post
> multi-megabyte patches to the mailing list every month, that totally
> doesn't help.

You're right, it's a pain for developers to have to keep shipping around
patches over and over. It means anyone else who wants to look at your code is
probably looking at an old version, there's no convenient way to convert from
incremental patches to cumulative patches, etc.

There are tools to solve this but they're not "process support", they're
revision control systems. If CVS didn't suck so hard I would suggest that the
natural thing to do would be to grant CVS access far and wide and have people
create branches for their proposed changes. The existing set of committers
would become the set of people allowed to commit to HEAD. Sadly CVS makes this
about as much work as the current system and the risks are much higher.

> We'd need ways to track the progress on these things:
> what was the specification for that patch, where was the discussion on
> it, what has changed in the patch since the last time, since the time
> before last time, what is left to be done, who has worked on it, etc.
> Figuring out the answer to those questions from a mailing list archive
> is tedious to the point that no one wants to do it.

Frankly these things all sound pretty trivial.

--
greg

Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
> > As an extreme example consider the new Linux release cycle. They have a
> > non-freeze period of a couple weeks, followed by months of frozen time. Users
> > who want to try out new features on different hardware can do so and often
> > turn up problems developers reviewing the patches missed. Developers spend the
> > months of frozen time working on new patches which, if they're ready on time,
> > all go into the queue in the first few weeks of a new release. If they miss
> > the window they'll make the next one.
>
> That doesn't seem particularly appealing for our purposes. What it
> sounds like is that all the development happens somewhere outside the
> main tree, and every so often everybody tries to land a bunch of large
> patches at once. That's going to be a mess if the patches interact at
> all, and it certainly isn't going to address my primary concern of
> encouraging a publicly-visible development process instead of having
> people hiding in a corner until they can drop a large patch on us.

Well that's basically what's been happening except they drop it on us a day
before feature freeze giving you a frantic month of trying to make it work.

The reason people go off on their own and develop without help are manifold
but I'm not sure the release process is one of them. They're not going to be
able to commit their incomplete code to the main CVS regardless of whether
there's a freeze or not anyways.

They can work openly on new features during the feature freeze and beta
period, they just won't have the expectation that they'll squeeze it into this
release. Instead they'll know that if it's ready for the patch landing period
it'll be in the next release.

> Here I agree - partly. With some continuing effort these patches could
> be in fine shape to apply by the time we branch CVS for 8.3 development.
> However, our traditional approach to beta period is that it's supposed
> to be a "quiet time" where people focus on testing, debugging, and
> documentation. Shepherding people who are doing major development
> during that period is likely to be a serious distraction from making the
> release ready and high-quality.

Well on the flip side you have months to do that work instead of having
pressure to rework whole patches in short order. Perhaps it helps that at
least in the case of Linux it's mostly different people handling the releases
and the raw development. I'm not sure we have enough manpower for that.

--
greg

Andrew Dunstan wrote:

> It would make the process more transparent, which is something several
> people have expressed a desire for.

Yes, the processes seems to work by having two of the most important
people waste time on getting information anyone else could collect, or
that the developer himself could quickly provide and keep up to date.

Furthermore the process expects these two people to know who to ask,
what to look for etc. Wouldn't it be great if someone could just decide:
"hey I know postgres, I have some unexpected spare time, I want to do
some code reviewing on patch x, y and z"

If this all works through some issue tracker, a wiki or a combination of
both, the end result is transparency, the opportunity to take load off
from people that have important other things to do and a chance at
getting unexpected help.

For example I have no expertise in coding on Postgres, but I think I
would be able to collect information from this mailinglist (like specs,
url's etc.) and put them in some issue tracker or wiki. I have done
exactly the same for PHP [1] (though there are rarely specs thrown
around in PHP, so my PHP todo list is not much more than a simple bullet
list of todo's with a name and occasional URL's to additional information).

regards,
Lukas

[1] http://oss.backendmedia.com/PHPTODO/

Lukas Kahwe Smith wrote:

> For example I have no expertise in coding on Postgres, but I think I
> would be able to collect information from this mailinglist (like specs,
> url's etc.) and put them in some issue tracker or wiki. I have done
> exactly the same for PHP [1] (though there are rarely specs thrown
> around in PHP, so my PHP todo list is not much more than a simple bullet
> list of todo's with a name and occasional URL's to additional information).

Actually I should add that I went ahead and created the PHP todo list on
my own, without any official blessing and one by one internals developer
have joined. Now its actively used in the entire release process.

This is probably the best approach to go about doing this for PostgreSQL
as well. That way there is no additional work, and instead it can show
if it actually helps people. And if it does more people will start to
contribute to it. If not then someone who did not contribute to the code
anyways wasted a bit of time to get to know the PostgreSQL todo items
really well. ;)

regards,
Lukas

On Fri, Sep 01, 2006 at 10:13:07PM -0400, Jonah H. Harris wrote:
> On 9/1/06, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> >I pummelled Jonah over the recursive query patch.
>
> He did. Trust me on this... think I still have some bruises too :)

That wasn't productive. Getting it out in public that your available
time didn't match the size of the project in, say, February or even
April, would have been.

> >Neither effort was very fruitful, but tracking wasn't what made
> >them fail. I am not saying tracking is wrong, but rather tracking
> >would not have helped make these things happen faster.
>
> This is correct. I just had too much stuff to do and there wasn't
> enough time to get the hierarchical query patch worthy of someone
> spending their time on a review.

I know it's hard to judge from the inside, which is why this process
needs more public visibility, early and often, to the rest of the
community. It does nobody any good to have Bruce bawl you out
repeatedly in private, and then spring it on the rest of us in June
that you weren't even close.

> IMHO, tracking occasionally is alright. However, as a developer,
> being "pummelled" does sometimes get irritating; even to the point
> that I will sometimes discontinue working on something because it's
> too much of a hassle for me to spend my free-time on.

That's an excellent outcome for the project, by the way. When
somebody is feeling "pummeled," it's frequently a signal that they
really don't have the time they thought they did, and the sooner and
more widely such a situation is known, the better.

> Though, this was not the case for hierarchical queries at all; that
> really was due to a lack of time.
>
> There's got to be a "happy medium" in which we can keep our status
> updated without it becoming an irritation. Has anyone looked at
> something like dotProject? It may be something we could use for
> development. Of course, there's lots of other tools... but it would
> be nice if we had a central location for each task's status so that
> we don't have to resort to searching email and/or archives.

...and we're back to the infamous tarbaby of bug/issue trackers. It's
time we took that head-on instead of dancing around it.

As expressed many times earlier, such a system must be accessible,
both for read and write, by email. What other things must it do?
Should it do? Must it avoid?

Cheers,
D
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
phone: +1 415 235 3778 AIM: dfetter666
Skype: davidfetter

Remember to vote!

On Fri, Sep 01, 2006 at 09:46:02PM -0400, Bruce Momjian wrote:
> Peter Eisentraut wrote:
> > Bruce Momjian wrote:
> > > It pulls my a mailbox file I use, and it does instant updates as
> > > soon as I change it. It is a URL. Why do people care where it
> > > is?
> >
> > The complaint has been that not enough people help with reviewing
> > patches. That would indicate that the problem is not location but
> > scalability. If everything has to go through your private
> > mailbox, then it's not a very obvious process to outsiders.
>
> Well, you can grab items from there and apply them. I will remove
> them from the mailbox when I see the commit.

This, in essence, is the problem. You're using the first person
singular here, and it's become obvious that one person can't keep up
with this any more. The project, thanks in part to your hard work,
has grown past that stage.

> You have another system that will not be significantly more work?

If it's significantly more work spread among many people, that is a
perfectly viable solution, just as long as it doesn't cause any one
person to be overloaded.

We need to get this project more bus-proof, whether it's the patch
queue or a server people depend on that's silently down for a week
with no ETAs in sight for getting any piece of it back on line.

Bruce, are you ready to help create a more bus-proof situation? If
so, what specific steps would you like to take in that direction?

Cheers,
D
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
phone: +1 415 235 3778 AIM: dfetter666
Skype: davidfetter

Remember to vote!

On Sat, Sep 02, 2006 at 09:00:35AM +0200, Lukas Kahwe Smith wrote:
> Actually I should add that I went ahead and created the PHP todo list on
> my own, without any official blessing and one by one internals developer
> have joined. Now its actively used in the entire release process.

That is the approach we should be taking. I see too many people wait
for an "official" solution when in fact the first person to come up
with a todo list that people like better than the current one will win.

I remember something about setting up a wiki for a todo list and pie in
the sky list. I thought it had promise, but until the wiki is there we
won't know...

Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Martijn van Oosterhout wrote:
> I remember something about setting up a wiki for a todo list and pie
> in the sky list. I thought it had promise, but until the wiki is
> there we won't know...

I think the wiki is the prerequisite for many ideas about alternative
tracking and documentation mechanisms. I just wonder what the hold up
is on it.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Saturday 02 September 2006 07:14, David Fetter wrote:
> On Fri, Sep 01, 2006 at 09:46:02PM -0400, Bruce Momjian wrote:
> > Peter Eisentraut wrote:
> > > Bruce Momjian wrote:
> > > > It pulls my a mailbox file I use, and it does instant updates as
> > > > soon as I change it. It is a URL. Why do people care where it
> > > > is?
> > >
> > > The complaint has been that not enough people help with reviewing
> > > patches. That would indicate that the problem is not location but
> > > scalability. If everything has to go through your private
> > > mailbox, then it's not a very obvious process to outsiders.
> >
> > Well, you can grab items from there and apply them. I will remove
> > them from the mailbox when I see the commit.
>
> This, in essence, is the problem. You're using the first person
> singular here, and it's become obvious that one person can't keep up
> with this any more. The project, thanks in part to your hard work,
> has grown past that stage.
>

AFAICT Bruce is not the bottleneck and our problem is not that multiple people
are reviewing the same item and duplicating effort, so I think your making
conclusions based not on the evidence but instead on a desired outcome.

No offense, a whole lot of this thread seems to be positioned that way, but
the real problem seems to be we do not have enough patch reviewers. ISTM the
questions we should be asking are who can actually help out with patch review
and then ask those people why they haven't done it. If folks like Peter,
Andrew, Magnus, Simon, Joe, and Niel all say that they are not reviewing
patches because they can't find the patches that need review, they can't
figure out who is reviewing what, or they don't think anyone is paying
attention when they do review something, then I think we have a serious
problem and we certainly need to change processes. What I think you'll find
is that they are all just busy working on other things, which in that case I
think we need to figure out how to motivate them to focus on the patch queue
rather than other items. IMHO

--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

Lukas Kahwe Smith <smith(at)pooteeweet(dot)org> writes:
>> For example I have no expertise in coding on Postgres, but I think I
>> would be able to collect information from this mailinglist (like specs,
>> url's etc.) and put them in some issue tracker or wiki. I have done
>> exactly the same for PHP [1] (though there are rarely specs thrown
>> around in PHP, so my PHP todo list is not much more than a simple bullet
>> list of todo's with a name and occasional URL's to additional information).

> Actually I should add that I went ahead and created the PHP todo list on
> my own, without any official blessing and one by one internals developer
> have joined. Now its actively used in the entire release process.

> This is probably the best approach to go about doing this for PostgreSQL
> as well.

I agree. Look at the most successful recent process change around here:
the buildfarm. Andrew Dunstan took it upon himself to make that happen.
He built it, and they came.

No bug/issue tracker, or anything else, is going to be successful unless
somebody commits enough time to make it so. I've noted a whole lot of
enthusiasm for having a tracker in these recent discussions, but a
remarkable shortage of individuals stepping up to do the work.

regards, tom lane

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tom Lane sagely noted:
> No bug/issue tracker, or anything else, is going to be successful unless
> somebody commits enough time to make it so. I've noted a whole lot of
> enthusiasm for having a tracker in these recent discussions, but a
> remarkable shortage of individuals stepping up to do the work.

There are a number of reasons for this, not least of which is the
enormous and ever-changing requirements such a system would have to
have. My company has even offered resources to get something going,
and I think I convinced Bruce at one point of the usefulness of a
PG bug/feature tracking system. However, the problem is that existing
software is never going to be good enough.

The buildfarm is an excellent example of this. If Andrew had tried
to create it with something "off-the-shelf", it probably would have
taken longer to get started, been more complicated than it needed
to be, and would not work as well as it now does. However, it's been
a smashing success, so much so that MySQL is now envious[1] and is
trying to make their own version of it[2].

[1] http://www.jpipes.com/index.php?/archives/93-MySQL-Build-Farm-Part-1.html
[2] http://forge.mysql.com/wiki/MySQL_Build_Farm_Initiative

I've been thinking about this a lot since before the Summit, and the
only solution I see is to design something specifically for us.
Rather than get bogged down in details about how it will work and
what technologies it will be using, I'd like to share my ideas on
how it will work from an end-user perspective. There's a simple web
interface. You can use it to get the answer to questions like this:

What are all the open bugs for the current release?
Who is working on the updateable views patch?
Which bugs were fixed in 8.1.3?
What's the feature history of psql since 2003?
What are some easy TODO items that need getting done?
Just when does Tom Lane sleep?
What features were added in 8.1?
What are the current unapplied patches?
What bugs were fixed in tsearch2 from version 7.4.5 to 7.4.12?

Yes, maintaining it will be a royal pain in the butt. But my theory has
been "if you build it, they will come". It will require a lot of human
interaction, as automation only takes you so far, especially when trying
to parse mailing list messages. But if we eventually get a decent-size
group of people who each put in a little work on it, it should be
quite possible to keep it up to date. (Also, this would be a great way
for people to help the project who don't have the time and/or skills
to do coding).

The idea is to allow everything to happen as it already does, e.g. no
extra burdens for Tom, no extra processes anywhere. However, all the
existing information is gathered into one centralized and searchable place.
The primary sources will include the code, the docs, and the mailing lists.
Especially the mailing lists.

So, rather than go off in a corner and start coding, how about I start a
pgfoundry project for this and document my existing ideas? Any objections
to giving this a try?

- --
Greg Sabino Mullane greg(at)endpoint(dot)com greg(at)turnstep(dot)com
End Point Corporation 610-983-9073
PGP Key: 0x14964AC8 200609021107
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iD8DBQFE+Z/1vJuQZxSWSsgRAu7rAKCJcIOiw4jty0slWuEd6fNsCV8xcgCfQpEj
MBp00ZxG9B6UVAoHnSDq0W0=
=zlQm
-----END PGP SIGNATURE-----

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> Here's a completely novel idea: accept incremental patches.

I don't think it's as novel as all that --- personally I've always
preferred to tackle large projects incrementally.

> I've been bitten by having stuff rejected because there was no immediate
> use for them;

Maybe you failed to make the larger picture clear. If you'd gotten the
pghackers list to agree to some plan like "I want to accomplish X, and
to get there I want to do A then B then C", I doubt we'd have rejected a
subsequent patch to do A (assuming it doesn't break anything else, of
course).

regards, tom lane

On Sep 2, 2006, at 11:28 AM, Tom Lane wrote:

> Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
>> Here's a completely novel idea: accept incremental patches.
>
> I don't think it's as novel as all that --- personally I've always
> preferred to tackle large projects incrementally.

I think that accepting incremental patches to a mainline is _bad_ and
that accepting them in general is good (as you have to go through
that process outside of version control sometimes anyway). Sticking
them in CVS however can get a bit messy. This is where other version
control systems that have a bit better branching and merging support
has an advantage as people can work in the repository on their
project in separate branches and pulling them all back together again
(once an overall satisfaction metric has been reached) is not
excruciatingly painful. Where am I going with this? From my
experience accepting incremental patches is a _bad_ idea unless you
have a VCS that has really makes it _easy_ to manage them. Sounds
like more work than worth on the postgres project as it is now.

Additionally, what problem is accepting incremental patches supposed
to solve?

// Theo Schlossnagle
// CTO -- http://www.omniti.com/~jesus/
// OmniTI Computer Consulting, Inc. -- http://www.omniti.com/

"Greg Sabino Mullane" <greg(at)turnstep(dot)com> writes:
> I've been thinking about this a lot since before the Summit, and the
> only solution I see is to design something specifically for us.

Well, nobody's going to accuse you of thinking too small ;-). Sounds
great to me, though, if you think you can pull it off. Have at it!

> Rather than get bogged down in details about how it will work and
> what technologies it will be using, I'd like to share my ideas on
> how it will work from an end-user perspective. There's a simple web
> interface. You can use it to get the answer to questions like this:

> What are all the open bugs for the current release?
> Who is working on the updateable views patch?
> Which bugs were fixed in 8.1.3?
> [etc]

That sounds fine for the "output" part of things, but where does the
"input" come from? ISTM that it's been the input side that's been the
real bone of contention in all the discussions of specific tracking
software.

BTW, another "output" thing you might consider is "having draft release
notes ready-to-go on demand". Currently, Bruce prepares the release
notes on the basis of a very tedious scan of the CVS commit logs.
If this sort of stuff were being dropped into a tracker as it went
into the CVS tree, at least the research part of making the notes would
be zero-effort (or perhaps better to say that the work would be spread
out instead of concentrated).

regards, tom lane

On Saturday 02 September 2006 11:42, Tom Lane wrote:
> BTW, another "output" thing you might consider is "having draft release
> notes ready-to-go on demand". Currently, Bruce prepares the release
> notes on the basis of a very tedious scan of the CVS commit logs.
> If this sort of stuff were being dropped into a tracker as it went
> into the CVS tree, at least the research part of making the notes would
> be zero-effort (or perhaps better to say that the work would be spread
> out instead of concentrated).

FWIW I have never understood why we don't require patch submitters/committers
to update the release notes when they do the patch. I know Bruce says that
it's not a problem for him to do it, but it sure would help out some of the
other folks.

--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

Greg Sabino Mullane wrote:
> I've been thinking about this a lot since before the Summit, and the
> only solution I see is to design something specifically for us.
> Rather than get bogged down in details about how it will work and
> what technologies it will be using, I'd like to share my ideas on
> how it will work from an end-user perspective. There's a simple web
> interface. You can use it to get the answer to questions like this:
>
> What are all the open bugs for the current release?
> Who is working on the updateable views patch?
> Which bugs were fixed in 8.1.3?
> What's the feature history of psql since 2003?
> What are some easy TODO items that need getting done?
> Just when does Tom Lane sleep?
> What features were added in 8.1?
> What are the current unapplied patches?
> What bugs were fixed in tsearch2 from version 7.4.5 to 7.4.12?
>
> Yes, maintaining it will be a royal pain in the butt. But my theory has
> been "if you build it, they will come". It will require a lot of human
> interaction, as automation only takes you so far, especially when trying
> to parse mailing list messages. But if we eventually get a decent-size
> group of people who each put in a little work on it, it should be
> quite possible to keep it up to date. (Also, this would be a great way
> for people to help the project who don't have the time and/or skills
> to do coding).
>
> The idea is to allow everything to happen as it already does, e.g. no
> extra burdens for Tom, no extra processes anywhere. However, all the
> existing information is gathered into one centralized and searchable place.
> The primary sources will include the code, the docs, and the mailing lists.
> Especially the mailing lists.
>
> So, rather than go off in a corner and start coding, how about I start a
> pgfoundry project for this and document my existing ideas? Any objections
> to giving this a try?

Good approach. Let me tell you what doesn't work well currently:

What is fixed in current CVS that was not on the TODO list
already, and marked with a dash?
What fixes/bugs are being worked on but not yet in the patch queue?
What are all the features/fixes in each release?

Right now, the release notes show a list of all the significant items in
each release, but it isn't available until the release, and it isn't
complete (because it would be unreadable by ordinary users). And there
is no tracking of individual items in progress except by individual
developers. Magnus, for example, tracks Win32 items, and Tom tracks
backend stuff. I track whatever no one else tracks.

As far as tracking, right now, +50% of items come in and are discussed
via email, not a bug form. How does that information get into a
tracking system without huge extra work? We can do it, and it would
give people nice feedback, but is the effort worth the benefit?

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Theo Schlossnagle <jesus(at)omniti(dot)com> writes:
> Additionally, what problem is accepting incremental patches supposed
> to solve?

Keeping the individual patches reviewable is one useful goal.

We may be talking at cross-purposes here. The sort of thing I think
Alvaro is imagining is something like what I did a year or two back when
I wanted to make the executor treat plan trees as read-only --- if
memory serves, I did that in three or four commits spread over a week or
two. I could have done it in one massive patch but it would have been
unreadable to me or anyone else. And for that matter, the reason for
doing it at all was as part of a much larger concept that's still
unfinished (better caching and invalidation of plans). When you're
talking about tasks of that magnitude, stepwise improvement is the only
way you are going to get there at all.

I really don't have any faith in the concept of doing very large tasks
on a development branch and expecting to land them in one huge merge.
For a nearby counterexample look at Postgres-R, which never has and
probably never will manage to produce a patch that could apply to the
core project's CVS head. At least not without thinking of some
incremental way to do it, because by the time they bring all their
changes up to HEAD, the tree has always drifted under them. There is
no magic version control system that will fix that.

regards, tom lane

Tom Lane wrote:
> "Greg Sabino Mullane" <greg(at)turnstep(dot)com> writes:
> > I've been thinking about this a lot since before the Summit, and the
> > only solution I see is to design something specifically for us.
>
> Well, nobody's going to accuse you of thinking too small ;-). Sounds
> great to me, though, if you think you can pull it off. Have at it!
>
> > Rather than get bogged down in details about how it will work and
> > what technologies it will be using, I'd like to share my ideas on
> > how it will work from an end-user perspective. There's a simple web
> > interface. You can use it to get the answer to questions like this:
>
> > What are all the open bugs for the current release?
> > Who is working on the updateable views patch?
> > Which bugs were fixed in 8.1.3?
> > [etc]
>
> That sounds fine for the "output" part of things, but where does the
> "input" come from? ISTM that it's been the input side that's been the
> real bone of contention in all the discussions of specific tracking
> software.
>
> BTW, another "output" thing you might consider is "having draft release
> notes ready-to-go on demand". Currently, Bruce prepares the release
> notes on the basis of a very tedious scan of the CVS commit logs.

Having an intermediate release note list, beyond the dashed TODO items,
might be helpful. So the idea is to track work in progress, and
completed items. The question is always whether the effort is worth the
benefit. We can try and see how it works.

> If this sort of stuff were being dropped into a tracker as it went
> into the CVS tree, at least the research part of making the notes would
> be zero-effort (or perhaps better to say that the work would be spread
> out instead of concentrated).

Well, multiple commits often end up as a single entry (and many commits
don't need to show up in the release notes), so the tracker would just
give me a backup when creating the release notes.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> Holy crap, Batman! This database can do
>
> INSERT INTO foo VALUES (1,1, 'so long'), (42, 2, 'and thanks'),
> (142857, 3, 'for all the fish')
>
> now! We should be talking to more people about that!
>
> That will make some MySQL users happy at least a bit :-)

It will certainly make porting applications from MySQL to
Postgres a lot easier, and such porting is where a lot more
effort needs to be put, IMO. (I mean more people working
on porting applications, not adding mysqlisms to the code :)

- --
Greg Sabino Mullane greg(at)turnstep(dot)com
End Point Corporation
PGP Key: 0x14964AC8 200609021200
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iD8DBQFE+as8vJuQZxSWSsgRAq9RAJwJzY9uTBmMjafX/96tyRgviN7TQQCaAt2S
bgXGiusM1R7RD8FoZsiYizo=
=3GI5
-----END PGP SIGNATURE-----

Robert Treat wrote:
> On Saturday 02 September 2006 11:42, Tom Lane wrote:
> > BTW, another "output" thing you might consider is "having draft release
> > notes ready-to-go on demand". Currently, Bruce prepares the release
> > notes on the basis of a very tedious scan of the CVS commit logs.
> > If this sort of stuff were being dropped into a tracker as it went
> > into the CVS tree, at least the research part of making the notes would
> > be zero-effort (or perhaps better to say that the work would be spread
> > out instead of concentrated).
>
> FWIW I have never understood why we don't require patch submitters/committers
> to update the release notes when they do the patch. I know Bruce says that
> it's not a problem for him to do it, but it sure would help out some of the
> other folks.

I can't even get documentation for many patches. I am hesitant to add
even more burden. I would prefer they concentrate on documentation.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Friday 01 September 2006 19:42, Tom Lane wrote:
> Gavin Sherry <swm(at)linuxworld(dot)com(dot)au> writes:
> > On Fri, 1 Sep 2006, Tom Lane wrote:
> >> My feeling is that we ought to bounce bitmap indexes and updatable views
> >> as not being ready, accept all the contrib stuff, and try to get the
> >> other items done in time for a beta at, say, the end of next week.
> >
> > For what it's worth, Jie and I hope to have finished the bitmap streaming
> > this weekend which is the only outstanding issue with the code as it
> > currently stands.
>
> AFAIR, you told me "it'll be done this weekend" last week. And the week
> before.
>
> In any case, it's going to be a large complicated patch, and if I spend
> all next week reviewing it then I won't make any progress on the other
> items on my to-do list. At some point we have to say "this has slipped
> too far, we have to hold it over for 8.3".
>

Perhaps if some other committers would take ownership of other items on
your "to be done list", that would spread the workload enough to keep
everything going? (Yes, this does require some of the other folks to step
up... responses to your original email with "I will commit to finishing this
item this week" would probably be enough)

--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

Tom Lane wrote:
> Theo Schlossnagle <jesus(at)omniti(dot)com> writes:
>> Additionally, what problem is accepting incremental patches supposed
>> to solve?
>
> Keeping the individual patches reviewable is one useful goal.
>
> We may be talking at cross-purposes here. The sort of thing I think
> Alvaro is imagining is something like what I did a year or two back when
> I wanted to make the executor treat plan trees as read-only --- if
> memory serves, I did that in three or four commits spread over a week or
> two.

To second this, Alvaro is constantly beating our (cmd) other developers
to do this, so I would guess that you are correct :).

I find also that this method allows someone like me, who can read C and
understand good parts of it to get the gist of what is going on without
trying to grok the whole thing. Large patches make it very difficult.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Greg Sabino Mullane wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> Tom Lane sagely noted:
>> No bug/issue tracker, or anything else, is going to be successful unless
>> somebody commits enough time to make it so. I've noted a whole lot of
>> enthusiasm for having a tracker in these recent discussions, but a
>> remarkable shortage of individuals stepping up to do the work.

Although I applaud Greg's efforts, I think it makes more sense to take
something and make it our own instead of trying to build something from
scratch.

I think that Bugzilla will probably serve our purpose for most things.

Greg, how about you and I (I am talking with Dunstan on this as well)
set up Bugzilla and just doing an small group audit of how it could (or
couldn't) work and report back?

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Robert Treat wrote:

> No offense, a whole lot of this thread seems to be positioned that way, but
> the real problem seems to be we do not have enough patch reviewers. ISTM the
> questions we should be asking are who can actually help out with patch review
> and then ask those people why they haven't done it. If folks like Peter,
> Andrew, Magnus, Simon, Joe, and Niel all say that they are not reviewing
> patches because they can't find the patches that need review, they can't
> figure out who is reviewing what, or they don't think anyone is paying
> attention when they do review something, then I think we have a serious
> problem and we certainly need to change processes. What I think you'll find
> is that they are all just busy working on other things, which in that case I
> think we need to figure out how to motivate them to focus on the patch queue
> rather than other items. IMHO

I think that if all the patches are listed with all relevant context
information on a webpage, then people can more easily jump in when they
unexpectedly have time (or prefer to procrastinate some other real world
thing they should rather work on). Right now if you have a few hours to
spare you do not have all the information readily available. Even worse
by inquiring for the information you might feel you are commiting more
than you really wanted to. This kind of information needs to be right
there without any person having to actively provide it upon inquiry.

regards,
Lukas

Robert Treat wrote:
> On Saturday 02 September 2006 07:14, David Fetter wrote:
> > On Fri, Sep 01, 2006 at 09:46:02PM -0400, Bruce Momjian wrote:
> > > Peter Eisentraut wrote:
> > > > Bruce Momjian wrote:
> > > > > It pulls my a mailbox file I use, and it does instant updates as
> > > > > soon as I change it. It is a URL. Why do people care where it
> > > > > is?
> > > >
> > > > The complaint has been that not enough people help with reviewing
> > > > patches. That would indicate that the problem is not location but
> > > > scalability. If everything has to go through your private
> > > > mailbox, then it's not a very obvious process to outsiders.
> > >
> > > Well, you can grab items from there and apply them. I will remove
> > > them from the mailbox when I see the commit.
> >
> > This, in essence, is the problem. You're using the first person
> > singular here, and it's become obvious that one person can't keep up
> > with this any more. The project, thanks in part to your hard work,
> > has grown past that stage.
> >
>
> AFAICT Bruce is not the bottleneck and our problem is not that multiple people
> are reviewing the same item and duplicating effort, so I think your making
> conclusions based not on the evidence but instead on a desired outcome.
>
> No offense, a whole lot of this thread seems to be positioned that way, but
> the real problem seems to be we do not have enough patch reviewers. ISTM the
> questions we should be asking are who can actually help out with patch review

Let me explain what I do. I subscribe to most PostgreSQL email lists.
I basically try to make sure every bug, patch, or idea is processed. If
it is a bug via user error, did they get correct information. If it is
valid PostgreSQL bug, was it fixed or added to the TODO list. If it was
a patch, was it applied or rejected. If it is a feature request, was it
added to the TODO list or rejected.

That's about it, folks. Not a lot of fancy magic. As other people take
up these items, my workload will be reduced. I do not take ownership of
any of these things. I merely jump on them when they are _not_ dealt
with by others.

What I am unclear about is how all of this traffic is going to be pushed
into a system that cleanly categorizes it so others can understand it.
It can be done, but what is the cost/benefit to it?

Perhaps if email could be pushed into the system, and somehow
categorized as bug, patch, or feature request, and somehow removed only
when it is dealt with. That is what I do now via my mailbox.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Peter Eisentraut wrote:
> Martijn van Oosterhout wrote:
>> I remember something about setting up a wiki for a todo list and pie
>> in the sky list. I thought it had promise, but until the wiki is
>> there we won't know...
>
> I think the wiki is the prerequisite for many ideas about alternative
> tracking and documentation mechanisms. I just wonder what the hold up
> is on it.

We have a wiki already:
http://wiki.postgresql.org/

It could be a bit faster, but its there.

I even already started on a little documentation effort:
http://wiki.postgresql.org/index.php/pgwiki:Variations

regards,
Lukas

Greg Sabino Mullane wrote:

> Yes, maintaining it will be a royal pain in the butt. But my theory has
> been "if you build it, they will come". It will require a lot of human
> interaction, as automation only takes you so far, especially when trying
> to parse mailing list messages. But if we eventually get a decent-size
> group of people who each put in a little work on it, it should be
> quite possible to keep it up to date. (Also, this would be a great way
> for people to help the project who don't have the time and/or skills
> to do coding).

I think we should start with the wiki and then add in whatever we notice
can be automated. This way we may not initially be able to cover all the
bases, but it will mean that whatever we do automate actually matches
real world requirements.

> The idea is to allow everything to happen as it already does, e.g. no
> extra burdens for Tom, no extra processes anywhere. However, all the
> existing information is gathered into one centralized and searchable place.
> The primary sources will include the code, the docs, and the mailing lists.
> Especially the mailing lists.

Yeah, let the ones work on this that do not do any of the pgsql coding.

regards,
Lukas

Greg Sabino Mullane wrote:
> There are a number of reasons for this, not least of which is the
> enormous and ever-changing requirements such a system would have to
> have.

> The buildfarm is an excellent example of this.

The build farm is not an example of this. There isn't any build-farm
software out there. There is, however, plenty of issue tracking
software out there.

Just pointing that out. Have at it. I for one would rather set up
existing issue trackers and work with them. If I could just find
hardware to host it. Not that I've looked, though.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Lukas Kahwe Smith wrote:
> Robert Treat wrote:
>
> > No offense, a whole lot of this thread seems to be positioned that way, but
> > the real problem seems to be we do not have enough patch reviewers. ISTM the
> > questions we should be asking are who can actually help out with patch review
> > and then ask those people why they haven't done it. If folks like Peter,
> > Andrew, Magnus, Simon, Joe, and Niel all say that they are not reviewing
> > patches because they can't find the patches that need review, they can't
> > figure out who is reviewing what, or they don't think anyone is paying
> > attention when they do review something, then I think we have a serious
> > problem and we certainly need to change processes. What I think you'll find
> > is that they are all just busy working on other things, which in that case I
> > think we need to figure out how to motivate them to focus on the patch queue
> > rather than other items. IMHO
>
> I think that if all the patches are listed with all relevant context
> information on a webpage, then people can more easily jump in when they
> unexpectedly have time (or prefer to procrastinate some other real world
> thing they should rather work on). Right now if you have a few hours to
> spare you do not have all the information readily available. Even worse
> by inquiring for the information you might feel you are commiting more
> than you really wanted to. This kind of information needs to be right
> there without any person having to actively provide it upon inquiry.

OK, how does that happen without a lot of work, or moving all discussion
on to a web-based system?

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Lukas Kahwe Smith wrote:
> Peter Eisentraut wrote:
>> Martijn van Oosterhout wrote:
>>> I remember something about setting up a wiki for a todo list and pie
>>> in the sky list. I thought it had promise, but until the wiki is
>>> there we won't know...
>>
>> I think the wiki is the prerequisite for many ideas about alternative
>> tracking and documentation mechanisms. I just wonder what the hold up
>> is on it.
>
> We have a wiki already:
> http://wiki.postgresql.org/
>
> It could be a bit faster, but its there.

That wiki is wrong. :) It was set up wrong and configured wrong. It was
supposed to be for developers only.

There is also another wiki that is a trac based that was set up at dave
pages request (for slaves_to_www).

>
> I even already started on a little documentation effort:
> http://wiki.postgresql.org/index.php/pgwiki:Variations
>

And note it has already been mentioned that, that wiki was the wrong
place to put it. Not that I am slamming your efforts, I think it was
really good that you did the docs :). They are just in the wrong place.

Sincerely,

Joshua D. Drake

> regards,
> Lukas
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
> choose an index scan if your joining column's datatypes do not
> match
>

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Bruce Momjian wrote:
> Lukas Kahwe Smith wrote:
>> Robert Treat wrote:
>>
>>> No offense, a whole lot of this thread seems to be positioned that way, but
>>> the real problem seems to be we do not have enough patch reviewers. ISTM the
>>> questions we should be asking are who can actually help out with patch review
>>> and then ask those people why they haven't done it. If folks like Peter,
>>> Andrew, Magnus, Simon, Joe, and Niel all say that they are not reviewing
>>> patches because they can't find the patches that need review, they can't
>>> figure out who is reviewing what, or they don't think anyone is paying
>>> attention when they do review something, then I think we have a serious
>>> problem and we certainly need to change processes. What I think you'll find
>>> is that they are all just busy working on other things, which in that case I
>>> think we need to figure out how to motivate them to focus on the patch queue
>>> rather than other items. IMHO
>> I think that if all the patches are listed with all relevant context
>> information on a webpage, then people can more easily jump in when they
>> unexpectedly have time (or prefer to procrastinate some other real world
>> thing they should rather work on). Right now if you have a few hours to
>> spare you do not have all the information readily available. Even worse
>> by inquiring for the information you might feel you are commiting more
>> than you really wanted to. This kind of information needs to be right
>> there without any person having to actively provide it upon inquiry.
>
> OK, how does that happen without a lot of work, or moving all discussion
> on to a web-based system?

No discussion takes place on the lists and IRC and busy bees make sure
that the progress/decisions etc get updated on the static wiki. Wiki's
suck at discussions, but they are great to store the decisions made so
that anyone can get himself upto speed and things do not get lost over time.

It seems that you have been the only busy bee so far, and we definitely
need more for this to work.

regards,
Lukas

Joshua D. Drake wrote:

> That wiki is wrong. :) It was set up wrong and configured wrong. It was
> supposed to be for developers only.
>
> There is also another wiki that is a trac based that was set up at dave
> pages request (for slaves_to_www).

Setup something better, until then we can start using it to generate
content.

>> I even already started on a little documentation effort:
>> http://wiki.postgresql.org/index.php/pgwiki:Variations
>>
>
> And note it has already been mentioned that, that wiki was the wrong
> place to put it. Not that I am slamming your efforts, I think it was
> really good that you did the docs :). They are just in the wrong place.

No problem. Its easy to move content, its much harder to generate it.

regards,
Lukas

Lukas Kahwe Smith wrote:
> Bruce Momjian wrote:
> > Lukas Kahwe Smith wrote:
> >> Robert Treat wrote:
> >>
> >>> No offense, a whole lot of this thread seems to be positioned that way, but
> >>> the real problem seems to be we do not have enough patch reviewers. ISTM the
> >>> questions we should be asking are who can actually help out with patch review
> >>> and then ask those people why they haven't done it. If folks like Peter,
> >>> Andrew, Magnus, Simon, Joe, and Niel all say that they are not reviewing
> >>> patches because they can't find the patches that need review, they can't
> >>> figure out who is reviewing what, or they don't think anyone is paying
> >>> attention when they do review something, then I think we have a serious
> >>> problem and we certainly need to change processes. What I think you'll find
> >>> is that they are all just busy working on other things, which in that case I
> >>> think we need to figure out how to motivate them to focus on the patch queue
> >>> rather than other items. IMHO
> >> I think that if all the patches are listed with all relevant context
> >> information on a webpage, then people can more easily jump in when they
> >> unexpectedly have time (or prefer to procrastinate some other real world
> >> thing they should rather work on). Right now if you have a few hours to
> >> spare you do not have all the information readily available. Even worse
> >> by inquiring for the information you might feel you are commiting more
> >> than you really wanted to. This kind of information needs to be right
> >> there without any person having to actively provide it upon inquiry.
> >
> > OK, how does that happen without a lot of work, or moving all discussion
> > on to a web-based system?
>
> No discussion takes place on the lists and IRC and busy bees make sure
> that the progress/decisions etc get updated on the static wiki. Wiki's
> suck at discussions, but they are great to store the decisions made so
> that anyone can get himself upto speed and things do not get lost over time.
>
> It seems that you have been the only busy bee so far, and we definitely
> need more for this to work.

Yea, I was afraid that was the answer. :-(

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Sat, Sep 02, 2006 at 06:18:13PM +0200, Lukas Kahwe Smith wrote:
> We have a wiki already:
> http://wiki.postgresql.org/

I must have missed the annoucement, oh well...

Now I'm only familiar with twiki so maybe this sounds silly but:

Does it support sections? Like can you have a developer section, a
patches section, a TODO section, etc. And then have differing
permissions per section?

So I was thinking a TODO section with the list as front page and each
tem having a page within that section. Create a parallel page
Wannabe-todo-tiems that can be edited by anyone and lock the main page
to just a few. Then all some developer has to do is scan the wannabe
list and copy the links to the main list.

Is this kind of setup ossible?

Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Bruce Momjian wrote:

>> It seems that you have been the only busy bee so far, and we definitely
>> need more for this to work.
>
> Yea, I was afraid that was the answer. :-(

But we have a few volunteers, like me for example.
Now don't say "I was afraid that was the answer" again or I might feel
insulted :)

regards,
Lukas

On 2/9/06 16:42, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> BTW, another "output" thing you might consider is "having draft release
> notes ready-to-go on demand". Currently, Bruce prepares the release
> notes on the basis of a very tedious scan of the CVS commit logs.
> If this sort of stuff were being dropped into a tracker as it went
> into the CVS tree, at least the research part of making the notes would
> be zero-effort (or perhaps better to say that the work would be spread
> out instead of concentrated).

We have the developers update the CHANGELOG file with each non-trivial
change in pgAdmin. It works very well for us and producing release
announcements etc. is a trivial task. The overhead on each developer is
virtually zero as well as they have to compose some text for the commit
message anyway.

http://www.pgadmin.org/development/changelog.php

Regards, Dave.

On 2/9/06 17:18, "Lukas Kahwe Smith" <smith(at)pooteeweet(dot)org> wrote:

> Peter Eisentraut wrote:
>> Martijn van Oosterhout wrote:
>>> I remember something about setting up a wiki for a todo list and pie
>>> in the sky list. I thought it had promise, but until the wiki is
>>> there we won't know...
>>
>> I think the wiki is the prerequisite for many ideas about alternative
>> tracking and documentation mechanisms. I just wonder what the hold up
>> is on it.
>
> We have a wiki already:
> http://wiki.postgresql.org/
>
> It could be a bit faster, but its there.
>
> I even already started on a little documentation effort:
> http://wiki.postgresql.org/index.php/pgwiki:Variations

The wiki will be going shortly as it's not been setup in the way that was
agreed, nor is it being used for it's intended purpose. I'll put it right
when I get time from dealing with releases of everything I seem to be
involved in.

Regards, Dave.

Martijn van Oosterhout wrote:
> On Sat, Sep 02, 2006 at 06:18:13PM +0200, Lukas Kahwe Smith wrote:
>> We have a wiki already:
>> http://wiki.postgresql.org/
>
> I must have missed the annoucement, oh well...
>
> Now I'm only familiar with twiki so maybe this sounds silly but:

wiki.postgresql.org is dead. :). It wasn't supposed to be an open wiki
like that.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

On Sat, Sep 02, 2006 at 06:38:30PM +0100, Dave Page wrote:
> The wiki will be going shortly as it's not been setup in the way that was
> agreed, nor is it being used for it's intended purpose. I'll put it right
> when I get time from dealing with releases of everything I seem to be
> involved in.

I beleive the idea was to not make the site editable by the public?
Does't that kind of defeat the purpose of getting more people to
takeover work?

Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

On 2/9/06 20:16, "Martijn van Oosterhout" <kleptog(at)svana(dot)org> wrote:

> On Sat, Sep 02, 2006 at 06:38:30PM +0100, Dave Page wrote:
>> The wiki will be going shortly as it's not been setup in the way that was
>> agreed, nor is it being used for it's intended purpose. I'll put it right
>> when I get time from dealing with releases of everything I seem to be
>> involved in.
>
> I beleive the idea was to not make the site editable by the public?
> Does't that kind of defeat the purpose of getting more people to
> takeover work?

That was one of the more flexible points as far as I'm concerned. What was
more important was to make it clear it was a *developer* resource and not an
alternative techdocs site or in fact a site intended in any way for end
users.

See my next email for more...

Regards, Dave.

Peter Eisentraut wrote:
> Greg Sabino Mullane wrote:
>
>> There are a number of reasons for this, not least of which is the
>> enormous and ever-changing requirements such a system would have to
>> have.
>>
>
>
>> The buildfarm is an excellent example of this.
>>
>
> The build farm is not an example of this. There isn't any build-farm
> software out there. There is, however, plenty of issue tracking
> software out there.
>
> Just pointing that out. Have at it. I for one would rather set up
> existing issue trackers and work with them. If I could just find
> hardware to host it. Not that I've looked, though.
>
>

What is more, a good bug tracking system is orders of magnitude more
complex that the buildfarm.

If we want something sooner rather than later, let's use something off
the shelf and tailor it.

cheers

andrew

Andrew Dunstan wrote:
>
>
> Peter Eisentraut wrote:
>> Greg Sabino Mullane wrote:
>>
>>> There are a number of reasons for this, not least of which is the
>>> enormous and ever-changing requirements such a system would have to
>>> have.
>>>
>>
>>
>>> The buildfarm is an excellent example of this.
>>>
>>
>> The build farm is not an example of this. There isn't any build-farm
>> software out there. There is, however, plenty of issue tracking
>> software out there.
>>
>> Just pointing that out. Have at it. I for one would rather set up
>> existing issue trackers and work with them. If I could just find
>> hardware to host it. Not that I've looked, though.
>>
>>
>
> What is more, a good bug tracking system is orders of magnitude more
> complex that the buildfarm.
>
> If we want something sooner rather than later, let's use something off
> the shelf and tailor it.

http://pgbugs.commandprompt.com (still need to configure email).

Joshua D. Drake

>
> cheers
>
> andrew
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
>

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Joshua D. Drake wrote:
> http://pgbugs.commandprompt.com (still need to configure email).

Thank you for that.

I think an issue tracking system for patches and such may need to be
distinct from a bug-tracking system such as bugzilla, but let's get one
thing after another up.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Joshua D. Drake wrote:
>> http://pgbugs.commandprompt.com (still need to configure email).
>
> Thank you for that.
>
> I think an issue tracking system for patches and such may need to be
> distinct from a bug-tracking system such as bugzilla, but let's get one
> thing after another up.

I made a mistake on this install and installed the dev version. I will
fix this weekend.

I was also thinking the domain: tracker.postgresql.org?

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Joshua D. Drake wrote:
> I was also thinking the domain: tracker.postgresql.org?

Certainly not for bugzilla.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Tom Lane wrote:
> Lukas Kahwe Smith <smith(at)pooteeweet(dot)org> writes:
>
>>> For example I have no expertise in coding on Postgres, but I think I
>>> would be able to collect information from this mailinglist (like specs,
>>> url's etc.) and put them in some issue tracker or wiki. I have done
>>> exactly the same for PHP [1] (though there are rarely specs thrown
>>> around in PHP, so my PHP todo list is not much more than a simple bullet
>>> list of todo's with a name and occasional URL's to additional information).
>>>
>
>
>> Actually I should add that I went ahead and created the PHP todo list on
>> my own, without any official blessing and one by one internals developer
>> have joined. Now its actively used in the entire release process.
>>
>
>
>> This is probably the best approach to go about doing this for PostgreSQL
>> as well.
>>
>
> I agree. Look at the most successful recent process change around here:
> the buildfarm. Andrew Dunstan took it upon himself to make that happen.
> He built it, and they came.
>

:-)

The difference is that the buildfarm could get going with effort only
from me and a handful of early adopters, while a tracker probably needs
higher level of initial community buyin. Nevertheless, I take your point.

> No bug/issue tracker, or anything else, is going to be successful unless
> somebody commits enough time to make it so. I've noted a whole lot of
> enthusiasm for having a tracker in these recent discussions, but a
> remarkable shortage of individuals stepping up to do the work.
>
>
>

You are right that it will need ongoing effort.

There are discussions happening about resources. Stay tuned.

cheers

andrew

Alvaro Herrera wrote:
> Joshua D. Drake wrote:
>
>
>> It does not mean all those features are useful, they definitely are. I
>> am just trying to look at it from at:
>>
>> WHIZ* BANG* POW* perspective.
>>
>
> Holy crap, Batman! This database can do
>
> INSERT INTO foo VALUES (1,1, 'so long'), (42, 2, 'and thanks'),
> (142857, 3, 'for all the fish')
>

I just lurk here at pgsql-hackers.

That function would be very cool.

Thanks!

Best Regards,

Carlo Florendo
Astra Philippines Inc.
www.astra.ph

> Right now, the release notes show a list of all the significant
> items in each release, but it isn't available until the release,
> and it isn't complete (because it would be unreadable by ordinary
> users). And there is no tracking of individual items in progress
> except by individual developers. Magnus, for example, tracks Win32
> items, and Tom tracks backend stuff. I track whatever no one else
> tracks.

Well, I *try*. I find myself not managing quite as well as I'd like ;-)
So I for one would definitely appreciate a tool that would help with
that - provided that the tool fits the development model, not the other
way around.

(most other things seems to have been said already this time around, so
I won't repeat arguments others have made)

//Magnus

Bruce Momjian wrote:
> Robert Treat wrote:
>> FWIW I have never understood why we don't require patch submitters/committers
>> to update the release notes when they do the patch.

I've suggested this more than once in the past -- I think it would be a
clear improvement over the status quo. Updating the release notes
incrementally would lead to more accurate and complete release notes:
more accurate because the description for a feature would be written at
the same time as the feature itself, and more complete because it would
be harder to unintentionally omit discussion of a new feature. It would
also help communicate to users what features will be in the next release
of Postgres, which is certainly good from a PR point of view (a certain
Swedish software company is very fond of talking about the features it
will be adding in future releases, for example...) Finally, it would
remove the need for a sequential scan of the CVS history, which I'm sure
is pretty time-consuming, and delays the beta process.

> I can't even get documentation for many patches. I am hesitant to add
> even more burden. I would prefer they concentrate on documentation.

The first revision of a patch often doesn't include documentation
updates, but in that case the submitter should be promptly told what
they need to fix; I think the same would apply here. In practice, if
you're committing a patch, you *should* understand it well enough to
write a release note entry for it, so the burden might end up falling on
committers, anyway.

-Neil

Tom Lane wrote:
> Bruce Momjian <bruce(at)momjian(dot)us> writes:
> > Tom Lane wrote:
> >> Well, it's taken us the full month to get this far through the queue, so
> >> I'd sure like to have more people on board reviewing next time. Alvaro
> >> helped but I miss Neil Conway, and some other people have helped in the
> >> past. However --- the present situation has nothing to do with lack of
> >> reviewers, it's lack of time to finish the patches.
>
> > I did try to get us additional help in reviewing. Neil was unavailable,
> > and Alvaro could only give part of his time.
>
> It strikes me that setting feature freeze in midsummer might not be the
> best strategy for having manpower available to review --- people tend to
> be on vacation in August. Maybe the answer is just to move the dates a
> bit one way or the other.

Hmm...but if you're going to do that, why not do that now: push the
beta date back by, say, a month (or however long you had in mind) for
this cycle. That way, the two major patches that are likely to be
dropped for this cycle stand a chance to make it into this release,
and you accomplish your goal of moving the dates a bit all at the same
time.

--
Kevin Brown kevin(at)sysexperts(dot)com

On Sat, Sep 02, 2006 at 06:31:51PM +0100, Dave Page wrote:
> > BTW, another "output" thing you might consider is "having draft release
> > notes ready-to-go on demand". Currently, Bruce prepares the release
> > notes on the basis of a very tedious scan of the CVS commit logs.
> > If this sort of stuff were being dropped into a tracker as it went
> > into the CVS tree, at least the research part of making the notes would
> > be zero-effort (or perhaps better to say that the work would be spread
> > out instead of concentrated).
>
> We have the developers update the CHANGELOG file with each non-trivial
> change in pgAdmin. It works very well for us and producing release
> announcements etc. is a trivial task. The overhead on each developer is
> virtually zero as well as they have to compose some text for the commit
> message anyway.

Another possibility would be to annotate commit messages that should get
added to the release notes and have something automagically cull those
out of CVS.
--
Jim C. Nasby, Database Architect jim(at)nasby(dot)net
512.569.9461 (cell) http://jim.nasby.net

On Sun, Sep 03, 2006 at 09:18:00AM +0200, Peter Eisentraut wrote:
> Joshua D. Drake wrote:
> > http://pgbugs.commandprompt.com (still need to configure email).
>
> Thank you for that.
>
> I think an issue tracking system for patches and such may need to be
> distinct from a bug-tracking system such as bugzilla, but let's get one
> thing after another up.

Actually, I've generally found bugzilla to be a decent tool for
general-purpose tracking. Feature requests and what-not need a lot of
the same capabilities of a bug tracker, and having everything in one
tool is often very handy.
--
Jim C. Nasby, Database Architect jim(at)nasby(dot)net
512.569.9461 (cell) http://jim.nasby.net

On Fri, Sep 01, 2006 at 03:28:36PM -0400, Stephen Frost wrote:
> Overall, I really think 8.2 is going to be an excellent release. I wish
> autovacuum could have been enabled by default and I'd just like to ask,
> now and I'll try to remember again once 8.2 is out, please let's turn it
> on by default for 8.3 (and early on so we get some good testing of it).

Can someone put this on the TODO, just so we (hopefully) don't forget
about it?
--
Jim C. Nasby, Database Architect jim(at)nasby(dot)net
512.569.9461 (cell) http://jim.nasby.net