small but useful patches for text search

Hi there,

I and Teodor have several small, but useful patches for text search:

1. Support of filtering dictionaries and unaccent dictionary/function.
This is often requested feature, which solves, for example,
problem with correct headlines for text with accents.
See example and docs http://www.sai.msu.su/~megera/wiki/unaccent

2. Add prefix search support to the synonym dictionary.
Star sign '*' at the end of definition word indicates,
that definition word is a prefix and to_tsquery() function will transform
that definition to the prefix search format.
Notice, it is ignored in to_tsvector().
Some examples http://www.sai.msu.su/~megera/wiki/2009-03-13

There are limitations - no support for thesaurus dictionary,
ts_debug doesn't support all these features (it needs to be rewritten
to C).

There is no problem with back compatibility.

We would like to have your opinion what to do with these patches - leave them
for 8.5 or provide them to hackers to review for 8.4.

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

Oleg Bartunov <oleg(at)sai(dot)msu(dot)su> writes:

> We would like to have your opinion what to do with these patches - leave them
> for 8.5 or provide them to hackers to review for 8.4.

In theory 8.5, though you wouldn't be the first to start sneaking in late
commits and given how long before the release I can't really think people
would complain too much.

Things have reached a perverse state. We've now been in a holding pattern for
4 1/2 months while development has basically ceased. Aside from committers, no
contributors have been able to make any progress on any work for already that
time and months remain before any reviewers will pay any attention to
submissions.

I have a bunch of ideas I wanted to follow up posix_fadvise with including
posix_fallocate and more uses of posix_fadvise. I also wanted to return to the
ordered merge-append which I still think is critical for partitioned table
plans.

I think it's clear that stretching feature freezes is a failed policy. Aside
from delaying everyone else's work, it hasn't helped the projects we held the
release for either. Those projects would have hit 8.5 in due course just fine
but now surely we'll delay 8.5 based on the 8.4 release date. So they'll be
delayed just like everyone else's work.

I think in the future we should adopt a policy that only minor patches can be
received after the first commitfest. Major patches are expected to submitted
in the *first* commitfest and reworked based on feedback on subsequent
commitfests. The last commitfest should be a real feature-freeze where only
bug-fixes and minor changes are accepted, not major features.

There seems to be a lot of resistance on the basis that there would be a long
wait until features are seen in a release. Personally I'm only interested in
when features get committed, not when they hit a release. But in any case I
think experience shows that this would result in hitting the same release
anyways and that release would be sooner as well.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's RemoteDBA services!

Gregory Stark <stark(at)enterprisedb(dot)com> writes:
> I think it's clear that stretching feature freezes is a failed policy.

Yeah, it's the same old same old --- once again, we've bent over
backwards to try to accommodate large patches at the end of a
development cycle. I'm not sure that that's ever going to stop,
because every time there are people cheerleading for said patches
and insisting that the release will be so much better if we wait
for them. Somehow we keep expecting that they'll get fixed and
landed in a short period of time.

A saner policy would mandate that large patches land near the start of
a development cycle, but I don't know how we get people to do that.

regards, tom lane

On Mon, Mar 16, 2009 at 8:41 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>  I'm not sure that that's ever going to stop,
> because every time there are people cheerleading for said patches
> and insisting that the release will be so much better if we wait
> for them.

Well 8.3 was better for having HOT. But I still feel we got lucky with
that. I was pretty nervous we would find some major data corruption
bug after 8.3 was out. Pavan did great work and others in EDB put a
lot of work into testing it, but systematic testing -- as great as it
is --just isn't the same as having many other people using it day in
and day out and doing *unexpected* things with it.

The problem I see isn't (just) that things take longer to land than we
expect. The problem is that working on them during feature freeze a)
means they have to be perfect since there won't be any more chance to
refine them and b) means nobody else can work on new development while
they're being worked on.

--
greg

On Mar 16, 2009, at 1:41 PM, Tom Lane wrote:

> A saner policy would mandate that large patches land near the start of
> a development cycle, but I don't know how we get people to do that.

I think that if you can strictly time-limit the final CommitFest in
the same way that you time-limit the others, then whatever doesn't get
in that last one will be deferred to the first fest for the next major
version.

For those patches that *do* get in for the last CommitFest, they have
to be ready within a month of the last day of the CommitFest month. So
if the last fest for 8.4 was December, then any patches not ready by
Feb 1 would be put off.

This doesn't guarantee that people will get those big patches in
earlier, but it will surely encourage them to do so.

Just an idea.

Best,

David

On Mon, Mar 16, 2009 at 04:41:29PM -0400, Tom Lane wrote:
> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
> > I think it's clear that stretching feature freezes is a failed
> > policy.
>
> Yeah, it's the same old same old --- once again, we've bent over
> backwards to try to accommodate large patches at the end of a
> development cycle. I'm not sure that that's ever going to stop,
> because every time there are people cheerleading for said patches
> and insisting that the release will be so much better if we wait for
> them. Somehow we keep expecting that they'll get fixed and landed
> in a short period of time.
>
> A saner policy would mandate that large patches land near the start
> of a development cycle, but I don't know how we get people to do
> that.

It's not easy, in the sense of timing and will, but it's not complex,
technically. Basically, we start from a largest size we'll allow at
the last commit fest, and increase it to linearly to, say, twice the
size of the largest patch proposed at the first commit fest.

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

David E. Wheeler wrote:
> On Mar 16, 2009, at 1:41 PM, Tom Lane wrote:
>
>> A saner policy would mandate that large patches land near the start of
>> a development cycle, but I don't know how we get people to do that.
>
> I think that if you can strictly time-limit the final CommitFest in the
> same way that you time-limit the others, then whatever doesn't get in
> that last one will be deferred to the first fest for the next major
> version.

The earlier commitfests were not time-limited either. They lasted until
all the patches were dealt with; either committed or bumped to next
commit fest. It's just that when you know there still at least one more
commitfest a couple of months ahead, everyone feels more happy to bump a
patch and to have one's patch bumped to the next one. In the last one,
it's a lot harder to do that because that means bumping to the next
release, and you don't even know when the next commitfest is.

The original plan was that anything not 100% ready to commit at the
beginning of the last commit fest will be bumped to the next release,
and beta would start right after the first commit fest, a week or two
after the submission deadline. We failed to enforce that. In the next
release cycle, I think we need to be more explicit about that policy
throughout the release cycle and really enforce it.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Hi,

Le 16 mars 09 à 21:41, Tom Lane a écrit :
> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
>> I think it's clear that stretching feature freezes is a failed
>> policy.
>
> A saner policy would mandate that large patches land near the start of
> a development cycle, but I don't know how we get people to do that.

I think Open Source showed that you can't have both time based
releases and feature based releases. Anytime any project try to
accomodate those two objectives, they fail at both.

It seems that PostgreSQL is leaning towards time based releases, which
I think made up the majority in our community. What I'd propose is to
refuse new patches posted in last two commit fests. Those are reserved
to bake what we release.

I'm not sure that's the best compromise you can do, even if it's
definitely one, but it has the merit to be quite clear and easy to
understand for contributors: you want your code to appear in X.Y, get
it reviewed at least once (with feedbacks) before we enter the last
but one commitfest. If you fail at this, you get to (try to) have your
code in X.Y+1, which won't be released that far away (about 1 year
away from a known date).

Regards,
--
dim

On Mon, Mar 16, 2009 at 5:10 PM, Heikki Linnakangas
<heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
> The earlier commitfests were not time-limited either. They lasted until all
> the patches were dealt with; either committed or bumped to next commit fest.
> It's just that when you know there still at least one more commitfest a
> couple of months ahead, everyone feels more happy to bump a patch and to
> have one's patch bumped to the next one. In the last one, it's a lot harder
> to do that because that means bumping to the next release, and you don't
> even know when the next commitfest is.
>
> The original plan was that anything not 100% ready to commit at the
> beginning of the last commit fest will be bumped to the next release, and
> beta would start right after the first commit fest, a week or two after the
> submission deadline. We failed to enforce that. In the next release cycle, I
> think we need to be more explicit about that policy throughout the release
> cycle and really enforce it.

I mostly agree with this, but there is one fly in the ointment: if a
patch really hasn't been seriously looked at by a committer, bumping
it recreates one of the problems CommitFests were designed to avoid -
patch limbo. I feel pretty good about the decisions to postpone Hot
Standby and SE-PostgreSQL (not that my personal opinion necessarily
counts for much, but that's how I feel); I would have felt less good
about it if the same decision had been made a month ago. But on the
other hand that means 8.4 will be a month later. If we could have
gone through the same process two months earlier, or if we could have
released 8.4beta and done those reviews on the side during the beta
period, that would have been best of all.

I basically agree with the idea of time-based releases, but if the
committers cherry-pick the patches that are easily dealt with and
never touch the more complex ones, it leads to a different set of
problems... I'm not sure what can be done about that, though.

...Robert

Robert Haas wrote:
> On Mon, Mar 16, 2009 at 5:10 PM, Heikki Linnakangas
> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
> > The earlier commitfests were not time-limited either. They lasted until all
> > the patches were dealt with; either committed or bumped to next commit fest.
> > It's just that when you know there still at least one more commitfest a
> > couple of months ahead, everyone feels more happy to bump a patch and to
> > have one's patch bumped to the next one. In the last one, it's a lot harder
> > to do that because that means bumping to the next release, and you don't
> > even know when the next commitfest is.
> >
> > The original plan was that anything not 100% ready to commit at the
> > beginning of the last commit fest will be bumped to the next release, and
> > beta would start right after the first commit fest, a week or two after the
> > submission deadline. We failed to enforce that. In the next release cycle, I
> > think we need to be more explicit about that policy throughout the release
> > cycle and really enforce it.
>
> I mostly agree with this, but there is one fly in the ointment: if a
> patch really hasn't been seriously looked at by a committer, bumping
> it recreates one of the problems CommitFests were designed to avoid -
> patch limbo. I feel pretty good about the decisions to postpone Hot
> Standby and SE-PostgreSQL (not that my personal opinion necessarily
> counts for much, but that's how I feel); I would have felt less good
> about it if the same decision had been made a month ago. But on the
> other hand that means 8.4 will be a month later. If we could have
> gone through the same process two months earlier, or if we could have
> released 8.4beta and done those reviews on the side during the beta
> period, that would have been best of all.

Well, we have been working on stuff for the past month so it was not
like we were waiting on SE-PG to move forward.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> writes:
> The original plan was that anything not 100% ready to commit at the
> beginning of the last commit fest will be bumped to the next release,
> and beta would start right after the first commit fest, a week or two
> after the submission deadline. We failed to enforce that.

Uh, no, that's historical revisionism, cf
http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Development_Plan
We expected and scheduled for a longer-than-normal final commitfest.
There's two months in the original schedule, whereas expectation was
that earlier ones would be less than a month (which mostly they were).

What we did say, and didn't enforce, was that patches too large to be
reviewed in a reasonably short time would be bounced. We thought we'd
be able to make that stick if large patches got reviewed and applied
in an incremental fashion over the series of commitfests. For one
reason or another that never happened for SEPostgres. We should try
to analyze exactly why not, although I think the bottom-line answer
there has to do with nobody being particularly eager to work on it.

Hot Standby had a different timeline, and quite frankly should have
never been seriously considered for 8.4 at all. But I think that
as long as SEPostgres was looming on the horizon, we didn't see the
point of being strict about deadlines ...

regards, tom lane

Tom Lane wrote:
> Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> writes:
> > The original plan was that anything not 100% ready to commit at the
> > beginning of the last commit fest will be bumped to the next release,
> > and beta would start right after the first commit fest, a week or two
> > after the submission deadline. We failed to enforce that.
>
> Uh, no, that's historical revisionism, cf
> http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Development_Plan
> We expected and scheduled for a longer-than-normal final commitfest.
> There's two months in the original schedule, whereas expectation was
> that earlier ones would be less than a month (which mostly they were).
>
> What we did say, and didn't enforce, was that patches too large to be
> reviewed in a reasonably short time would be bounced. We thought we'd
> be able to make that stick if large patches got reviewed and applied
> in an incremental fashion over the series of commitfests. For one
> reason or another that never happened for SEPostgres. We should try
> to analyze exactly why not, although I think the bottom-line answer
> there has to do with nobody being particularly eager to work on it.

I think SE-Postgres development timeline of going from feature-complete
to "give us the features we want" really hampred things, and the fact
that we didn't give SE-Postgres much feedback earlier, for the same
reason (feature complete to "give us the features we want").
>
> Hot Standby had a different timeline, and quite frankly should have
> never been seriously considered for 8.4 at all. But I think that
> as long as SEPostgres was looming on the horizon, we didn't see the
> point of being strict about deadlines ...

Hot Standby wasn't in the original plan for 8.4, but someone suggested
"Hey, let's try.", and we did.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian <bruce(at)momjian(dot)us> writes:
> Tom Lane wrote:
>> Hot Standby had a different timeline, and quite frankly should have
>> never been seriously considered for 8.4 at all. But I think that
>> as long as SEPostgres was looming on the horizon, we didn't see the
>> point of being strict about deadlines ...

> Hot Standby wasn't in the original plan for 8.4, but someone suggested
> "Hey, let's try.", and we did.

Simon certainly made a heroic try at it, and I give him full marks for
that. But HS was obviously not ready on 1 November. The point I was
trying to make was that if SEPostgres had not been there, we'd have
probably said on 1 November "sorry, this has to wait for 8.5". As it
was, we let him carry on trying to get the patch to a committable state.

And of course all these things feed on each other --- when it's obvious
that there is no immediate deadline, it's easy to let things slide a
bit further.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <bruce(at)momjian(dot)us> writes:
> > Tom Lane wrote:
> >> Hot Standby had a different timeline, and quite frankly should have
> >> never been seriously considered for 8.4 at all. But I think that
> >> as long as SEPostgres was looming on the horizon, we didn't see the
> >> point of being strict about deadlines ...
>
> > Hot Standby wasn't in the original plan for 8.4, but someone suggested
> > "Hey, let's try.", and we did.
>
> Simon certainly made a heroic try at it, and I give him full marks for
> that. But HS was obviously not ready on 1 November. The point I was
> trying to make was that if SEPostgres had not been there, we'd have
> probably said on 1 November "sorry, this has to wait for 8.5". As it
> was, we let him carry on trying to get the patch to a committable state.

Well, we had many other patches in November so it isn't clear that SE-PG
was somehow what kept hot standby in-play.

> And of course all these things feed on each other --- when it's obvious
> that there is no immediate deadline, it's easy to let things slide a
> bit further.

True, but we haven't been sitting around doing nothing, and we had to do
most of what we have done since November whether we had SE-PG or host
standby in play.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Mon, Mar 16, 2009 at 6:20 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Robert Haas wrote:
>> On Mon, Mar 16, 2009 at 5:10 PM, Heikki Linnakangas
>> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>> > The earlier commitfests were not time-limited either. They lasted until all
>> > the patches were dealt with; either committed or bumped to next commit fest.
>> > It's just that when you know there still at least one more commitfest a
>> > couple of months ahead, everyone feels more happy to bump a patch and to
>> > have one's patch bumped to the next one. In the last one, it's a lot harder
>> > to do that because that means bumping to the next release, and you don't
>> > even know when the next commitfest is.
>> >
>> > The original plan was that anything not 100% ready to commit at the
>> > beginning of the last commit fest will be bumped to the next release, and
>> > beta would start right after the first commit fest, a week or two after the
>> > submission deadline. We failed to enforce that. In the next release cycle, I
>> > think we need to be more explicit about that policy throughout the release
>> > cycle and really enforce it.
>>
>> I mostly agree with this, but there is one fly in the ointment: if a
>> patch really hasn't been seriously looked at by a committer, bumping
>> it recreates one of the problems CommitFests were designed to avoid -
>> patch limbo.  I feel pretty good about the decisions to postpone Hot
>> Standby and SE-PostgreSQL (not that my personal opinion necessarily
>> counts for much, but that's how I feel); I would have felt less good
>> about it if the same decision had been made a month ago.  But on the
>> other hand that means 8.4 will be a month later.  If we could have
>> gone through the same process two months earlier, or if we could have
>> released 8.4beta and done those reviews on the side during the beta
>> period, that would have been best of all.
>
> Well, we have been working on stuff for the past month so it was not
> like we were waiting on SE-PG to move forward.

Stuff related to the CommitFest?

AFAICS, the only committer who has done any significant review or
committing of patches in the last month is Heikki, who extensively
reworked and then committed infrastructure changes for recovery on
February 18th (2 days shy of a month ago) and then extensively
reviewed Hot Standby and SE-PostgreSQL. It's really, really good that
those patches have finally received some extensive review, both
because now some version of each of them will likely make it into 8.5,
and because now we have only a handful of patches left that Tom has
said are pretty close to being committable. But I don't see how you
can say it didn't delay the release.

Frankly, if we'd rejected on January 1st all of the patches that (1)
were submitted on time, (2) had been reviewed in a timely fashion, and
(3) had not been resubmitted in largely committable form, we would
have bounced infrastructure changes for recovery, hot standby,
column-level privileges, autovacuum and reloptions, and probably
parallel restore. If the committers who subsequently worked to get
those changes into the tree had devoted an equal amount of effort to
what would have been left in the commitfest after so doing, we would
long since be done with this commitfest and into beta. But committers
are free to spend their time on the projects they think are most
interesting or which their employer is willing to pay them to work on,
so that didn't happen.

This is kind of understandable from the point of view of the
committers, and it's even sorta good for the project as a whole to the
extent that it prioritizes more interesting features over less
interesting ones, but it's pretty frustrating for non-committer
contributors. Any non-committer who pushes for a hard limit on the
length of the commitfest is taking the risk that next time the patch
that none of the committers are interested in will be theirs. So
nobody really wants to argue for that. But the alternative is to
argue that the commitfest should continue until all the patches have
gotten some reasonable minimum amount of review, and that means the
commitfest can continue for a very, very long time during which
non-committers can't expect to get any new patches looked at. That's
not very appealing either. The commitfest process really only works
if the committers review all of the patches submitted within some
reasonable period of time - if that isn't possible, there really isn't
going to be a satisfactory solution.

...Robert

Robert Haas wrote:
> > Well, we have been working on stuff for the past month so it was not
> > like we were waiting on SE-PG to move forward.
>
> Stuff related to the CommitFest?
>
> AFAICS, the only committer who has done any significant review or
> committing of patches in the last month is Heikki, who extensively
> reworked and then committed infrastructure changes for recovery on
> February 18th (2 days shy of a month ago) and then extensively
> reviewed Hot Standby and SE-PostgreSQL. It's really, really good that
> those patches have finally received some extensive review, both
> because now some version of each of them will likely make it into 8.5,
> and because now we have only a handful of patches left that Tom has
> said are pretty close to being committable. But I don't see how you
> can say it didn't delay the release.

You are assuming that only commit-fest work is required to get us to
beta. You might remember the long list of open items I faced in January
that I have whittled down, but I still have about twenty left.

Tom has done work fixing optimizer bugs introduced in 8.4. I have had
EnterpriseDB work to do and am working on the release notes now. The
bottom line is that there is lots of cleanup required to get to beta
independent of the last commit fest work.

> Frankly, if we'd rejected on January 1st all of the patches that (1)
> were submitted on time, (2) had been reviewed in a timely fashion, and
> (3) had not been resubmitted in largely committable form, we would
> have bounced infrastructure changes for recovery, hot standby,
> column-level privileges, autovacuum and reloptions, and probably
> parallel restore. If the committers who subsequently worked to get
> those changes into the tree had devoted an equal amount of effort to
> what would have been left in the commitfest after so doing, we would
> long since be done with this commitfest and into beta. But committers
> are free to spend their time on the projects they think are most
> interesting or which their employer is willing to pay them to work on,
> so that didn't happen.

I agree if we had said "no" to those patches we could be farther now,
but I am not sure how much farther.

> This is kind of understandable from the point of view of the
> committers, and it's even sorta good for the project as a whole to the
> extent that it prioritizes more interesting features over less
> interesting ones, but it's pretty frustrating for non-committer
> contributors. Any non-committer who pushes for a hard limit on the
> length of the commitfest is taking the risk that next time the patch
> that none of the committers are interested in will be theirs. So
> nobody really wants to argue for that. But the alternative is to
> argue that the commitfest should continue until all the patches have
> gotten some reasonable minimum amount of review, and that means the
> commitfest can continue for a very, very long time during which
> non-committers can't expect to get any new patches looked at. That's
> not very appealing either. The commitfest process really only works
> if the committers review all of the patches submitted within some
> reasonable period of time - if that isn't possible, there really isn't
> going to be a satisfactory solution.

Again, this assumes the commit-fest is the only old on beta starting.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

> You are assuming that only commit-fest work is required to get us to
> beta.  You might remember the long list of open items I faced in January
> that I have whittled down, but I still have about twenty left.
>
> Tom has done work fixing optimizer bugs introduced in 8.4.  I have had
> EnterpriseDB work to do and am working on the release notes now.  The
> bottom line is that there is lots of cleanup required to get to beta
> independent of the last commit fest work.

Sure. I don't have a problem with that. But I don't see what it has
to do with the point of my original post, which is that it we can
either make the release happen on time, or we can get all of the
patches reviewed, but we can only do both if the committers have the
time and energy to make that happen. Do you disagree?

...Robert

Robert Haas wrote:
> > You are assuming that only commit-fest work is required to get us to
> > beta. ?You might remember the long list of open items I faced in January
> > that I have whittled down, but I still have about twenty left.
> >
> > Tom has done work fixing optimizer bugs introduced in 8.4. ?I have had
> > EnterpriseDB work to do and am working on the release notes now. ?The
> > bottom line is that there is lots of cleanup required to get to beta
> > independent of the last commit fest work.
>
> Sure. I don't have a problem with that. But I don't see what it has
> to do with the point of my original post, which is that it we can
> either make the release happen on time, or we can get all of the
> patches reviewed, but we can only do both if the committers have the
> time and energy to make that happen. Do you disagree?

I agree. My point was that we let hot standby and se-pg stay around
longer than necessary because we were involved in other things. We
could have said they had sufficient review in January if those were the
only things holding us up.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

It's not like my opinion has any weight but still,

On Tuesday 17 March 2009 16:15:21 Robert Haas wrote:
> either make the release happen on time, or we can get all of the
> patches reviewed, but we can only do both if the committers have the
> time and energy to make that happen. Do you disagree?

Sure I do.
You can either have time based release or feature based one. Full stop.

That do not depend on time and energy of people involved, nor to their role in
the community (contributor, reviewer, commiter, beta tester, etc)

Regards,
--
dim

On Tue, Mar 17, 2009 at 11:18 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Robert Haas wrote:
>> > You are assuming that only commit-fest work is required to get us to
>> > beta. ?You might remember the long list of open items I faced in January
>> > that I have whittled down, but I still have about twenty left.
>> >
>> > Tom has done work fixing optimizer bugs introduced in 8.4. ?I have had
>> > EnterpriseDB work to do and am working on the release notes now. ?The
>> > bottom line is that there is lots of cleanup required to get to beta
>> > independent of the last commit fest work.
>>
>> Sure.  I don't have a problem with that.  But I don't see what it has
>> to do with the point of my original post, which is that it we can
>> either make the release happen on time, or we can get all of the
>> patches reviewed, but we can only do both if the committers have the
>> time and energy to make that happen.  Do you disagree?
>
> I agree.  My point was that we let hot standby and se-pg stay around
> longer than necessary because we were involved in other things.  We
> could have said they had sufficient review in January if those were the
> only things holding us up.

Right, that's what I think too. Or at least we could have said that
we weren't going to review them any more right now, leaving aside the
question of sufficiency.

Basically, for the project to grow, it needs more committers, and the
precondition for being added as a committer should be a promise to
spend a certain amount of time reviewing and committing patches other
than your own. According to the wiki, we have 15 committers, which is
more than enough, but most of those are inactive or are just there as
maintainers for very specific portions of the code.

Failing that, we will continue to argue about whether to slip releases
or ignore patches. I don't like either of those options very much.

...Robert

On Tuesday 17 March 2009 18:00:57 Robert Haas wrote:
> Basically, for the project to grow, it needs more committers, and the
> precondition for being added as a committer should be a promise to
> spend a certain amount of time reviewing and committing patches other
> than your own. According to the wiki, we have 15 committers, which is
> more than enough, but most of those are inactive or are just there as
> maintainers for very specific portions of the code.

The limiting factor is mainly time, not talent or dedication. Contributing to
PostgreSQL requires a lot of time for learning and staying up to date, and
very few people have lives that permit that.

The alternatives are making the code more modular and improving the tools, but
both can only go so far.

On Tuesday 17 March 2009 09:38:59 Bruce Momjian wrote:
> You are assuming that only commit-fest work is required to get us to
> beta. You might remember the long list of open items I faced in January
> that I have whittled down, but I still have about twenty left.
>

I think part of the perception of the project sitting around doing nothing
isn't so much that you/tom are doing nothing, but others who were doing
review or coding features are now caught in limbo. One of the things the
commitfest has been successful at is helping delegate code review. Perhaps we
need to take a fresh look at your list of twenty things and see what can be
delegated out to others.

--
Robert Treat
Conjecture: http://www.xzilla.net
Consulting: http://www.omniti.com

Robert Treat wrote:
> On Tuesday 17 March 2009 09:38:59 Bruce Momjian wrote:
> > You are assuming that only commit-fest work is required to get us to
> > beta. You might remember the long list of open items I faced in January
> > that I have whittled down, but I still have about twenty left.
> >
>
> I think part of the perception of the project sitting around doing nothing
> isn't so much that you/tom are doing nothing, but others who were doing
> review or coding features are now caught in limbo. One of the things the
> commitfest has been successful at is helping delegate code review. Perhaps we
> need to take a fresh look at your list of twenty things and see what can be
> delegated out to others.

Yep, I agree. The problem is that last time I put out a list that
wasn't clensed I got a lot of compaints so I am only going to put out a
list that is 100% accurate, and that will take hours to produce, time I
don't have now because I am working on the release notes.

And things are getting addressed, like the pg_restore -j/-m flag and GIN
index changes, so there is movement, there just isn't documented
movement.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Tue, Mar 17, 2009 at 09:38:59AM -0400, Bruce Momjian wrote:
> Robert Haas wrote:
> > > Well, we have been working on stuff for the past month so it was not
> > > like we were waiting on SE-PG to move forward.
> >
> > Stuff related to the CommitFest?
> >
> > AFAICS, the only committer who has done any significant review or
> > committing of patches in the last month is Heikki, who extensively
> > reworked and then committed infrastructure changes for recovery on
> > February 18th (2 days shy of a month ago) and then extensively
> > reviewed Hot Standby and SE-PostgreSQL. It's really, really good that
> > those patches have finally received some extensive review, both
> > because now some version of each of them will likely make it into 8.5,
> > and because now we have only a handful of patches left that Tom has
> > said are pretty close to being committable. But I don't see how you
> > can say it didn't delay the release.
>
> You are assuming that only commit-fest work is required to get us to
> beta. You might remember the long list of open items I faced in January
> that I have whittled down, but I still have about twenty left.

What are those, and are there any you can delegate, or at least shout
out for help on?

> Tom has done work fixing optimizer bugs introduced in 8.4. I have had
> EnterpriseDB work to do and am working on the release notes now. The
> bottom line is that there is lots of cleanup required to get to beta
> independent of the last commit fest work.

How much of this cleanup work can be distributed?

> I agree if we had said "no" to those patches we could be farther
> now, but I am not sure how much farther.

One way to find out is to make a list of all the things that happen
and see how to get more people, productively, on it :)

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Bruce Momjian <bruce(at)momjian(dot)us> writes:
> Robert Treat wrote:
>> ... Perhaps we
>> need to take a fresh look at your list of twenty things and see what can be
>> delegated out to others.

> Yep, I agree. The problem is that last time I put out a list that
> wasn't clensed I got a lot of compaints so I am only going to put out a
> list that is 100% accurate, and that will take hours to produce, time I
> don't have now because I am working on the release notes.

I complained to Bruce about this off-list a couple days ago --- doing
the release notes before putting up the TODO list is backwards,
precisely because it serializes everybody else on the release note work.

Personally I've been distracted by some Red Hat responsibilities, but
am now clear of the main one and will get back to reviewing the last
couple of patches today. I think we should be focusing on getting
things done with the idea that we are trying to be ready for beta
in a week or two.

regards, tom lane

On Fri, Mar 20, 2009 at 10:34 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Robert Treat wrote:
>> On Tuesday 17 March 2009 09:38:59 Bruce Momjian wrote:
>> > You are assuming that only commit-fest work is required to get us to
>> > beta.  You might remember the long list of open items I faced in January
>> > that I have whittled down, but I still have about twenty left.
>> >
>>
>> I think part of the perception of the project sitting around doing nothing
>> isn't so much that you/tom are doing nothing, but others who were doing
>> review or coding features are now caught in limbo. One of the things the
>> commitfest has been successful at is helping delegate code review. Perhaps we
>> need to take a fresh look at your list of twenty things and see what can be
>> delegated out to others.
>
> Yep, I agree.  The problem is that last time I put out a list that
> wasn't clensed I got a lot of compaints so I am only going to put out a
> list that is 100% accurate, and that will take hours to produce, time I
> don't have now because I am working on the release notes.

I don't want to be a grump, but this is a false dichotomy. What you
put out last time was a dump of 700 emails, much of which was
irrelevant and most of the rest of which was duplicative of itself or
the CommitFest wiki. Now, it may be true that even if your list was
80% accurate, people would still have complained about the other 20%,
but we don't know that, because the actual list was at best 10%
accurate, and of course people are going to complain about that.

I personally think that the way pgsql-hackers organizes itself using
email is completely insane. The only reason that you need to write
the release notes instead of, say, me, is because the only information
on what needs to go into them is buried in a thicket of CVS commit
messages that I am not nearly brave enough to attempt to penetrate. I
suggested putting them in CVS yesterday; Tom didn't like that, but
what about a wiki page or a database? grep 'release notes'
/last/six/months/of/email can't possibly be the best way to do this.
Given any sort of list to work from, even one that is totally
disorganized and written in broken English, I can't believe this is
more than an hour or two of work, and I'd be more than happy to take a
crack at it (I'm probably not the only one, either).

Similarly, the only reason we don't have a workable TODO list is
because you're attempting to extract it from a disorganized jumble of
email after the fact, instead of maintaining it publicly and adding
and removing items along the way. It might be slightly more work to
think up a reasonable label for an action item at the time you learn
about it than to just dump the email in a folder, but I think the time
you didn't have to spend sorting through it later would more than make
up for it. Plus then items could be worked on along the way instead
of waiting until the bitter end when the TODO list materializes and we
all say "Oh, really?".

...Robert

Robert Haas wrote:
> Similarly, the only reason we don't have a workable TODO list is
> because you're attempting to extract it from a disorganized jumble of
> email after the fact, instead of maintaining it publicly and adding
> and removing items along the way. It might be slightly more work to
> think up a reasonable label for an action item at the time you learn
> about it than to just dump the email in a folder, but I think the time
> you didn't have to spend sorting through it later would more than make
> up for it. Plus then items could be worked on along the way instead
> of waiting until the bitter end when the TODO list materializes and we
> all say "Oh, really?".

The TODO list is already on the Wiki. I've edited it a few times when
I've spotted TODO-worthy ideas on the mailing lists.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Robert Haas escribió:

> I personally think that the way pgsql-hackers organizes itself using
> email is completely insane.

Note that during the 8.4 timeframe we've stolen a lot of work from
Bruce. The TODO list was moved to the wiki, for one; the "patch queue"
was also moved to the wiki. Now the FAQ has moved to wiki (and has
already seen lots of improvement, so it was clearly a good move).
Previously this was all handled as email boxes, so while some
inefficiences in the process still remain, we're a lot better than we
were in the 8.3 cycle.

> Similarly, the only reason we don't have a workable TODO list is
> because you're attempting to extract it from a disorganized jumble of
> email after the fact, instead of maintaining it publicly and adding
> and removing items along the way.

We do have an alternative "open items" list,
http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items
However, it's incomplete. It is a bit sad that nobody can complete it,
because Bruce has taken "pgpatches" out of the air. (Of course, anybody
could go fetch all the pgsql-hackers emails and dig up the remaining
open items to add them there, but that would be duplicative of the
effort Bruce has already put into his own queue).

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

This is about the reaction I expected, and is again so far off the mark
that I will just continue doing what I think is best.

Why doesn't someone offer to take my mbox file and generate a list from
that?

---------------------------------------------------------------------------

Robert Haas wrote:
> On Fri, Mar 20, 2009 at 10:34 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > Robert Treat wrote:
> >> On Tuesday 17 March 2009 09:38:59 Bruce Momjian wrote:
> >> > You are assuming that only commit-fest work is required to get us to
> >> > beta. ?You might remember the long list of open items I faced in January
> >> > that I have whittled down, but I still have about twenty left.
> >> >
> >>
> >> I think part of the perception of the project sitting around doing nothing
> >> isn't so much that you/tom are doing nothing, but others who were doing
> >> review or coding features are now caught in limbo. One of the things the
> >> commitfest has been successful at is helping delegate code review. Perhaps we
> >> need to take a fresh look at your list of twenty things and see what can be
> >> delegated out to others.
> >
> > Yep, I agree. ?The problem is that last time I put out a list that
> > wasn't clensed I got a lot of compaints so I am only going to put out a
> > list that is 100% accurate, and that will take hours to produce, time I
> > don't have now because I am working on the release notes.
>
> I don't want to be a grump, but this is a false dichotomy. What you
> put out last time was a dump of 700 emails, much of which was
> irrelevant and most of the rest of which was duplicative of itself or
> the CommitFest wiki. Now, it may be true that even if your list was
> 80% accurate, people would still have complained about the other 20%,
> but we don't know that, because the actual list was at best 10%
> accurate, and of course people are going to complain about that.
>
> I personally think that the way pgsql-hackers organizes itself using
> email is completely insane. The only reason that you need to write
> the release notes instead of, say, me, is because the only information
> on what needs to go into them is buried in a thicket of CVS commit
> messages that I am not nearly brave enough to attempt to penetrate. I
> suggested putting them in CVS yesterday; Tom didn't like that, but
> what about a wiki page or a database? grep 'release notes'
> /last/six/months/of/email can't possibly be the best way to do this.
> Given any sort of list to work from, even one that is totally
> disorganized and written in broken English, I can't believe this is
> more than an hour or two of work, and I'd be more than happy to take a
> crack at it (I'm probably not the only one, either).
>
> Similarly, the only reason we don't have a workable TODO list is
> because you're attempting to extract it from a disorganized jumble of
> email after the fact, instead of maintaining it publicly and adding
> and removing items along the way. It might be slightly more work to
> think up a reasonable label for an action item at the time you learn
> about it than to just dump the email in a folder, but I think the time
> you didn't have to spend sorting through it later would more than make
> up for it. Plus then items could be worked on along the way instead
> of waiting until the bitter end when the TODO list materializes and we
> all say "Oh, really?".
>
> ...Robert

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Alvaro Herrera wrote:
> Robert Haas escribi?:
>
> > I personally think that the way pgsql-hackers organizes itself using
> > email is completely insane.
>
> Note that during the 8.4 timeframe we've stolen a lot of work from
> Bruce. The TODO list was moved to the wiki, for one; the "patch queue"
> was also moved to the wiki. Now the FAQ has moved to wiki (and has
> already seen lots of improvement, so it was clearly a good move).
> Previously this was all handled as email boxes, so while some
> inefficiences in the process still remain, we're a lot better than we
> were in the 8.3 cycle.

Please take as much work from me as you can. However, looking at the
TODO commits, I am still the one doing most of the changes:

http://wiki.postgresql.org/index.php?title=Todo&action=history

> > Similarly, the only reason we don't have a workable TODO list is
> > because you're attempting to extract it from a disorganized jumble of
> > email after the fact, instead of maintaining it publicly and adding
> > and removing items along the way.
>
> We do have an alternative "open items" list,
> http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items
> However, it's incomplete. It is a bit sad that nobody can complete it,
> because Bruce has taken "pgpatches" out of the air. (Of course, anybody
> could go fetch all the pgsql-hackers emails and dig up the remaining
> open items to add them there, but that would be duplicative of the
> effort Bruce has already put into his own queue).

Yep, anyone can do what I am doing now; there is no magic in my
fingers.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian escribió:
> Alvaro Herrera wrote:

> > Note that during the 8.4 timeframe we've stolen a lot of work from
> > Bruce. The TODO list was moved to the wiki, for one; the "patch queue"
> > was also moved to the wiki. Now the FAQ has moved to wiki (and has
> > already seen lots of improvement, so it was clearly a good move).
> > Previously this was all handled as email boxes, so while some
> > inefficiences in the process still remain, we're a lot better than we
> > were in the 8.3 cycle.
>
> Please take as much work from me as you can. However, looking at the
> TODO commits, I am still the one doing most of the changes:
>
> http://wiki.postgresql.org/index.php?title=Todo&action=history

Yes, but there are commits from others too. I think the TODO is seen as
a delicate document and other people is reluctant to edit it.

> > We do have an alternative "open items" list,
> > http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items
> > However, it's incomplete. It is a bit sad that nobody can complete it,
> > because Bruce has taken "pgpatches" out of the air.
>
> Yep, anyone can do what I am doing now; there is no magic in my
> fingers.

So are you going to publish your mbox?

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera wrote:
> Bruce Momjian escribi?:
> > Alvaro Herrera wrote:
>
> > > Note that during the 8.4 timeframe we've stolen a lot of work from
> > > Bruce. The TODO list was moved to the wiki, for one; the "patch queue"
> > > was also moved to the wiki. Now the FAQ has moved to wiki (and has
> > > already seen lots of improvement, so it was clearly a good move).
> > > Previously this was all handled as email boxes, so while some
> > > inefficiences in the process still remain, we're a lot better than we
> > > were in the 8.3 cycle.
> >
> > Please take as much work from me as you can. However, looking at the
> > TODO commits, I am still the one doing most of the changes:
> >
> > http://wiki.postgresql.org/index.php?title=Todo&action=history
>
> Yes, but there are commits from others too. I think the TODO is seen as
> a delicate document and other people is reluctant to edit it.
>
>
> > > We do have an alternative "open items" list,
> > > http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items
> > > However, it's incomplete. It is a bit sad that nobody can complete it,
> > > because Bruce has taken "pgpatches" out of the air.
> >
> > Yep, anyone can do what I am doing now; there is no magic in my
> > fingers.
>
> So are you going to publish your mbox?

After the complaints last time, no. If everyone says they are not going
to complain, I will, or email it to people who ask for it (and if you
ask for it, don't complain either).

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian escribió:
> Alvaro Herrera wrote:
> > Bruce Momjian escribi?:

> > > > We do have an alternative "open items" list,
> > > > http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items
> > > > However, it's incomplete. It is a bit sad that nobody can complete it,
> > > > because Bruce has taken "pgpatches" out of the air.
> > >
> > > Yep, anyone can do what I am doing now; there is no magic in my
> > > fingers.
> >
> > So are you going to publish your mbox?
>
> After the complaints last time, no. If everyone says they are not going
> to complain, I will, or email it to people who ask for it (and if you
> ask for it, don't complain either).

Well, I spent some time pointing out items to prune, and several others
did too. Are you saying that we wasted our time doing that?

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera wrote:
> Bruce Momjian escribi?:
> > Alvaro Herrera wrote:
> > > Bruce Momjian escribi?:
>
> > > > > We do have an alternative "open items" list,
> > > > > http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items
> > > > > However, it's incomplete. It is a bit sad that nobody can complete it,
> > > > > because Bruce has taken "pgpatches" out of the air.
> > > >
> > > > Yep, anyone can do what I am doing now; there is no magic in my
> > > > fingers.
> > >
> > > So are you going to publish your mbox?
> >
> > After the complaints last time, no. If everyone says they are not going
> > to complain, I will, or email it to people who ask for it (and if you
> > ask for it, don't complain either).
>
> Well, I spent some time pointing out items to prune, and several others
> did too. Are you saying that we wasted our time doing that?

Yes, I did get some helpful feedback from publishing the list, but most
felt it was a waste in showing it (including Robert Haas), so I have no
desire to repeat that.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Fri, Mar 20, 2009 at 1:08 PM, Heikki Linnakangas
<heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
> The TODO list is already on the Wiki. I've edited it a few times when I've
> spotted TODO-worthy ideas on the mailing lists.

Well, Bruce and Tom both made reference to something that Bruce was
going to produce along these lines... I think what we are talking
about is open items for 8.4beta, not dev projects in general.

...Robert

On Fri, Mar 20, 2009 at 1:08 PM, Alvaro Herrera
<alvherre(at)commandprompt(dot)com> wrote:
>> I personally think that the way pgsql-hackers organizes itself using
>> email is completely insane.
> Note that during the 8.4 timeframe we've stolen a lot of work from
> Bruce.  The TODO list was moved to the wiki, for one; the "patch queue"
> was also moved to the wiki.  Now the FAQ has moved to wiki (and has
> already seen lots of improvement, so it was clearly a good move).
> Previously this was all handled as email boxes, so while some
> inefficiences in the process still remain, we're a lot better than we
> were in the 8.3 cycle.

Good point.

>> Similarly, the only reason we don't have a workable TODO list is
>> because you're attempting to extract it from a disorganized jumble of
>> email after the fact, instead of maintaining it publicly and adding
>> and removing items along the way.
>
> We do have an alternative "open items" list,
> http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items
> However, it's incomplete.  It is a bit sad that nobody can complete it,
> because Bruce has taken "pgpatches" out of the air.  (Of course, anybody
> could go fetch all the pgsql-hackers emails and dig up the remaining
> open items to add them there, but that would be duplicative of the
> effort Bruce has already put into his own queue).

I don't even understand why we're interested in doing this. If the
patches weren't important enough for someone to add them to the
CommitFest wiki in October, why are we delaying the release to hunt
for them in March? I personally spent hours and hours of time in late
October dredging up every patch that looked anywhere close to
reviewable (not committable, reviewable!) and put them all on the
wiki. Now, it's certainly possible that I missed something, but
there's been plenty of time between then and now and I think there's
only been 1 or 2 complaints about things being overlooked (and those
were very, very old patches from before I started reading the mailing
list regularly).

...Robert

Robert Haas escribió:

> I don't even understand why we're interested in doing this. If the
> patches weren't important enough for someone to add them to the
> CommitFest wiki in October, why are we delaying the release to hunt
> for them in March?

The problem is not patches that were not committed, but rather loose
ends in patches that were.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

On Fri, Mar 20, 2009 at 1:10 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> This is about the reaction I expected, and is again so far off the mark
> that I will just continue doing what I think is best.

Would you like to explain WHY it's off the mark?

> Why doesn't someone offer to take my mbox file and generate a list from
> that?

A list of what? CVS commit messages that needed to be added to the
release notes? Patches that were committed but need follow-up work?
Patches that were missed and never added to the Wiki? Other
release-related tasks that need to be done?

...Robert

On Fri, Mar 20, 2009 at 1:40 PM, Alvaro Herrera
<alvherre(at)commandprompt(dot)com> wrote:
> Robert Haas escribió:
>> I don't even understand why we're interested in doing this.  If the
>> patches weren't important enough for someone to add them to the
>> CommitFest wiki in October, why are we delaying the release to hunt
>> for them in March?
> The problem is not patches that were not committed, but rather loose
> ends in patches that were.

There seems to be a reasonably well-maintained list of open items here:

http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items

The only thing I can recall that is outstanding but not mentioned here
is the controversy over the behavior of the various \d commands. But
even that is something that can be changed after getting feedback from
beta-users and beta-testers, and we might even have a better idea what
to do about it at that point. Once we release, we're probably stuck
with whatever the behavior is at that point, but I think we've got
enough time between now and then that we don't need to get too worried
about it now.

Of course, if this list is radically incomplete, then it doesn't help
much, but does anyone think that's the case? My impression is that
most of the major open items (e.g. follow-on cleanup patch for
column-level permissions, Kevin Grittner's planner issues) were tied
up some time ago. If there are other outstanding issues, why can't
they just wait to 8.5? I guess I'm just confused as to how this
process works (I'm new around here?). As far as I can tell, the
committers are very careful about not committing stuff to the tree, so
that means that the tree pretty much always works and doesn't usually
contain too much that's half-baked. So it would seem like that would
make going to beta mostly a matter of finishing all the committing,
and maybe addressing the documentation issues mentioned on the open
items list linked above. I gather from Bruce's comments, and Tom's,
that there's more to it than that, but I'm sort of in the dark on the
details.

...Robert

Robert Haas escribió:
> On Fri, Mar 20, 2009 at 1:40 PM, Alvaro Herrera
> <alvherre(at)commandprompt(dot)com> wrote:
> > Robert Haas escribió:
> >> I don't even understand why we're interested in doing this.  If the
> >> patches weren't important enough for someone to add them to the
> >> CommitFest wiki in October, why are we delaying the release to hunt
> >> for them in March?
> > The problem is not patches that were not committed, but rather loose
> > ends in patches that were.
>
> There seems to be a reasonably well-maintained list of open items here:
>
> http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items

Hey, dude, I pointed that URL in a followup to a message from you,
earlier today.

> Of course, if this list is radically incomplete, then it doesn't help
> much, but does anyone think that's the case?

We don't know -- Bruce's list may contain something, but since it's
hidden we can't do anything. Maybe it is all already-completed items,
or redundant with the list on wiki, or maybe they're things that can be
postponed for 8.5. No way to tell.

My guess is that Bruce's queue contains a reasonably small number of
minor fixups that need to happen before release.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

On Fri, Mar 20, 2009 at 3:13 PM, Alvaro Herrera
<alvherre(at)commandprompt(dot)com> wrote:
> Robert Haas escribió:
>> On Fri, Mar 20, 2009 at 1:40 PM, Alvaro Herrera
>> <alvherre(at)commandprompt(dot)com> wrote:
>> > Robert Haas escribió:
>> >> I don't even understand why we're interested in doing this.  If the
>> >> patches weren't important enough for someone to add them to the
>> >> CommitFest wiki in October, why are we delaying the release to hunt
>> >> for them in March?
>> > The problem is not patches that were not committed, but rather loose
>> > ends in patches that were.
>>
>> There seems to be a reasonably well-maintained list of open items here:
>>
>> http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items
>
> Hey, dude, I pointed that URL in a followup to a message from you,
> earlier today.

So you did. Sorry... I seem to be rubbing everyone the wrong way
today. (Did I hear someone say, "who said anything about today"?)

>> Of course, if this list is radically incomplete, then it doesn't help
>> much, but does anyone think that's the case?
>
> We don't know -- Bruce's list may contain something, but since it's
> hidden we can't do anything.  Maybe it is all already-completed items,
> or redundant with the list on wiki, or maybe they're things that can be
> postponed for 8.5.  No way to tell.
>
> My guess is that Bruce's queue contains a reasonably small number of
> minor fixups that need to happen before release.

Well, I guess there's no point in speculating. We'll just have to
wait and see. Bruce seems to feel that what he's doing is something
that no one else has done or is willing to do, and I guess I'm
wondering whether that's really the case. I think a lot of people
have worked hard to make sure that all of the patches that were
submitted got dealt with and the open items resulting from those
patches were closed. There is always more to do, of course.

...Robert

Robert Haas wrote:
> >> Of course, if this list is radically incomplete, then it doesn't help
> >> much, but does anyone think that's the case?
> >
> > We don't know -- Bruce's list may contain something, but since it's
> > hidden we can't do anything. ?Maybe it is all already-completed items,
> > or redundant with the list on wiki, or maybe they're things that can be
> > postponed for 8.5. ?No way to tell.
> >
> > My guess is that Bruce's queue contains a reasonably small number of
> > minor fixups that need to happen before release.
>
> Well, I guess there's no point in speculating. We'll just have to
> wait and see. Bruce seems to feel that what he's doing is something
> that no one else has done or is willing to do, and I guess I'm
> wondering whether that's really the case. I think a lot of people
> have worked hard to make sure that all of the patches that were
> submitted got dealt with and the open items resulting from those
> patches were closed. There is always more to do, of course.

I did offer to post my mbox file so people could see what I have as open
8.4 items, but the "no complaining" requirement seems to have eliminated
volunteers.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Fri, Mar 20, 2009 at 3:23 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> I did offer to post my mbox file so people could see what I have as open
> 8.4 items, but the "no complaining" requirement seems to have eliminated
> volunteers.

IIRC, the biggest problem we had last time (apart from the
complaining) was that there was no easy way to refer to particular
items in the list. If you could find a way to make the numbering
stable (first 8 characters of the SHA-1 hash of the contents, a la
git? drop every message into a file whose name is the initial number
of that messages, and keep the numbers the same as files are
removed?), it would be easier for people tell you which items could be
removed and why. I think you suggested referencing by subject lines
last time, but the problem is that you had so many duplicates that it
was hard to explain which ones should be kept and which one should be
left, especially because you were making a lot of edits at the same
time (subject A three down from subject B ceases to be clear when
subject B has in the meantime been removed, or when one of the
intervening messages has been removed, thus making the one three down
a different message with the same subject).

Lest I be accused of offering suggestions in place of actual help, if
you send me whatever script you're using to put this on the web now I
would be willing to take a crack at making the changes described
above. I really don't believe dumping everything into an mbox file is
the best way of keeping track of open to-do items - but if that's the
only choice, then we might as well try to make it as painless as
possible. (I offered to write a webapp to attempt to improve upon the
CommitFest wiki a while ago, so we could eventually do things like...
run a command to dump the latest version of every outstanding patch
into a given directory... and a whole lot of people offered
suggestions for features... but I haven't been able to connect up
with Magnus, so that's gone nowhere.)

...Robert

Robert Haas wrote:
> On Fri, Mar 20, 2009 at 3:23 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > I did offer to post my mbox file so people could see what I have as open
> > 8.4 items, but the "no complaining" requirement seems to have eliminated
> > volunteers.
>
> IIRC, the biggest problem we had last time (apart from the
> complaining) was that there was no easy way to refer to particular
> items in the list. If you could find a way to make the numbering
> stable (first 8 characters of the SHA-1 hash of the contents, a la
> git? drop every message into a file whose name is the initial number
> of that messages, and keep the numbers the same as files are
> removed?), it would be easier for people tell you which items could be
> removed and why. I think you suggested referencing by subject lines
> last time, but the problem is that you had so many duplicates that it
> was hard to explain which ones should be kept and which one should be
> left, especially because you were making a lot of edits at the same
> time (subject A three down from subject B ceases to be clear when
> subject B has in the meantime been removed, or when one of the
> intervening messages has been removed, thus making the one three down
> a different message with the same subject).
>
> Lest I be accused of offering suggestions in place of actual help, if
> you send me whatever script you're using to put this on the web now I
> would be willing to take a crack at making the changes described
> above. I really don't believe dumping everything into an mbox file is
> the best way of keeping track of open to-do items - but if that's the
> only choice, then we might as well try to make it as painless as
> possible. (I offered to write a webapp to attempt to improve upon the
> CommitFest wiki a while ago, so we could eventually do things like...
> run a command to dump the latest version of every outstanding patch
> into a given directory... and a whole lot of people offered
> suggestions for features... but I haven't been able to connect up
> with Magnus, so that's gone nowhere.)

You can use my emails to make a master list --- there is no need to make
mine the master.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Fri, Mar 20, 2009 at 3:50 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> You can use my emails to make a master list --- there is no need to make
> mine the master.

OK, good enough. Can you post a link to the raw mbox file?

...Robert

Robert Haas wrote:
> On Fri, Mar 20, 2009 at 3:50 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > You can use my emails to make a master list --- there is no need to make
> > mine the master.
>
> OK, good enough. Can you post a link to the raw mbox file?

OK, done:

http://momjian.us/cgi-bin/pgsql/open

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Fri, Mar 20, 2009 at 4:03 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Robert Haas wrote:
>> On Fri, Mar 20, 2009 at 3:50 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> > You can use my emails to make a master list --- there is no need to make
>> > mine the master.
>>
>> OK, good enough.  Can you post a link to the raw mbox file?
>
> OK, done:
>
>        http://momjian.us/cgi-bin/pgsql/open

The "Download in mbox format" link doesn't work.

...Robert

Here are some of the emails I consider open for 8.4:

http://momjian.us/cgi-bin/pgsql/open

(Same email, using updated subject line.)

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Fri, 2009-03-20 at 16:03 -0400, Robert Haas wrote:
> On Fri, Mar 20, 2009 at 4:03 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > Robert Haas wrote:
> >> On Fri, Mar 20, 2009 at 3:50 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> >> > You can use my emails to make a master list --- there is no need to make
> >> > mine the master.
> >>
> >> OK, good enough. Can you post a link to the raw mbox file?
> >
> > OK, done:
> >
> > http://momjian.us/cgi-bin/pgsql/open
>
> The "Download in mbox format" link doesn't work.

Worked for me.

Joshua D. Drake

>
> ...Robert
>
--
PostgreSQL - XMPP: jdrake(at)jabber(dot)postgresql(dot)org
Consulting, Development, Support, Training
503-667-4564 - http://www.commandprompt.com/
The PostgreSQL Company, serving since 1997

Bruce Momjian wrote:
> Here are some of the emails I consider open for 8.4:
>
> http://momjian.us/cgi-bin/pgsql/open
>
> (Same email, using updated subject line.)

mbox download URL fixed.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Fri, Mar 20, 2009 at 4:06 PM, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
> On Fri, 2009-03-20 at 16:03 -0400, Robert Haas wrote:
>> On Fri, Mar 20, 2009 at 4:03 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> > Robert Haas wrote:
>> >> On Fri, Mar 20, 2009 at 3:50 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> >> > You can use my emails to make a master list --- there is no need to make
>> >> > mine the master.
>> >>
>> >> OK, good enough.  Can you post a link to the raw mbox file?
>> >
>> > OK, done:
>> >
>> >        http://momjian.us/cgi-bin/pgsql/open
>>
>> The "Download in mbox format" link doesn't work.
>
> Worked for me.

Ah, it's working now.

...Robert

Bruce Momjian wrote:
> Bruce Momjian wrote:
> > Here are some of the emails I consider open for 8.4:
> >
> > http://momjian.us/cgi-bin/pgsql/open
> >
> > (Same email, using updated subject line.)
>
> mbox download URL fixed.

Let me add that my item tracking is more of a blob that expands and
contracts every day, rather than some fixed list.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Fri, Mar 20, 2009 at 4:08 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Bruce Momjian wrote:
>> Bruce Momjian wrote:
>> > Here are some of the emails I consider open for 8.4:
>> >
>> >     http://momjian.us/cgi-bin/pgsql/open
>> >
>> > (Same email, using updated subject line.)
>>
>> mbox download URL fixed.
>
> Let me add that my item tracking is more of a blob that expands and
> contracts every day, rather than some fixed list.

how many of these things are really blocking 8.4? or is that not known?

merlin

Merlin Moncure wrote:
> On Fri, Mar 20, 2009 at 4:08 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > Bruce Momjian wrote:
> >> Bruce Momjian wrote:
> >> > Here are some of the emails I consider open for 8.4:
> >> >
> >> > ? ? http://momjian.us/cgi-bin/pgsql/open
> >> >
> >> > (Same email, using updated subject line.)
> >>
> >> mbox download URL fixed.
> >
> > Let me add that my item tracking is more of a blob that expands and
> > contracts every day, rather than some fixed list.
>
> how many of these things are really blocking 8.4? or is that not known?

Not known, and changes every day.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On 3/20/09, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Here are some of the emails I consider open for 8.4:
>
> http://momjian.us/cgi-bin/pgsql/open
>

Item #3 Extending grant insert on tables to sequences

is not for 8.4, i haven't reviewed it to adjust the patch after column
privileges was applied. i will update it and put a patch for 8.5 ASAP

--
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. +59387171157

Alvaro Herrera wrote:
> Bruce Momjian escribi?:
> > Alvaro Herrera wrote:
>
> > > Note that during the 8.4 timeframe we've stolen a lot of work from
> > > Bruce. The TODO list was moved to the wiki, for one; the "patch queue"
> > > was also moved to the wiki. Now the FAQ has moved to wiki (and has
> > > already seen lots of improvement, so it was clearly a good move).
> > > Previously this was all handled as email boxes, so while some
> > > inefficiences in the process still remain, we're a lot better than we
> > > were in the 8.3 cycle.
> >
> > Please take as much work from me as you can. However, looking at the
> > TODO commits, I am still the one doing most of the changes:
> >
> > http://wiki.postgresql.org/index.php?title=Todo&action=history
>
> Yes, but there are commits from others too. I think the TODO is seen as
> a delicate document and other people is reluctant to edit it.

Agreed. My only point is that many thought that putting the TODO in a
wiki would significantly reduce my TODO maintenance effort, which it has
not; which is fine, and I expected this outcome, but others did not.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Robert Haas wrote:
> I personally think that the way pgsql-hackers organizes itself using
> email is completely insane. The only reason that you need to write
> the release notes instead of, say, me, is because the only information
> on what needs to go into them is buried in a thicket of CVS commit
> messages that I am not nearly brave enough to attempt to penetrate. I
> suggested putting them in CVS yesterday; Tom didn't like that, but
> what about a wiki page or a database? grep 'release notes'
> /last/six/months/of/email can't possibly be the best way to do this.
> Given any sort of list to work from, even one that is totally
> disorganized and written in broken English, I can't believe this is
> more than an hour or two of work, and I'd be more than happy to take a
> crack at it (I'm probably not the only one, either).

Let me just add, as a way of macro-understanding our approach to
development, that the Postgres community has always been set up to get
the maximum feedback from the community, even if it sometimes increases
the work required by a few core folks to keep things going.

So some of the things we do that seem inefficient are done because many
feel a more structured approach would limit our ability to harness the
strengths of our community. For example, moving to a bug tracking
system would make some things much easier, but would probably dampen our
momentum. Handling discussions via web forums instead of email would
probably have the same effect. Of course, I might be wrong, but that is
what many in the community think.

I think the example of moving the TODO list to a wiki, that was supposed
to relive a lot of the burden I carry to maintain the TODO list, has
really not affected my workload much, which kind of reinforces the
feeling that our existing setup is probably the best we are going to do.
Of course, the commit fest wikis have helped, so I guess there is room
for improvement in some places.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Fri, Mar 20, 2009 at 11:59 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Robert Haas wrote:
>> I personally think that the way pgsql-hackers organizes itself using
>> email is completely insane.  The only reason that you need to write
>> the release notes instead of, say, me, is because the only information
>> on what needs to go into them is buried in a thicket of CVS commit
>> messages that I am not nearly brave enough to attempt to penetrate.  I
>> suggested putting them in CVS yesterday; Tom didn't like that, but
>> what about a wiki page or a database?  grep 'release notes'
>> /last/six/months/of/email can't possibly be the best way to do this.
>> Given any sort of list to work from, even one that is totally
>> disorganized and written in broken English, I can't believe this is
>> more than an hour or two of work, and I'd be more than happy to take a
>> crack at it (I'm probably not the only one, either).
>
> Let me just add, as a way of macro-understanding our approach to
> development, that the Postgres community has always been set up to get
> the maximum feedback from the community, even if it sometimes increases
> the work required by a few core folks to keep things going.
>
> So some of the things we do that seem inefficient are done because many
> feel a more structured approach would limit our ability to harness the
> strengths of our community.  For example, moving to a bug tracking
> system would make some things much easier, but would probably dampen our
> momentum.  Handling discussions via web forums instead of email would
> probably have the same effect.  Of course, I might be wrong, but that is
> what many in the community think.

Oh, I'm not objecting to email as a way of communicating. I think a
bug tracking system or web forums would increase the amount of effort
required to keep up to date on what is going on, and I can't imagine
what the corresponding advantage would be. What I don't like is the
use of email as an *organizational* tool, because (even with Google)
it's hard to go back to a pile of email and fish out the items that
are still relevant. If there's a list of things that need to be put
into the release notes or a list of things that need to be done by 8.4
or a list of patches that need to be reviewed, I think it makes sense
to have an explicit list of some kind.

I think there is near-universal agreement that the CommitFest wiki has
been very succesful. I've certainly spent a lot of time keeping it up
to date, which wouldn't have been possible with the old system, and I
at least find it much easier to refer to. I don't see why the same
thing couldn't be done with release notes. Heikki asked this week
where he should document an item to mentioned in the release notes,
and the answer was in the CVS commit message. If the answer had been,
in a wiki page, he wouldn't have minded, and if we did that
consistently for a whole release cycle, it would probably save you
quite a bit of time finding everything again at the end. Or so it
seems to me; but I might be wrong.

> I think the example of moving the TODO list to a wiki, that was supposed
> to relive a lot of the burden I carry to maintain the TODO list, has
> really not affected my workload much, which kind of reinforces the
> feeling that our existing setup is probably the best we are going to do.
> Of course, the commit fest wikis have helped, so I guess there is room
> for improvement in some places.

Well, the TODO list, because it's traditionally been filtered by you,
carries the implication that the items therein are not just any old
thing that's been suggested by someone, but things where there was
some level agreement (from you, if not from anyone else, but that
carries some weight all by itself) that they might be worthwhile.
Maybe that wasn't your intention, but I think people see it that way
to some degree.

My concern with the list of outstanding items for 8.4 based on a quick
look is that I think many of those things are not, in fact,
outstanding items for 8.4, and those that are may not be important
enough to hold up beta for. Now since I haven't read through them all
yet, I'm not 100% sure of that, but that's my concern for what it's
worth.

...Robert

Robert Haas wrote:
> Oh, I'm not objecting to email as a way of communicating. I think a
> bug tracking system or web forums would increase the amount of effort
> required to keep up to date on what is going on, and I can't imagine
> what the corresponding advantage would be. What I don't like is the
> use of email as an *organizational* tool, because (even with Google)
> it's hard to go back to a pile of email and fish out the items that
> are still relevant. If there's a list of things that need to be put
> into the release notes or a list of things that need to be done by 8.4
> or a list of patches that need to be reviewed, I think it makes sense
> to have an explicit list of some kind.
>
> I think there is near-universal agreement that the CommitFest wiki has
> been very succesful. I've certainly spent a lot of time keeping it up
> to date, which wouldn't have been possible with the old system, and I
> at least find it much easier to refer to. I don't see why the same
> thing couldn't be done with release notes. Heikki asked this week
> where he should document an item to mentioned in the release notes,
> and the answer was in the CVS commit message. If the answer had been,
> in a wiki page, he wouldn't have minded, and if we did that
> consistently for a whole release cycle, it would probably save you
> quite a bit of time finding everything again at the end. Or so it
> seems to me; but I might be wrong.

Collecting the release note items takes less than a day; it takes 5-6
days to research and reword them all to make a consistent set of release
notes. I don't see how pushing the burden of release _item_ tracking to
every committer is going to significantly reduce the job of creating
the release notes. I can see it increasing the burden on the community.

Remember the audience for a commit message (backend technical) is not
the same as the for release notes (end user), so you would have to
educate everyone and make sure they are all consistent.

> > I think the example of moving the TODO list to a wiki, that was supposed
> > to relive a lot of the burden I carry to maintain the TODO list, has
> > really not affected my workload much, which kind of reinforces the
> > feeling that our existing setup is probably the best we are going to do.
> > Of course, the commit fest wikis have helped, so I guess there is room
> > for improvement in some places.
>
> Well, the TODO list, because it's traditionally been filtered by you,
> carries the implication that the items therein are not just any old
> thing that's been suggested by someone, but things where there was
> some level agreement (from you, if not from anyone else, but that
> carries some weight all by itself) that they might be worthwhile.
> Maybe that wasn't your intention, but I think people see it that way
> to some degree.

It was not my intention. Once an item gets a concensus community
opinion, I add it to the TODO list, and hopefully others can as well.

> My concern with the list of outstanding items for 8.4 based on a quick
> look is that I think many of those things are not, in fact,
> outstanding items for 8.4, and those that are may not be important
> enough to hold up beta for. Now since I haven't read through them all
> yet, I'm not 100% sure of that, but that's my concern for what it's
> worth.

Certainly not all are valid, and some are things we _should_ fix for
8.4, even if not necessary, but again it is based on time and manpower,
but again, once the release notes are done I can focus on the list.
Hopefully the emails will jog people's memory that we do have some open
stuff.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Saturday 21 March 2009 09:04:12 Robert Haas wrote:
> On Fri, Mar 20, 2009 at 11:59 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > Robert Haas wrote:
<snip>
> My concern with the list of outstanding items for 8.4 based on a quick
> look is that I think many of those things are not, in fact,
> outstanding items for 8.4, and those that are may not be important
> enough to hold up beta for. Now since I haven't read through them all
> yet, I'm not 100% sure of that, but that's my concern for what it's
> worth.
>

Robert, this has been discussed many times before, and most people agree with
you, but Bruce doesn't. I think the ony way this will change is if someone
takes on the role of "release notes manager", subscrbes to pgsql-commits, and
then starts updating a wiki page as each item is committed. Once other
committers see that, I'm guessing they will start helping, and eventually
Bruce will join in. Outside of that I think we're wasting our time on this.

--
Robert Treat
Conjecture: http://www.xzilla.net
Consulting: http://www.omniti.com

On Sat, 2009-03-21 at 12:02 -0400, Robert Treat wrote:
> On Saturday 21 March 2009 09:04:12 Robert Haas wrote:
> > On Fri, Mar 20, 2009 at 11:59 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > > Robert Haas wrote:
> <snip>
> > My concern with the list of outstanding items for 8.4 based on a quick
> > look is that I think many of those things are not, in fact,
> > outstanding items for 8.4, and those that are may not be important
> > enough to hold up beta for. Now since I haven't read through them all
> > yet, I'm not 100% sure of that, but that's my concern for what it's
> > worth.
> >
>
> Robert, this has been discussed many times before, and most people agree with
> you, but Bruce doesn't. I think the ony way this will change is if someone
> takes on the role of "release notes manager", subscrbes to pgsql-commits, and
> then starts updating a wiki page as each item is committed. Once other
> committers see that, I'm guessing they will start helping, and eventually
> Bruce will join in. Outside of that I think we're wasting our time on this.

This is actually a good point regardless of Bruce. The run down is this:

We have an individual that does stuff a certain way. That way works for
him. That individual is the one actually doing the work.

We have other individuals who would like to do it a different way.

I invite those individuals to do so. If there way proves to be more
efficient the community will move in that direction.

Joshua D. Drake

>
> --
> Robert Treat
> Conjecture: http://www.xzilla.net
> Consulting: http://www.omniti.com
>
--
PostgreSQL - XMPP: jdrake(at)jabber(dot)postgresql(dot)org
Consulting, Development, Support, Training
503-667-4564 - http://www.commandprompt.com/
The PostgreSQL Company, serving since 1997

On Sat, Mar 21, 2009 at 12:55 PM, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
>> Robert,  this has been discussed many times before, and most people agree with
>> you, but Bruce doesn't. I think the ony way this will change is if someone
>> takes on the role of "release notes manager", subscrbes to pgsql-commits, and
>> then starts updating a wiki page as each item is committed. Once other
>> committers see that, I'm guessing they will start helping, and eventually
>> Bruce will join in. Outside of that I think we're wasting our time on this.
>
> This is actually a good point regardless of Bruce. The run down is this:
>
> We have an individual that does stuff a certain way. That way works for
> him. That individual is the one actually doing the work.
>
> We have other individuals who would like to do it a different way.
>
> I invite those individuals to do so. If there way proves to be more
> efficient the community will move in that direction.

Sadly, this approach seems to have a high percentage probability of
being completely wasted effort. So I agree with Robert Treat: we're
wasting our time on this. As I said upthread, I am still willing to
help edit release notes, either now or most likely in the future, if
there is a list of commits to start from (and although I may find it
odd, Bruce feels it's the easier half of the job, so, fine). If that
is helpful, great. If it's not, that's OK too.

...Robert

Robert Haas wrote:
> On Sat, Mar 21, 2009 at 12:55 PM, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
> >> Robert, ?this has been discussed many times before, and most people agree with
> >> you, but Bruce doesn't. I think the ony way this will change is if someone
> >> takes on the role of "release notes manager", subscrbes to pgsql-commits, and
> >> then starts updating a wiki page as each item is committed. Once other
> >> committers see that, I'm guessing they will start helping, and eventually
> >> Bruce will join in. Outside of that I think we're wasting our time on this.
> >
> > This is actually a good point regardless of Bruce. The run down is this:
> >
> > We have an individual that does stuff a certain way. That way works for
> > him. That individual is the one actually doing the work.
> >
> > We have other individuals who would like to do it a different way.
> >
> > I invite those individuals to do so. If there way proves to be more
> > efficient the community will move in that direction.
>
> Sadly, this approach seems to have a high percentage probability of
> being completely wasted effort. So I agree with Robert Treat: we're
> wasting our time on this. As I said upthread, I am still willing to
> help edit release notes, either now or most likely in the future, if
> there is a list of commits to start from (and although I may find it
> odd, Bruce feels it's the easier half of the job, so, fine). If that
> is helpful, great. If it's not, that's OK too.

So far taking the CVS logs and making a list of only the items we want
for the release notes took one day; researching and rewording the items
so they are ready for the release notes took five days; grouping them
into sections and rewording/combining, 1/2 a day, and adding SGML markup
will be another few hours.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Mar 24, 2009, at 9:41 AM, Bruce Momjian wrote:

> So far taking the CVS logs and making a list of only the items we want
> for the release notes took one day; researching and rewording the
> items
> so they are ready for the release notes took five days; grouping them
> into sections and rewording/combining, 1/2 a day, and adding SGML
> markup
> will be another few hours.

FWIW, this is a thankless job, and I commend you for taking it on,
Bruce. Others may feel that there are better ways to do it, or to
track changes through time so it's easier to generate the changes list
at release time, but I sure as hell wouldn't want to have to do what
you do for this, so many thanks for putting together a useful list of
changes every release.

Thanks,

David

David E. Wheeler wrote:
> On Mar 24, 2009, at 9:41 AM, Bruce Momjian wrote:
>
> > So far taking the CVS logs and making a list of only the items we want
> > for the release notes took one day; researching and rewording the
> > items
> > so they are ready for the release notes took five days; grouping them
> > into sections and rewording/combining, 1/2 a day, and adding SGML
> > markup
> > will be another few hours.
>
> FWIW, this is a thankless job, and I commend you for taking it on,
> Bruce. Others may feel that there are better ways to do it, or to
> track changes through time so it's easier to generate the changes list
> at release time, but I sure as hell wouldn't want to have to do what
> you do for this, so many thanks for putting together a useful list of
> changes every release.

A preliminary version of the 8.4 release note are now online; the
details are on my blog, including the release note creation process:

http://momjian.us/main/blogs/pgblog.html#March_25_2009

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

OK, I am all wet. I now understand why the editing is the
time-consuming part of this job. On the plus side it is probably
possible to parallelize it to some degree by splitting the list into N
pieces after the "remove insignificant items" step.

With respect to this item:
Disable appending of the epoch date/time when '%' escapes are missing
in log_filename (Robert Haas)
I might suggest explaining it this way:
This change makes it easier to use PostgreSQL in conjunction with an
external log rotation tool.

The following item uses "of" where it should say "if":
Throw an error of an escape character is the last character in a LIKE
pattern (nothing to escape) (Tom)

...Robert

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> OK, I am all wet. I now understand why the editing is the
> time-consuming part of this job. On the plus side it is probably
> possible to parallelize it to some degree by splitting the list into N
> pieces after the "remove insignificant items" step.

The advantage of having one person do it (and do it over a short period
of time) is that you end up with a fairly uniform "voice" across the
whole set of notes. Since we lack a professional copy editor, we'd have
a hard time coming out with something that wasn't pretty obviously a
patchwork if several people did bits of it.

In any case, the release notes aren't normally a bottleneck. I still
think that Bruce had his priorities out of whack in not cleaning up
his open-items list before doing this. If he had done so, nobody
would have noticed how long the notes took.

regards, tom lane

All,

> In any case, the release notes aren't normally a bottleneck. I still
> think that Bruce had his priorities out of whack in not cleaning up
> his open-items list before doing this. If he had done so, nobody
> would have noticed how long the notes took.

Yes, although Bruce *has* asked for help in cleaning up the open-items list.

--Josh

Josh Berkus <josh(at)agliodbs(dot)com> writes:
> Yes, although Bruce *has* asked for help in cleaning up the open-items list.

I spent several hours on that on Saturday, and more or less got the bird
in response... the way Bruce has that page set up, only he can do any
actual item removal, the rest of us can only comment.

regards, tom lane

On Wednesday 25 March 2009 23:17:41 Robert Haas wrote:
> With respect to this item:
> Disable appending of the epoch date/time when '%' escapes are missing
> in log_filename (Robert Haas)
> I might suggest explaining it this way:
> This change makes it easier to use PostgreSQL in conjunction with an
> external log rotation tool.
>

Hey! We were just complaining about this behavior the other day at $dayjob. We
were considering hacking our build to make it stop doing this ourselves, but
decided to use syslog in the end. Nice to see this "feature" disappear. :-)

--
Robert Treat
Conjecture: http://www.xzilla.net
Consulting: http://www.omniti.com

Robert Haas wrote:
> OK, I am all wet. I now understand why the editing is the
> time-consuming part of this job. On the plus side it is probably
> possible to parallelize it to some degree by splitting the list into N
> pieces after the "remove insignificant items" step.
>
> With respect to this item:
> Disable appending of the epoch date/time when '%' escapes are missing
> in log_filename (Robert Haas)
> I might suggest explaining it this way:
> This change makes it easier to use PostgreSQL in conjunction with an
> external log rotation tool.
>
> The following item uses "of" where it should say "if":
> Throw an error of an escape character is the last character in a LIKE
> pattern (nothing to escape) (Tom)

Thanks, I think Tom fixed that one already.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Tom Lane wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> > OK, I am all wet. I now understand why the editing is the
> > time-consuming part of this job. On the plus side it is probably
> > possible to parallelize it to some degree by splitting the list into N
> > pieces after the "remove insignificant items" step.
>
> The advantage of having one person do it (and do it over a short period
> of time) is that you end up with a fairly uniform "voice" across the
> whole set of notes. Since we lack a professional copy editor, we'd have
> a hard time coming out with something that wasn't pretty obviously a
> patchwork if several people did bits of it.
>
> In any case, the release notes aren't normally a bottleneck. I still
> think that Bruce had his priorities out of whack in not cleaning up
> his open-items list before doing this. If he had done so, nobody
> would have noticed how long the notes took.

Ah, but the open items list is never done; it is always in flux and
will be probably until final release. Also, you can't just put out the
open items list becuase then there is a flurry of activity and people
want you to keep the list current.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Josh Berkus wrote:
> All,
>
> > In any case, the release notes aren't normally a bottleneck. I still
> > think that Bruce had his priorities out of whack in not cleaning up
> > his open-items list before doing this. If he had done so, nobody
> > would have noticed how long the notes took.
>
> Yes, although Bruce *has* asked for help in cleaning up the open-items list.

Uh, not now; I put up the list only so people wouldn't think I was
hiding things, and I said it wasn't cleaned up. What is so hard for
people to understand about that. This is quite annoying.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Tom Lane wrote:
> Josh Berkus <josh(at)agliodbs(dot)com> writes:
> > Yes, although Bruce *has* asked for help in cleaning up the open-items list.
>
> I spent several hours on that on Saturday, and more or less got the bird
> in response... the way Bruce has that page set up, only he can do any
> actual item removal, the rest of us can only comment.

Sorry you felt that way.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

> In any case, the release notes aren't normally a bottleneck.
...

Could I respectfully request people make an effort to change the
subject lines when the thread radically moves away from its
original purpose? Modern mail systems don't thread by subject
lines anyway, so no worries there. Thanks.

- --
Greg Sabino Mullane greg(at)turnstep(dot)com
PGP Key: 0x14964AC8 200903262040
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAknMIKYACgkQvJuQZxSWSsi3TgCfQJn+kbfWEJOMbj9WfPpXpgPN
GHUAoIcCnekhlDrDxaqpfJ9lAnMtxQDj
=icok
-----END PGP SIGNATURE-----

On Thu, Mar 26, 2009 at 8:30 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Tom Lane wrote:
>> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> > OK, I am all wet.  I now understand why the editing is the
>> > time-consuming part of this job.  On the plus side it is probably
>> > possible to parallelize it to some degree by splitting the list into N
>> > pieces after the "remove insignificant items" step.
>>
>> The advantage of having one person do it (and do it over a short period
>> of time) is that you end up with a fairly uniform "voice" across the
>> whole set of notes.  Since we lack a professional copy editor, we'd have
>> a hard time coming out with something that wasn't pretty obviously a
>> patchwork if several people did bits of it.
>>
>> In any case, the release notes aren't normally a bottleneck.  I still
>> think that Bruce had his priorities out of whack in not cleaning up
>> his open-items list before doing this.  If he had done so, nobody
>> would have noticed how long the notes took.
>
> Ah, but the open items list is never done;  it is always in flux and
> will be probably until final release.  Also, you can't just put out the
> open items list becuase then there is a flurry of activity and people
> want you to keep the list current.

At this point I think we are just trying to get a list of items that
need to be done before we can release beta. Very little, if anything,
should be getting added to that list at this point.

...Robert

On Thu, Mar 26, 2009 at 8:31 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Josh Berkus wrote:
>> All,
>>
>> > In any case, the release notes aren't normally a bottleneck.  I still
>> > think that Bruce had his priorities out of whack in not cleaning up
>> > his open-items list before doing this.  If he had done so, nobody
>> > would have noticed how long the notes took.
>>
>> Yes, although Bruce *has* asked for help in cleaning up the open-items list.
>
> Uh, not now;  I put up the list only so people wouldn't think I was
> hiding things, and I said it wasn't cleaned up.  What is so hard for
> people to understand about that.  This is quite annoying.

If I understand correctly, you're saying that you didn't want to have
help with the release notes, you don't want help cleaning up your
mailbox, but you do want beta to wait until you are done doing those
things and any resulting action items have been completed. Is that
right?

I don't think anyone thinks you're hiding anything, but I think there
is a general desire to move this thing along as quickly as reasonably
possible.

...Robert

Robert Haas wrote:
> On Thu, Mar 26, 2009 at 8:30 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > Tom Lane wrote:
> >> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> >> > OK, I am all wet. ?I now understand why the editing is the
> >> > time-consuming part of this job. ?On the plus side it is probably
> >> > possible to parallelize it to some degree by splitting the list into N
> >> > pieces after the "remove insignificant items" step.
> >>
> >> The advantage of having one person do it (and do it over a short period
> >> of time) is that you end up with a fairly uniform "voice" across the
> >> whole set of notes. ?Since we lack a professional copy editor, we'd have
> >> a hard time coming out with something that wasn't pretty obviously a
> >> patchwork if several people did bits of it.
> >>
> >> In any case, the release notes aren't normally a bottleneck. ?I still
> >> think that Bruce had his priorities out of whack in not cleaning up
> >> his open-items list before doing this. ?If he had done so, nobody
> >> would have noticed how long the notes took.
> >
> > Ah, but the open items list is never done; ?it is always in flux and
> > will be probably until final release. ?Also, you can't just put out the
> > open items list becuase then there is a flurry of activity and people
> > want you to keep the list current.
>
> At this point I think we are just trying to get a list of items that
> need to be done before we can release beta. Very little, if anything,
> should be getting added to that list at this point.

You can say that, but things are going to be uncovered during beta
regularly.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Robert Haas wrote:
> On Thu, Mar 26, 2009 at 8:31 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > Josh Berkus wrote:
> >> All,
> >>
> >> > In any case, the release notes aren't normally a bottleneck. ?I still
> >> > think that Bruce had his priorities out of whack in not cleaning up
> >> > his open-items list before doing this. ?If he had done so, nobody
> >> > would have noticed how long the notes took.
> >>
> >> Yes, although Bruce *has* asked for help in cleaning up the open-items list.
> >
> > Uh, not now; ?I put up the list only so people wouldn't think I was
> > hiding things, and I said it wasn't cleaned up. ?What is so hard for
> > people to understand about that. ?This is quite annoying.
>
> If I understand correctly, you're saying that you didn't want to have
> help with the release notes, you don't want help cleaning up your
> mailbox, but you do want beta to wait until you are done doing those
> things and any resulting action items have been completed. Is that
> right?
>
> I don't think anyone thinks you're hiding anything, but I think there
> is a general desire to move this thing along as quickly as reasonably
> possible.

I can't start on the release notes, stop to update an open items list,
jump back to the release notes, update open items as people do stuff,
etc. The release notes have to be done in one big chunk of time, pretty
much.

I will need help cleaning out the open items list once I get a final
list together, though Tom's comments on the existing list has helped
greatly.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Thu, Mar 26, 2009 at 9:28 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> At this point I think we are just trying to get a list of items that
>> need to be done before we can release beta.  Very little, if anything,
>> should be getting added to that list at this point.
>
> You can say that, but things are going to be uncovered during beta
> regularly.

I'm sure they will. But the current problem is getting beta released
in the first place, and AFAICS we're all waiting for you.

...Robert

Robert Haas wrote:
> On Thu, Mar 26, 2009 at 8:31 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > Josh Berkus wrote:
> >> All,
> >>
> >> > In any case, the release notes aren't normally a bottleneck. ?I still
> >> > think that Bruce had his priorities out of whack in not cleaning up
> >> > his open-items list before doing this. ?If he had done so, nobody
> >> > would have noticed how long the notes took.
> >>
> >> Yes, although Bruce *has* asked for help in cleaning up the open-items list.
> >
> > Uh, not now; ?I put up the list only so people wouldn't think I was
> > hiding things, and I said it wasn't cleaned up. ?What is so hard for
> > people to understand about that. ?This is quite annoying.
>
> If I understand correctly, you're saying that you didn't want to have
> help with the release notes, you don't want help cleaning up your
> mailbox, but you do want beta to wait until you are done doing those
> things and any resulting action items have been completed. Is that
> right?
>
> I don't think anyone thinks you're hiding anything, but I think there
> is a general desire to move this thing along as quickly as reasonably
> possible.

And to answer your question about the release notes, Tom is right that
it probably has to be done by one person because it needs a consistent
voice, and someone has to get the entire release notes in their head so
they can see logical sections/groupings, etc. Once it is done, as it is
now, people can jump in and add improvements.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Thu, Mar 26, 2009 at 9:35 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Robert Haas wrote:
>> On Thu, Mar 26, 2009 at 8:31 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> > Josh Berkus wrote:
>> >> All,
>> >>
>> >> > In any case, the release notes aren't normally a bottleneck. ?I still
>> >> > think that Bruce had his priorities out of whack in not cleaning up
>> >> > his open-items list before doing this. ?If he had done so, nobody
>> >> > would have noticed how long the notes took.
>> >>
>> >> Yes, although Bruce *has* asked for help in cleaning up the open-items list.
>> >
>> > Uh, not now; ?I put up the list only so people wouldn't think I was
>> > hiding things, and I said it wasn't cleaned up. ?What is so hard for
>> > people to understand about that. ?This is quite annoying.
>>
>> If I understand correctly, you're saying that you didn't want to have
>> help with the release notes, you don't want help cleaning up your
>> mailbox, but you do want beta to wait until you are done doing those
>> things and any resulting action items have been completed.  Is that
>> right?
>>
>> I don't think anyone thinks you're hiding anything, but I think there
>> is a general desire to move this thing along as quickly as reasonably
>> possible.
>
> And to answer your question about the release notes, Tom is right that
> it probably has to be done by one person because it needs a consistent
> voice, and someone has to get the entire release notes in their head so
> they can see logical sections/groupings, etc.  Once it is done, as it is
> now, people can jump in and add improvements.

Fair enough. I'm just looking for a way to help out so we can get
this done. I am not coming up with much that seems to be helpful. I
was planning to do some comments on your mailbox file but Tom beat me
to the punch.

...Robert

Robert Haas wrote:
> On Thu, Mar 26, 2009 at 9:28 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> >> At this point I think we are just trying to get a list of items that
> >> need to be done before we can release beta. ?Very little, if anything,
> >> should be getting added to that list at this point.
> >
> > You can say that, but things are going to be uncovered during beta
> > regularly.
>
> I'm sure they will. But the current problem is getting beta released
> in the first place, and AFAICS we're all waiting for you.

As Tom said, it is more the open items that we are waiting on, not the
release notes, but if if you are waiting for me to generate a list of
open items, I already published in inaccurate list that at least
_includes_ all the open items, plus lots of stuff that isn't open.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Oleg Bartunov <oleg(at)sai(dot)msu(dot)su> writes:
> I and Teodor have several small, but useful patches for text search:
> ...
> We would like to have your opinion what to do with these patches - leave them
> for 8.5 or provide them to hackers to review for 8.4.

I think the general consensus is that these were submitted too late for
8.4. Sorry ... the unaccent filter sounds particularly useful. Please
submit them as patches for CommitFest 2009-First.

regards, tom lane