Re: Not ready for 8.3

Based on our progress during this feature freeze, we will not complete
the feature freeze until August/September. I think we need adjust
expectations about an 8.3 release date, and decide if we want to
radically change our work process.

Basically, to make a release anywhere near July, we need 10x more
progress than we have had, which is unlikely.

We could ship what is in CVS now, but that just pushes the patches for
8.4, and isn't fair to patch submitters. We could shove what we have
now into CVS, but that is unwise.

I think the only other thing we _could_ do is to re-open normal 8.3
development, so we aren't hampering updates to trivial parts of the
code. Many of the patches now in the queue had been developed for months
before 8.3 started, so the hope is that we wouldn't have many more new
large patches, but several small ones we could deal with while we
whittle away at the larger patches during the next few months.

The question is whether it is healthy for us to remain in feature freeze
for months, and if it is unhealthy, what are our options?

Patch status:

http://developer.postgresql.org/index.php/Todo:PatchStatus

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

"Bruce Momjian" <bruce(at)momjian(dot)us> writes:

> I think the only other thing we _could_ do is to re-open normal 8.3
> development, so we aren't hampering updates to trivial parts of the
> code. Many of the patches now in the queue had been developed for months
> before 8.3 started, so the hope is that we wouldn't have many more new
> large patches, but several small ones we could deal with while we
> whittle away at the larger patches during the next few months.
>
> The question is whether it is healthy for us to remain in feature freeze
> for months, and if it is unhealthy, what are our options?

I don't see any reason development has to stop while the tree is in feature
freeze. If it led to patches being ready for review and getting reviewed and
committed early in the cycle rather than just before release I think it would
actually be extremely healthy.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com

Bruce Momjian wrote:
> Based on our progress during this feature freeze, we will not complete
> the feature freeze until August/September. I think we need adjust
> expectations about an 8.3 release date, and decide if we want to
> radically change our work process.
>
> Basically, to make a release anywhere near July, we need 10x more
> progress than we have had, which is unlikely.

Or we have another short release cycle, basically accepting what we have
now that can be worked through in the next 3 weeks and committed. If it
can't be done in that time, it waits. Then we have beta etc...

>
> We could ship what is in CVS now, but that just pushes the patches for
> 8.4, and isn't fair to patch submitters. We could shove what we have
> now into CVS, but that is unwise.

Sure it is, if we have a short release cycle. There are plenty of things
out there that are not quite ready yet, that could be if we released
again next January...

>
> I think the only other thing we _could_ do is to re-open normal 8.3
> development, so we aren't hampering updates to trivial parts of the
> code. Many of the patches now in the queue had been developed for months

Gah, that sounds like a horrible solution. Sorry Bruce.

>
> The question is whether it is healthy for us to remain in feature freeze
> for months, and if it is unhealthy, what are our options?
>
> Patch status:
>
> http://developer.postgresql.org/index.php/Todo:PatchStatus

If... this is actually a problem (I leave to other committers and
reviewers to comment) then I suggest we push all patches without a
reviewer as of now to 8.4.

Leaving only those patches that have confirmed reviewers to be worked
through.

FYI, whoever did that Todo:Patch status, Bravo! That is easily one of
the smallest but best improvements to the process I have seen in recent
memory.

Sincerely,

Joshua D. Drake

Joshua D. Drake wrote:
> Bruce Momjian wrote:
> > Based on our progress during this feature freeze, we will not complete
> > the feature freeze until August/September. I think we need adjust
> > expectations about an 8.3 release date, and decide if we want to
> > radically change our work process.
> >
> > Basically, to make a release anywhere near July, we need 10x more
> > progress than we have had, which is unlikely.
>
> Or we have another short release cycle, basically accepting what we have
> now that can be worked through in the next 3 weeks and committed. If it
> can't be done in that time, it waits. Then we have beta etc...

That is not fair to patch submitters, and pushes the problem to 8.4,
where it will be no better.

> > We could ship what is in CVS now, but that just pushes the patches for
> > 8.4, and isn't fair to patch submitters. We could shove what we have
> > now into CVS, but that is unwise.
>
> Sure it is, if we have a short release cycle. There are plenty of things
> out there that are not quite ready yet, that could be if we released
> again next January...

Huh, you didn't answer my issue about unfair.

> > The question is whether it is healthy for us to remain in feature freeze
> > for months, and if it is unhealthy, what are our options?
> >
> > Patch status:
> >
> > http://developer.postgresql.org/index.php/Todo:PatchStatus
>
> If... this is actually a problem (I leave to other committers and
> reviewers to comment) then I suggest we push all patches without a
> reviewer as of now to 8.4.
>
> Leaving only those patches that have confirmed reviewers to be worked
> through.
>
> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of
> the smallest but best improvements to the process I have seen in recent
> memory.

Fairness and not pushing our problems out to the future --- address
those issues.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Joshua D. Drake wrote:
> > Patch status:
> >
> > http://developer.postgresql.org/index.php/Todo:PatchStatus
>
> If... this is actually a problem (I leave to other committers and
> reviewers to comment) then I suggest we push all patches without a
> reviewer as of now to 8.4.
>
> Leaving only those patches that have confirmed reviewers to be worked
> through.
>
> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of
> the smallest but best improvements to the process I have seen in recent
> memory.

I did one of those for previous releases. I guess you forgot. It was
done by someone else this time only because I was going to be traveling.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

"Bruce Momjian" <bruce(at)momjian(dot)us> writes:

> Joshua D. Drake wrote:
>> Bruce Momjian wrote:
>> > Based on our progress during this feature freeze, we will not complete
>> > the feature freeze until August/September. I think we need adjust
>> > expectations about an 8.3 release date, and decide if we want to
>> > radically change our work process.
>> >
>> > Basically, to make a release anywhere near July, we need 10x more
>> > progress than we have had, which is unlikely.
>>
>> Or we have another short release cycle, basically accepting what we have
>> now that can be worked through in the next 3 weeks and committed. If it
>> can't be done in that time, it waits. Then we have beta etc...
>
> That is not fair to patch submitters, and pushes the problem to 8.4,
> where it will be no better.

It's nice to be fair but it's not really the goal here.

However I think the latter point is key. This *was* a short release cycle. If
we push off these patches to later there's no reason to think the situation
will be any different in a few months.

I suspect as much as Tom finds reviewing boring he would rather get it out of
the way and then have a chance to do development before more major patches
arrive than start the new cycle with major patches sitting in the review queue
and spend the whole cycle reviewing them.

As much as I hate that the patches sit in the queue until feature freeze
there's nothing else that will really force us to make a yea or nay decision
on them. If we start the next release with things in the patch queue they're
liable to sit there with no feedback for months again.

I think the key here is to give feedback for the authors but not allow the
time to respond to that feedback to grow. If you review a patch and find
problems with it and there's only three weeks for the author to deal with
those problems and it's not enough time then he can deal with them and submit
it for next release. But letting the patch ride over without giving any
feedback just seems pointless.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com

Joshua D. Drake wrote:

> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of
> the smallest but best improvements to the process I have seen in recent
> memory.

well bruce asked for something like that:
http://archives.postgresql.org/pgsql-hackers/2007-05/msg00249.php

and I simply went ahead and did it:

http://archives.postgresql.org/pgsql-hackers/2007-05/msg00265.php

Stefan

Bruce,

Realistically I just don't see getting everything in the ToDo patch list in;
my vote is that we start deferring stuff for 8.4 if it doesn't have a
reviewer, except for items which were submitted early in the cycle (and to
whom it would be unfair).

If that means shortening the 8.4 cycle somewhat, I'm for that ... feature
freeze in Feburary would be even better than April, because it means we could
be in Beta for the May-June-July conferences, and increase our probability of
being able to release at a major conference or PostgreSQL conference.

Obviously for 8.4 reviewers need to start reviewing stuff from the first week
of the development cycle. I also don't actually see anything wrong with a
3-month feature freeze if we can somehow branch development earlier. Easy
for me to say, I know.

--
Josh Berkus
PostgreSQL @ Sun
San Francisco

> That is not fair to patch submitters, and pushes the problem to 8.4,
> where it will be no better.

If it isn't done, it isn't done. We aren't dropping the patch. The patch
has been accepted, just not reviewed. It is just delayed.

>> Sure it is, if we have a short release cycle. There are plenty of things
>> out there that are not quite ready yet, that could be if we released
>> again next January...
>
> Huh, you didn't answer my issue about unfair.

Sure I did see above :). Unfair would be to let go all together. We are
just delaying. If we don't have reviewers, we don't have reviewers and
as I look at the open list we have more reviewers than we did for 8.2 so
I would suspect that 8.4 is going to go smoother as it is.

>> Leaving only those patches that have confirmed reviewers to be worked
>> through.
>>
>> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of
>> the smallest but best improvements to the process I have seen in recent
>> memory.
>
> Fairness and not pushing our problems out to the future --- address
> those issues.

Life isn't fair. It isn't like we are telling these people to bugger
off. We are just delaying the review to what is a reasonable workload
for the current set of reviewers.

Life... is a perpetual problem. We do what we can, when we can, and the
best that we can.

Taking my suggestion above for leaving patches that don't have reviewers
let's look at some of them.

Tsearch2 in core. I know that Tom has some reservations, he I and Treat
all commented on how it was done and to my knowledge those reservations
have not been resolved.

HOT is a huge patch that has gone round and round and round. Lots of
people want it, including me but it is invasive enough to where it may
due it good to work through another cycle.

Grouped Index Tuples. I like the result of this patch. I tested the
performance and it did work as advertised but we didn't get much
feedback as a whole from known hackers. Either there isn't much interest
or we are too busy. Either way it is certainly not a critical patch
and can be pushed.

Concurrent psql, nifty but still trying to decide on actual interface.

Full Page Writes Improvement, doesn't actually do anything *yet* (as far
as I can tell) it just makes it so something can be done in the future.
It is also apparently a small patch.

UTF8 text matching performance improvements (todo wiki link broke), so I
don't have much comment.

On Disk Bitmap indexes (todo wiki link broke): Doesn't support Vacuum
(to my knowledge), community member tested, reported problems... no
response yet from author. The author is known to be time constrained,
boot it till 8.4.

Posix shared memory, not usable in current state per Tom's comments and
Apples, agreement. Dumped till 8.4 or further.

Stream bitmaps, apparently needed more info from Gavin (see previous
comment about author's time). No feed back since March 9th. Dumped till 8.4.

Bitmapscan change, this one is unfortunate because at the time Heikki
had resources and desire but was basically ignored. Sometimes we have to
punt although Heikki is doing patch review right now and it is not
unheard of for a patch reviewer to commit his own patch (in this case we
would need a comitter to actually accept it.).

Updateable cursors, apparently breaks explain and patch has been
updated. Punt!

Group Commit, Heikki has already suggested that it may be a good idea to
push for 8.4 on this patch: http://momjian.us/mhonarc/patches/msg00127.html

Index Advisor.. I think the link is wrong:

http://momjian.us/mhonarc/patches/msg00119.html

Ctid chain patch, apparently no discussion since last January even
though requests had been made to change the patch to some degree. Punt.
I will note that no one was negative about the patch, it just doesn't
appear that the requests were ever finished.

Error correct for n_dead_tuples, patch was requested for hold in Feb. No
discussion since. Punt!

DSM, apparently includes n_dead_tuples, please remove n_dead_tuples line
on todo. Significant memory allocation enhancements are expected in 8.4
for this. Discussion died in April. Concerns were raised about how
memory is allocated (fixed, shared) which author already admints to
wanting to change for 8.4.

PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
developed outside of core. Don't get me wrong I like the feature but it
can take advantage of facilities outside of core.

So there ya go... thoughts, flames?

Sincerely,

Joshua D. Drake

>

Gregory Stark wrote:
> "Bruce Momjian" <bruce(at)momjian(dot)us> writes:
>
>
>> I think the only other thing we _could_ do is to re-open normal 8.3
>> development, so we aren't hampering updates to trivial parts of the
>> code. Many of the patches now in the queue had been developed for months
>> before 8.3 started, so the hope is that we wouldn't have many more new
>> large patches, but several small ones we could deal with while we
>> whittle away at the larger patches during the next few months.
>>
>> The question is whether it is healthy for us to remain in feature freeze
>> for months, and if it is unhealthy, what are our options?
>>
>
> I don't see any reason development has to stop while the tree is in feature
> freeze. If it led to patches being ready for review and getting reviewed and
> committed early in the cycle rather than just before release I think it would
> actually be extremely healthy.
>
>

If we had multiple dev branches it might be more possible. That has its
own costs, of course - having a single dev branch makes management much
easier, and we never have to worry about things like merging.

Patches seem to be getting larger, more complex, and harder to review.
That is stressing our processes more than somewhat.

Short cycles would only make matters worse - the frictional cost of each
dev cycle is growing. I think at least we have learned not to repeat
this exercise.

cheers

andrew

We have new patch available
http://www.sigaev.ru/misc/tsearch_core-0.47.gz
to sync with CVS HEAD.

Oleg
On Tue, 15 May 2007, Joshua D. Drake wrote:

> Bruce Momjian wrote:
>> Based on our progress during this feature freeze, we will not complete
>> the feature freeze until August/September. I think we need adjust
>> expectations about an 8.3 release date, and decide if we want to
>> radically change our work process.
>>
>> Basically, to make a release anywhere near July, we need 10x more
>> progress than we have had, which is unlikely.
>
> Or we have another short release cycle, basically accepting what we have now
> that can be worked through in the next 3 weeks and committed. If it can't be
> done in that time, it waits. Then we have beta etc...
>
>>
>> We could ship what is in CVS now, but that just pushes the patches for
>> 8.4, and isn't fair to patch submitters. We could shove what we have
>> now into CVS, but that is unwise.
>
> Sure it is, if we have a short release cycle. There are plenty of things out
> there that are not quite ready yet, that could be if we released again next
> January...
>
>
>>
>> I think the only other thing we _could_ do is to re-open normal 8.3
>> development, so we aren't hampering updates to trivial parts of the
>> code. Many of the patches now in the queue had been developed for months
>
> Gah, that sounds like a horrible solution. Sorry Bruce.
>
>>
>> The question is whether it is healthy for us to remain in feature freeze
>> for months, and if it is unhealthy, what are our options?
>>
>> Patch status:
>>
>> http://developer.postgresql.org/index.php/Todo:PatchStatus
>
> If... this is actually a problem (I leave to other committers and reviewers
> to comment) then I suggest we push all patches without a reviewer as of now
> to 8.4.
>
> Leaving only those patches that have confirmed reviewers to be worked
> through.
>
> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of the
> smallest but best improvements to the process I have seen in recent memory.
>
> Sincerely,
>
> Joshua D. Drake
>
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org
>

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

josh(at)agliodbs(dot)com (Josh Berkus) writes:
> Bruce,
>
> Realistically I just don't see getting everything in the ToDo patch
> list in; my vote is that we start deferring stuff for 8.4 if it
> doesn't have a reviewer, except for items which were submitted early
> in the cycle (and to whom it would be unfair).
>
> If that means shortening the 8.4 cycle somewhat, I'm for that
> ... feature freeze in Feburary would be even better than April,
> because it means we could be in Beta for the May-June-July
> conferences, and increase our probability of being able to release
> at a major conference or PostgreSQL conference.

If we're pushing stuff planned for 8.3 off into 8.4, then doesn't that
mean that we'd be inclined to _lengthen_ the 8.4 cycle?

If we were to shorten the 8.4 cycle, that seems likely to me to worsen
the problem, so that I'd expect to see even more stuff deferred for
8.5 than was the case with 8.3...

> Obviously for 8.4 reviewers need to start reviewing stuff from the
> first week of the development cycle. I also don't actually see
> anything wrong with a 3-month feature freeze if we can somehow
> branch development earlier. Easy for me to say, I know.

Well, if people are essentially already working on patches for 8.4
*now*, but just deferring merging until after 8.3 is committed +
released, then we've got 8.4 work underway already.

Unfortunately, that's also likely to worsen the problem of the
reviewers' queues being even more overflowing.

I'd sorta like to see the Tom Lanes and Bruce Momjians of the project
getting to have some time to work on things that *they'd* like add, as
opposed to turning things into a spiral cycle where they keep spending
more and more of their time reviewing patches, as opposed to doing
things they find neat and new. Too many iterations of that sort of
thing, and they'll not want to ever see a patch again...

--
output = ("cbbrowne" "@" "linuxdatabases.info")
http://cbbrowne.com/info/wp.html
"...It is also possible to post imbecilic articles with any browser,
especially when you toppost and omit snippage." -- CBFalconer
<cbfalconer(at)yahoo(dot)com> - seen on comp.lang.c

On Tue, 15 May 2007, Joshua D. Drake wrote:

> Tsearch2 in core. I know that Tom has some reservations, he I and Treat all
> commented on how it was done and to my knowledge those reservations have not
> been resolved.

We'd like to know about these reservations. If I understand you mean there
are issues with the patch ? Our patch is several months old. We permanently
keep it in sync with CVS HEAD, latest version is 0.47.

We're really bored with the whole process of development. We're not
full-time developers, we just devote our spare time which we withdraw from
time we should spend on many other things. We have no time even to discuss
those very useful threads about community management, patches, etc. We just
rely on community decision.

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

Joshua D. Drake wrote:

[...]
> Concurrent psql, nifty but still trying to decide on actual interface.
>
> Full Page Writes Improvement, doesn't actually do anything *yet* (as far
> as I can tell) it just makes it so something can be done in the future.
> It is also apparently a small patch.
>
> UTF8 text matching performance improvements (todo wiki link broke), so I
> don't have much comment.

url should be fixed now

>
> On Disk Bitmap indexes (todo wiki link broke): Doesn't support Vacuum
> (to my knowledge), community member tested, reported problems... no
> response yet from author. The author is known to be time constrained,
> boot it till 8.4.

url fixed too

>
> Bitmapscan change, this one is unfortunate because at the time Heikki
> had resources and desire but was basically ignored. Sometimes we have to
> punt although Heikki is doing patch review right now and it is not
> unheard of for a patch reviewer to commit his own patch (in this case we
> would need a comitter to actually accept it.).
>
> Updateable cursors, apparently breaks explain and patch has been
> updated. Punt!
>
> Group Commit, Heikki has already suggested that it may be a good idea to
> push for 8.4 on this patch: http://momjian.us/mhonarc/patches/msg00127.html
>
> Index Advisor.. I think the link is wrong:
>
> http://momjian.us/mhonarc/patches/msg00119.html

url fixed

> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> developed outside of core. Don't get me wrong I like the feature but it
> can take advantage of facilities outside of core.

url fixed - I wonder why we had so much of them and all those pointing
to the patch list bruce maintains - are the urls to the mails there not
stable somehow ?

Stefan

Oleg Bartunov wrote:
> On Tue, 15 May 2007, Joshua D. Drake wrote:
>
>> Tsearch2 in core. I know that Tom has some reservations, he I and
>> Treat all commented on how it was done and to my knowledge those
>> reservations have not been resolved.
>
> We'd like to know about these reservations. If I understand you mean
> there are issues with the patch ? Our patch is several months old. We
> permanently
> keep it in sync with CVS HEAD, latest version is 0.47.

http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php

Sincerely,

Joshua D. Drake

Stefan Kaltenbrunner wrote:
> Joshua D. Drake wrote:

> > PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> > developed outside of core. Don't get me wrong I like the feature but it
> > can take advantage of facilities outside of core.
>
> url fixed - I wonder why we had so much of them and all those pointing
> to the patch list bruce maintains - are the urls to the mails there not
> stable somehow ?

They are not stable. The items should point to the archives, which are
supposedly more stable. (I had already fixed one item in PatchStatus
this morning). Really it would be much nicer to have links using the
Message-Id but I doubt that's at all doable.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera wrote:
> Stefan Kaltenbrunner wrote:
>> Joshua D. Drake wrote:
>
>>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
>>> developed outside of core. Don't get me wrong I like the feature but it
>>> can take advantage of facilities outside of core.
>> url fixed - I wonder why we had so much of them and all those pointing
>> to the patch list bruce maintains - are the urls to the mails there not
>> stable somehow ?
>
> They are not stable. The items should point to the archives, which are
> supposedly more stable. (I had already fixed one item in PatchStatus
> this morning). Really it would be much nicer to have links using the
> Message-Id but I doubt that's at all doable.

I think we discussed it once... I don't remember the reason why we
didn't go that direction.

Joshua D. Drake

>

Alvaro Herrera wrote:
> Stefan Kaltenbrunner wrote:
>> Joshua D. Drake wrote:
>
>>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
>>> developed outside of core. Don't get me wrong I like the feature but it
>>> can take advantage of facilities outside of core.
>> url fixed - I wonder why we had so much of them and all those pointing
>> to the patch list bruce maintains - are the urls to the mails there not
>> stable somehow ?
>
> They are not stable. The items should point to the archives, which are
> supposedly more stable. (I had already fixed one item in PatchStatus
> this morning). Really it would be much nicer to have links using the
> Message-Id but I doubt that's at all doable.

hrm - I see so is there a particular reason for that behaviour ?

I will work on pointing to the archives tomorrow ...

Stefan

> They are not stable. The items should point to the archives, which are
> supposedly more stable. (I had already fixed one item in PatchStatus
> this morning). Really it would be much nicer to have links using the
> Message-Id but I doubt that's at all doable.

I use this all the time:
http://news.gmane.org/find-root.php?message_id=$MSGID

It uses non-PostgreSQL infrastructure, but hey, I think gmane does list
archives better than PostgreSQL right now ;-)

Try it:
http://news.gmane.org/find-root(dot)php?message_id=20070515201925(dot)GL12731(at)alvh(dot)no-ip(dot)org

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Stefan Kaltenbrunner wrote:
> Alvaro Herrera wrote:
> > Stefan Kaltenbrunner wrote:
> >> Joshua D. Drake wrote:
> >
> >>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> >>> developed outside of core. Don't get me wrong I like the feature but it
> >>> can take advantage of facilities outside of core.
> >> url fixed - I wonder why we had so much of them and all those pointing
> >> to the patch list bruce maintains - are the urls to the mails there not
> >> stable somehow ?
> >
> > They are not stable. The items should point to the archives, which are
> > supposedly more stable. (I had already fixed one item in PatchStatus
> > this morning). Really it would be much nicer to have links using the
> > Message-Id but I doubt that's at all doable.
>
> hrm - I see so is there a particular reason for that behaviour ?
>
> I will work on pointing to the archives tomorrow ...

Bruce adds and removes emails from the "pgpatches" inbox and I assume he
regenerates the MHonarc archives when he does, which may change the URL
for each specific item. I don't think it was ever intended that the
URLs were to be stable.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Stefan Kaltenbrunner wrote:
>> They are not stable. The items should point to the archives, which are
>> supposedly more stable. (I had already fixed one item in PatchStatus
>> this morning). Really it would be much nicer to have links using the
>> Message-Id but I doubt that's at all doable.
>
> hrm - I see so is there a particular reason for that behaviour ?

They're stable until Bruce removes something from the queue. When
something is removed, it's renumbered.

It's how mhonarc works. It's the same with the archives - if we delete a
mail, they get renumbered. So we never should delete, we should just
blank out, but it has happened a couple of times.

//Magnus

Aidan Van Dyk wrote:
>
> > They are not stable. The items should point to the archives, which are
> > supposedly more stable. (I had already fixed one item in PatchStatus
> > this morning). Really it would be much nicer to have links using the
> > Message-Id but I doubt that's at all doable.
>
> I use this all the time:
> http://news.gmane.org/find-root.php?message_id=$MSGID
>
> It uses non-PostgreSQL infrastructure, but hey, I think gmane does list
> archives better than PostgreSQL right now ;-)
>
> Try it:
> http://news.gmane.org/find-root(dot)php?message_id=20070515201925(dot)GL12731(at)alvh(dot)no-ip(dot)org

Wow, this is really nice, thanks.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

On Tue, 15 May 2007, Joshua D. Drake wrote:

> Oleg Bartunov wrote:
>> On Tue, 15 May 2007, Joshua D. Drake wrote:
>>
>>> Tsearch2 in core. I know that Tom has some reservations, he I and Treat
>>> all commented on how it was done and to my knowledge those reservations
>>> have not been resolved.
>>
>> We'd like to know about these reservations. If I understand you mean there
>> are issues with the patch ? Our patch is several months old. We permanently
>> keep it in sync with CVS HEAD, latest version is 0.47.
>
> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php

there are several others threads.

well, we did answer all raised questions and I don't want to begin
another wave. The only problem I see is that some people doesn't like
introducing SQL syntax, they claim functions would be enough. SQL is a
nice language to manipulate db objects, finally.
Also, select to fts_create_dictionary(bla-bla....) looks pretty ugly,
pretty artificial.

CREATE FULLTEXT DICTIONARY dictname
[ { INIT init_function
| LEXIZE lexize_function
| OPTION opt_text }
[ ... ]] LIKE template_dictname;

select

>
> Sincerely,
>
> Joshua D. Drake
>

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

Gregory Stark wrote:
> "Bruce Momjian" <bruce(at)momjian(dot)us> writes:
>
> > I think the only other thing we _could_ do is to re-open normal 8.3
> > development, so we aren't hampering updates to trivial parts of the
> > code. Many of the patches now in the queue had been developed for months
> > before 8.3 started, so the hope is that we wouldn't have many more new
> > large patches, but several small ones we could deal with while we
> > whittle away at the larger patches during the next few months.
> >
> > The question is whether it is healthy for us to remain in feature freeze
> > for months, and if it is unhealthy, what are our options?
>
> I don't see any reason development has to stop while the tree is in feature
> freeze. If it led to patches being ready for review and getting reviewed and
> committed early in the cycle rather than just before release I think it would
> actually be extremely healthy.

So you are saying just let people keep developing for 8.4 and we will
use it as soon as we start for 8.4. OK. We might get to a point where
we can just open development for 8.4, apply outstanding patches, and
head for beta. ;-)

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Josh Berkus wrote:
> Bruce,
>
> Realistically I just don't see getting everything in the ToDo patch list in;
> my vote is that we start deferring stuff for 8.4 if it doesn't have a
> reviewer, except for items which were submitted early in the cycle (and to
> whom it would be unfair).

It seems unfair to disinguish between early/last in cycle postings.

> If that means shortening the 8.4 cycle somewhat, I'm for that ... feature
> freeze in Feburary would be even better than April, because it means we could
> be in Beta for the May-June-July conferences, and increase our probability of
> being able to release at a major conference or PostgreSQL conference.
>
> Obviously for 8.4 reviewers need to start reviewing stuff from the first week
> of the development cycle. I also don't actually see anything wrong with a
> 3-month feature freeze if we can somehow branch development earlier. Easy
> for me to say, I know.

Yea, this is just pushing off work until later, which I don't support.
There is no easy out here and I am afraid we will just have to accept a
3-month feature freeze.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Joshua D. Drake wrote:
> > That is not fair to patch submitters, and pushes the problem to 8.4,
> > where it will be no better.
>
> If it isn't done, it isn't done. We aren't dropping the patch. The patch
> has been accepted, just not reviewed. It is just delayed.

Delayed isn't rejected, but it isn't accepted either. I see that as
unfair to patch submitters, and not making things any easier. It just
pushes our problem into 8.4.

> >> Sure it is, if we have a short release cycle. There are plenty of things
> >> out there that are not quite ready yet, that could be if we released
> >> again next January...
> >
> > Huh, you didn't answer my issue about unfair.
>
> Sure I did see above :). Unfair would be to let go all together. We are
> just delaying. If we don't have reviewers, we don't have reviewers and
> as I look at the open list we have more reviewers than we did for 8.2 so
> I would suspect that 8.4 is going to go smoother as it is.

And even more patches for 8.4.

Given your list below, let me match it up with Tom's comments and see if
I can defer some patches that are not ready for application. I think
bitmapped indexes are one of them.

---------------------------------------------------------------------------

> >> Leaving only those patches that have confirmed reviewers to be worked
> >> through.
> >>
> >> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of
> >> the smallest but best improvements to the process I have seen in recent
> >> memory.
> >
> > Fairness and not pushing our problems out to the future --- address
> > those issues.
>
> Life isn't fair. It isn't like we are telling these people to bugger
> off. We are just delaying the review to what is a reasonable workload
> for the current set of reviewers.

Again, it just pushed things off. I see just keeping going as a better
approach.

> Life... is a perpetual problem. We do what we can, when we can, and the
> best that we can.

Yea, but we have to do that best we can, not give up and hope something
improves for 8.4, because I see no basis for such a belief.

> Taking my suggestion above for leaving patches that don't have reviewers
> let's look at some of them.
>
> Tsearch2 in core. I know that Tom has some reservations, he I and Treat
> all commented on how it was done and to my knowledge those reservations
> have not been resolved.
>
> HOT is a huge patch that has gone round and round and round. Lots of
> people want it, including me but it is invasive enough to where it may
> due it good to work through another cycle.
>
> Grouped Index Tuples. I like the result of this patch. I tested the
> performance and it did work as advertised but we didn't get much
> feedback as a whole from known hackers. Either there isn't much interest
> or we are too busy. Either way it is certainly not a critical patch
> and can be pushed.
>
> Concurrent psql, nifty but still trying to decide on actual interface.
>
> Full Page Writes Improvement, doesn't actually do anything *yet* (as far
> as I can tell) it just makes it so something can be done in the future.
> It is also apparently a small patch.
>
> UTF8 text matching performance improvements (todo wiki link broke), so I
> don't have much comment.
>
> On Disk Bitmap indexes (todo wiki link broke): Doesn't support Vacuum
> (to my knowledge), community member tested, reported problems... no
> response yet from author. The author is known to be time constrained,
> boot it till 8.4.
>
> Posix shared memory, not usable in current state per Tom's comments and
> Apples, agreement. Dumped till 8.4 or further.
>
> Stream bitmaps, apparently needed more info from Gavin (see previous
> comment about author's time). No feed back since March 9th. Dumped till 8.4.
>
> Bitmapscan change, this one is unfortunate because at the time Heikki
> had resources and desire but was basically ignored. Sometimes we have to
> punt although Heikki is doing patch review right now and it is not
> unheard of for a patch reviewer to commit his own patch (in this case we
> would need a comitter to actually accept it.).
>
> Updateable cursors, apparently breaks explain and patch has been
> updated. Punt!
>
> Group Commit, Heikki has already suggested that it may be a good idea to
> push for 8.4 on this patch: http://momjian.us/mhonarc/patches/msg00127.html
>
> Index Advisor.. I think the link is wrong:
>
> http://momjian.us/mhonarc/patches/msg00119.html
>
> Ctid chain patch, apparently no discussion since last January even
> though requests had been made to change the patch to some degree. Punt.
> I will note that no one was negative about the patch, it just doesn't
> appear that the requests were ever finished.
>
> Error correct for n_dead_tuples, patch was requested for hold in Feb. No
> discussion since. Punt!
>
> DSM, apparently includes n_dead_tuples, please remove n_dead_tuples line
> on todo. Significant memory allocation enhancements are expected in 8.4
> for this. Discussion died in April. Concerns were raised about how
> memory is allocated (fixed, shared) which author already admints to
> wanting to change for 8.4.
>
> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> developed outside of core. Don't get me wrong I like the feature but it
> can take advantage of facilities outside of core.
>
> So there ya go... thoughts, flames?
>
> Sincerely,
>
> Joshua D. Drake
>
>
>
>
>
>
> >
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
> message can get through to the mailing list cleanly

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Andrew Dunstan wrote:
> >> I think the only other thing we _could_ do is to re-open normal 8.3
> >> development, so we aren't hampering updates to trivial parts of the
> >> code. Many of the patches now in the queue had been developed for months
> >> before 8.3 started, so the hope is that we wouldn't have many more new
> >> large patches, but several small ones we could deal with while we
> >> whittle away at the larger patches during the next few months.
> >>
> >> The question is whether it is healthy for us to remain in feature freeze
> >> for months, and if it is unhealthy, what are our options?
> >>
> >
> > I don't see any reason development has to stop while the tree is in feature
> > freeze. If it led to patches being ready for review and getting reviewed and
> > committed early in the cycle rather than just before release I think it would
> > actually be extremely healthy.
> >
> >
>
> If we had multiple dev branches it might be more possible. That has its
> own costs, of course - having a single dev branch makes management much
> easier, and we never have to worry about things like merging.
>
> Patches seem to be getting larger, more complex, and harder to review.
> That is stressing our processes more than somewhat.
>
> Short cycles would only make matters worse - the frictional cost of each
> dev cycle is growing. I think at least we have learned not to repeat
> this exercise.

Agreed. Good summary. Let's look at our problems honestly now and find
a direction, rather than pushing them off for later.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> Gregory Stark wrote:
> > "Bruce Momjian" <bruce(at)momjian(dot)us> writes:
> >
> > > I think the only other thing we _could_ do is to re-open normal 8.3
> > > development, so we aren't hampering updates to trivial parts of the
> > > code. Many of the patches now in the queue had been developed for months
> > > before 8.3 started, so the hope is that we wouldn't have many more new
> > > large patches, but several small ones we could deal with while we
> > > whittle away at the larger patches during the next few months.
> > >
> > > The question is whether it is healthy for us to remain in feature freeze
> > > for months, and if it is unhealthy, what are our options?
> >
> > I don't see any reason development has to stop while the tree is in feature
> > freeze. If it led to patches being ready for review and getting reviewed and
> > committed early in the cycle rather than just before release I think it would
> > actually be extremely healthy.
>
> So you are saying just let people keep developing for 8.4 and we will
> use it as soon as we start for 8.4. OK. We might get to a point where
> we can just open development for 8.4, apply outstanding patches, and
> head for beta. ;-)

This is what happens with the Linux kernel. They have hundreds of
developers getting their hands dirty during a previous period. Then
2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of
patches are flooded in. The merge window closes some time after that
(it's something like 2 or 3 weeks), and the stabilization period follows
(2 months?), during which 2.6.21-rc1, -rc2 etc are released. When
stability is reached, 2.6.21 is released, and the cycle starts again.

Sadly, we are missing the hundreds of developers. We are nowhere near
the scale of Linux.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Chris Browne wrote:
> josh(at)agliodbs(dot)com (Josh Berkus) writes:
> > Bruce,
> >
> > Realistically I just don't see getting everything in the ToDo patch
> > list in; my vote is that we start deferring stuff for 8.4 if it
> > doesn't have a reviewer, except for items which were submitted early
> > in the cycle (and to whom it would be unfair).
> >
> > If that means shortening the 8.4 cycle somewhat, I'm for that
> > ... feature freeze in Feburary would be even better than April,
> > because it means we could be in Beta for the May-June-July
> > conferences, and increase our probability of being able to release
> > at a major conference or PostgreSQL conference.
>
> If we're pushing stuff planned for 8.3 off into 8.4, then doesn't that
> mean that we'd be inclined to _lengthen_ the 8.4 cycle?
>
> If we were to shorten the 8.4 cycle, that seems likely to me to worsen
> the problem, so that I'd expect to see even more stuff deferred for
> 8.5 than was the case with 8.3...

And, as I remember, we already deferred some stuff during 8.2 for 8.3.
:-(

> > Obviously for 8.4 reviewers need to start reviewing stuff from the
> > first week of the development cycle. I also don't actually see
> > anything wrong with a 3-month feature freeze if we can somehow
> > branch development earlier. Easy for me to say, I know.
>
> Well, if people are essentially already working on patches for 8.4
> *now*, but just deferring merging until after 8.3 is committed +
> released, then we've got 8.4 work underway already.
>
> Unfortunately, that's also likely to worsen the problem of the
> reviewers' queues being even more overflowing.
>
> I'd sorta like to see the Tom Lanes and Bruce Momjians of the project
> getting to have some time to work on things that *they'd* like add, as
> opposed to turning things into a spiral cycle where they keep spending
> more and more of their time reviewing patches, as opposed to doing
> things they find neat and new. Too many iterations of that sort of
> thing, and they'll not want to ever see a patch again...

Another good summary.

Myself, I have always done what no one wanted to do so that more people
can do what they want to do, and I am OK with that. It helps the
project move forward faster than if I did only what I wanted to do.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Oleg Bartunov wrote:
> On Tue, 15 May 2007, Joshua D. Drake wrote:
>
>> Oleg Bartunov wrote:
>>> On Tue, 15 May 2007, Joshua D. Drake wrote:
>>>
>>>> Tsearch2 in core. I know that Tom has some reservations, he I and
>>>> Treat all commented on how it was done and to my knowledge those
>>>> reservations have not been resolved.
>>>
>>> We'd like to know about these reservations. If I understand you mean
>>> there are issues with the patch ? Our patch is several months old. We
>>> permanently
>>> keep it in sync with CVS HEAD, latest version is 0.47.
>>
>> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php
>
> there are several others threads.
>
> well, we did answer all raised questions and I don't want to begin
> another wave.

Did you address them? For example:

http://archives.postgresql.org/pgsql-hackers/2007-03/msg00914.php

Sincerely,

Joshua d. drake

The only problem I see is that some people doesn't like
> introducing SQL syntax, they claim functions would be enough. SQL is a
> nice language to manipulate db objects, finally. Also, select to
> fts_create_dictionary(bla-bla....) looks pretty ugly,
> pretty artificial.
>
> CREATE FULLTEXT DICTIONARY dictname [ { INIT init_function
> | LEXIZE lexize_function
> | OPTION opt_text } [ ... ]] LIKE template_dictname;
>
> select
>
>>
>> Sincerely,
>>
>> Joshua D. Drake
>>
>
> Regards,
> Oleg
> _____________________________________________________________
> Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
> Sternberg Astronomical Institute, Moscow University, Russia
> Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
> phone: +007(495)939-16-83, +007(495)939-23-83
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: You can help support the PostgreSQL project by donating at
>
> http://www.postgresql.org/about/donate
>

Joshua D. Drake wrote:
> Oleg Bartunov wrote:
> > On Tue, 15 May 2007, Joshua D. Drake wrote:
> >
> >> Tsearch2 in core. I know that Tom has some reservations, he I and
> >> Treat all commented on how it was done and to my knowledge those
> >> reservations have not been resolved.
> >
> > We'd like to know about these reservations. If I understand you mean
> > there are issues with the patch ? Our patch is several months old. We
> > permanently
> > keep it in sync with CVS HEAD, latest version is 0.47.
>
> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php

Oleg has been producing new versions of his patch, and no one has
objected to any of it, so I assume all the issues were addressed.

As far was whether tsearch2 should be in core, I think most agreed that
SQL support for full-text would make it easier to use, and that it is a
fundamental technology. As I remember, the only holdback was full
multi-byte support, but that is done.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Tue, 15 May 2007, Joshua D. Drake wrote:

> Oleg Bartunov wrote:
>> On Tue, 15 May 2007, Joshua D. Drake wrote:
>>
>>> Oleg Bartunov wrote:
>>>> On Tue, 15 May 2007, Joshua D. Drake wrote:
>>>>
>>>>> Tsearch2 in core. I know that Tom has some reservations, he I and Treat
>>>>> all commented on how it was done and to my knowledge those reservations
>>>>> have not been resolved.
>>>>
>>>> We'd like to know about these reservations. If I understand you mean
>>>> there are issues with the patch ? Our patch is several months old. We
>>>> permanently
>>>> keep it in sync with CVS HEAD, latest version is 0.47.
>>>
>>> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php
>>
>> there are several others threads.
>>
>> well, we did answer all raised questions and I don't want to begin
>> another wave.
>
> Did you address them? For example:
>
> http://archives.postgresql.org/pgsql-hackers/2007-03/msg00914.php

we added support of gin index for text data type, so one could just

create index fts_idx on TABLE using gin(body);

and should be able to do full-text search. Of course, ranking is not
available, since index doesn't store positional information.

About this syntax see
http://archives.postgresql.org/pgsql-hackers/2007-03/msg00936.php

>
> Sincerely,
>
> Joshua d. drake
>
>
> The only problem I see is that some people doesn't like
>> introducing SQL syntax, they claim functions would be enough. SQL is a
>> nice language to manipulate db objects, finally. Also, select to
>> fts_create_dictionary(bla-bla....) looks pretty ugly,
>> pretty artificial.
>>
>> CREATE FULLTEXT DICTIONARY dictname [ { INIT init_function
>> | LEXIZE lexize_function
>> | OPTION opt_text } [ ... ]] LIKE template_dictname;
>>
>> select
>>
>>>
>>> Sincerely,
>>>
>>> Joshua D. Drake
>>>
>>
>> Regards,
>> Oleg
>> _____________________________________________________________
>> Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
>> Sternberg Astronomical Institute, Moscow University, Russia
>> Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
>> phone: +007(495)939-16-83, +007(495)939-23-83
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 7: You can help support the PostgreSQL project by donating at
>>
>> http://www.postgresql.org/about/donate
>>
>

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

Oleg Bartunov wrote:
> On Tue, 15 May 2007, Joshua D. Drake wrote:
>
> > Tsearch2 in core. I know that Tom has some reservations, he I and Treat all
> > commented on how it was done and to my knowledge those reservations have not
> > been resolved.
>
> We'd like to know about these reservations. If I understand you mean there
> are issues with the patch ? Our patch is several months old. We permanently
> keep it in sync with CVS HEAD, latest version is 0.47.
>
> We're really bored with the whole process of development. We're not
> full-time developers, we just devote our spare time which we withdraw from
> time we should spend on many other things. We have no time even to discuss
> those very useful threads about community management, patches, etc. We just
> rely on community decision.

This is a good example of how developers can get frustrated. Pushing a
patch to 8.4 that was completed before 8.3 feature freeze is guaranteed
to add to that --- and if we lose our developers, we might as well shut
down the PostgreSQL project.

One good thing is that we have community discussion this now, so at
least we are focusing on it.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Stefan Kaltenbrunner wrote:
> > PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> > developed outside of core. Don't get me wrong I like the feature but it
> > can take advantage of facilities outside of core.
>
> url fixed - I wonder why we had so much of them and all those pointing
> to the patch list bruce maintains - are the urls to the mails there not
> stable somehow ?

Yep, the URLs are unstable. I am using mhonarc and regenerate the page
when the patches mbox file changes. When I append to the mailbox, the
URLs don't change, but when I delete, they do.

Does anyone know a way to make those URLs stable?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Alvaro Herrera wrote:
> Stefan Kaltenbrunner wrote:
> > Alvaro Herrera wrote:
> > > Stefan Kaltenbrunner wrote:
> > >> Joshua D. Drake wrote:
> > >
> > >>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> > >>> developed outside of core. Don't get me wrong I like the feature but it
> > >>> can take advantage of facilities outside of core.
> > >> url fixed - I wonder why we had so much of them and all those pointing
> > >> to the patch list bruce maintains - are the urls to the mails there not
> > >> stable somehow ?
> > >
> > > They are not stable. The items should point to the archives, which are
> > > supposedly more stable. (I had already fixed one item in PatchStatus
> > > this morning). Really it would be much nicer to have links using the
> > > Message-Id but I doubt that's at all doable.
> >
> > hrm - I see so is there a particular reason for that behaviour ?
> >
> > I will work on pointing to the archives tomorrow ...
>
> Bruce adds and removes emails from the "pgpatches" inbox and I assume he
> regenerates the MHonarc archives when he does, which may change the URL
> for each specific item. I don't think it was ever intended that the
> URLs were to be stable.

Right. Is there no way to make the mhonarc URLs stable?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> Stefan Kaltenbrunner wrote:
> > > PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> > > developed outside of core. Don't get me wrong I like the feature but it
> > > can take advantage of facilities outside of core.
> >
> > url fixed - I wonder why we had so much of them and all those pointing
> > to the patch list bruce maintains - are the urls to the mails there not
> > stable somehow ?
>
> Yep, the URLs are unstable. I am using mhonarc and regenerate the page
> when the patches mbox file changes. When I append to the mailbox, the
> URLs don't change, but when I delete, they do.
>
> Does anyone know a way to make those URLs stable?

Does it matter? I assume that we should not consider pgpatches to be a
stable source anyway ... those emails are there just as a reminder for
short-lived information anyway, so why bother?

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Bruce Momjian wrote:
> Oleg Bartunov wrote:
> This is a good example of how developers can get frustrated. Pushing a
> patch to 8.4 that was completed before 8.3 feature freeze is guaranteed
> to add to that --- and if we lose our developers, we might as well shut
> down the PostgreSQL project.

Let's not call the sky falling just yet. We are doing what we can. There
are several sub projects afoot trying to eliminate some of this pain. If
you recall, Lukas and I both spent time before feature freeze helping
keep people in communication.

Secondly and although you aren't 100% on board, there is also the
tracker project.

Lastly, more people are starting to review code. Even people that can't
code are starting to at least participate in testing features.

>
> One good thing is that we have community discussion this now, so at
> least we are focusing on it.
>

Agreed, but it certainly is not a critical mass problem either. We are
starting to bounce off the wall, and are starting to take a step back to
reflect what size ladder or rope we need to get over it.

Joshua D. Drake

Alvaro Herrera wrote:
> Bruce Momjian wrote:
>> Stefan Kaltenbrunner wrote:
>>>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
>>>> developed outside of core. Don't get me wrong I like the feature but it
>>>> can take advantage of facilities outside of core.
>>> url fixed - I wonder why we had so much of them and all those pointing
>>> to the patch list bruce maintains - are the urls to the mails there not
>>> stable somehow ?
>> Yep, the URLs are unstable. I am using mhonarc and regenerate the page
>> when the patches mbox file changes. When I append to the mailbox, the
>> URLs don't change, but when I delete, they do.
>>
>> Does anyone know a way to make those URLs stable?
>
> Does it matter? I assume that we should not consider pgpatches to be a
> stable source anyway ... those emails are there just as a reminder for
> short-lived information anyway, so why bother?

Yeah let's just source archives, the urls are reasonable stable there.

Joshua D. Drake

>

Joshua D. Drake wrote:
> Bruce Momjian wrote:
> > Oleg Bartunov wrote:
> > This is a good example of how developers can get frustrated. Pushing a
> > patch to 8.4 that was completed before 8.3 feature freeze is guaranteed
> > to add to that --- and if we lose our developers, we might as well shut
> > down the PostgreSQL project.
>
> Let's not call the sky falling just yet. We are doing what we can. There
> are several sub projects afoot trying to eliminate some of this pain. If
> you recall, Lukas and I both spent time before feature freeze helping
> keep people in communication.
>
> Secondly and although you aren't 100% on board, there is also the
> tracker project.
>
> Lastly, more people are starting to review code. Even people that can't
> code are starting to at least participate in testing features.
>
> >
> > One good thing is that we have community discussion this now, so at
> > least we are focusing on it.
> >
>
> Agreed, but it certainly is not a critical mass problem either. We are
> starting to bounce off the wall, and are starting to take a step back to
> reflect what size ladder or rope we need to get over it.

Right. I just don't want a short-term solution (push patches for 8.4)
that avoids a long-term analysis.

Lots of people complained we didn't have a patches status page. Someone
has done that and it is current, but it is not moving us forward fast
enough.

What is our next move, or do we just keep moving and settle on August?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> Joshua D. Drake wrote:

>>> One good thing is that we have community discussion this now, so at
>>> least we are focusing on it.
>>>
>> Agreed, but it certainly is not a critical mass problem either. We are
>> starting to bounce off the wall, and are starting to take a step back to
>> reflect what size ladder or rope we need to get over it.
>
> Right. I just don't want a short-term solution (push patches for 8.4)
> that avoids a long-term analysis.
>
> Lots of people complained we didn't have a patches status page. Someone
> has done that and it is current, but it is not moving us forward fast
> enough.
>
> What is our next move, or do we just keep moving and settle on August?

Well I think what you said about reviewing my list versus Tom's triage
is a good start. If there are things we can push, let's push them. Plus
based on this conversation I think we need reviewers to step up and
say... "Hey! this is where I am at...".

Sincerely,

Joshua D. Drake

Joshua D. Drake wrote:
> Bruce Momjian wrote:
> > Joshua D. Drake wrote:
>
> >>> One good thing is that we have community discussion this now, so at
> >>> least we are focusing on it.
> >>>
> >> Agreed, but it certainly is not a critical mass problem either. We are
> >> starting to bounce off the wall, and are starting to take a step back to
> >> reflect what size ladder or rope we need to get over it.
> >
> > Right. I just don't want a short-term solution (push patches for 8.4)
> > that avoids a long-term analysis.
> >
> > Lots of people complained we didn't have a patches status page. Someone
> > has done that and it is current, but it is not moving us forward fast
> > enough.
> >
> > What is our next move, or do we just keep moving and settle on August?
>
> Well I think what you said about reviewing my list versus Tom's triage
> is a good start. If there are things we can push, let's push them. Plus
> based on this conversation I think we need reviewers to step up and
> say... "Hey! this is where I am at...".

Agreed. I have already contacted all reviewers individually to ask them
for a status, and mentioned I was concerned.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Tue, May 15, 2007 at 12:42:28PM -0400, Bruce Momjian wrote:
> Joshua D. Drake wrote:
> > > Patch status:
> > >
> > > http://developer.postgresql.org/index.php/Todo:PatchStatus
> >
> > If... this is actually a problem (I leave to other committers and
> > reviewers to comment) then I suggest we push all patches without a
> > reviewer as of now to 8.4.
> >
> > Leaving only those patches that have confirmed reviewers to be worked
> > through.
> >
> > FYI, whoever did that Todo:Patch status, Bravo! That is easily one of
> > the smallest but best improvements to the process I have seen in recent
> > memory.
>
> I did one of those for previous releases. I guess you forgot. It was
> done by someone else this time only because I was going to be traveling.

Unless you're really in love with doing that sort of thing it's really
good that someone else did it. You're one of a handful of folks that can
actually review patches, while there's any number of us that can update
a wiki.

Speaking of reviewers... should we put some thought into how we can
increase the number of people who can review code? It seems that's one
of our biggest bottlenecks...
--
Jim Nasby decibel(at)decibel(dot)org
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

On Tue, May 15, 2007 at 10:32:14PM +0200, Magnus Hagander wrote:
> Stefan Kaltenbrunner wrote:
> >> They are not stable. The items should point to the archives, which are
> >> supposedly more stable. (I had already fixed one item in PatchStatus
> >> this morning). Really it would be much nicer to have links using the
> >> Message-Id but I doubt that's at all doable.
> >
> > hrm - I see so is there a particular reason for that behaviour ?
>
> They're stable until Bruce removes something from the queue. When
> something is removed, it's renumbered.
>
> It's how mhonarc works. It's the same with the archives - if we delete a
> mail, they get renumbered. So we never should delete, we should just
> blank out, but it has happened a couple of times.

Isn't there any other archiver we could use? The lack of URL stability
in mhonarc is bad enough, but the cross-month issue is just horrible.
--
Jim Nasby decibel(at)decibel(dot)org
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

Jim C. Nasby wrote:
> On Tue, May 15, 2007 at 12:42:28PM -0400, Bruce Momjian wrote:
> > Joshua D. Drake wrote:
> > > > Patch status:
> > > >
> > > > http://developer.postgresql.org/index.php/Todo:PatchStatus
> > >
> > > If... this is actually a problem (I leave to other committers and
> > > reviewers to comment) then I suggest we push all patches without a
> > > reviewer as of now to 8.4.
> > >
> > > Leaving only those patches that have confirmed reviewers to be worked
> > > through.
> > >
> > > FYI, whoever did that Todo:Patch status, Bravo! That is easily one of
> > > the smallest but best improvements to the process I have seen in recent
> > > memory.
> >
> > I did one of those for previous releases. I guess you forgot. It was
> > done by someone else this time only because I was going to be traveling.
>
> Unless you're really in love with doing that sort of thing it's really
> good that someone else did it. You're one of a handful of folks that can
> actually review patches, while there's any number of us that can update
> a wiki.

Sure. Good idea. All I had to do was ask (10 times, though). ;-)

> Speaking of reviewers... should we put some thought into how we can
> increase the number of people who can review code? It seems that's one
> of our biggest bottlenecks...

Sure. That is going to move us forward.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Jim C. Nasby wrote:
> On Tue, May 15, 2007 at 10:32:14PM +0200, Magnus Hagander wrote:
> > Stefan Kaltenbrunner wrote:
> > >> They are not stable. The items should point to the archives, which are
> > >> supposedly more stable. (I had already fixed one item in PatchStatus
> > >> this morning). Really it would be much nicer to have links using the
> > >> Message-Id but I doubt that's at all doable.
> > >
> > > hrm - I see so is there a particular reason for that behaviour ?
> >
> > They're stable until Bruce removes something from the queue. When
> > something is removed, it's renumbered.
> >
> > It's how mhonarc works. It's the same with the archives - if we delete a
> > mail, they get renumbered. So we never should delete, we should just
> > blank out, but it has happened a couple of times.
>
> Isn't there any other archiver we could use? The lack of URL stability
> in mhonarc is bad enough, but the cross-month issue is just horrible.

Agreed. It is just crippled.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Tue, May 15, 2007 at 07:01:39PM -0400, Bruce Momjian wrote:
> > Unless you're really in love with doing that sort of thing it's really
> > good that someone else did it. You're one of a handful of folks that can
> > actually review patches, while there's any number of us that can update
> > a wiki.
>
> Sure. Good idea. All I had to do was ask (10 times, though). ;-)

Even having to ask 10 times undoubtedly was a lot faster than doing it
yourself though. :)

> > Speaking of reviewers... should we put some thought into how we can
> > increase the number of people who can review code? It seems that's one
> > of our biggest bottlenecks...
>
> Sure. That is going to move us forward.

So... any ideas? :)
--
Jim Nasby decibel(at)decibel(dot)org
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

Jim C. Nasby wrote:
> On Tue, May 15, 2007 at 07:01:39PM -0400, Bruce Momjian wrote:
> > > Unless you're really in love with doing that sort of thing it's really
> > > good that someone else did it. You're one of a handful of folks that can
> > > actually review patches, while there's any number of us that can update
> > > a wiki.
> >
> > Sure. Good idea. All I had to do was ask (10 times, though). ;-)
>
> Even having to ask 10 times undoubtedly was a lot faster than doing it
> yourself though. :)

Very true.

> > > Speaking of reviewers... should we put some thought into how we can
> > > increase the number of people who can review code? It seems that's one
> > > of our biggest bottlenecks...
> >
> > Sure. That is going to move us forward.
>
> So... any ideas? :)

No, that is my problem. I was hoping others would come up with some. I
think communicating the problem is part of the solution.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

> Stefan Kaltenbrunner wrote:
> >> They are not stable. The items should point to the archives, which are
> >> supposedly more stable. (I had already fixed one item in PatchStatus
> >> this morning). Really it would be much nicer to have links using the
> >> Message-Id but I doubt that's at all doable.
> >
> > hrm - I see so is there a particular reason for that behaviour ?
>
> They're stable until Bruce removes something from the queue. When
> something is removed, it's renumbered.
>
> It's how mhonarc works. It's the same with the archives - if we delete a
> mail, they get renumbered. So we never should delete, we should just
> blank out, but it has happened a couple of times.

As I proposed for many times, why don't we add message number to each
subject line in mail? For example like this:

[HACKERS: 12345] Re: Not ready for 8.3

This way, we could always obtain stable (logical) pointer, without
reling on particular archival infrastructure.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

On 5/15/07, Tatsuo Ishii <ishii(at)postgresql(dot)org> wrote:
>
> > Stefan Kaltenbrunner wrote:
> > >> They are not stable. The items should point to the archives, which
> are
> > >> supposedly more stable. (I had already fixed one item in PatchStatus
> > >> this morning). Really it would be much nicer to have links using the
> > >> Message-Id but I doubt that's at all doable.
> > >
> > > hrm - I see so is there a particular reason for that behaviour ?
> >
> > They're stable until Bruce removes something from the queue. When
> > something is removed, it's renumbered.
> >
> > It's how mhonarc works. It's the same with the archives - if we delete a
> > mail, they get renumbered. So we never should delete, we should just
> > blank out, but it has happened a couple of times.
>
> As I proposed for many times, why don't we add message number to each
> subject line in mail? For example like this:
>
> [HACKERS: 12345] Re: Not ready for 8.3
>
> This way, we could always obtain stable (logical) pointer, without
> reling on particular archival infrastructure.

This sounds like a good idea to me - though I'm just a lurker on the list.

Nathan

Tatsuo Ishii wrote:
> > Stefan Kaltenbrunner wrote:
> > >> They are not stable. The items should point to the archives, which are
> > >> supposedly more stable. (I had already fixed one item in PatchStatus
> > >> this morning). Really it would be much nicer to have links using the
> > >> Message-Id but I doubt that's at all doable.
> > >
> > > hrm - I see so is there a particular reason for that behaviour ?
> >
> > They're stable until Bruce removes something from the queue. When
> > something is removed, it's renumbered.
> >
> > It's how mhonarc works. It's the same with the archives - if we delete a
> > mail, they get renumbered. So we never should delete, we should just
> > blank out, but it has happened a couple of times.
>
> As I proposed for many times, why don't we add message number to each
> subject line in mail? For example like this:
>
> [HACKERS: 12345] Re: Not ready for 8.3
>
> This way, we could always obtain stable (logical) pointer, without
> reling on particular archival infrastructure.

While that helps, it does not actually allow software to link the
threads together, i.e. it can't tell which is the head and tail of the
thread. The actual software needs to improve because the thread
information is already part of the email headers.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Tuesday 15 May 2007 16:48, Bruce Momjian wrote:
> Joshua D. Drake wrote:
> > > That is not fair to patch submitters, and pushes the problem to 8.4,
> > > where it will be no better.
> >
> > If it isn't done, it isn't done. We aren't dropping the patch. The patch
> > has been accepted, just not reviewed. It is just delayed.
>
> Delayed isn't rejected, but it isn't accepted either. I see that as
> unfair to patch submitters, and not making things any easier. It just
> pushes our problem into 8.4.
>

So, I'm not sure that these are actual rules, but it's how I have always seen
the process working:

a) if we cannot resolve a submitted patch of technical issues within a
reasonable time, it gets pushed to the next release.

it is at the discression of -core/reviewer to determine both what are
unresolvable issues and what is an unreasonable time... but I think patches
that are not making progress within a period of months probably qualify on
both counts.

b) the development branch for the next release is not opened until delayed
patches are dealt with

dealing with a patch may not mean committing it, if it determined that the
flaws are enough that it just isnt ready, or the author decides they aren't
ready to devote time to it.

i think everyone understands we all want everyones patches to get in, but it
isn't realistic to think that will happen every release.

one of the reasons we have a feature freeze is because the community has
decided that holding up a release for feature X isn't desirable; aiui all of
the patches in the current queue have been given a once over, and some more
than that; i'm sure that cycle could go on quite some time for some patches,
so as unfortunate as it is, I think at some point you have to just suck it up
and call it day. all imho.

--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

Bruce,

> It seems unfair to disinguish between early/last in cycle postings.

I think it's fair. A patch which was submitted for 8.2 and didn't get in
should get consideration over a patch which was still in prototype form a
week before feature freeze. Also, I really don't think it's a crime to push
back patches which are under active redevelopment (seqscan) if that
redevelopment isn't a result of review neglect.

> > Obviously for 8.4 reviewers need to start reviewing stuff from the first
> > week of the development cycle. I also don't actually see anything wrong
> > with a 3-month feature freeze if we can somehow branch development
> > earlier. Easy for me to say, I know.
>
> Yea, this is just pushing off work until later, which I don't support.
> There is no easy out here and I am afraid we will just have to accept a
> 3-month feature freeze.

I think that may be where we're heading. In that case, we may need to talk
about branching earlier so that developers can work on the new version who
are frozen out of the in-process one.

> Oleg has been producing new versions of his patch, and no one has
> objected to any of it, so I assume all the issues were addressed.
>
> As far was whether tsearch2 should be in core, I think most agreed that
> SQL support for full-text would make it easier to use, and that it is a
> fundamental technology. As I remember, the only holdback was full
> multi-byte support, but that is done.

I'm going to echo Bruce on this; I've mentioned that TSearch was going into
Core at conferences and the reaction from existing users has been very
enthusiastic, ranging from "yippee!" to "about time!". Our users hate the
fact that FTS is a separate module.

--
Josh Berkus
PostgreSQL @ Sun
San Francisco

> Tatsuo Ishii wrote:
> > > Stefan Kaltenbrunner wrote:
> > > >> They are not stable. The items should point to the archives, which are
> > > >> supposedly more stable. (I had already fixed one item in PatchStatus
> > > >> this morning). Really it would be much nicer to have links using the
> > > >> Message-Id but I doubt that's at all doable.
> > > >
> > > > hrm - I see so is there a particular reason for that behaviour ?
> > >
> > > They're stable until Bruce removes something from the queue. When
> > > something is removed, it's renumbered.
> > >
> > > It's how mhonarc works. It's the same with the archives - if we delete a
> > > mail, they get renumbered. So we never should delete, we should just
> > > blank out, but it has happened a couple of times.
> >
> > As I proposed for many times, why don't we add message number to each
> > subject line in mail? For example like this:
> >
> > [HACKERS: 12345] Re: Not ready for 8.3
> >
> > This way, we could always obtain stable (logical) pointer, without
> > reling on particular archival infrastructure.
>
> While that helps, it does not actually allow software to link the
> threads together, i.e. it can't tell which is the head and tail of the
> thread. The actual software needs to improve because the thread
> information is already part of the email headers.

I know. My point is, subject-with-sequence will guarantee at least we
do not lose the pointer completely.

Also it will help us searching particular message in the private mail
box.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

Josh Berkus wrote:
>> Yea, this is just pushing off work until later, which I don't support.
>> There is no easy out here and I am afraid we will just have to accept a
>> 3-month feature freeze.
>>
>
> I think that may be where we're heading. In that case, we may need to talk
> about branching earlier so that developers can work on the new version who
> are frozen out of the in-process one.
>
>

I've argued this in the past. But be aware that it will make more work
for committers. It is very far from cost-free.

cheers

andrew

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, May 16, 2007 at 10:16:43AM +0900, Tatsuo Ishii wrote:
> > Stefan Kaltenbrunner wrote:
> > >> They are not stable. [...]

> As I proposed for many times, why don't we add message number to each
> subject line in mail? For example like this:
>
> [HACKERS: 12345] Re: Not ready for 8.3

What I don't understand is why mailing list software doesn't use message
ID as primary index. Granted, it's ugly, but it is globally stable (and
hopefully unique) _across all mailboxes_

Sorry for the random rant
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFGSrYGBcgs9XrR2kYRAlY2AJ9Vq5JZZEqX/y/DkN/HJ4Wg47RMyQCfbdgh
Z6KnR4eJHh/oDHr7GI/OiPU=
=Nxev
-----END PGP SIGNATURE-----

Jim C. Nasby wrote:
> On Tue, May 15, 2007 at 10:32:14PM +0200, Magnus Hagander wrote:
>> Stefan Kaltenbrunner wrote:
>>>> They are not stable. The items should point to the archives, which are
>>>> supposedly more stable. (I had already fixed one item in PatchStatus
>>>> this morning). Really it would be much nicer to have links using the
>>>> Message-Id but I doubt that's at all doable.
>>> hrm - I see so is there a particular reason for that behaviour ?
>> They're stable until Bruce removes something from the queue. When
>> something is removed, it's renumbered.
>>
>> It's how mhonarc works. It's the same with the archives - if we delete a
>> mail, they get renumbered. So we never should delete, we should just
>> blank out, but it has happened a couple of times.
>
> Isn't there any other archiver we could use? The lack of URL stability
> in mhonarc is bad enough, but the cross-month issue is just horrible.

Nothing useful last time I looked (a year or two back admittedley). I
have a design for one in mind that I was looking to prototype - there
are some php classes that would make it quite simple to get messages
into a database either via procmail, or from an mbox.

the stumbling block I was running into was rewriting the old archives
URLs to the new ones.

Regards, Dave

* Tatsuo Ishii <ishii(at)postgresql(dot)org> [070515 21:19]:

> As I proposed for many times, why don't we add message number to each
> subject line in mail? For example like this:
>
> [HACKERS: 12345] Re: Not ready for 8.3
>
> This way, we could always obtain stable (logical) pointer, without
> reling on particular archival infrastructure.

Isn't that what the "Message-Id" field is for?

http://news.gmane.org/find-root(dot)php?message_id=20070516(dot)101643(dot)94564776(dot)t-ishii(at)sraoss(dot)co(dot)jp
a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

> * Tatsuo Ishii <ishii(at)postgresql(dot)org> [070515 21:19]:
>
> > As I proposed for many times, why don't we add message number to each
> > subject line in mail? For example like this:
> >
> > [HACKERS: 12345] Re: Not ready for 8.3
> >
> > This way, we could always obtain stable (logical) pointer, without
> > reling on particular archival infrastructure.
>
> Isn't that what the "Message-Id" field is for?
>
> http://news.gmane.org/find-root(dot)php?message_id=20070516(dot)101643(dot)94564776(dot)t-ishii(at)sraoss(dot)co(dot)jp
> a.

Maybe. However I think "subject-sequence" has some advantages over
Message-Id:

- Easy to identify. Message-Id may not appear on some MUA with default
setting

- More handy than lengthy message Id

- Easy to detect messages not delivered, by knowing that the sequence
number was skipped
--
Tatsuo Ishii
SRA OSS, Inc. Japan

> I'm going to echo Bruce on this; I've mentioned that TSearch was going

> into Core at conferences and the reaction from existing users has been

> very enthusiastic, ranging from "yippee!" to "about time!". Our users

> hate the fact that FTS is a separate module.

Here here.

And with respect to the debate about syntax, who cares? I think I
prefer introducing real SQL-ish syntax over a bunch of pg_* functions,
but doing it either way is IMHO better than doing nothing.

...Robert

Am Mittwoch, 16. Mai 2007 05:37 schrieb Josh Berkus:
> In that case, we may need to talk
> about branching earlier so that developers can work on the new version who
> are frozen out of the in-process one.

Well, we could branch right now, but who is going to commit anything into that
new head branch? The committers are already claimed to be falling behind.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Am Mittwoch, 16. Mai 2007 05:37 schrieb Josh Berkus:
> I'm going to echo Bruce on this; I've mentioned that TSearch was going into
> Core at conferences and the reaction from existing users has been very
> enthusiastic, ranging from "yippee!" to "about time!".  Our users hate the
> fact that FTS is a separate module.

At the same time they are subconsciously counting on us to make decisions
based on rationality, not on enthusiasm.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Robert Haas wrote:
>> Our users hate the fact that FTS is a separate module.
>>
>
> Here here.
>

Where? Where? Oh, you mean "Hear Hear." (sorry - one of my pet peeves)
> And with respect to the debate about syntax, who cares? I think I
> prefer introducing real SQL-ish syntax over a bunch of pg_* functions,
> but doing it either way is IMHO better than doing nothing.
>
>
>

We do have a responsibility, I think, to keep the grammar fairly clean,
so the answer to your question "who cares?" is "we do."

That said, last time I looked most of the warts seemed to have been
knocked off, IIRC, and the functional syntax would have been
sufficiently ugly and cumbersome to weigh against it. So, like most
others, I'm in favor of going with this unless there is some
overwhelming reason I haven't heard of yet not to.

cheers

andrew

tomas(at)tuxteam(dot)de wrote:
> On Wed, May 16, 2007 at 10:16:43AM +0900, Tatsuo Ishii wrote:
> > > Stefan Kaltenbrunner wrote:
> > > >> They are not stable. [...]
>
> > As I proposed for many times, why don't we add message number to each
> > subject line in mail? For example like this:
> >
> > [HACKERS: 12345] Re: Not ready for 8.3
>
> What I don't understand is why mailing list software doesn't use message
> ID as primary index. Granted, it's ugly, but it is globally stable (and
> hopefully unique) _across all mailboxes_

It does. The problem is that it only considers messages posted in the
last calendar month. So on the 1st of each month, all threads "start
again" as far as the archive goes.

There is just one remaining problem: Outlook and derivatives don't set
the In-Reply-To: nor References: headers. This breaks the threads (the
best the software can do is group the messages by subject, but it works
only partially).

I've tried a coupld of times to decode how the Thread-Id stuff works,
which is what Outlook seems to use, but I haven't been able to detect a
pattern.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Tatsuo Ishii wrote:
> > * Tatsuo Ishii <ishii(at)postgresql(dot)org> [070515 21:19]:
> >
> > > As I proposed for many times, why don't we add message number to each
> > > subject line in mail? For example like this:
> > >
> > > [HACKERS: 12345] Re: Not ready for 8.3
> > >
> > > This way, we could always obtain stable (logical) pointer, without
> > > reling on particular archival infrastructure.
> >
> > Isn't that what the "Message-Id" field is for?
> >
> > http://news.gmane.org/find-root(dot)php?message_id=20070516(dot)101643(dot)94564776(dot)t-ishii(at)sraoss(dot)co(dot)jp
> > a.
>
> Maybe. However I think "subject-sequence" has some advantages over
> Message-Id:
>
> - Easy to identify. Message-Id may not appear on some MUA with default
> setting

Message-Ids are present in all messages. When the MUA doesn't set it,
the MTA does. The problem starts when the MUA doesn't set the
In-Reply-To header.

> - More handy than lengthy message Id

True.

> - Easy to detect messages not delivered, by knowing that the sequence
> number was skipped

The problem is that the number would be possibly set at a later stage of
email delivery by the list software, so it doesn't help if the message
is skipped in an earlier stage (spam filter, etc).

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

* Tatsuo Ishii <ishii(at)postgresql(dot)org> [070516 07:23]:

> Maybe. However I think "subject-sequence" has some advantages over
> Message-Id:
>
> - Easy to identify. Message-Id may not appear on some MUA with default
> setting
>
> - More handy than lengthy message Id
>
> - Easy to detect messages not delivered, by knowing that the sequence
> number was skipped

IMNSHO, we should be encouraging lists to move *away* from subject
munging. Adding more characters into a subject line which can already
be quite long is just making the situation worse.

And of course, once you realize that subject munging is wrong, putting
the "list-sequence" into a header of no real value, unless you believe
your MTA/MUA filtering to be somehow dropping list messages, and your
sequence numbers will prove to you if that's true or not. Oh - but
wait, don't we all, as PostgreSQL users realize that sequences aren't
generally guaranteed to be gapless anyways (sure, there are work
"solutions", but I'm not about to audit any MTA/list software for
that...)

;-)

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, May 16, 2007 at 10:03:47AM -0400, Alvaro Herrera wrote:
> tomas(at)tuxteam(dot)de wrote:

[...]
> There is just one remaining problem: Outlook and derivatives don't set
> the In-Reply-To: nor References: headers. This breaks the threads (the
> best the software can do is group the messages by subject, but it works
> only partially).
>
> I've tried a coupld of times to decode how the Thread-Id stuff works,
> which is what Outlook seems to use, but I haven't been able to detect a
> pattern.

Heh. Seems to be a well-known problem:

<http://osdir.com/ml/discuss/2003-10/threads.html#00194>

Ackthpttp.

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFGSxVVBcgs9XrR2kYRAsRrAJ9GC7fHJgaS424yoylnzB/9YPtE/wCbBCRE
C62mzkj1BVXKlIau0TKlOS4=
=GVAM
-----END PGP SIGNATURE-----

> > Here here.
> >
> Where? Where? Oh, you mean "Hear Hear." (sorry - one of my pet peeves)

Oops. Of course since it's in written form perhaps I should be writing
"Read! Read!" instead.

> We do have a responsibility, I think, to keep the grammar fairly
clean,
> so the answer to your question "who cares?" is "we do."

Sure. I'm asking that rhetorical question under the assumption that
none of the options on the table are so bad as to constitute an
abrogation of that responsibility. If that's not the case, then it's an
issue; otherwise, we shouldn't let differences of opinion over a
question that may not have only one right answer prevent us from doing
anything at all.

...Robert

On Wed, May 16, 2007 at 08:58:44AM +0100, Dave Page wrote:
> Jim C. Nasby wrote:
> > On Tue, May 15, 2007 at 10:32:14PM +0200, Magnus Hagander wrote:
> >> Stefan Kaltenbrunner wrote:
> >>>> They are not stable. The items should point to the archives, which are
> >>>> supposedly more stable. (I had already fixed one item in PatchStatus
> >>>> this morning). Really it would be much nicer to have links using the
> >>>> Message-Id but I doubt that's at all doable.
> >>> hrm - I see so is there a particular reason for that behaviour ?
> >> They're stable until Bruce removes something from the queue. When
> >> something is removed, it's renumbered.
> >>
> >> It's how mhonarc works. It's the same with the archives - if we delete a
> >> mail, they get renumbered. So we never should delete, we should just
> >> blank out, but it has happened a couple of times.
> >
> > Isn't there any other archiver we could use? The lack of URL stability
> > in mhonarc is bad enough, but the cross-month issue is just horrible.
>
> Nothing useful last time I looked (a year or two back admittedley). I
> have a design for one in mind that I was looking to prototype - there
> are some php classes that would make it quite simple to get messages
> into a database either via procmail, or from an mbox.
>
> the stumbling block I was running into was rewriting the old archives
> URLs to the new ones.

How much visibility do we have into the mhonarc database? We should be
able to come up with a simple redirector that would point the old
mhonarc URLs to URLs for the new system...
--
Jim Nasby decibel(at)decibel(dot)org
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

Jim C. Nasby wrote:

> How much visibility do we have into the mhonarc database? We should be
> able to come up with a simple redirector that would point the old
> mhonarc URLs to URLs for the new system...

<cough>database?</cough>

It's a file system. It simply generates HTML (or in our case) PHP files
from each message in an mbox. That's one of the reasons for the monthly
break - without it, the directories would be unusably full of files.

I the current URLs represent the month, and the ID of the message as it
comes out of the mbox I believe. We could probably write a script to
dump a list of message IDs, directories and mbox positions I imagine,
and then import that into a new database.

It's been on my list to rewrite the whole archive system for a while for
various reasons. There is quite a bit of crossover with the patch
tracker I proposed so I was hoping to look at both together.

Regards, Dave

Aidan Van Dyk wrote:
> * Tatsuo Ishii <ishii(at)postgresql(dot)org> [070516 07:23]:
>
> > Maybe. However I think "subject-sequence" has some advantages over
> > Message-Id:
> >
> > - Easy to identify. Message-Id may not appear on some MUA with default
> > setting
> >
> > - More handy than lengthy message Id
> >
> > - Easy to detect messages not delivered, by knowing that the sequence
> > number was skipped
>
> IMNSHO, we should be encouraging lists to move *away* from subject
> munging. Adding more characters into a subject line which can already
> be quite long is just making the situation worse.

+1 on that. It gets worse when messages are crossposted to multiple
lists and so multiple [FOO] strings are put into the subject. On the
other hand I know there is people who remove the [FOO] strings in their
local machines! (I don't do it just because I have been too lazy to
write a procmail rule about it).

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

On Wed, May 16, 2007 at 04:34:56PM +0100, Dave Page wrote:
> >How much visibility do we have into the mhonarc database? We should be
> >able to come up with a simple redirector that would point the old
> >mhonarc URLs to URLs for the new system...
>
> <cough>database?</cough>

And here I thought the reason we used tho POS was because it was the
only archiver that used PostgreSQL as it's backend...

> It's a file system. It simply generates HTML (or in our case) PHP files
> from each message in an mbox. That's one of the reasons for the monthly
> break - without it, the directories would be unusably full of files.
>
> I the current URLs represent the month, and the ID of the message as it
> comes out of the mbox I believe. We could probably write a script to
> dump a list of message IDs, directories and mbox positions I imagine,
> and then import that into a new database.

Yeah, if the files still resemble real emails then we can probably come
up with a way to pull the data in.

> It's been on my list to rewrite the whole archive system for a while for
> various reasons. There is quite a bit of crossover with the patch
> tracker I proposed so I was hoping to look at both together.

Let me know when you start on that...
--
Jim Nasby decibel(at)decibel(dot)org
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

aidan(at)highrise(dot)ca (Aidan Van Dyk) writes:
> * Tatsuo Ishii <ishii(at)postgresql(dot)org> [070516 07:23]:
>
>> Maybe. However I think "subject-sequence" has some advantages over
>> Message-Id:
>>
>> - Easy to identify. Message-Id may not appear on some MUA with default
>> setting
>>
>> - More handy than lengthy message Id
>>
>> - Easy to detect messages not delivered, by knowing that the sequence
>> number was skipped
>
> IMNSHO, we should be encouraging lists to move *away* from subject
> munging. Adding more characters into a subject line which can already
> be quite long is just making the situation worse.
>
> And of course, once you realize that subject munging is wrong, putting
> the "list-sequence" into a header of no real value, unless you believe
> your MTA/MUA filtering to be somehow dropping list messages, and your
> sequence numbers will prove to you if that's true or not. Oh - but
> wait, don't we all, as PostgreSQL users realize that sequences aren't
> generally guaranteed to be gapless anyways (sure, there are work
> "solutions", but I'm not about to audit any MTA/list software for
> that...)
>
> ;-)

The message ID *ought* to be usable for at least some of the desired
purposes; having a web client that uses message IDs as a query
mechanism, and where switching months doesn't ludicrously break
things, would seem to be one part of the solution.

Adding some sort of (I don't know...) database that associates bug
tracking system items with message IDs would seem like an "along-side"
way of linking in relevant information.

Adding x-???: tags might be a way of adding bug tracking information
into messages; it wouldn't help with original copies, but could be a
reasonable change to make in a given message repository.
--
output = ("cbbrowne" "@" "acm.org")
http://cbbrowne.com/info/rdbms.html
If nothing ever sticks to Teflon, how do they make Teflon stick to the
pan?

Robert Haas wrote:
>
>> hate the fact that FTS is a separate module.
>
> Here here.
>
> And with respect to the debate about syntax, who cares? I think I
> prefer introducing real SQL-ish syntax over a bunch of pg_* functions,
> but doing it either way is IMHO better than doing nothing.
>

I care. I want a professional easy to understand and easy to maintain
that doesn't cause potential conflict with future and past development
syntax.

Or should we be writing SQL in assembly?

Joshua D. Drake

> ...Robert
>

> I care. I want a professional easy to understand and easy to maintain
> that doesn't cause potential conflict with future and past development

> syntax.

I agree with this. The point of my comment was that ISTM that an
arbitrary amount of time can be consumed determining the optimal syntax,
during which Oleg is obligated to continually update his patch without
any guarantee that it will be accepted in anything like its current
form, or at all. If people have strong opinions about the syntax, why
were they not ALL expressed when the proposal was originally laid on the
table? Sure, some people offered opinions, but it doesn't appear that
there was any real consensus, and there certainly wasn't any clear
guidance of the form "this is what you need to do to get your patch
accepted", which leaves everything in limbo and Oleg doing a lot of
extra work to keep updating his patch.

I haven't studied the proposed syntaxes in detail, but ISTM that if
there is no consensus then probably all of the alternatives being
advocated are reasonable. Again, if that's not the case, then let's
eliminate the unreasonable ones and pick one of the remaining choices.
But if committer A thinks that X is the only reasonable options and
committer B thinks that Y is the only reasonable option, does that mean
that the patch just sits there forever, or do we find a way to move
forward?

...Robert

Jim C. Nasby wrote:
> On Wed, May 16, 2007 at 04:34:56PM +0100, Dave Page wrote:
>>> How much visibility do we have into the mhonarc database? We should be
>>> able to come up with a simple redirector that would point the old
>>> mhonarc URLs to URLs for the new system...
>> <cough>database?</cough>
>
> And here I thought the reason we used tho POS was because it was the
> only archiver that used PostgreSQL as it's backend...

You're thinking of the old search engine.

>> It's a file system. It simply generates HTML (or in our case) PHP files
>> from each message in an mbox. That's one of the reasons for the monthly
>> break - without it, the directories would be unusably full of files.
>>
>> I the current URLs represent the month, and the ID of the message as it
>> comes out of the mbox I believe. We could probably write a script to
>> dump a list of message IDs, directories and mbox positions I imagine,
>> and then import that into a new database.
>
> Yeah, if the files still resemble real emails then we can probably come
> up with a way to pull the data in.

We have all the mbox files, so we can import them from there as raw
messages.

>> It's been on my list to rewrite the whole archive system for a while for
>> various reasons. There is quite a bit of crossover with the patch
>> tracker I proposed so I was hoping to look at both together.
>
> Let me know when you start on that...

Roger.

/D

Dave Page wrote:
>>> I the current URLs represent the month, and the ID of the message as
>>> it comes out of the mbox I believe. We could probably write a script
>>> to dump a list of message IDs, directories and mbox positions I
>>> imagine, and then import that into a new database.
>>
>> Yeah, if the files still resemble real emails then we can probably come
>> up with a way to pull the data in.
>
> We have all the mbox files, so we can import them from there as raw
> messages.

yeah, that's clearly the best source to work from. It's *possible* work
from the mhonarc files (I've done it before), but it's more work.

>>> It's been on my list to rewrite the whole archive system for a while
>>> for various reasons. There is quite a bit of crossover with the patch
>>> tracker I proposed so I was hoping to look at both together.
>>
>> Let me know when you start on that...
>
> Roger.

Same here - I've done something similar (off mhonarc files and in much
smaller scale) before, and I'm definitely interested in helping out.

//Magnus

Andrew Dunstan wrote:
> Josh Berkus wrote:
>> I think that may be where we're heading. In that case, we may need to
>> talk about branching earlier so that developers can work on the new
>> version who are frozen out of the in-process one.
>
> I've argued this in the past. But be aware that it will make more work
> for committers. It is very far from cost-free.

I hate to bring up the CMS flamewar again, but IMHO the one advantage
the distributed systems (Git, Monotone, Darcs, etc) have over CVS &
subversion is that they push this branch management cost down to the
developer who wants a branch - rather from the guys maintaining the
official CMS.

Ron Mayer wrote:
> Andrew Dunstan wrote:
>> Josh Berkus wrote:
>>> I think that may be where we're heading. In that case, we may need to
>>> talk about branching earlier so that developers can work on the new
>>> version who are frozen out of the in-process one.
>> I've argued this in the past. But be aware that it will make more work
>> for committers. It is very far from cost-free.
>
> I hate to bring up the CMS flamewar again, but IMHO the one advantage
> the distributed systems (Git, Monotone, Darcs, etc) have over CVS &
> subversion is that they push this branch management cost down to the
> developer who wants a branch - rather from the guys maintaining the
> official CMS.

One step at a time :). We are now married, and have just begun to have
have children. We need to buy a house in the next year, and figure out
how to pay off some debt without going into hock up to our ears with a
long term low interest loan.

Joshua D. Drake

>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org
>

On 5/16/07, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
> I care. I want a professional easy to understand and easy to maintain
> that doesn't cause potential conflict with future and past development
> syntax.

You've just tempted me to create embedded SQL in assembly :)

--
Jonah H. Harris, Software Architect | phone: 732.331.1324
EnterpriseDB Corporation | fax: 732.331.1301
33 Wood Ave S, 3rd Floor | jharris(at)enterprisedb(dot)com
Iselin, New Jersey 08830 | http://www.enterprisedb.com/

Magnus Hagander wrote:
>>>> It's been on my list to rewrite the whole archive system for a while
>>>> for various reasons. There is quite a bit of crossover with the patch
>>>> tracker I proposed so I was hoping to look at both together.
>>> Let me know when you start on that...
>> Roger.
>
> Same here - I've done something similar (off mhonarc files and in much
> smaller scale) before, and I'm definitely interested in helping out.

Is everyone aware of this system that runs on a well-known open-source
database?
http://www.archiveopteryx.org/
I've used it in a small way, and while I don't claim to have looked at
it in detail it seems to pretty much do what it claims to.

--
Richard Huxton
Archonet Ltd

Richard Huxton wrote:
> Magnus Hagander wrote:
>>>>> It's been on my list to rewrite the whole archive system for a while
>>>>> for various reasons. There is quite a bit of crossover with the patch
>>>>> tracker I proposed so I was hoping to look at both together.
>>>> Let me know when you start on that...
>>> Roger.
>>
>> Same here - I've done something similar (off mhonarc files and in much
>> smaller scale) before, and I'm definitely interested in helping out.
>
> Is everyone aware of this system that runs on a well-known open-source
> database?
> http://www.archiveopteryx.org/
> I've used it in a small way, and while I don't claim to have looked at
> it in detail it seems to pretty much do what it claims to.
>

Yeah, I looked at it in the past. The database storage part is actually
pretty simple - it's the web front end that's going to take more effort,
and thats what that product doesn't do (or if it does, it's a secondary
function they don't shout about).

Regards, Dave.

Dave Page wrote:
> Richard Huxton wrote:
>> Magnus Hagander wrote:
>>>>>> It's been on my list to rewrite the whole archive system for a while
>>>>>> for various reasons. There is quite a bit of crossover with the patch
>>>>>> tracker I proposed so I was hoping to look at both together.
>>>>> Let me know when you start on that...
>>>> Roger.
>>> Same here - I've done something similar (off mhonarc files and in much
>>> smaller scale) before, and I'm definitely interested in helping out.
>> Is everyone aware of this system that runs on a well-known open-source
>> database?
>> http://www.archiveopteryx.org/
>> I've used it in a small way, and while I don't claim to have looked at
>> it in detail it seems to pretty much do what it claims to.
>>
>
> Yeah, I looked at it in the past. The database storage part is actually
> pretty simple - it's the web front end that's going to take more effort,
> and thats what that product doesn't do (or if it does, it's a secondary
> function they don't shout about).

It's supposed to have something in the latest version, I think. I used
it as backing store for a small workflow app, so I've got some simple
views/functions I added and PHP code (cake-php framework) for displaying
messages if it'll be of any use.

My one concern with the schema was that there didn't seem to be a way to
partition archives (e.g. by year) to make maintenance a little simpler
for large databases.

--
Richard Huxton
Archonet Ltd

On Wed, May 16, 2007 at 07:48:10PM +0200, Magnus Hagander wrote:
> Dave Page wrote:
> >>> I the current URLs represent the month, and the ID of the message as
> >>> it comes out of the mbox I believe. We could probably write a script
> >>> to dump a list of message IDs, directories and mbox positions I
> >>> imagine, and then import that into a new database.
> >>
> >> Yeah, if the files still resemble real emails then we can probably come
> >> up with a way to pull the data in.
> >
> > We have all the mbox files, so we can import them from there as raw
> > messages.
>
> yeah, that's clearly the best source to work from. It's *possible* work
> from the mhonarc files (I've done it before), but it's more work.

We'd want the old URLs to be redirected too, so at some point we'll have
to deal with mhonarc.
--
Jim Nasby decibel(at)decibel(dot)org
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

On Wed, May 16, 2007 at 09:32:44PM +0100, Richard Huxton wrote:
> Dave Page wrote:
> >Richard Huxton wrote:
> >>Magnus Hagander wrote:
> >>>>>>It's been on my list to rewrite the whole archive system for a while
> >>>>>>for various reasons. There is quite a bit of crossover with the patch
> >>>>>>tracker I proposed so I was hoping to look at both together.
> >>>>>Let me know when you start on that...
> >>>>Roger.
> >>>Same here - I've done something similar (off mhonarc files and in much
> >>>smaller scale) before, and I'm definitely interested in helping out.
> >>Is everyone aware of this system that runs on a well-known open-source
> >>database?
> >> http://www.archiveopteryx.org/
> >>I've used it in a small way, and while I don't claim to have looked at
> >>it in detail it seems to pretty much do what it claims to.
> >>
> >
> >Yeah, I looked at it in the past. The database storage part is actually
> >pretty simple - it's the web front end that's going to take more effort,
> >and thats what that product doesn't do (or if it does, it's a secondary
> >function they don't shout about).
>
> It's supposed to have something in the latest version, I think. I used
> it as backing store for a small workflow app, so I've got some simple
> views/functions I added and PHP code (cake-php framework) for displaying
> messages if it'll be of any use.
>
> My one concern with the schema was that there didn't seem to be a way to
> partition archives (e.g. by year) to make maintenance a little simpler
> for large databases.

Luckily I happen to know of a database that would make that transparent
to the app...

But I tend to agree with Dave; the storage part is pretty easy. If we've
still got to write our own front-end ISTM it'd be better to just make it
exactly what we want...
--
Jim Nasby decibel(at)decibel(dot)org
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

On Tue, 15 May 2007, Jim C. Nasby wrote:

> Speaking of reviewers... should we put some thought into how we can
> increase the number of people who can review code? It seems that's one
> of our biggest bottlenecks...

Having recently dragged myself from never seeing the code before to being
able to provide an early review to help the committers out, I can tell you
a few things that would have sped up that process and therefore improve
the odds more people will navigate the trail.

My main difficulty was figuring out a workable way to build a local mirror
of the code (so I could get off-line diffs), keep it in sync with the
development tree, while branching out working areas to evaluate patches or
do new development in. A good example walkthrough of how to do that using
standard tools would be a big help. Hint: if you suggest CVsup I'll
smack you.

Overall, though, I don't think there would have been any substitute for
what I learned by putting together my own small patches, so in some
respects thinking about how to lower the bar for doing that would also
ultimately expand the review pool. The main issues I had as a newcomer to
the codebase was getting data in/out of the new code I added that was
buried inside the system. Here are some examples of what I kept
hoping to find documentation on:

-Examples and usage guidelines for eLog and eReport (the stuff in the FAQ
needs some more meat)

-How to add a GUC variable. Once I've got that, now I can adjust the code
in real-time just by updating it.

-How to add to the statistics for some part of the system that track a new
variable and follow how that moves through the collector process.

If you put people into a position where they easily can poke data in at
one end, see how it rattles around inside the engine by logging, and get
some statistics on the result, now you've got someone who can build their
own simple patches without being so frustrated.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Jim C. Nasby wrote:
> On Wed, May 16, 2007 at 07:48:10PM +0200, Magnus Hagander wrote:
>> Dave Page wrote:
>>>>> I the current URLs represent the month, and the ID of the message as
>>>>> it comes out of the mbox I believe. We could probably write a script
>>>>> to dump a list of message IDs, directories and mbox positions I
>>>>> imagine, and then import that into a new database.
>>>>
>>>> Yeah, if the files still resemble real emails then we can probably come
>>>> up with a way to pull the data in.
>>> We have all the mbox files, so we can import them from there as raw
>>> messages.
>> yeah, that's clearly the best source to work from. It's *possible* work
>> from the mhonarc files (I've done it before), but it's more work.
>
> We'd want the old URLs to be redirected too, so at some point we'll have
> to deal with mhonarc.

Right. Grabbing the msgid alone from them shouldn't be too hard though.
It's included in the "meta-headers" mhonarc sticks in each file, so it
should be a simple regex to find it.

//Magnus

On Wed, May 16, 2007 at 11:50:26PM -0400, Greg Smith wrote:
> On Tue, 15 May 2007, Jim C. Nasby wrote:
>
> >Speaking of reviewers... should we put some thought into how we can
> >increase the number of people who can review code? It seems that's one
> >of our biggest bottlenecks...
>
> Having recently dragged myself from never seeing the code before to being
> able to provide an early review to help the committers out, I can tell you
> a few things that would have sped up that process and therefore improve
> the odds more people will navigate the trail.
>
> My main difficulty was figuring out a workable way to build a local mirror
> of the code (so I could get off-line diffs), keep it in sync with the
> development tree, while branching out working areas to evaluate patches or
> do new development in. A good example walkthrough of how to do that using
> standard tools would be a big help. Hint: if you suggest CVsup I'll
> smack you.
>
> Overall, though, I don't think there would have been any substitute for
> what I learned by putting together my own small patches, so in some
> respects thinking about how to lower the bar for doing that would also
> ultimately expand the review pool. The main issues I had as a newcomer to
> the codebase was getting data in/out of the new code I added that was
> buried inside the system. Here are some examples of what I kept
> hoping to find documentation on:
>
> -Examples and usage guidelines for eLog and eReport (the stuff in the FAQ
> needs some more meat)
>
> -How to add a GUC variable. Once I've got that, now I can adjust the code
> in real-time just by updating it.
>
> -How to add to the statistics for some part of the system that track a new
> variable and follow how that moves through the collector process.
>
> If you put people into a position where they easily can poke data in at
> one end, see how it rattles around inside the engine by logging, and get
> some statistics on the result, now you've got someone who can build their
> own simple patches without being so frustrated.

Would you be interested in providing this meat? You're uniquely
qualified because your shins still smart from all the things you
barked them on :)

Cheers,
D
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
phone: +1 415 235 3778 AIM: dfetter666
Skype: davidfetter

Remember to vote!
Consider donating to PostgreSQL: http://www.postgresql.org/about/donate

Magnus Hagander wrote:
> Right. Grabbing the msgid alone from them shouldn't be too hard though.
> It's included in the "meta-headers" mhonarc sticks in each file, so it
> should be a simple regex to find it.

Should be easier than that - when we import the existing messages from
the mbox files we should be able to figure out the current URL for each
message based on the year/month/list of the mbox file, and the message
number within the mbox.

/D

David Fetter wrote:
>>
>> My main difficulty was figuring out a workable way to build a local mirror
>> of the code (so I could get off-line diffs), keep it in sync with the
>> development tree, while branching out working areas to evaluate patches or
>> do new development in. A good example walkthrough of how to do that using
>> standard tools would be a big help. Hint: if you suggest CVsup I'll
>> smack you.
>>
>>
> Would you be interested in providing this meat? You're uniquely
> qualified because your shins still smart from all the things you
> barked them on :)
>
>
>

For this item at least the work has been done in showing how to set up a
local mirror using rsync instead of CVSup. It's mentioned in the dev
version of the docs at
http://developer.postgresql.org/pgdocs/postgres/rsync.html - although
frankly it would be better to import the information rather than just
refer to the buildfarm HOWTO.

cheers

andredw

On Wednesday 16 May 2007 13:02, Robert Haas wrote:
> > I care. I want a professional easy to understand and easy to maintain
> > that doesn't cause potential conflict with future and past development
> >
> > syntax.
>
> <snip>
> If people have strong opinions about the syntax, why
> were they not ALL expressed when the proposal was originally laid on the
> table?
<snip>
> I haven't studied the proposed syntaxes in detail,
<snip>
>

lol... just thought this was ironic. :-)

--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

On Thursday 17 May 2007 08:16, Andrew Dunstan wrote:
> David Fetter wrote:
> >> My main difficulty was figuring out a workable way to build a local
> >> mirror of the code (so I could get off-line diffs), keep it in sync with
> >> the development tree, while branching out working areas to evaluate
> >> patches or do new development in. A good example walkthrough of how to
> >> do that using standard tools would be a big help. Hint: if you suggest
> >> CVsup I'll smack you.
> >
> > Would you be interested in providing this meat? You're uniquely
> > qualified because your shins still smart from all the things you
> > barked them on :)
>
> For this item at least the work has been done in showing how to set up a
> local mirror using rsync instead of CVSup. It's mentioned in the dev
> version of the docs at
> http://developer.postgresql.org/pgdocs/postgres/rsync.html - although
> frankly it would be better to import the information rather than just
> refer to the buildfarm HOWTO.
>

And really should probably be written up into FAQ section or a full on guide
for "how do i get started hacking on postgresql?"

--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

On Thu, 17 May 2007, Andrew Dunstan wrote:

> For this item at least the work has been done in showing how to set up a
> local mirror using rsync instead of CVSup...although frankly it would be
> better to import the information rather than just refer to the buildfarm
> HOWTO.

The information in the buildfarm HOWTO has maybe 1/2 of what you need to
know--you get a local mirror out of it but no idea how to then use that on
the client side to branch and generate patches. And what it does provide
is less helpful than it might be because it includes diversions specific
to the buildfarm application I found confusing.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

On Thu, 17 May 2007, Robert Treat wrote:

> And really should probably be written up into FAQ section or a full on guide
> for "how do i get started hacking on postgresql?"

To be clear here: while there are guides for new hackers out there, they
focus on the big picture. I walked from those with a good idea of how the
major pieces fit together and was able to navigate the source code. It
was the little details of how to actually work with the code and the
repository that were the biggest drag on my progress.

I used to wonder if it was just me, but watching the discussion that went
along with the "OS/X startup scripts" thread recently made me realize how
many other people struggle(d) with this as well.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Greg Smith wrote:
> On Thu, 17 May 2007, Andrew Dunstan wrote:
>
>> For this item at least the work has been done in showing how to set
>> up a local mirror using rsync instead of CVSup...although frankly it
>> would be better to import the information rather than just refer to
>> the buildfarm HOWTO.
>
> The information in the buildfarm HOWTO has maybe 1/2 of what you need
> to know--you get a local mirror out of it but no idea how to then use
> that on the client side to branch and generate patches. And what it
> does provide is less helpful than it might be because it includes
> diversions specific to the buildfarm application I found confusing.
>
>

Sure. And of course the creation of branches and generation of patches
is irrelevant to the buildfarm. All this just reinforces my point that
we should remove that reference and fill in the blanks on the docs, FAQs
etc.

Incidentally, I don't use a repo mirror for online or offline work,
although I do for buildfarm work. What I do is to have a pristine
checkout of each live stable branch plus HEAD. I have a cron job that
keeps these moderately up to date. When I'm working on a feature or
patch, I make a copy (using cp -a) of the relevant branch and then go to
work on that. If the work lasts a while, every so often I run 'cvs
update' in that copy. And of course I run 'cvs update' before cutting a
patch or doing a commit. When the work is done I blow away the work
directory. If the work is large and I need to checkpoint certain files
locally so I can roll back, I occasionally use RCS.

There's no right answer on how to work - everyone uses tools they feel
comfortable with.

cheers

andrew

On Tue, May 15, 2007 at 04:52:16PM -0400, Alvaro Herrera wrote:

> This is what happens with the Linux kernel. They have hundreds of
> developers getting their hands dirty during a previous period. Then
> 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of
> patches are flooded in.

I hasten to point out that the Linux kernel has also had several
"stable" releases with huge bugs -- things like massive filesystem
corruption, bizarre failure cases, and nasty compatibility problems
with modules across versions. I am not entirely sure that the Linux
model is the one to ape. PostgreSQL has a history with remarkably
few of those blunders, and I'd hate to give that up.

A

--
Andrew Sullivan | ajs(at)crankycanuck(dot)ca
If they don't do anything, we don't need their acronym.
--Josh Hamilton, on the US FEMA

Andrew Sullivan wrote:
> On Tue, May 15, 2007 at 04:52:16PM -0400, Alvaro Herrera wrote:
>
>> This is what happens with the Linux kernel. They have hundreds of
>> developers getting their hands dirty during a previous period. Then
>> 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of
>> patches are flooded in.
>
> I hasten to point out that the Linux kernel has also had several
> "stable" releases with huge bugs --

/me fondly remembers kernel 2.4.

things like massive filesystem
> corruption, bizarre failure cases, and nasty compatibility problems
> with modules across versions. I am not entirely sure that the Linux
> model is the one to ape. PostgreSQL has a history with remarkably
> few of those blunders, and I'd hate to give that up.
>
> A
>

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

Joshua D. Drake wrote:
> Andrew Sullivan wrote:
>> On Tue, May 15, 2007 at 04:52:16PM -0400, Alvaro Herrera wrote:
>>
>>> This is what happens with the Linux kernel. They have hundreds of
>>> developers getting their hands dirty during a previous period. Then
>>> 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of
>>> patches are flooded in.
>>
>> I hasten to point out that the Linux kernel has also had several
>> "stable" releases with huge bugs --
>
> /me fondly remembers kernel 2.4.
>
>

We keep focusing on process. I am on record as saying we can improve our
processes, but the fact is our major immediate problem is person-power,
not process. We need more qualified reviewers. Qualified means (to me,
at least) you have to have done enough visible PostgreSQL hacking that a
committer can reasonably place some level of trust in your review,
thereby saving some time. That's not to say that others can't or
shouldn't do reviews - every little bit helps, but if Freda Bloggs comes
along with a review of some new, large, feature, she isn't helping to
make the process shorter, although she might be helping to make it more
robust.

cheers

andrew

Andrew Sullivan wrote:
> On Tue, May 15, 2007 at 04:52:16PM -0400, Alvaro Herrera wrote:
>
> > This is what happens with the Linux kernel. They have hundreds of
> > developers getting their hands dirty during a previous period. Then
> > 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of
> > patches are flooded in.
>
> I hasten to point out that the Linux kernel has also had several
> "stable" releases with huge bugs -- things like massive filesystem
> corruption, bizarre failure cases, and nasty compatibility problems
> with modules across versions. I am not entirely sure that the Linux
> model is the one to ape. PostgreSQL has a history with remarkably
> few of those blunders, and I'd hate to give that up.

Sorry, I wasn't trying to say that we should follow the Linux model all
that closely. I know there are regressions in the "point zero"
releases, and that there are bugs.

But then, we're nowhere near the scale of Linux. In the press release
for 8.0 we mentioned something about "hundreds of developers". This was
true -- but it was not in the same league as Linux's "hundreds of
developers". We're nowhere near the manpower that Linux has, nowhere
near the amount of code these guys change. I think the figure is around
9000 lines of code changed per day, _every day_[1]. They have new
drivers all the time, internal interfaces are cleaned up, new facilities
are invented to support new kinds of hardware, performance improvements
are made all over the place. So there are a hundred of strange machines
where the thing does not boot; yes, but those bugs are fixed in 2.6.2x.1
or subsequent "stable branch" releases.

I dare not think of the 2.2 or 2.4 disasters, where distributions were
fond of backporting huge patches from the 2.3 or 2.5 development
branches. I compiled my own kernel from Linus' tree back then, which
worked without the strange behavior the other kernels had (maybe the
distro kernels stabilized at some point, but I didn't try later in the
game -- I was too used to compiling my own). Now with 2.6 I don't do
that anymore (of course I don't run 2.6.22 as soon as it is out either).

In my opinion Linux 2.6 is much better than Linux 2.4.

I don't think we should follow either model though. We have different
problems and different people.

[1] http://lwn.net/Articles/232379/

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

On Thu, 17 May 2007, David Fetter wrote:

> Would you be interested in providing this meat? You're uniquely
> qualified because your shins still smart from all the things you
> barked them on :)

Unfortunately I'm temporarily on the other side of this problem; all the
time I have to spare right now is going to into reviewing other people's
patches.

Even more unfortunately, I'm not 100% happy with what I'm doing, and
certainly wouldn't want to present the shoddy techniques I'm using as
recommended. If anyone else has useful info on this subject (keeping a
local repository in sync with HEAD via rsync while working on branches)
they'd like to pass along, please drop me a message with whatever level of
documentation you might have--even rough notes would be a help. Once I'm
finished with the reviews I'm trying to contribute, I will sort through
any suggestions I get and turn that into formal documentation. This has
left enough shin damage that I'd enjoy completely slaying the cause.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Greg Smith wrote:
> On Thu, 17 May 2007, David Fetter wrote:
>
>> Would you be interested in providing this meat? You're uniquely
>> qualified because your shins still smart from all the things you
>> barked them on :)
>
> Unfortunately I'm temporarily on the other side of this problem; all the
> time I have to spare right now is going to into reviewing other people's
> patches.
>
> Even more unfortunately, I'm not 100% happy with what I'm doing, and
> certainly wouldn't want to present the shoddy techniques I'm using as
> recommended. If anyone else has useful info on this subject (keeping a
> local repository in sync with HEAD via rsync while working on branches)
> they'd like to pass along, please drop me a message with whatever level
> of documentation you might have--even rough notes would be a help. Once
> I'm finished with the reviews I'm trying to contribute, I will sort
> through any suggestions I get and turn that into formal documentation.
> This has left enough shin damage that I'd enjoy completely slaying the
> cause.

Let me describe my method. It's not ideal by any means, but I've been
satisfied:

First of all, having a local CVS repository is a must. I used to use
cvsup until a couple of months ago, but it suddenly stopped working, so
I switched to rsync and I haven't looked back. I have a one line shell
script to update it:

rsync --progress -avzCH --delete anoncvs.postgresql.org::pgsql-cvs
/home/hlinnaka/pgcvsrepository

When I start working on a subject, I make a new checkout from the local
repository to a designated directory under "pg_sandbox"-directory. For
example, when I started working on the mvcc-safe cluster patch:

~/pg_sandbox$ cvs -d /home/hlinnaka/pgcvsrepository cvs co pgsql
~/pg_sandbox$ mv pgsql pgsql.cluster
~/pg_sandbox$ cd pgsql.cluster
~/pg_sandbox/pgsql.cluster$ cvs update -dP # Remove empty dirs.
There's a checkout-flag for that as well, but I never remember it...
~/pg_sandbox/pgsql.cluster$ ./configure --enable-depend --enable-cassert
--enable-debug --prefix=/home/hlinnaka/pgsql.cluster

For primitive version control, I make a diff after any significant changes:

~/pg_sandbox/pgsql.cluster$ cvs diff -cN > cluster-mvcc-1.patch

The last number is a revision number, increment by one after each diff.

That's it!

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On 5/18/07, Heikki Linnakangas <heikki(at)enterprisedb(dot)com> wrote:
>
>
>
> When I start working on a subject, I make a new checkout from the local
> repository to a designated directory under "pg_sandbox"-directory. For
> example, when I started working on the mvcc-safe cluster patch:
>
> ~/pg_sandbox$ cvs -d /home/hlinnaka/pgcvsrepository cvs co pgsql
> ~/pg_sandbox$ mv pgsql pgsql.cluster

You can use cvs co -d pgsql.cluster to save the above step.

~/pg_sandbox$ cd pgsql.cluster
> ~/pg_sandbox/pgsql.cluster$ cvs update -dP # Remove empty dirs.
> There's a checkout-flag for that as well, but I never remember it...
> ~/pg_sandbox/pgsql.cluster$ ./configure --enable-depend --enable-cassert
> --enable-debug --prefix=/home/hlinnaka/pgsql.cluster

I find configuring with CFLAGS="-g -O0" more useful.

For primitive version control, I make a diff after any significant changes:
>
> ~/pg_sandbox/pgsql.cluster$ cvs diff -cN > cluster-mvcc-1.patch

I usually commit each version and tag the tree. That helps me to get
diff between two versions as well.

Thanks,
Pavan

--
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com

Pavan Deolasee wrote:
> For primitive version control, I make a diff after any significant changes:
>>
>> ~/pg_sandbox/pgsql.cluster$ cvs diff -cN > cluster-mvcc-1.patch
>
> I usually commit each version and tag the tree. That helps me to get
> diff between two versions as well.

Doesn't that confuse rsync?

I use "diff patch-1 patch-2" for diffs between versions, but it's really
hard to read.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On 5/18/07, Heikki Linnakangas <heikki(at)enterprisedb(dot)com> wrote:
>
> Pavan Deolasee wrote:
> > For primitive version control, I make a diff after any significant
> changes:
> >>
> >> ~/pg_sandbox/pgsql.cluster$ cvs diff -cN > cluster-mvcc-1.patch
> >
> > I usually commit each version and tag the tree. That helps me to get
> > diff between two versions as well.
>
> Doesn't that confuse rsync?

I use cvsup. But thats not relevant here. Yes, cvsup would delete any
local tags/branches and sync the tree. But then I don't sync the tree very
often. And when I need to sync my work with CVS HEAD, I create
all the required diffs, sync the tree and reapply the patches (and commit
and tag) the tree again. Thats some work, but is I am OK with that
because as I said we don't need to sync with head that often.

Thanks

--
Pavan Deolasee
EnterpriseDB http://www.enterprisedb.com

On Thu, May 17, 2007 at 11:02:31PM -0400, Greg Smith wrote:
> On Thu, 17 May 2007, David Fetter wrote:
>
> >Would you be interested in providing this meat? You're uniquely
> >qualified because your shins still smart from all the things you
> >barked them on :)
>
> Unfortunately I'm temporarily on the other side of this problem; all the
> time I have to spare right now is going to into reviewing other people's
> patches.
>
> Even more unfortunately, I'm not 100% happy with what I'm doing, and
> certainly wouldn't want to present the shoddy techniques I'm using as
> recommended. If anyone else has useful info on this subject (keeping a
> local repository in sync with HEAD via rsync while working on branches)
> they'd like to pass along, please drop me a message with whatever level of
> documentation you might have--even rough notes would be a help. Once I'm
> finished with the reviews I'm trying to contribute, I will sort through
> any suggestions I get and turn that into formal documentation. This has
> left enough shin damage that I'd enjoy completely slaying the cause.

Rather than trying to get something "FAQ-perfect" right now, can you at
least brain-dump what your current process is so that others can learn?
I've created a wiki page for this and added Heikki's tips:
http://developer.postgresql.org/index.php/Working_with_CVS
--
Jim Nasby decibel(at)decibel(dot)org
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

On Fri, 18 May 2007, Jim C. Nasby wrote:

> Rather than trying to get something "FAQ-perfect" right now, can you at
> least brain-dump what your current process is so that others can learn?
> I've created a wiki page for this and added Heikki's tips:
> http://developer.postgresql.org/index.php/Working_with_CVS

Heikki's great summary helps (I think the one piece I was screwing up is
covered there), and Pavan's comments adds some useful bits. The still
missing part is how to make a real branch to work in, which is much easier
to work with once you figure out how to do it than either using just diffs
or tagging.

I agree the wiki is the right place to hash this out; when that's
converged to something useful, then take a snapshot and convert for the
official manual. I'll work on it.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Greg Smith wrote:
>
> Heikki's great summary helps (I think the one piece I was screwing up
> is covered there), and Pavan's comments adds some useful bits. The
> still missing part is how to make a real branch to work in, which is
> much easier to work with once you figure out how to do it than either
> using just diffs or tagging.
>
>

What would making a branch actually do for you? The only advantage I can
see is that it will give you a way of checkpointing your files. As I
remarked upthread, I occasionally use RCS for that. But mostly I don't
actually bother. I don't see how you can do it reasonably off a local
cvs mirror - rsync will just blow away any changes you have checked in
next time you sync with the master.

I don't think we can make CVS behave like a distributed SCM system, and
ability to create local branches seems to me one of the fundamental
points of such systems. If that's what the demand is for, then we should
look again at moving to something like Mercurial.

cheers

andrew

Andrew Dunstan wrote:
> What would making a branch actually do for you? The only advantage I can
> see is that it will give you a way of checkpointing your files. As I
> remarked upthread, I occasionally use RCS for that. But mostly I don't
> actually bother. I don't see how you can do it reasonably off a local
> cvs mirror - rsync will just blow away any changes you have checked in
> next time you sync with the master.
>
> I don't think we can make CVS behave like a distributed SCM system, and
> ability to create local branches seems to me one of the fundamental
> points of such systems. If that's what the demand is for, then we should
> look again at moving to something like Mercurial.

I think the great thing about DCVS systems is that not everybody
necessarily needs to use the *same* system. And it doesn't really
matter what the central repository runs on - I think they are
gateway from/to nearly everything available...

I currently use GIT for my SoC project, and it works quite well -
I can create an abitrary number of local branches, and syncing
the currently active branch with CVS is archived by just doing
"cg-update pgsql-head".

greetings, Florian Pflug

On Sat, 19 May 2007, Andrew Dunstan wrote:

> What would making a branch actually do for you? The only advantage I can see
> is that it will give you a way of checkpointing your files.

Exactly. It's not as bad now, but when I was getting started there were
lots of times I got something working and I wanted a clean way to save it
in that state before I started messing with anything else--such that I
could backtrack what I did if I later broke it. This style of work has
some advantages for working on other people's patches as well, especially
if you're trying to review two at once and it takes you a while to
finish--the situation I always find myself in.

> I don't see how you can do it reasonably off a local cvs mirror - rsync
> will just blow away any changes you have checked in next time you sync
> with the master.

It's certainly not easy. I try not to let the fact that what I'd like to
do may actually be impossible ever discourage me.

There are at least three ways to approach this general problem just
counting the rsync/CVS variations, each with some obvious and some subtle
advantages/disadvantages. I'm still in the stage where I'm mapping out
the options.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD