Re: Getting a move on for 8.2 beta

bruce wrote:
> Tom Lane wrote:
> > Bruce Momjian <bruce(at)momjian(dot)us> writes:
> > > Tom Lane wrote:
> > >> Peter has made it pretty clear that he didn't care for the
> > >> refactorization aspect of that patch.
> >
> > > Peter asked why it was done, a good answer was given, and Peter did not
> > > reply.
> >
> > Au contraire, he's reiterated since then that he didn't like it.
>
> The thread order was: patch, Peter comments, submitter gives reasons,
> patch put in the queue, Peter comments again, I reply that the change is
> not just "refactoring" but is needed based on submitters comments, and
> no reply from Peter:
>
> http://archives.postgresql.org/pgsql-patches/2006-08/msg00334.php
>
> Without a reply from Peter, I have to assume the patch is valid.

This is also an interesting example for a tracker. If we had one, all
discussion on the patch would be in one place, but I am thinking that
would require all posting to happen in a browser, or somehow have emails
tagged to attach to each item. Is that something that can happen
easily? I don't know. Would the repost of a patch be attached to the
original submission?

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Sun, Sep 03, 2006 at 03:40:40PM -0400, Bruce Momjian wrote:
> This is also an interesting example for a tracker. If we had one, all
> discussion on the patch would be in one place, but I am thinking that
> would require all posting to happen in a browser, or somehow have emails
> tagged to attach to each item. Is that something that can happen
> easily? I don't know. Would the repost of a patch be attached to the
> original submission?

In debbugs for example, the bug gets an email address
<bugnumber>@bugs.project.site. Anything sent to that address gets
recorded in the bug. So as long as people hit "reply-to-all" the whole
thread will be archived in the bug.

I assume other bug trackers have a similar feature...

Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Martijn van Oosterhout <kleptog(at)svana(dot)org> writes:

> I assume other bug trackers have a similar feature...

Sadly no. That's precisely why I was pushing debbugs so hard earlier.

The weakest of them will send a content-free email saying *something* happened
to your issue and you have to click a link to find out what. Bugzilla is one
step above that: it includes the latest comment though it doesn't thread
comments and it can't handle replies. RT is intended to be email based but it
requires a fair amount of work to get the transparent behaviour you want.

Generally they make you click a link the email and fill in your comments in a
text widget in a browser. That might be acceptable (not to me personally
but...) when you're talking about requests made in some structured environment
where requests are supposed to go through specific workflow. It'll never fly
if you want it to track all the development discussions. I don't see old
school unix hackers used to discussing things in email switching over to some
web based interface to replace development mailing lists.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com

Gregory Stark wrote:
> Martijn van Oosterhout <kleptog(at)svana(dot)org> writes:
>
>
>> I assume other bug trackers have a similar feature...
>>
>
> Sadly no. That's precisely why I was pushing debbugs so hard earlier.
>
> The weakest of them will send a content-free email saying *something* happened
> to your issue and you have to click a link to find out what. Bugzilla is one
> step above that: it includes the latest comment though it doesn't thread
> comments and it can't handle replies. RT is intended to be email based but it
> requires a fair amount of work to get the transparent behaviour you want.
>
> Generally they make you click a link the email and fill in your comments in a
> text widget in a browser. That might be acceptable (not to me personally
> but...) when you're talking about requests made in some structured environment
> where requests are supposed to go through specific workflow. It'll never fly
> if you want it to track all the development discussions. I don't see old
> school unix hackers used to discussing things in email switching over to some
> web based interface to replace development mailing lists.
>
>

Bz has a contrib module which I believe is supposed to handle replies.
I have not used it and don't know how good it is, but there is nothing
to say we can't make it work if we want to.

Maybe you need to take a new look at bz, instead of retailing out of
date info.

As for remarks about old school unix hackers not liking web interfaces,
I think Tom's recent remarks relating to the 21st century were more than
apposite.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:

> As for remarks about old school unix hackers not liking web interfaces, I
> think Tom's recent remarks relating to the 21st century were more than
> apposite.

I like web interfaces well enough for the things they're good at.
Replacing e-mail is not one of them.

--
greg

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> As for remarks about old school unix hackers not liking web interfaces,
> I think Tom's recent remarks relating to the 21st century were more than
> apposite.

I don't see a big problem with using a web interface to search for
information --- they're good at that. If you're thinking that the
hackers community is willing to abandon email as a discussion medium
for a web forum, however, you're going to be sadly disappointed ... and
I don't see how it would help anyway: a pile of text is a pile of text,
no matter where it came from.

The hard part of this problem is finding a convenient way to capture
status data out of the community's conversations. I think when you find
a solution to that, you'll notice that email is not the problem.

regards, tom lane

Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> As for remarks about old school unix hackers not liking web interfaces,
>> I think Tom's recent remarks relating to the 21st century were more than
>> apposite.
>
> I don't see a big problem with using a web interface to search for
> information --- they're good at that. If you're thinking that the
> hackers community is willing to abandon email as a discussion medium
> for a web forum, however, you're going to be sadly disappointed ... and
> I don't see how it would help anyway: a pile of text is a pile of text,
> no matter where it came from.

I don't think anyone would expect that (at least I wouldn't). Heck, even
I wouldn't use a web forum.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>
>> As for remarks about old school unix hackers not liking web interfaces,
>> I think Tom's recent remarks relating to the 21st century were more than
>> apposite.
>>
>
> I don't see a big problem with using a web interface to search for
> information --- they're good at that. If you're thinking that the
> hackers community is willing to abandon email as a discussion medium
> for a web forum, however, you're going to be sadly disappointed ... and
> I don't see how it would help anyway: a pile of text is a pile of text,
> no matter where it came from.
>
> The hard part of this problem is finding a convenient way to capture
> status data out of the community's conversations. I think when you find
> a solution to that, you'll notice that email is not the problem.
>
>
>

You have put your finger on the central problem. Email is a wonderful
way of conducting an ongoing conversation and a horrid way of recording
it (mail archives and search engines notwithstanding).

I live by email, but I am also (sometimes painfully) aware of its
limitations.

Anyway, the larger point in my email was that we might well be able to
have an email conversational mode with bugzilla, if indeed it doesn't
already exist.

cheers

andrew

Martijn van Oosterhout wrote:
-- Start of PGP signed section.
> On Sun, Sep 03, 2006 at 03:40:40PM -0400, Bruce Momjian wrote:
> > This is also an interesting example for a tracker. If we had one, all
> > discussion on the patch would be in one place, but I am thinking that
> > would require all posting to happen in a browser, or somehow have emails
> > tagged to attach to each item. Is that something that can happen
> > easily? I don't know. Would the repost of a patch be attached to the
> > original submission?
>
> In debbugs for example, the bug gets an email address
> <bugnumber>@bugs.project.site. Anything sent to that address gets
> recorded in the bug. So as long as people hit "reply-to-all" the whole
> thread will be archived in the bug.
>
> I assume other bug trackers have a similar feature...

Oh, so the bug is tracked by being part of the email reply list. That
is a good idea. Now, how does that get assigned for non-bugs, like
patches? Does any email sent to the lists that doesn't already have a
bug number get one? That might be really valuable.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Gregory Stark wrote:
>
> Martijn van Oosterhout <kleptog(at)svana(dot)org> writes:
>
> > I assume other bug trackers have a similar feature...
>
> Sadly no. That's precisely why I was pushing debbugs so hard earlier.

Oh. That is bad.

> The weakest of them will send a content-free email saying *something* happened
> to your issue and you have to click a link to find out what. Bugzilla is one
> step above that: it includes the latest comment though it doesn't thread
> comments and it can't handle replies. RT is intended to be email based but it
> requires a fair amount of work to get the transparent behaviour you want.
>
> Generally they make you click a link the email and fill in your comments in a
> text widget in a browser. That might be acceptable (not to me personally
> but...) when you're talking about requests made in some structured environment
> where requests are supposed to go through specific workflow. It'll never fly
> if you want it to track all the development discussions. I don't see old
> school unix hackers used to discussing things in email switching over to some
> web based interface to replace development mailing lists.

Double-ouch. No threads, reply has to be via web browser. I can see
that really killing communication. That bugs email address seemed very
interesting. Could we create something ourselves, and have emails that
don't have a bug id that are sent to the list assigned on?

One problem I have now is that threads are not tracked across months,
and if a new email is started, it doesn't attached to the old threads.
The bug-id might not fix that, but perhaps it would.

Of course one easier fix would be to get our threads to track across
months, but it still doesn't handle cases where new email threads are
started that apply to earlier threads.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> > As for remarks about old school unix hackers not liking web interfaces,
> > I think Tom's recent remarks relating to the 21st century were more than
> > apposite.
>
> I don't see a big problem with using a web interface to search for
> information --- they're good at that. If you're thinking that the
> hackers community is willing to abandon email as a discussion medium
> for a web forum, however, you're going to be sadly disappointed ... and
> I don't see how it would help anyway: a pile of text is a pile of text,
> no matter where it came from.
>
> The hard part of this problem is finding a convenient way to capture
> status data out of the community's conversations. I think when you find
> a solution to that, you'll notice that email is not the problem.

And somehow binding threads of email together into bundles. That is
also something we can't do now. The only clean fix seems to require
manual work to attach them, and I doubt that is every going to happen
reliably.

And our email threads wander around quite a bit, with patches, ideas,
and bugs sometimes all thrown in --- see the interval
multiplication/division thread as a good example. How do you capture
that?

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian <bruce(at)momjian(dot)us> writes:
> Oh, so the bug is tracked by being part of the email reply list. That
> is a good idea. Now, how does that get assigned for non-bugs, like
> patches? Does any email sent to the lists that doesn't already have a
> bug number get one? That might be really valuable.

*Any* email? I hope not ... otherwise it's hard to see what you've got
that you don't get with a full-text search on a mailing list archive.

AFAICS the bottom line here is that we need some intelligent filtering.
In the short run I doubt that we can have that except through human
gruntwork to filter the mail traffic and update a tracker database.
Maybe after we see such a system in operation for awhile, we can start
to automate some obvious bits. But if we start with the assumption that
it's going to be mostly automated on day zero, I predict a resounding
failure.

It strikes me that the CERT CVE database might be a useful analogy.
AFAIK there is little or no automated entry into that database ---
every change has a human reviewer in front. Of course, they have some
darn good security reasons for wanting strong filters in front of their
database, but still it's a case worth thinking about. They have the
same problem of pulling status information from a lot of not-very-
well-standardized input sources.

regards, tom lane

Bruce Momjian wrote:
> Oh, so the bug is tracked by being part of the email reply list. That
> is a good idea. Now, how does that get assigned for non-bugs, like
> patches? Does any email sent to the lists that doesn't already have a
> bug number get one? That might be really valuable.
>
>

It would just clog the bug tracker and make it unusable.

I spent months on a working party on these and similar issues a few
years back, in a commercial setting. One of the big issues is when you
start tracking something. Bugs are a pretty simple case. Features are
much harder to handle. Someone comes up with an idea. There is a lot of
discussion. a consensus is arrived at to go forward. I think that's the
point at which we start tracking, but it's a judgement call. What is we
decide not to go ahead? Do we capture that in the tracker (with a
resolution of "rejected")?

One useful idea, not implemented by any of the OSS tracking / SCM
systems that I know of, is to tie the two together, i.e every commit
relates to a tracker item which captures the major discussions
surrounding it. (If there isn't a bug, it prompts you to start one,
which you can immediately mark as fixed if necessary). That way you can
go backwards and forwards between the code and the discussion that led
to it easily. Trust me, once you have worked with such a system you
appreciate the benefits.

Anyway, let me reiterate: bz has at least the start of an email reply
facility, and making that work if it doesn't already should not be
beyond us.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Anyway, let me reiterate: bz has at least the start of an email reply
> facility, and making that work if it doesn't already should not be
> beyond us.

I agree that starting from scratch seems like a good way to waste a
lot of time. I don't have a preference for BZ over debbugs or the
others that have been mentioned, but I'd suggest starting with something
that exists and trying to adapt it. If you end up throwing it away and
starting over, well, you'll have spent a lot less time on the trial
version than if you insist on coding from scratch.

regards, tom lane

Tom Lane wrote:
>
> AFAICS the bottom line here is that we need some intelligent filtering.
> In the short run I doubt that we can have that except through human
> gruntwork to filter the mail traffic and update a tracker database.
> Maybe after we see such a system in operation for awhile, we can start
> to automate some obvious bits. But if we start with the assumption that
> it's going to be mostly automated on day zero, I predict a resounding
> failure.
>
>
>

+1

cheers

andrew

Tom Lane wrote:
> Bruce Momjian <bruce(at)momjian(dot)us> writes:
> > Oh, so the bug is tracked by being part of the email reply list. That
> > is a good idea. Now, how does that get assigned for non-bugs, like
> > patches? Does any email sent to the lists that doesn't already have a
> > bug number get one? That might be really valuable.
>
> *Any* email? I hope not ... otherwise it's hard to see what you've got
> that you don't get with a full-text search on a mailing list archive.

Yes, I was thinking any email. +50% of features/bugs/patches don't come
in via the bug form. If we don't capture everything, will it be useful?

> AFAICS the bottom line here is that we need some intelligent filtering.
> In the short run I doubt that we can have that except through human
> gruntwork to filter the mail traffic and update a tracker database.
> Maybe after we see such a system in operation for awhile, we can start
> to automate some obvious bits. But if we start with the assumption that
> it's going to be mostly automated on day zero, I predict a resounding
> failure.
>
> It strikes me that the CERT CVE database might be a useful analogy.
> AFAIK there is little or no automated entry into that database ---
> every change has a human reviewer in front. Of course, they have some
> darn good security reasons for wanting strong filters in front of their
> database, but still it's a case worth thinking about. They have the
> same problem of pulling status information from a lot of not-very-
> well-standardized input sources.

Oh, lots of grunt work. I can see that working, but at a high cost.

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
>
> Oh, lots of grunt work. I can see that working, but at a high cost.
>
>

I doubt it. Let's just start with bugs, since that's the easy case
anyway. Our real volume is pretty low, so the cost of maintaining it
should not be high. I am assuming we would not be including HEAD, but
only stable branches. With HEAD the volume would be quite a bit higher,
but not impossibly so.

cheers

andrew

Andrew Dunstan wrote:
>
>
> Bruce Momjian wrote:
>>
>> Oh, lots of grunt work. I can see that working, but at a high cost.
>>
>>
>
> I doubt it. Let's just start with bugs, since that's the easy case
> anyway. Our real volume is pretty low, so the cost of maintaining it
> should not be high. I am assuming we would not be including HEAD, but
> only stable branches. With HEAD the volume would be quite a bit higher,
> but not impossibly so.

The email interface to bugzilla has potential. I *think* with a nominal
amount of work, we should be able to modify it, so that a person could
reply to the bug and it would correct associate as a bug.

Joshua D. Drake

>
> cheers
>
> andrew
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
> choose an index scan if your joining column's datatypes do not
> match
>

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Bruce Momjian wrote:
>> Oh, lots of grunt work. I can see that working, but at a high cost.

> I doubt it. Let's just start with bugs, since that's the easy case
> anyway. Our real volume is pretty low, so the cost of maintaining it
> should not be high. I am assuming we would not be including HEAD, but
> only stable branches. With HEAD the volume would be quite a bit higher,
> but not impossibly so.

Maybe we're working from different assumptions, but I thought the entire
basis for this discussion is that the project has grown to the point
where we have more resources than we used to --- and in particular,
we can find people who don't feel able to fix deep backend bugs, but are
ready and willing to track bug details and status with a goodly amount
of cluefulness. If that resource doesn't actually exist, then I fear
this whole discussion will come to naught. If it does exist, we should
call upon it.

This is not that far different from the premise upon which you built
the buildfarm: that there were people out there able to provide machine
resources and a certain amount of admin time. The resources this
project requires are not those exactly, but why shouldn't we expect
that some people will answer the call?

regards, tom lane

> This is not that far different from the premise upon which you built
> the buildfarm: that there were people out there able to provide machine
> resources and a certain amount of admin time. The resources this
> project requires are not those exactly, but why shouldn't we expect
> that some people will answer the call?

I concur, more then one person has complained on this thread and others
about wanting these tools. It is time that those people step up.

Sincerely,

Joshua D. Drake

>
> regards, tom lane
>

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Tom Lane wrote:

> AFAICS the bottom line here is that we need some intelligent filtering.
> In the short run I doubt that we can have that except through human
> gruntwork to filter the mail traffic and update a tracker database.
> Maybe after we see such a system in operation for awhile, we can start
> to automate some obvious bits. But if we start with the assumption that
> it's going to be mostly automated on day zero, I predict a resounding
> failure.

I agree 100%. Lets start off with grunt workers doing their magic in
parallel with whatever systems we currently have. They will one by one
figure out what to automate, what cannot be automated, and maybe provide
value that is promising enough for people to slightly modify their modus
operanti for those aspects that cannot be automated. However there will
probably always be a great deal of grunt work.

Again Email's are great for discussions and I think its great to link up
discussions with a bug or issue tracker id. However Email discussions
also often go in circles, are side tracked by IRC discussions etc. So
its really hard to figure out what decisions have been made if you look
things up later on. So the task of the grunt workers is to make sure
that there is a summary of the relevant information available, even if
all they do is flag the important decision emails.

regards,
Lukas

Andrew Dunstan wrote:

> I spent months on a working party on these and similar issues a few
> years back, in a commercial setting. One of the big issues is when you
> start tracking something. Bugs are a pretty simple case. Features are
> much harder to handle. Someone comes up with an idea. There is a lot of
> discussion. a consensus is arrived at to go forward. I think that's the
> point at which we start tracking, but it's a judgement call. What is we
> decide not to go ahead? Do we capture that in the tracker (with a
> resolution of "rejected")?

Exactly its a judgement call. The idea would be to try and pick up each
of the proposals in the discussion, summarize them in the issue tracker
or via a link to the wiki. This way people do not argue in circles all
that much (hopefully) and there is something to vote on. More
importantly there is something to point to if the topic ever comes up
again and the previous discussion did not lead to a decision. The
classic "read the archives" is just very suboptimal.

regards,
Lukas

Am Montag, 4. September 2006 04:19 schrieb Bruce Momjian:
> And our email threads wander around quite a bit, with patches, ideas,
> and bugs sometimes all thrown in --- see the interval
> multiplication/division thread as a good example. How do you capture
> that?

It's easy: Those who put in the care to capture all that have a better chance
of getting their issue resolved.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>
>> Bruce Momjian wrote:
>>
>>> Oh, lots of grunt work. I can see that working, but at a high cost.
>>>
>
>
>> I doubt it. Let's just start with bugs, since that's the easy case
>> anyway. Our real volume is pretty low, so the cost of maintaining it
>> should not be high. I am assuming we would not be including HEAD, but
>> only stable branches. With HEAD the volume would be quite a bit higher,
>> but not impossibly so.
>>
>
> Maybe we're working from different assumptions, but I thought the entire
> basis for this discussion is that the project has grown to the point
> where we have more resources than we used to --- and in particular,
> we can find people who don't feel able to fix deep backend bugs, but are
> ready and willing to track bug details and status with a goodly amount
> of cluefulness. If that resource doesn't actually exist, then I fear
> this whole discussion will come to naught. If it does exist, we should
> call upon it.
>
> This is not that far different from the premise upon which you built
> the buildfarm: that there were people out there able to provide machine
> resources and a certain amount of admin time. The resources this
> project requires are not those exactly, but why shouldn't we expect
> that some people will answer the call?
>
>
>

I am not saying there is no work involved. In fact, throughout this
thread I have agreed with you that a tracker that is not given regular
maintenance effort is doomed to fail. But I don't think the level of
effort required is undoable, nor that the cost is too high.

Unlike running buildfarm, this is not largely automatable.

Incidentally, the buildfarm has taken far more of my time and energy
than I originally expected. It has probably been worth it - I doubt I
could have delivered anything else as valuable in its place, but in that
sense the cost to me has been quite high. I hope that by spreading the
load a bit maintenance of a tracker will not be as burdensome to anybody.

cheers

andrew

On Sun, Sep 03, 2006 at 07:42:02PM -0400, Tom Lane wrote:
> The hard part of this problem is finding a convenient way to capture
> status data out of the community's conversations. I think when you find
> a solution to that, you'll notice that email is not the problem.

In private groups (like companies) that do this well, that sort of
"convenient way" turns out to be someone who is willing to do the
summarisation and post it. Perhaps what is needed is a small group
of people who would like to contribute, who can't contribute code,
but who can spend some time doing summaries, documents, and that sort
of thing? (Yes, I'll put my money where my mouth is.)

A

--
Andrew Sullivan | ajs(at)crankycanuck(dot)ca
Users never remark, "Wow, this software may be buggy and hard
to use, but at least there is a lot of code underneath."
--Damien Katz

Andrew Sullivan wrote:
> On Sun, Sep 03, 2006 at 07:42:02PM -0400, Tom Lane wrote:
>
>> The hard part of this problem is finding a convenient way to capture
>> status data out of the community's conversations. I think when you find
>> a solution to that, you'll notice that email is not the problem.
>>
>
> In private groups (like companies) that do this well, that sort of
> "convenient way" turns out to be someone who is willing to do the
> summarisation and post it. Perhaps what is needed is a small group
> of people who would like to contribute, who can't contribute code,
> but who can spend some time doing summaries, documents, and that sort
> of thing? (Yes, I'll put my money where my mouth is.)
>
>

Excellent! You are just the sort of person for this task, I think.

cheers

andrew