Re: Managing the community information stream

Let me give you my approach to tracking. It might help set the stage
for moving forward. My goal has always been to foster discussion and
pull as many TODO items and patches from the discussion as possible (and
others do that as well by saying "Please add to TODO" or applying
patches).

I see the process much more as pulling things from a stream of data,
rather than tracking every event. We already record everything in the
archive. The current discussion is how and who should summarize/track
that information.

Right now, the TODO list is a good summary, and URLs help to give
detail. I am not sure seeing all treads of a TODO item would help. In
a way, the summarization is more valuable than the details for most
people. Again, the question is what is the cost of summarizing the
stream at a more detailed level vs. its value.

Because I see us operating on a stream, it is unclear when to
pull an item from the stream and track it off-stream, such as in a bug
tracker database. I am also concerned that tracking itself not inhibit
the volume of the stream, particularly if discussion participants have
to do something more difficult than what they do now.

The idea of the patch number in the subject line works with that
streaming model because it merely marks streams so they can be grouped.
The defining event that marks the stream is a post to the patches list.
We already number posts to the bugs list, so in a way we could improve
tracking there and somehow link it to TODO items and patch submissions,
but because many TODO items are not the result of bug reports but come
out of general discussions, I am not sure tracking would work as well
there. And what about features? Do you start assigning numbers there,
and what is your trigger event? In my opinion, as you start trying to
place more structure on the stream, the stream itself starts to degrade
in its dynamism and ease of use. To me, that is the fundamental issue,
and risk.

I think a lot of this relates to the volume of work we do per
participant. I think we are probably near the top for open source
projects, and while more detailed tracking might help, it also might
hurt.

I am hoping the "stream" analogy might help people understand why we do
what we do, why we are so successful, and how we can improve what we
currently have.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> The idea of the patch number in the subject line works with that
> streaming model because it merely marks streams so they can be grouped.
> The defining event that marks the stream is a post to the patches list.
> We already number posts to the bugs list, so in a way we could improve
> tracking there and somehow link it to TODO items and patch submissions,
> but because many TODO items are not the result of bug reports but come
> out of general discussions, I am not sure tracking would work as well
> there. And what about features? Do you start assigning numbers there,
> and what is your trigger event? In my opinion, as you start trying to
> place more structure on the stream, the stream itself starts to degrade
> in its dynamism and ease of use. To me, that is the fundamental issue,
> and risk.

Bruce,

I cannot really add to that except to say that you neatly summarized
what I've completely failed to in my last few emails to Andrew. I agree
completely.

Regards, Dave.

Dave Page wrote:
> Bruce Momjian wrote:
>> The idea of the patch number in the subject line works with that
>> streaming model because it merely marks streams so they can be grouped.
>> The defining event that marks the stream is a post to the patches list.
>> We already number posts to the bugs list, so in a way we could improve
>> tracking there and somehow link it to TODO items and patch submissions,
>> but because many TODO items are not the result of bug reports but come
>> out of general discussions, I am not sure tracking would work as well
>> there. And what about features? Do you start assigning numbers there,
>> and what is your trigger event? In my opinion, as you start trying to
>> place more structure on the stream, the stream itself starts to degrade
>> in its dynamism and ease of use. To me, that is the fundamental issue,
>> and risk.
>
> Bruce,
>
> I cannot really add to that except to say that you neatly summarized
> what I've completely failed to in my last few emails to Andrew. I
> agree completely.
>
>

Frankly, this strikes me as painting lipstick on a pig.

Try searching the mailing list archives to find information. It's hard.
It sucks badly. So often you have to post a query on a mailing list,
which you have to join unless you want your query to sit in limbo for
days. If you think this is treating users nicely then you have a
different idea from me of what that means. Yes, what I'm proposing means
work, and no it can't be fully automated. That doesn't mean it's not
worth doing.

Case 1 (bug): Recently I had a problem with Gaim/Pidgin on my fc6 boxes.
I went to the bug site, clicked a few buttons and found that our own
Devrim Gunduz had reported the problem. Later, when I found out some
more information, I went back and added it to the bug. When the
RedHat/Fedora guys get around to fixing it they will know what the
problem is and what the solution is. They will have all the info
gathered in one spot.

Case 2 (feature): Several years ago I wanted to find out what had
happened about BZ support for Postgres. It was in their roadmap doc, so
I went and looked at the tracking item. Nothing seemed to be happening,
so I asked. Then I reviewed the patches (plural, note - another reason
why tracking patches rather than action items is not necessarily good)
that related to the item. I didn't like the direction they were going so
I did some work and proposed an alternative. That got picked up by Ed
Sobol and Max Kanat-Alexander (iirc) and the result is that today there
is full support for Postgres in BZ mainline. If someone wants to review
the history it is all there, with patches and comments all gathered neatly.

Oh, the answer to Bruce's question about when to create a feature item?
You could well do it at the time when today you create a TODO item.
However, we might even do better. For example, we might well add feature
requests that are denied. That would help people to see if something has
been proposed before.

I could go on but I'm actually trying to get some code written today :-)

cheers

andrew

On May 6, 2007, at 8:18 AM, Andrew Dunstan wrote:
> Oh, the answer to Bruce's question about when to create a feature
> item? You could well do it at the time when today you create a TODO
> item. However, we might even do better. For example, we might well
> add feature requests that are denied. That would help people to see
> if something has been proposed before.

The problem with our current TODO process is that whether an item
makes it onto the list is essentially determined by did the idea
catch a committer's attention, and did that committer happen to think
it was a good idea. That sets the bar pretty high for getting stuff
on the list (which you need for a simple list like TODO), but it also
means it's very subjective. (Of course 98% of the time that committer
is Bruce, but I don't think that matters here...)

The subjectivity is because we don't have an effective means to get
information about how PostgreSQL is used in the field. Sometimes you
can mine that information out of the archives, but that's a pretty
tedious process (and therefor one that's unlikely to happen). But
that's also not necessarily representative... many people will try
and find an answer to something on their own and not post anything to
the lists at all, even if the 'answer' they find isn't very optimal.

Instead, if all feature requests are tracked then users can vote on
what's most important to them. That provides immediate feedback to
the community on how important something is to the users. http://
lnk.nu/bugzilla.mozilla.org/edc.cgi is an example of that for Firefox.
--
Jim Nasby jim(at)nasby(dot)net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

On Mon, May 07, 2007 at 07:36:55AM -0500, Jim Nasby wrote:

> Instead, if all feature requests are tracked then users can vote on
> what's most important to them.

I am sympathetic to the issues you and Andrew are describing (I
understand Bruce's stream analogy, but I think Andrew is right that
from the user's point of view, it's not usable). But I am not
convinced that users voting on desired features will get us the
users' desired features. The features we get are mostly the features
that have attracted developers. The method by which that attraction
happens is interesting, but I don't think it's democratic.

A

--
Andrew Sullivan | ajs(at)crankycanuck(dot)ca
Information security isn't a technological problem. It's an economics
problem.
--Bruce Schneier

Andrew Sullivan wrote:
> On Mon, May 07, 2007 at 07:36:55AM -0500, Jim Nasby wrote:
>
>
>> Instead, if all feature requests are tracked then users can vote on
>> what's most important to them.
>>
>
> I am sympathetic to the issues you and Andrew are describing (I
> understand Bruce's stream analogy, but I think Andrew is right that
> from the user's point of view, it's not usable). But I am not
> convinced that users voting on desired features will get us the
> users' desired features. The features we get are mostly the features
> that have attracted developers. The method by which that attraction
> happens is interesting, but I don't think it's democratic.
>
>
>

Getting votes might provide a useful point of information, not a way of
making decisions, though.

I certainly don't regard it as a must-have feature.

cheers

andrew

On May 8, 2007, at 9:50 AM, Andrew Sullivan wrote:
> On Mon, May 07, 2007 at 07:36:55AM -0500, Jim Nasby wrote:
>> Instead, if all feature requests are tracked then users can vote on
>> what's most important to them.
>
> I am sympathetic to the issues you and Andrew are describing (I
> understand Bruce's stream analogy, but I think Andrew is right that
> from the user's point of view, it's not usable). But I am not
> convinced that users voting on desired features will get us the
> users' desired features. The features we get are mostly the features
> that have attracted developers. The method by which that attraction
> happens is interesting, but I don't think it's democratic.

It may... it may not. If a high-demand feature sits around long
enough it could well attract someone capable of working on it, but
who isn't a current contributor. Or it could attract a bounty.

I'm also not sure if PostgreSQL is quite the same as other OSS
projects. My impression is that we have quite a few developers who no
longer do much if any database development (ie: they're not serious
users); they continue to contribute because of other reasons. I
suspect developers like that are not unlikely to scratch an itch that
isn't their own.
--
Jim Nasby jim(at)nasby(dot)net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

Jim Nasby wrote:
> On May 8, 2007, at 9:50 AM, Andrew Sullivan wrote:
>> On Mon, May 07, 2007 at 07:36:55AM -0500, Jim Nasby wrote:
>>> Instead, if all feature requests are tracked then users can vote on
>>> what's most important to them.
>>
>> I am sympathetic to the issues you and Andrew are describing (I
>> understand Bruce's stream analogy, but I think Andrew is right that
>> from the user's point of view, it's not usable). But I am not
>> convinced that users voting on desired features will get us the
>> users' desired features. The features we get are mostly the features
>> that have attracted developers. The method by which that attraction
>> happens is interesting, but I don't think it's democratic.
>
> It may... it may not. If a high-demand feature sits around long enough
> it could well attract someone capable of working on it, but who isn't a
> current contributor. Or it could attract a bounty.

Also keep in mind that many of the developers are working for companies
that ensure that resources get allocated according to what users need
and not only by what developers are motivated to work on.

That being said, it seems obvious that so far PostgreSQL has been mainly
driven by what developers feel like implementing. I think this is also
what ensured the high level of standards compliance of PostgreSQL, since
features were tailored for experienced DBA types, rather than end users
that are less experienced in how to leverage these standards.

regards,
Lukas

Hi,

guess I missed hackers on my initial reply. So I am re-sending the reply
I send to Joshua based on the reply I send to him in regards to a
hackers@ posting.

Read below.

regards,
Lukas

Joshua D. Drake wrote:
>
>>
>> That being said, it seems obvious that so far PostgreSQL has been
>> mainly driven by what developers feel like implementing. I think this
>> is also what ensured the high level of standards compliance of
>> PostgreSQL, since features were tailored for experienced DBA types,
>> rather than end users that are less experienced in how to leverage
>> these standards.
>
> PostgreSQL has *never* been developed with the DBA in mind. Keep in mind
> that most of the postgresql developers have *zero* real world
> experience. Nor do they run postgresql themselves in real world
> production environments.

Well, certainly more with a DBA in mind than a middle tier developer?

regards,
Lukas

PS: Did you mean to only reply to me?

Jim,

> I am sympathetic to the issues you and Andrew are describing (I
> understand Bruce's stream analogy, but I think Andrew is right that
> from the user's point of view, it's not usable). But I am not
> convinced that users voting on desired features will get us the
> users' desired features. The features we get are mostly the features
> that have attracted developers. The method by which that attraction
> happens is interesting, but I don't think it's democratic.

Further, our community has always operated by consensus and public
mailing list poll when applicable, and not by "majority rules" vote or
anything similar. The only advantage I can see to allowing "voting" on
TODOs would be to quickly answer the question "does anyone t all care
about this", but I personally am not convinced that offering
Bugzilla-style voting would help that at all. On other projects, my
experience is that people don't use the BZ voting, even projects which
otherwise use BZ extensively.

--Josh Berkus

Jim Nasby wrote:
> On May 6, 2007, at 8:18 AM, Andrew Dunstan wrote:
> > Oh, the answer to Bruce's question about when to create a feature
> > item? You could well do it at the time when today you create a TODO
> > item. However, we might even do better. For example, we might well
> > add feature requests that are denied. That would help people to see
> > if something has been proposed before.
>
> The problem with our current TODO process is that whether an item
> makes it onto the list is essentially determined by did the idea
> catch a committer's attention, and did that committer happen to think
> it was a good idea. That sets the bar pretty high for getting stuff
> on the list (which you need for a simple list like TODO), but it also
> means it's very subjective. (Of course 98% of the time that committer
> is Bruce, but I don't think that matters here...)

Users often request items be added to the TODO list, and I usually comply.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Jim Nasby wrote:
> On May 6, 2007, at 8:18 AM, Andrew Dunstan wrote:
> > Oh, the answer to Bruce's question about when to create a feature
> > item? You could well do it at the time when today you create a TODO
> > item. However, we might even do better. For example, we might well
> > add feature requests that are denied. That would help people to see
> > if something has been proposed before.

Uh, TODO has that:

Features We Do _Not_ Want

Do we need more items on that list?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

To follow up on Andrew's idea of tracking things back to the TODO or bug
number:

We could have a universal developer number, something like PGD#23432 as
a PostgreSQL Developer number. We could assign them for submissions to
the bugs list, where we already assign a number. I could easily add
them to TODO items that already don't have a number from the bugs list,
and we could use the number for postings to the patches list that again
don't already have a number. (The PGD numbers would have value ranges
assigned for specific uses, like 0-100000 are bugs, 100001-200000 are
assigned as TODO items, +300000 are patches, etc.)

The idea is that if you are working on a TODO item you mention that
number in the email subject discussing it, and for postings to the
patches list. A web application could then read from the email stream
and pull out information about any item. The only overhead is people
mentioning the assigned number consistently.

One problem is that our development isn't linear --- often TODO items
are the result of several email threads, and TODO items are split and
merged regularly, meaning that a PGD number could be partially complete
or be merged with another number. When this happens, the number might
cause confusion, and I don't see a way to fix that easily.

---------------------------------------------------------------------------

Dave Page wrote:
> Bruce Momjian wrote:
> > The idea of the patch number in the subject line works with that
> > streaming model because it merely marks streams so they can be grouped.
> > The defining event that marks the stream is a post to the patches list.
> > We already number posts to the bugs list, so in a way we could improve
> > tracking there and somehow link it to TODO items and patch submissions,
> > but because many TODO items are not the result of bug reports but come
> > out of general discussions, I am not sure tracking would work as well
> > there. And what about features? Do you start assigning numbers there,
> > and what is your trigger event? In my opinion, as you start trying to
> > place more structure on the stream, the stream itself starts to degrade
> > in its dynamism and ease of use. To me, that is the fundamental issue,
> > and risk.
>
> Bruce,
>
> I cannot really add to that except to say that you neatly summarized
> what I've completely failed to in my last few emails to Andrew. I agree
> completely.
>
> Regards, Dave.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +