Re: Managing the community information stream

To follow up on this, if you look at how TODO items are created, they
often come out of discussion threads, and sometimes more than one idea
comes from a discussion thread. If we moved to a trackers system, how
would we handle that?

Also, if I want to discuss renaming something or cleaning up some code,
do we create a tracker item for that or do we have a developer email
list to discuss such issues? And if we have a developer email list, how
do we make sure everything that happens there gets into the tracker if
needed?

Basically, right now, the steam ignores non-TODO items that are
discussed, while with a trackers, I am afraid you have to explicitly
mark every discussion thread as uninteresting/closed, and I am worried
about the manpower and participant overhead of doing that.

---------------------------------------------------------------------------

bruce wrote:
> Let me give you my approach to tracking. It might help set the stage
> for moving forward. My goal has always been to foster discussion and
> pull as many TODO items and patches from the discussion as possible (and
> others do that as well by saying "Please add to TODO" or applying
> patches).
>
> I see the process much more as pulling things from a stream of data,
> rather than tracking every event. We already record everything in the
> archive. The current discussion is how and who should summarize/track
> that information.
>
> Right now, the TODO list is a good summary, and URLs help to give
> detail. I am not sure seeing all treads of a TODO item would help. In
> a way, the summarization is more valuable than the details for most
> people. Again, the question is what is the cost of summarizing the
> stream at a more detailed level vs. its value.
>
> Because I see us operating on a stream, it is unclear when to
> pull an item from the stream and track it off-stream, such as in a bug
> tracker database. I am also concerned that tracking itself not inhibit
> the volume of the stream, particularly if discussion participants have
> to do something more difficult than what they do now.
>
> The idea of the patch number in the subject line works with that
> streaming model because it merely marks streams so they can be grouped.
> The defining event that marks the stream is a post to the patches list.
> We already number posts to the bugs list, so in a way we could improve
> tracking there and somehow link it to TODO items and patch submissions,
> but because many TODO items are not the result of bug reports but come
> out of general discussions, I am not sure tracking would work as well
> there. And what about features? Do you start assigning numbers there,
> and what is your trigger event? In my opinion, as you start trying to
> place more structure on the stream, the stream itself starts to degrade
> in its dynamism and ease of use. To me, that is the fundamental issue,
> and risk.
>
> I think a lot of this relates to the volume of work we do per
> participant. I think we are probably near the top for open source
> projects, and while more detailed tracking might help, it also might
> hurt.
>
> I am hoping the "stream" analogy might help people understand why we do
> what we do, why we are so successful, and how we can improve what we
> currently have.
>
> --
> Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
> EnterpriseDB http://www.enterprisedb.com
>
> + If your life is a hard drive, Christ can be your backup. +

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> To follow up on this, if you look at how TODO items are created, they
> often come out of discussion threads, and sometimes more than one idea
> comes from a discussion thread. If we moved to a trackers system, how
> would we handle that?
>

We have the discussion on list, if it warrants a todo, we create a todo.

> Also, if I want to discuss renaming something or cleaning up some code,
> do we create a tracker item for that or do we have a developer email
> list to discuss such issues?

In the most conformist sense yes, but I can tell you that generally
isn't how CMD does it. How we general do it, is to create a ticket basic
on a topic, that ticket cc's a mailing list and discussion happens in
reply to that cc. So the workflow doesn't actually change. Once
everything is decided we may update the ticket with the final solution,
and then when the work is done we close the ticket.

However, we do it the way we do, because we don't have email
integration. Supposedly (which a small group is currently reviewing) BZ
3.0 does have email integration so this may change a bit.

> And if we have a developer email list, how
> do we make sure everything that happens there gets into the tracker if
> needed?

See above.

>
> Basically, right now, the steam ignores non-TODO items that are
> discussed, while with a trackers, I am afraid you have to explicitly
> mark every discussion thread as uninteresting/closed, and I am worried
> about the manpower and participant overhead of doing that.

Oh good lord, yeah I wouldn't want to do that either. Email is obviously
going to be the predominant medium of communication. I think what would
end up happening, if we were able to tightly integrate with email and bz
would that at some point all discussions die off, it would be up to the
person that opened the discussion or an bz admin to close or change the
status of the ticket.

The nice thing is if someone comes back to the thread at any point
(which happens all the time) the ticket should automatically re-open.

Joshua D. Drake

>
> ---------------------------------------------------------------------------
>
> bruce wrote:
>> Let me give you my approach to tracking. It might help set the stage
>> for moving forward. My goal has always been to foster discussion and
>> pull as many TODO items and patches from the discussion as possible (and
>> others do that as well by saying "Please add to TODO" or applying
>> patches).
>>
>> I see the process much more as pulling things from a stream of data,
>> rather than tracking every event. We already record everything in the
>> archive. The current discussion is how and who should summarize/track
>> that information.
>>
>> Right now, the TODO list is a good summary, and URLs help to give
>> detail. I am not sure seeing all treads of a TODO item would help. In
>> a way, the summarization is more valuable than the details for most
>> people. Again, the question is what is the cost of summarizing the
>> stream at a more detailed level vs. its value.
>>
>> Because I see us operating on a stream, it is unclear when to
>> pull an item from the stream and track it off-stream, such as in a bug
>> tracker database. I am also concerned that tracking itself not inhibit
>> the volume of the stream, particularly if discussion participants have
>> to do something more difficult than what they do now.
>>
>> The idea of the patch number in the subject line works with that
>> streaming model because it merely marks streams so they can be grouped.
>> The defining event that marks the stream is a post to the patches list.
>> We already number posts to the bugs list, so in a way we could improve
>> tracking there and somehow link it to TODO items and patch submissions,
>> but because many TODO items are not the result of bug reports but come
>> out of general discussions, I am not sure tracking would work as well
>> there. And what about features? Do you start assigning numbers there,
>> and what is your trigger event? In my opinion, as you start trying to
>> place more structure on the stream, the stream itself starts to degrade
>> in its dynamism and ease of use. To me, that is the fundamental issue,
>> and risk.
>>
>> I think a lot of this relates to the volume of work we do per
>> participant. I think we are probably near the top for open source
>> projects, and while more detailed tracking might help, it also might
>> hurt.
>>
>> I am hoping the "stream" analogy might help people understand why we do
>> what we do, why we are so successful, and how we can improve what we
>> currently have.
>>
>> --
>> Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
>> EnterpriseDB http://www.enterprisedb.com
>>
>> + If your life is a hard drive, Christ can be your backup. +
>

Joshua D. Drake wrote:

> >Also, if I want to discuss renaming something or cleaning up some code,
> >do we create a tracker item for that or do we have a developer email
> >list to discuss such issues?
>
> In the most conformist sense yes, but I can tell you that generally
> isn't how CMD does it. How we general do it, is to create a ticket basic
> on a topic, that ticket cc's a mailing list and discussion happens in
> reply to that cc. So the workflow doesn't actually change. Once
> everything is decided we may update the ticket with the final solution,
> and then when the work is done we close the ticket.
>
> However, we do it the way we do, because we don't have email
> integration. Supposedly (which a small group is currently reviewing) BZ
> 3.0 does have email integration so this may change a bit.

Well, with email integration (as I am envisioning -- I don't know what
BZ actually implements) it is even better, because you just create a
ticket, and that sends an email to the list. Other people can respond
to that email, which gets saved into the bug without need for further
action.

In Debian's bug tracking system, when the bug is created (which is done
by sending an email to a certain address) it gets a number, and the
email is distributed to certain lists. People can then reply to that
mail, and send messages to 12345(at)bugs(dot)debian(dot)org and it gets tracked in
the bug, and you can see all those messages in the bug report. I
ass-ume that BZ 3.0 does something similar.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro Herrera wrote:
> Joshua D. Drake wrote:
>
> > >Also, if I want to discuss renaming something or cleaning up some code,
> > >do we create a tracker item for that or do we have a developer email
> > >list to discuss such issues?
> >
> > In the most conformist sense yes, but I can tell you that generally
> > isn't how CMD does it. How we general do it, is to create a ticket basic
> > on a topic, that ticket cc's a mailing list and discussion happens in
> > reply to that cc. So the workflow doesn't actually change. Once
> > everything is decided we may update the ticket with the final solution,
> > and then when the work is done we close the ticket.
> >
> > However, we do it the way we do, because we don't have email
> > integration. Supposedly (which a small group is currently reviewing) BZ
> > 3.0 does have email integration so this may change a bit.
>
> Well, with email integration (as I am envisioning -- I don't know what
> BZ actually implements) it is even better, because you just create a
> ticket, and that sends an email to the list. Other people can respond
> to that email, which gets saved into the bug without need for further
> action.
>
> In Debian's bug tracking system, when the bug is created (which is done
> by sending an email to a certain address) it gets a number, and the
> email is distributed to certain lists. People can then reply to that
> mail, and send messages to 12345(at)bugs(dot)debian(dot)org and it gets tracked in
> the bug, and you can see all those messages in the bug report. I
> ass-ume that BZ 3.0 does something similar.

But often a TODO item has multiple threads containing details (often
months apart), and it isn't obvious at the time the thread is started
that this will happen. Note the number of TODO items that now have
multiple URLs. How is that handled?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> Alvaro Herrera wrote:

> > In Debian's bug tracking system, when the bug is created (which is done
> > by sending an email to a certain address) it gets a number, and the
> > email is distributed to certain lists. People can then reply to that
> > mail, and send messages to 12345(at)bugs(dot)debian(dot)org and it gets tracked in
> > the bug, and you can see all those messages in the bug report. I
> > ass-ume that BZ 3.0 does something similar.
>
> But often a TODO item has multiple threads containing details (often
> months apart), and it isn't obvious at the time the thread is started
> that this will happen. Note the number of TODO items that now have
> multiple URLs. How is that handled?

Just add the bug address to CC and reply to it, just like when you reply
to say "added to TODO", only that you don't need to manually go and
modify the TODO file by hand. The bug tracking system puts that mail
into the bug report. Subsequent followups keep the bug address in CC
and thus the whole discussion is saved in the bug report.

This is even better than our archives due to the problem that the
archives don't have links to messages crossing month boundaries. Have
you noticed that if you go to the archives, some discussions appear
truncated at a point, but you can go to the archive for the next month
and it continues there? I find that artifact somewhat annoying. The
bug report would continue receiving the CC'ed mails, so it would record
them all in a single place.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Bruce Momjian wrote:

>> In Debian's bug tracking system, when the bug is created (which is done
>> by sending an email to a certain address) it gets a number, and the
>> email is distributed to certain lists. People can then reply to that
>> mail, and send messages to 12345(at)bugs(dot)debian(dot)org and it gets tracked in
>> the bug, and you can see all those messages in the bug report. I
>> ass-ume that BZ 3.0 does something similar.
>
> But often a TODO item has multiple threads containing details (often
> months apart), and it isn't obvious at the time the thread is started
> that this will happen. Note the number of TODO items that now have
> multiple URLs. How is that handled?

Well you can certainly merge tickets, but one of the ideas would be to
help stop that :)...

Hey what about foo... oh we discussed that *here*...

Joshua D. Drake

>

Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > Alvaro Herrera wrote:
>
> > > In Debian's bug tracking system, when the bug is created (which is done
> > > by sending an email to a certain address) it gets a number, and the
> > > email is distributed to certain lists. People can then reply to that
> > > mail, and send messages to 12345(at)bugs(dot)debian(dot)org and it gets tracked in
> > > the bug, and you can see all those messages in the bug report. I
> > > ass-ume that BZ 3.0 does something similar.
> >
> > But often a TODO item has multiple threads containing details (often
> > months apart), and it isn't obvious at the time the thread is started
> > that this will happen. Note the number of TODO items that now have
> > multiple URLs. How is that handled?
>
> Just add the bug address to CC and reply to it, just like when you reply
> to say "added to TODO", only that you don't need to manually go and
> modify the TODO file by hand. The bug tracking system puts that mail
> into the bug report. Subsequent followups keep the bug address in CC
> and thus the whole discussion is saved in the bug report.

Right, but you are adding the bug addresss at the end of the email
thread. How do you point to the email you want to reference?

> This is even better than our archives due to the problem that the
> archives don't have links to messages crossing month boundaries. Have
> you noticed that if you go to the archives, some discussions appear
> truncated at a point, but you can go to the archive for the next month
> and it continues there? I find that artifact somewhat annoying. The
> bug report would continue receiving the CC'ed mails, so it would record
> them all in a single place.

Not crossing month boundaries is super-annoying.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Joshua D. Drake wrote:
> Bruce Momjian wrote:
>
> >> In Debian's bug tracking system, when the bug is created (which is done
> >> by sending an email to a certain address) it gets a number, and the
> >> email is distributed to certain lists. People can then reply to that
> >> mail, and send messages to 12345(at)bugs(dot)debian(dot)org and it gets tracked in
> >> the bug, and you can see all those messages in the bug report. I
> >> ass-ume that BZ 3.0 does something similar.
> >
> > But often a TODO item has multiple threads containing details (often
> > months apart), and it isn't obvious at the time the thread is started
> > that this will happen. Note the number of TODO items that now have
> > multiple URLs. How is that handled?
>
> Well you can certainly merge tickets, but one of the ideas would be to
> help stop that :)...
>
> Hey what about foo... oh we discussed that *here*...

Our thought process is not linear --- often an item changes as our
surrounding code changes too. The multiple URLs are not because no one
knows about the previous discussion.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Tue, May 15, 2007 at 01:18:42PM -0400, Bruce Momjian wrote:
> > In Debian's bug tracking system, when the bug is created (which is done
> > by sending an email to a certain address) it gets a number, and the
> > email is distributed to certain lists. People can then reply to that
> > mail, and send messages to 12345(at)bugs(dot)debian(dot)org and it gets tracked in
> > the bug, and you can see all those messages in the bug report. I
> > ass-ume that BZ 3.0 does something similar.
>
> But often a TODO item has multiple threads containing details (often
> months apart), and it isn't obvious at the time the thread is started
> that this will happen. Note the number of TODO items that now have
> multiple URLs. How is that handled?

Worst-case, for those cases we add URLs to the tracker manually like you
do now. The big advantage is that most of the time that's not needed.
And in cases where it's not automatic we can grant any number of people
permission to add that information to the tracker, because that
permission wouldn't be tied to CVS commit privs.
--
Jim Nasby decibel(at)decibel(dot)org
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

Bruce Momjian wrote:
> Alvaro Herrera wrote:
> > Bruce Momjian wrote:
> > > Alvaro Herrera wrote:
> >
> > > > In Debian's bug tracking system, when the bug is created (which is done
> > > > by sending an email to a certain address) it gets a number, and the
> > > > email is distributed to certain lists. People can then reply to that
> > > > mail, and send messages to 12345(at)bugs(dot)debian(dot)org and it gets tracked in
> > > > the bug, and you can see all those messages in the bug report. I
> > > > ass-ume that BZ 3.0 does something similar.
> > >
> > > But often a TODO item has multiple threads containing details (often
> > > months apart), and it isn't obvious at the time the thread is started
> > > that this will happen. Note the number of TODO items that now have
> > > multiple URLs. How is that handled?
> >
> > Just add the bug address to CC and reply to it, just like when you reply
> > to say "added to TODO", only that you don't need to manually go and
> > modify the TODO file by hand. The bug tracking system puts that mail
> > into the bug report. Subsequent followups keep the bug address in CC
> > and thus the whole discussion is saved in the bug report.
>
> Right, but you are adding the bug addresss at the end of the email
> thread. How do you point to the email you want to reference?

I am not sure. We will have to investigate more the capabilities of the
bug tracking system we intend to use. In the worst case one could add
the URL for the archived message copy; second worst would be bouncing
(hopefully not forward) the interesting messages to the bug address.

If we had our own method for fetching a message by Message-Id, we could
add Message-Ids to bugs reports. In the meantime we could use Gmane's.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > Alvaro Herrera wrote:
> > > Bruce Momjian wrote:
> > > > Alvaro Herrera wrote:
> > >
> > > > > In Debian's bug tracking system, when the bug is created (which is done
> > > > > by sending an email to a certain address) it gets a number, and the
> > > > > email is distributed to certain lists. People can then reply to that
> > > > > mail, and send messages to 12345(at)bugs(dot)debian(dot)org and it gets tracked in
> > > > > the bug, and you can see all those messages in the bug report. I
> > > > > ass-ume that BZ 3.0 does something similar.
> > > >
> > > > But often a TODO item has multiple threads containing details (often
> > > > months apart), and it isn't obvious at the time the thread is started
> > > > that this will happen. Note the number of TODO items that now have
> > > > multiple URLs. How is that handled?
> > >
> > > Just add the bug address to CC and reply to it, just like when you reply
> > > to say "added to TODO", only that you don't need to manually go and
> > > modify the TODO file by hand. The bug tracking system puts that mail
> > > into the bug report. Subsequent followups keep the bug address in CC
> > > and thus the whole discussion is saved in the bug report.
> >
> > Right, but you are adding the bug addresss at the end of the email
> > thread. How do you point to the email you want to reference?
>
> I am not sure. We will have to investigate more the capabilities of the
> bug tracking system we intend to use. In the worst case one could add
> the URL for the archived message copy; second worst would be bouncing
> (hopefully not forward) the interesting messages to the bug address.

Sounds like what I do with the TODO list now.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> Alvaro Herrera wrote:
> > Bruce Momjian wrote:
> > > Alvaro Herrera wrote:
> > > > Bruce Momjian wrote:
> > > > > Alvaro Herrera wrote:
> > > >
> > > > > > In Debian's bug tracking system, when the bug is created (which is done
> > > > > > by sending an email to a certain address) it gets a number, and the
> > > > > > email is distributed to certain lists. People can then reply to that
> > > > > > mail, and send messages to 12345(at)bugs(dot)debian(dot)org and it gets tracked in
> > > > > > the bug, and you can see all those messages in the bug report. I
> > > > > > ass-ume that BZ 3.0 does something similar.
> > > > >
> > > > > But often a TODO item has multiple threads containing details (often
> > > > > months apart), and it isn't obvious at the time the thread is started
> > > > > that this will happen. Note the number of TODO items that now have
> > > > > multiple URLs. How is that handled?
> > > >
> > > > Just add the bug address to CC and reply to it, just like when you reply
> > > > to say "added to TODO", only that you don't need to manually go and
> > > > modify the TODO file by hand. The bug tracking system puts that mail
> > > > into the bug report. Subsequent followups keep the bug address in CC
> > > > and thus the whole discussion is saved in the bug report.
> > >
> > > Right, but you are adding the bug addresss at the end of the email
> > > thread. How do you point to the email you want to reference?
> >
> > I am not sure. We will have to investigate more the capabilities of the
> > bug tracking system we intend to use. In the worst case one could add
> > the URL for the archived message copy; second worst would be bouncing
> > (hopefully not forward) the interesting messages to the bug address.
>
> Sounds like what I do with the TODO list now.

Except that this is the *worst case* with the bug tracker, whereas for
the TODO list it is not only the worst case, it is also the best case
and the only case at all.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera wrote:
> > > I am not sure. We will have to investigate more the capabilities of the
> > > bug tracking system we intend to use. In the worst case one could add
> > > the URL for the archived message copy; second worst would be bouncing
> > > (hopefully not forward) the interesting messages to the bug address.
> >
> > Sounds like what I do with the TODO list now.
>
> Except that this is the *worst case* with the bug tracker, whereas for
> the TODO list it is not only the worst case, it is also the best case
> and the only case at all.

And it requires no additional work to ignore threads.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Wed, May 16, 2007 at 10:46:50AM -0400, Alvaro Herrera wrote:
> > > I am not sure. We will have to investigate more the capabilities of the
> > > bug tracking system we intend to use. In the worst case one could add
> > > the URL for the archived message copy; second worst would be bouncing
> > > (hopefully not forward) the interesting messages to the bug address.
> >
> > Sounds like what I do with the TODO list now.
>
> Except that this is the *worst case* with the bug tracker, whereas for
> the TODO list it is not only the worst case, it is also the best case
> and the only case at all.

And any number of people can manage it (just like the wiki).
--
Jim Nasby decibel(at)decibel(dot)org
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

Bruce Momjian <bruce(at)momjian(dot)us> writes:
> Alvaro Herrera wrote:
>> This is even better than our archives due to the problem that the
>> archives don't have links to messages crossing month boundaries. Have
>> you noticed that if you go to the archives, some discussions appear
>> truncated at a point, but you can go to the archive for the next month
>> and it continues there? I find that artifact somewhat annoying. The
>> bug report would continue receiving the CC'ed mails, so it would record
>> them all in a single place.

> Not crossing month boundaries is super-annoying.

Indeed, but that should be fixed. I can't imagine that one
presumably-fixable deficiency is grounds for changing our entire
discussion infrastructure. Or do you think we will find something
else that has no deficiencies of its own?

regards, tom lane

Bruce Momjian <bruce(at)momjian(dot)us> writes:
> Alvaro Herrera wrote:
>> This is even better than our archives due to the problem that the
>> archives don't have links to messages crossing month boundaries. Have
>> you noticed that if you go to the archives, some discussions appear
>> truncated at a point, but you can go to the archive for the next month
>> and it continues there? I find that artifact somewhat annoying. The
>> bug report would continue receiving the CC'ed mails, so it would record
>> them all in a single place.

> Not crossing month boundaries is super-annoying.

Indeed, but that should be fixed. I can't imagine that one
presumably-fixable deficiency is grounds for changing our entire
discussion infrastructure. Or do you think we will find something
else that has no deficiencies of its own?

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <bruce(at)momjian(dot)us> writes:
>> Alvaro Herrera wrote:
>>> This is even better than our archives due to the problem that the
>>> archives don't have links to messages crossing month boundaries. Have
>>> you noticed that if you go to the archives, some discussions appear
>>> truncated at a point, but you can go to the archive for the next month
>>> and it continues there? I find that artifact somewhat annoying. The
>>> bug report would continue receiving the CC'ed mails, so it would record
>>> them all in a single place.
>
>> Not crossing month boundaries is super-annoying.
>
> Indeed, but that should be fixed. I can't imagine that one
> presumably-fixable deficiency is grounds for changing our entire
> discussion infrastructure. Or do you think we will find something
> else that has no deficiencies of its own?

Very much agreed, however, changing how it's done might open up ways to
change other things for the better - things we can't do now. But getting
rid of that annoying thing alone does not change anything else, or
require changing of anything else.

//Magnus