Re: Patch committers

bruce wrote:
> This brings up a concern I have --- that the number of patch
> committers/managers is decreasing while our patch volume is increasing.
> Consider that Heikki is working mostly on Hot Standby and Streaming
> Replication, Alvaro isn't as involved in applying patches, Neil Conway
> isn't involved with Postgres anymore, I am in a 42-day travel period,
> and Robert Haas is feeling burnt-out --- that is not a pretty sight.
>
> Much of the patch burden is falling on Tom. Now, Tom isn't complaining,
> but I am concerned about placing too much of a burden on him. I know we
> are growing new patch reviewers who will eventually be able to review
> _and_ apply patches on their own, but getting them to that point is
> going to take time.
>
> I have no real answers, but I am concerned. I have talked to many of
> you privately about this to get your input.

There is a more worrisome/sinister possibility that I didn't want to
mention in my first email --- that companies are hiring our most
experienced developers and having them work almost exclusively on
company-related or closed-source projects.

Unfortunately I can think of at least half-a-dozen cases of this
happening. Now, this was expected, but the hope was that this kind of
skill siphoning would be offset by additional people being paid to be
involved in Postgres development, and that clearly is happening. What I
am worried about is that this is not happening as much among our most
experienced people --- that in fact they are the most likely to be hired
and perhaps placed into roles where they are less involved in the
community than they were before. (Of course, there are counter-examples
where experience developers are hired to work on community Postgres
full-time.)

There is not much we as a community can do to prevent skill siphoning,
except perhaps publicly complaining about companies that do this.

If this is indeed a pattern, it has serious long-term consequences
because it means we will always have an unnaturally small pool of very
skilled people.

Up to this point we have been able to maintain a happy group of
developers. If things become unbalanced and people are regularly
required to do things they don't like doing, it will lead to Postgres no
longer being fun for them, which leads to burn-out and them leaving the
project. That happens frequently in other open source projects, but it
has been a very rare occurance for us, and I hope it stays that way.
Our ability to retain people for many years has benefitted us in
countless ways.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On ons, 2009-11-11 at 10:25 -0500, Bruce Momjian wrote:
> There is a more worrisome/sinister possibility that I didn't want to
> mention in my first email --- that companies are hiring our most
> experienced developers and having them work almost exclusively on
> company-related or closed-source projects.

I can't claim to know everyone's employment terms, but I think it's a
bit of an illusion to think that the above hasn't always happened.
Except for a handful of people who have very special job arrangements,
*everyone* is working almost exclusively on company-related or
closed-source projects. The patches that get sent in are almost always
either fallout from a customer/company project, or stuff that one of the
closed-sourced forks has developed that they don't want to maintain, or
stuff people do "at night" to make their lives easier in the distant
future. All of those things are already special arrangements that
people need to make with their employers and their lives, but they have
discernible benefits. But you can't expect a lot of people or employers
to reserve time on top of that for handholding other people's patches
and for other "community" stuff that has no easy to measure benefits.

Peter Eisentraut wrote:
> On ons, 2009-11-11 at 10:25 -0500, Bruce Momjian wrote:
> > There is a more worrisome/sinister possibility that I didn't want to
> > mention in my first email --- that companies are hiring our most
> > experienced developers and having them work almost exclusively on
> > company-related or closed-source projects.
>
> I can't claim to know everyone's employment terms, but I think it's a
> bit of an illusion to think that the above hasn't always happened.
> Except for a handful of people who have very special job arrangements,

Yes, it has always happened. I think the big difference is that those
"special employments" are currently a fixed size, and the volume of
patches has increased as our user-base has increased.

I also think the bad economy is making it harder for people/companies to
devote time to community stuff when paid work is available.

I always had an assumption that those special employments would grow as
the community grew, but I am not sure I am seeing that. Combine that
with skill siphoning, and I get concerned.

> *everyone* is working almost exclusively on company-related or
> closed-source projects. The patches that get sent in are almost always
> either fallout from a customer/company project, or stuff that one of the
> closed-sourced forks has developed that they don't want to maintain, or
> stuff people do "at night" to make their lives easier in the distant
> future. All of those things are already special arrangements that
> people need to make with their employers and their lives, but they have
> discernible benefits. But you can't expect a lot of people or employers
> to reserve time on top of that for handholding other people's patches
> and for other "community" stuff that has no easy to measure benefits.

Totally agree. It is that zero-return work that is hard to justify for
people and companies. It is clearly something that requires
self-sacrifice, and personally I think a culture of self-sacrifice is
what has given us such great success and such a nuturing community
environment.

Historically, there was a "golden era" when Tom, Heikki, Neil Conway,
Alvaro, and others all handled patches and things ran with much less of
an individual burden than we have now, but that might have been an
aberration.

Personally, I don't think we have a major problem now, but I do think
there is the chance of this getting worse in the future, and making
people unhappy. I am brining it up now in case there is something we
can do to avoid such unhappiness.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce,

Well, I think the answer is to promote some new committers. When was
the last time we added a committer?

We have added a bunch of new reviewers and major contributors over the
last 2 years. It's time to review their work and see who can be
promoted to more responsibility for other people's patches.

--Josh Berkus

Josh Berkus wrote:
> Bruce,
>
> Well, I think the answer is to promote some new committers. When was
> the last time we added a committer?
>
> We have added a bunch of new reviewers and major contributors over the
> last 2 years. It's time to review their work and see who can be
> promoted to more responsibility for other people's patches.

Yes, I did consider that, and it might be a solution. The larger
problem is that with many of the patches, it isn't commit-rights that is
preventing people from being involved, but rather the expert knowledge
necessary to make sure the patch works properly.

For example, probably only Simon and Heikki are knowledge enough to
apply the HS and SR patchs --- but those patch is handled. The harder
part is the other patches where there are only a small pool of people
knowledgeable enough to apply the patch, but many of those knowledgeable
people are tied up with non-community things.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

> For example, probably only Simon and Heikki are knowledge enough to
> apply the HS and SR patchs --- but those patch is handled. The harder
> part is the other patches where there are only a small pool of people
> knowledgeable enough to apply the patch, but many of those knowledgeable
> people are tied up with non-community things.

People learn to do this by doing it. That's how you learned, and Tom
and Heikki. Ultimately, it's a source code control system; if someone
screws up, we can always back it out.

--Josh Berkus

Josh Berkus wrote:
>
> > For example, probably only Simon and Heikki are knowledge enough to
> > apply the HS and SR patchs --- but those patch is handled. The harder
> > part is the other patches where there are only a small pool of people
> > knowledgeable enough to apply the patch, but many of those knowledgeable
> > people are tied up with non-community things.
>
> People learn to do this by doing it. That's how you learned, and Tom
> and Heikki. Ultimately, it's a source code control system; if someone
> screws up, we can always back it out.

True, but even I avoid patches I don't understand, and "practicing" by
applying them could lead to a very undesirable outcome, e.g.
instability.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

> True, but even I avoid patches I don't understand, and "practicing" by
> applying them could lead to a very undesirable outcome, e.g.
> instability.

Right, but being responsible for applying the patch is the only real
incentive anyone has to learn enough to understand its effects. If a
contributor is not responsible, they can always think "oh, Tom will get
it, I'll learn that later."

--Josh Berkus

Couldn't you have a policy that every patch is reviewed first by someone who wants to be an expert in that area, and then by someone who is currently an expert. Then you have the best of both worlds. The patch is reviewed by someone will make sure it won't cause problems, and the want to be expert gets training and will eventually be able to be the expert.

On Nov 11, 2009, at 12:58 PM, Josh Berkus wrote:

>
>> True, but even I avoid patches I don't understand, and "practicing" by
>> applying them could lead to a very undesirable outcome, e.g.
>> instability.
>
> Right, but being responsible for applying the patch is the only real
> incentive anyone has to learn enough to understand its effects. If a
> contributor is not responsible, they can always think "oh, Tom will get
> it, I'll learn that later."
>
> --Josh Berkus
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

Rick Gigger wrote:
> Couldn't you have a policy that every patch is reviewed first by someone who wants to be an expert in that area, and then by someone who is currently an expert. Then you have the best of both worlds. The patch is reviewed by someone will make sure it won't cause problems, and the want to be expert gets training and will eventually be able to be the expert.

The wannabe-experts can do all the reviewing they want today (and they
do). But we cannot make this a hard requirement, or we risk stalling
the project for patches that are not reviewed timely by non-experts.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

On Wed, Nov 11, 2009 at 12:45 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> The patches that get sent in are almost always
> either fallout from a customer/company project, or stuff that one of the
> closed-sourced forks has developed that they don't want to maintain, or
> stuff people do "at night" to make their lives easier in the distant
> future.  All of those things are already special arrangements that
> people need to make with their employers and their lives, but they have
> discernible benefits.  But you can't expect a lot of people or employers
> to reserve time on top of that for handholding other people's patches
> and for other "community" stuff that has no easy to measure benefits.

Well, I'm perfectly willing to handhold other people's patches and do
community stuff that has no easy to measure benefits. I'm willing to
do it precisely because I believe it will make it easier to get my own
patches accepted. AIUI, the whole point of CommitFests is that I
agree to review other people's patches and they in turn agree to
review mine, and we all get to scratch our respective itches. In
fact, because I enjoy community work, I'm even willing to spend MORE
time helping other people get their patches in than I do writing my
own patches - but I'm only willing to spend 2x or 3x the time on other
people's stuff that I spend on my own, not 10x.

In the last two CommitFests it has become evident that we have a
problem with freeloading. During the September CommitFest, the set of
reviewers and the set of patch submitters were almost disjoint. Now,
fortunately, we still had enough reviewers; the real shortage was of
committers. But I think that some of the reviews were not as good as
they could have been had they been reviewed by more experienced
contributors, or even just by multiple people. A fair amount of easy
stuff slipped through the review process. Tom cleaned it up, but he
shouldn't have to be responsible for things that a read-through of the
patch by an experienced C programmer should have caught. I tried to
help, but I was fairly tied up with overall CommitFest management and
did not have time for a full read-through of every patch.

...Robert

On Wed, Nov 11, 2009 at 2:51 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Josh Berkus wrote:
>>
>> > For example, probably only Simon and Heikki are knowledge enough to
>> > apply the HS and SR patchs --- but those patch is handled.  The harder
>> > part is the other patches where there are only a small pool of people
>> > knowledgeable enough to apply the patch, but many of those knowledgeable
>> > people are tied up with non-community things.
>>
>> People learn to do this by doing it.  That's how you learned, and Tom
>> and Heikki.  Ultimately, it's a source code control system; if someone
>> screws up, we can always back it out.
>
> True, but even I avoid patches I don't understand, and "practicing" by
> applying them could lead to a very undesirable outcome, e.g.
> instability.

Well, if we're going to add committers (and I'm generally in favor of
that) we need to pick people who won't apply patches that they don't
understand. In a way, that's really the only HARD requirement for a
committer, I think. If someone doesn't understand any aspect of the
system well enough to commit patches, but they know that, then having
them as a committer will be useless, but not actively harmful (beyond
being a waste of time). On the other hand, someone who actually does
know some parts of the system well enough to commit patches but
doesn't have the discretion to avoid things that are more than they
can tackle will create chaos.

In other words, we should be looking for people who (1) know enough to
be able to commit at least some patches without breaking the world and
(2) are responsible enough to know which things they can commit and
which they should leave alone.

Of course, anyone you pick will probably make some mistakes, but
that's the general idea...

...Robert

Bruce Momjian wrote:
> True, but even I avoid patches I don't understand, and "practicing" by
> applying them could lead to a very undesirable outcome, e.g.
> instability.
>
The usual type of practice here should come from applying trivial
patches, or ones that don't impact code quality. Docs patches come to
mind as a good way someone could get used to the commit process without
introducing much potential mayhem along the way. As far as keeping new
people away from complicated patches, ultimately you just have to trust
that anyone who can commit has a reasonable idea of their own
capabilities. I seriously doubt you're going to find a new committer
jumping right in by committing hot standby out of the gate just because
they could do so.

--
Greg Smith 2ndQuadrant Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com www.2ndQuadrant.com

Robert Haas wrote:
> I tried to help, but I was fairly tied up with overall CommitFest management and
> did not have time for a full read-through of every patch.
>
I think it's completely unreasonable to expect the CF manager to do any
patch review themselves. It's a hard enough job to keep going without
actually getting your hands into the details.

--
Greg Smith 2ndQuadrant Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com www.2ndQuadrant.com

On Wed, Nov 11, 2009 at 3:57 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> Robert Haas wrote:
>>
>> I tried to help, but I was fairly tied up with overall CommitFest
>> management and
>> did not have time for a full read-through of every patch.
>>
>
> I think it's completely unreasonable to expect the CF manager to do any
> patch review themselves.  It's a hard enough job to keep going without
> actually getting your hands into the details.

Well, that's another reason I'd kind of like to take a break from CF
management - it would free up more of my time for actual patch review.
I have been doing some review anyhow, but given as how I don't get
paid for any of this, there's only a few hours a day that are
potentially available for this, and it has to compete with the
dishwasher breaking and my wife wanting to play Pandemic. By the way,
if anyone wants to buy me a new dishwasher, I will put reviewing your
patch at the top of my "to do" list. :-)

...Robert

Bruce Momjian wrote:
> I also think the bad economy is making it harder for people/companies to
> devote time to community stuff when paid work is available.
>
I think this explains away more of the recent situation than you're
giving it credit for. When everybody's fat and happy and it's easy to
generate/raise money, it's also easy to throw money toward the
community. When times are tight, giving away work that you might charge
for (or have already charged for) is harder for a company to justify.
It's easy to plan to have someone do community work when you hire them,
only to realize down the line that business has dried up enough that
you're stuck with the choice between them doing that and a job that will
make or break and upcoming payroll. And that's where a lot more
businesses are at right now than at any time in a long while.

After looking for an example of the boom/bust cycle impacting this
community's work that's old enough to be clearer in hindsight, I would
suggest noting that Great Bridge was officially announced in May of 2000
and was gone by the end of 2001. Overlay those dates on top of
http://www.google.com/finance?q=INDEXNASDAQ:.IXIC after switching "Zoom"
to show 10 years.

--
Greg Smith 2ndQuadrant Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com www.2ndQuadrant.com

On 11/11/09 12:55 PM, Greg Smith wrote:
> I seriously doubt you're going to find a new committer
> jumping right in by committing hot standby out of the gate just because
> they could do so.
>

We'd be lucky to get them to *read* the Hot Standby patch and comment on it.

--Josh Berkus

The following email expresses my personal opinion and does not reflect
the opinion of my employers.

Bruce Momjian wrote:
> I also think the bad economy is making it harder for people/companies to
> devote time to community stuff when paid work is available.
>
Actually the bad economy should be a booster for open source projects.
There should be more developers with time to acquire new skills on
projects that will get them a better job when the economy comes back.

I think the problems are more rooted in the developer community itself.
The pg-hackers mailing list is probably the less socially skilled
developer community I have ever seen in all the open source projects I
have been involved with. A very high standard is set for contributions,
which is good for the quality of the code, but the lack of development
process and clear decision chain turns every new contributor into
endless frustration. For a patch to be committed, a vague consensus has
to arise among the strong technical voice(s) (usually convincing Tom is
enough). If a more complex feature needs to be implemented, the lack of
decision process ends up in a first long round of emails until everybody
gets tired of it. Then sometimes later someone tries to re-activate the
debate for another round and so on (partitioning is a good example). You
lost potential committers at each of these rounds.

The way I see it, most companies try to push their agenda, contribute
their patches back to the community if it works and just go with their
own fork and closed implementation if this is too much work or burden.
Whatever the economy, very few people can commit to an indefinite amount
of time to get a feature integrated in Postgres.

Now you should probably ask yourself what you should do as a community
to get more committers? Like it was said at PGCon, to get a patch in, it
is going to be hard and painful. How do you make it less hard and less
painful?

On the other end, how do we, simple developers, become better to reach
the status of committers? How can we endure the constant bashing
especially in the early stages of our learning phase (especially if you
are not paid to do it)? How do we justify to our employers that they
should persist through this long process without visibility, so that
eventually their contribution will make it back to the community? How do
we make it easier for companies to contribute code?

The lightness of the development process (no project manager, no
steering committee, no voting, no product management, ...) is both a
strength and a weakness that makes Postgres what it is today. The
commitfest started to address some of the most obvious issues but there
is probably much more that can be done by looking at what is happening
in the other open source communities.

Even if the economy is hard, I found time to invest my own 2 cents in
this email ;-)

Emmanuel

Hi,

> True, but even I avoid patches I don't understand, and "practicing" by
> applying them could lead to a very undesirable outcome, e.g.
> instability.
>

How about having a staging server to help around novice committers?
Basically the selected new band of people can take a patch, review it
and if they deem it fit for checkin, they could check it into the
staging server.

Then they should try out the regression runs (we do have very few test
cases) and other sanity checks to ensure that things are sane.
Probably the buildfarm infrastructure can also be used to run against
this staging server on all platforms.

That ways, we can avoid the initial instability upto some extent and
then noob committer can then muster courage to atleast tell the
core-committers what they think about the staged patch commit.

Regards,
Nikhils
--
http://www.enterprisedb.com

On ons, 2009-11-11 at 22:30 -0500, Emmanuel Cecchet wrote:
> On the other end, how do we, simple developers, become better to
> reach
> the status of committers? How can we endure the constant bashing
> especially in the early stages of our learning phase (especially if
> you
> are not paid to do it)? How do we justify to our employers that they
> should persist through this long process without visibility, so that
> eventually their contribution will make it back to the community?

That's basically just it: Assume bashing is part of the process. Don't
think of it as bashing. Take the constructive criticism from it, ignore
the rest. Assume only one out of three feature ideas will make it.
Apply the prerequisite amount of gamesmanship to the system and tune
your bikeshedding detectors. Don't take anything personally. Live and
learn.

Bruce Momjian <bruce(at)momjian(dot)us> writes:

> Peter Eisentraut wrote:
>> discernible benefits. But you can't expect a lot of people or employers
>> to reserve time on top of that for handholding other people's patches
>> and for other "community" stuff that has no easy to measure benefits.
>
> Totally agree. It is that zero-return work that is hard to justify for
> people and companies. It is clearly something that requires
> self-sacrifice, and personally I think a culture of self-sacrifice is
> what has given us such great success and such a nuturing community
> environment.

I don't know what you're talking about here, or refering to.

The easy way to explain Open Source contributing to PHB and finance
people, in my mind, is the following: Open Source is not free as in free
beer. At all. Rather than paying licences costs, what you have to pay
for is your employees salary when they take the time to participate into
this community things. And if you're using Open Source Software, you
*need* employees that take part of the community processes.

That also mean you have real experts now, and ones able to drive the
project in a direction that will be to your profit as an employer. For
example, this hairy bug in your production environment will get fixed
the day you report about it, rather that when the licencing company
think it matches their roadmap, if ever.

Oh and some other niceties about it. Let's try the "story telling"
style... When I worked on preprepare, I though we had an urgent need
here to run it. It happened that we had yet more urgent to do, so I
barely had time to have it working and commit it on some pgfoundry
CVS. Less than a week after, I'm told on IRC that the product is running
fine in a production environment somewhere, taking real load. Now I
don't have to make tests before deploying. That has been done for
me. That's Open Source.

Regards,
--
dim

Is it 2009 and we're still having this discussion about how
participating into Open Source communities is a good way to spend your
money as an employer? Wow.

> That's basically just it: Assume bashing is part of the process. Don't
> think of it as bashing. Take the constructive criticism from it, ignore
> the rest. Assume only one out of three feature ideas will make it.
> Apply the prerequisite amount of gamesmanship to the system and tune
> your bikeshedding detectors. Don't take anything personally. Live and
> learn.

I think what Emmanuel is complaining about here ... with some
justification ... is not the revision requirements of our patch process
but the extremely confusing and frustrating nature of it for new
contributors. For example, how exactly is a new contributor supposed to
know the difference between "bikeshedding" and "constructive criticism",
when (a) they don't know the people involved, and (b) even our more
dedicated committers engage in bikeshedding periodically?

This isn't just Emmanuel; I've heard this complaint from numerous
first-time contributors, and have seen several talented people walk away
from our project because of it. Even people who have stuck with us,
such as Josh Tolley, have remarked on the "hazing ritual" inherent in
getting a first-time contribution in.

Probably, the only possible solution is for each new contributor to have
a mentor who sticks with them throughout the process of getting their
first contribution accepted, explaining the process, pre-reviewing their
submissions, and explaining to them which criticism they should be
paying attention to and which they shouldn't. I *thought* that Bruce
was doing that for AsterData, but apparently not.

This would require a pool of experienced contributors volunteering to be
mentors, which I'm not sure we have.

It would also help if people on this list *in general*, were to be a bit
more consistent about phrasing criticism as constructive criticism.
I've seen far too much "how stupid are you?" on this list for the good
health of our developer community.

--Josh Berkus

On Thu, Nov 12, 2009 at 1:27 PM, Josh Berkus <josh(at)agliodbs(dot)com> wrote:
>
>> That's basically just it:  Assume bashing is part of the process.  Don't
>> think of it as bashing.  Take the constructive criticism from it, ignore
>> the rest.  Assume only one out of three feature ideas will make it.
>> Apply the prerequisite amount of gamesmanship to the system and tune
>> your bikeshedding detectors.  Don't take anything personally.  Live and
>> learn.
>
> I think what Emmanuel is complaining about here ... with some
> justification ... is not the revision requirements of our patch process
> but the extremely confusing and frustrating nature of it for new
> contributors.  For example, how exactly is a new contributor supposed to
> know the difference between "bikeshedding" and "constructive criticism",
> when (a) they don't know the people involved, and (b) even our more
> dedicated committers engage in bikeshedding periodically?
>
> This isn't just Emmanuel; I've heard this complaint from numerous
> first-time contributors, and have seen several talented people walk away
> from our project because of it.  Even people who have stuck with us,
> such as Josh Tolley, have remarked on the "hazing ritual" inherent in
> getting a first-time contribution in.

I used to feel this way, too. I'm not sure whether it's really worse
at first, or whether it just seems worse a first until you get used to
it. There is no getting around the fact that this is a community of
very smart people. I work at a company where I'm the only developer.
Guess what - my development ideas are all brilliant - or at least
there's no contrary evidence! Then I come here, where there are
plenty of other developers just as smart as I am, or possibly smarter,
and here some of my ideas crash and burn. On the other hand,
sometimes someone objects to my idea and the *objections* crash and
burn. You can't expect to win them all when you're dealing with
people who are as smart as you and more knowledgeable about the code.

I try to be gentle with newcomers but it's not always easy to figure
out what that means. When someone submits a bad patch and I review
it, they sometimes get two or three pages of feedback detailing
inadequacies of various types. I understand that can be hard on the
ego, but I don't think it does anyone any favors to give them a
PARTIAL list of things they need to fix - in fact I think breaking up
the pain into multiple rounds is quite a bit worse, because then you
create the illusion of progress without the actuality. I don't give
people feedback because I want to hurt them or cause them pain, but I
know from experience that until those things are fixed, it *will*
*not* *get* *committed*. If I say nothing now, someone else will say
it later.

I am afraid that the difference between bikeshedding and legitimate
criticism is something that contributors have to learn for themselves.
I can tell you which things *I* think are bikeshedding, but someone
else may have a longer, shorter, or same-size non-overlapping list.
No one can speak with authority on this topic because no one is the
boss.

All that having been said, I feel your pain. I don't want to scare
off new contributors, but I don't necessarily know how to improve the
situation.

> Probably, the only possible solution is for each new contributor to have
> a mentor who sticks with them throughout the process of getting their
> first contribution accepted, explaining the process, pre-reviewing their
> submissions, and explaining to them which criticism they should be
> paying attention to and which they shouldn't.  I *thought* that Bruce
> was doing that for AsterData, but apparently not.

Well, this is a good idea, although I'm not sure how well it will play
out in practice. I'm always game if anyone wants to email me off-list
to discuss issues of this type, or...

> This would require a pool of experienced contributors volunteering to be
> mentors, which I'm not sure we have.

...I'm willing to do this.

> It would also help if people on this list *in general*, were to be a bit
> more consistent about phrasing criticism as constructive criticism.
> I've seen far too much "how stupid are you?" on this list for the good
> health of our developer community.

I don't see very much of this at all, so maybe this is another area
where it boils down to perspective. I find this list to be quite
civilized most of the time; I wouldn't hang around if I didn't.

...Robert

I'm not addressing the complaints Emmanuel has about patch submission,
but was inspired by what he said. I hesitated to post this to
-hackers, but my hope is that my comments are taken in the spirit of
reflection.

I've been thinking about the Postgres community and things that
differentiate it from other FOSS projects for a few years. We lack
both a benevolent dictator, or a single-corporate backer... We have an
incredible reputation for technical expertise -- most knowledgeable
FOSS developers fear and are in awe of the quality of code our
community produces. I hear criticism for the way in which our
development community operates, and how we communicate with
"outsiders". But few people are able to offer concrete suggestions on
/how/ we might change, as opposed to the /particular things/ that they
think should change.

So on this point:

On Wed, Nov 11, 2009 at 7:30 PM, Emmanuel Cecchet <manu(at)frogthinker(dot)org> wrote:

> The commitfest started to
> address some of the most obvious issues but there is probably much more that
> can be done by looking at what is happening in the other open source
> communities.

Speaking from a cultural, and organizational standpoint:

* Distributed revision control as standard for the project

Using a distributed revision control system, and working from
developer branches which are pulled & pushed directly between
developers is consistently reported to have a positive impact on
solving problems and sharing information. Summer of Code mentors
consistently mention DVCS as having a huge positive impact on
productivity for their students - both because it's easy and cheap for
students to commit to their local repos, and because the merge/pull
tools are intuitive for the mentors and students.

This is how people work and share code "out there". I realize that
Postgres has an established cultural norm of posting patches to the
mailing list, and I'm not suggesting that people stop doing that.

I think more developers should be sharing branches as well.
git.postgresql.org should probably be reorganized to clearly divide
repos between postgresql-core and other projects (contrib, add-ons,
whatever).

We should also publish more information about how our developers are
sharing code with each other through repos.

* Decision making

Is there a project you'd point to as a model for decision making?
(serious question)

This is a volunteer organization, and as such, we're subject to who
has the expertise + time/energy to devote to the patches and projects.
Going back to what Bruce has said about skill drain -- if more
individuals were able to get their time sponsored to devote to the
technical challenges posed by the patches, then it might be possible
to have more coordinated/streamlined responses and decision making.

My experience in FOSS projects is that Postgres does fairly well on
execution compared to other projects (getting a new release out the
door every year, and now the alphas every couple months), but has some
issues with a couple large, important patches. Part of the problem
there appears to be people being stretched too thin, and inconsistent
communication practices.

More involvement from people really focusing on improving
communication between developers would be helpful to make those issues
easier to work around.

* Volunteers, funding and burnout

It's challenging to keep up with the technical discussions on -hackers
even if you are a very knowledgeable developer. If you're only doing
that on your free time, and say, you have a family? Most of the
developers that I know in other projects in leadership/commit manager
roles don't have young children, and very few are women.

So, another thing that we could use are more people like Bruce that
can devote a great deal of *work* time to thinking about and acting on
the issues around community management and recruitment. Robert Haas'
time spent on commitfest, for example, if sponsored by a foundation
would be very helpful. Maybe he could have bought a new dishwasher. :)

Also, we could use an individual dedicated to raising funds and
consistently executing on distributing the money out for non-code
"infrastructure" projects. Like:

* release management,
* marketing,
* dedicated public relations (driving things like new software release
articles, organizing interviews with press, and getting new case
studies written), and
* keeping track of our relationships with other organizations (Summer
of Code, conferences, commercial forks).

A lot of this work falls on Josh Berkus to coordinate. There are many
more tasks, and some are delegated to others & coordinated on
-advocacy. Again, this is work that could be funded through a
foundation, or as part of positions in particular companies.

IMO, it's about providing incentives that encourage people to use
their time wisely, and to help prevent burnout on the difficult tasks
around people management. And to say definitively that the people
tasks require organization and management, and we as a development
community value that.

-selena

--
http://chesnok.com/daily - me
http://endpoint.com - work

Robert Haas escribió:

> I used to feel this way, too. I'm not sure whether it's really worse
> at first, or whether it just seems worse a first until you get used to
> it. There is no getting around the fact that this is a community of
> very smart people. I work at a company where I'm the only developer.
> Guess what - my development ideas are all brilliant - or at least
> there's no contrary evidence! Then I come here, where there are
> plenty of other developers just as smart as I am, or possibly smarter,
> and here some of my ideas crash and burn. On the other hand,
> sometimes someone objects to my idea and the *objections* crash and
> burn. You can't expect to win them all when you're dealing with
> people who are as smart as you and more knowledgeable about the code.

Maybe this paragraph (and some others from this email and Josh's) should
be part of the "how to submit a patch" wiki page.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

On the subject topic, I have to say that I don't see where Robert
hasn't met the qualifications mentioned so far on this thread as
required to promote someone to the committer level; are there some
requirements which exist but haven't been mentioned?

Regarding the specific issues below, which seem to be more about the
related topic of drawing in new contributors...

Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> Josh Berkus <josh(at)agliodbs(dot)com> wrote:
>>
>>> That's basically just it: Assume bashing is part of the process.

>> I've heard this complaint from numerous first-time contributors,
>> and have seen several talented people walk away from our project
>> because of it. Even people who have stuck with us, such as Josh
>> Tolley, have remarked on the "hazing ritual" inherent in getting a
>> first-time contribution in.
>
> I used to feel this way, too. I'm not sure whether it's really
> worse at first, or whether it just seems worse a first until you get
> used to it.

I would say that I don't often see anything I would consider as
"bashing", but that I have seen newcomers who appeared to want to help
contribute to the project handled with less finesse than one could
perhaps expect. It seems to me that those needing assistance
(especially if they are in dire straights or are trying to convert to
PostgreSQL) are treated very gently. Someone with a question, if it
isn't something too obviously documented in the FAQ or the online
manual, is given a polite and informative response. The responses get
pretty blunt if the person offers suggestions which show that the
person has (or assumes) technical competence, when those suggestions
aren't acceptable to the community. At that point it takes a pretty
thick skin to find some other way to contribute, rather than just
walking away. Again, I wouldn't call it bashing, but a level of
bluntness which could be disconcerting if the person hasn't followed
these lists long enough to understand how it is intended.

> When someone submits a bad patch and I review it, they sometimes get
> two or three pages of feedback detailing inadequacies of various
> types.

When done in a professional manner, which is typical of this list, I
don't think there's any reasonable alternative. I don't see that as a
problem.

Well, there is one thing which must be frustrating. If I was seated
near others who read this list, I would start a pool every time a
newcomer submitted a large patch -- how long until someone tells them
to break it into smaller patches, and how long until someone else
tells them it makes no sense separated like that and should be put
back together?

>> the only possible solution is for each new contributor to have a
>> mentor who sticks with them throughout the process of getting their
>> first contribution accepted

I don't know how common it is, but I felt that Bruce did that for me
when I worked on the standard_conforming_strings TODO item for 8.2.
(Perhaps part of the reason was that it was urgently needed to address
problems we found in production after we converted a critical database
to PostgreSQL; but whatever the reason, I was grateful for the help.)
Having gone through that experience, I feel confident that such
assistance for first-time contributors can make a difference both in
the success of their efforts (which will then, presumably, benefit the
community in general) and in their willingness to contribute in other
ways.

>> It would also help if people on this list *in general*, were to be
>> a bit more consistent about phrasing criticism as constructive
>> criticism. I've seen far too much "how stupid are you?" on this
>> list for the good health of our developer community.
>
> I don't see very much of this at all, so maybe this is another area
> where it boils down to perspective.

I don't see much of that either, although describing another's
statement as "nonsense" or something similar isn't unheard of. It's
not usually an unfair characterization, but some could take such
bluntness as hostility.

-Kevin

"Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov> writes:
> On the subject topic, I have to say that I don't see where Robert
> hasn't met the qualifications mentioned so far on this thread as
> required to promote someone to the committer level; are there some
> requirements which exist but haven't been mentioned?

He hasn't actually written all that much *code* ... yet, anyway.

Historically, we've promoted people to being committers when
(a) the existing committers got tired of applying patches from them
and (b) their patches were usually going in without significant
modification. (a) speaks to whether you do enough work regularly enough
that there's actually a benefit to the project from giving you the keys
to the car, and (b) speaks to whether you're ready in terms of technical
knowledge of the code base. (I'm not saying you have to be competent
in the whole PG code base, just whatever part you like to work on.)

Now those criteria were developed to deal mainly with people committing
their own patches. What we have at the moment is a lot of patches
coming in from people who aren't ready to be committers, and maybe don't
ever want to be. The question is how to do an adequate job of reviewing
their patches, when only a fraction of the existing committers are
willing to put time into reviewing other people's patches. (Let's face
it, that's a lot less fun than writing your own code.) While I'm not
against promoting more committers to deal with the influx of patches,
the only way I know for people to get to the skill level of being fully
competent reviewers is to have done a lot of patch writing themselves.

Years ago, somebody (I think one of the original Berkeley crew) remarked
"this project doesn't need a lot of people with a little time, it needs
a few people with a lot of time". I'm afraid that's still true, and
it's still hard to find those people.

regards, tom lane

On Thu, Nov 12, 2009 at 6:53 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Now those criteria were developed to deal mainly with people committing
> their own patches.  What we have at the moment is a lot of patches
> coming in from people who aren't ready to be committers, and maybe don't
> ever want to be.  The question is how to do an adequate job of reviewing
> their patches, when only a fraction of the existing committers are
> willing to put time into reviewing other people's patches.  (Let's face
> it, that's a lot less fun than writing your own code.)

I kind of like reviewing, actually. It's a good way to get familiar
with new parts of the code. That's part of the coolness of open
source: other people do a lot of your work for you. Of course, that
doesn't mean I would want to ONLY review other people's patches and
never write any of my own.

> While I'm not
> against promoting more committers to deal with the influx of patches,
> the only way I know for people to get to the skill level of being fully
> competent reviewers is to have done a lot of patch writing themselves.
>
> Years ago, somebody (I think one of the original Berkeley crew) remarked
> "this project doesn't need a lot of people with a little time, it needs
> a few people with a lot of time".  I'm afraid that's still true, and
> it's still hard to find those people.

No, I think it's hard to find those people's salaries. I feel like
I've managed a fairly good stream of patches given that this is
something I do mostly between 8 PM and midnight after working a full
day and in between other things that I need to get done, but if I were
getting paid to hack on PostgreSQL full time (or even one or two days
a week) that stream would be a whole lot bigger.

...Robert

Josh Berkus wrote:
> paying attention to and which they shouldn't. I *thought* that Bruce
> was doing that for AsterData, but apparently not.

Josh, two days ago you complained I that I mentioned 'search_path' when
we were talking about postgresql.conf, now you have another complaint
about me. Did you get appointed network nanny recently? I never got
that email.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Tom Lane wrote:
> While I'm not against promoting more committers to deal with the influx of patches,
> the only way I know for people to get to the skill level of being fully
> competent reviewers is to have done a lot of patch writing themselves.
>
The dynamic going on right now is that many people who might otherwise
be writing their own patches are instead doing patch review to try and
keep the project as a whole moving forward. I actually had two off-list
discussions about that just today, that topic pops up pretty regularly
as I talk with contributors at all levels.

Since most people have an upper limit on how much community time they
can spend, every minute spent reviewing is one you're not working on
your own patches during. The way you're describing the qualification
process, it would be easy to conclude that there's a reviewer ladder,
and a developer ladder, and only climbing the latter leads to being a
committer--that no matter how much review you do, it doesn't really
count as a committer grade skill.

I'm not sure that's the message you want to be sending, because anyone
who dreams of being a committer is going to stay as far away from doing
review as they can if that notion spreads. Based on the growing
frustration with "doing review doesn't leave me with time for my own
patches" I keep hearing, that perception is already something to be wary
of. If the primary criteria is generating patches that apply with
minimal changes, you could make a case that someone who's gotten skilled
enough as a reviewer to only pass through patches of that quality should
get some recognition even if they didn't write them. That's clearly a
useful subset of the skills needed to commit patches only if they look
to be ready for it.

--
Greg Smith 2ndQuadrant Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com www.2ndQuadrant.com

On Thu, Nov 12, 2009 at 8:55 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> I'm not sure that's the message you want to be sending, because anyone who
> dreams of being a committer is going to stay as far away from doing review
> as they can if that notion spreads.

To say nothing of CommitFest management.

...Robert

Greg Smith <greg(at)2ndquadrant(dot)com> writes:
> Since most people have an upper limit on how much community time they
> can spend, every minute spent reviewing is one you're not working on
> your own patches during. The way you're describing the qualification
> process, it would be easy to conclude that there's a reviewer ladder,
> and a developer ladder, and only climbing the latter leads to being a
> committer--that no matter how much review you do, it doesn't really
> count as a committer grade skill.

Well, we only instituted the commitfest system last year; before that
there wasn't any formal concept of people reviewing patches at all.
Nobody has yet been promoted to committer on the basis of their
reviewing work, but I don't wish to suggest that it couldn't happen.

> I'm not sure that's the message you want to be sending, because anyone
> who dreams of being a committer is going to stay as far away from doing
> review as they can if that notion spreads. Based on the growing
> frustration with "doing review doesn't leave me with time for my own
> patches" I keep hearing, that perception is already something to be wary
> of.

I'll let you in on a secret: it feels about like that in the committer
trenches too.

At this point it seems that one of the major practical reasons for
appointing new committers would be their willingness to help with
review/commit of incoming patches. If anyone is thinking there's
a ladder they can climb that doesn't have that responsibility at
the top, I'm afraid they're mistaken.

regards, tom lane

(I'm not sure I should piping up here, so feel free to ignore, but
perhaps I have something small to offer. I've been following the list
for a while, but try to keep my mouth shut.)

On 13/11/2009 3:07 AM, Selena Deckelmann wrote:

> * Distributed revision control as standard for the project

This would also make it a lot easier to track in-progress work on
particular features of interest, allowing interested users to help with
advance testing of early versions of major feature work. Chasing patches
on a mailing list is not an attractive way to try to keep up with
someone's in-progress work, and is demotivating to people interested in
testing that work. Think: HOT, warm standby, etc.

It also helps with the issue where a patch is posted, followed by short
thread of corrections and changes you have to manually apply to reach
(you hope) the same codebase others are testing. Sure, sometimes a
follow-up patch is posted with the changes, but often not.

I also found one comment of Tom Lane's really interesting:

> The question is how to do an adequate job of reviewing
> their patches, when only a fraction of the existing committers are
> willing to put time into reviewing other people's patches. (Let's face
> it, that's a lot less fun than writing your own code.)

Perhaps I'm strange, but I personally enjoy patch review. I often think
others are smarter and better at creative, new development - but their
work often needs polish and cleanup, makes mistakes with internal APIs,
etc. It feels like something I can contribute usefully, and it's a role
I've had in PoDoFo among other projects. I naturally gravitate toward
cleanup and patch review, but I'm not generally much good at major
feature work.

Is it possible that it's worth inviting committers who're primarily
interested in patch review, not their own original work? I'd be
surprised if there weren't interested, competent people. I might be one
of them if I didn't fail quite so profoundly on the latter point. If
somebody isn't feeling constantly frustrated by the perceived burden of
reviewing others' work taking time out of their own, they might be less
prone to burn-out.

As for gaining the required competence without doing a lot of first-hand
development on the codebase: Much as a committer may focus mainly on one
area of Pg's codebase and may not be competent to make unsupervised /
unassisted changes in another, surely patch reviewer-committers could
also have a few areas of interest and focus on patches touching those
areas? Especially with review of patches by several different people
with different areas of interest touched on by a given patch, it may not
require globally competent reviewers to ensure good patch quality.

--
Craig Ringer

On Sat, Nov 14, 2009 at 04:55, Craig Ringer <craig(at)postnewspapers(dot)com(dot)au> wrote:
> (I'm not sure I should piping up here, so feel free to ignore, but
> perhaps I have something small to offer. I've been following the list
> for a while, but try to keep my mouth shut.)

Meh. All constructive input is welcome!

> On 13/11/2009 3:07 AM, Selena Deckelmann wrote:
>
>> * Distributed revision control as standard for the project
>
> This would also make it a lot easier to track in-progress work on
> particular features of interest, allowing interested users to help with
> advance testing of early versions of major feature work. Chasing patches
> on a mailing list is not an attractive way to try to keep up with
> someone's in-progress work, and is demotivating to people interested in
> testing that work. Think: HOT, warm standby, etc.
>
> It also helps with the issue where a patch is posted, followed by short
> thread of corrections and changes you have to manually apply to reach
> (you hope) the same codebase others are testing. Sure, sometimes a
> follow-up patch is posted with the changes, but often not.

This is probably most important for large patches, but the line where
it becomes useful is very fuzzy. I think it's helpful at a much lower
complexity level than most people realize.

I think we should encourage more poeple to use this. How can we do that?

Perhaps we can put an encouragement in the description how to submit a patch?

How about we add specific feature(s) about tihs to the commitfest
management tool? Like the possibility to directly link a git
repo/branch with the patch?

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On Sat, Nov 14, 2009 at 4:11 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> How about we add specific feature(s) about tihs to the commitfest
> management tool? Like the possibility to directly link a git
> repo/branch with the patch?

So two fields, one for the repo URL and one for the branch name?

...Robert

On Sat, Nov 14, 2009 at 13:35, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Sat, Nov 14, 2009 at 4:11 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>> How about we add specific feature(s) about tihs to the commitfest
>> management tool? Like the possibility to directly link a git
>> repo/branch with the patch?
>
> So two fields, one for the repo URL and one for the branch name?

Yeah, I think that's it. It might actually be interesting to pull the
latest version date and make a note in the cf management stuff
automagically in case there the git repo has a more updated version
than the one that was submitted. I think that could be quite useful -
shouldn't be too hard to do, I think. Probably just a cron job that
updates a third col in the db?

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Magnus Hagander wrote:
> On Sat, Nov 14, 2009 at 13:35, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> > On Sat, Nov 14, 2009 at 4:11 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> >> How about we add specific feature(s) about tihs to the commitfest
> >> management tool? Like the possibility to directly link a git
> >> repo/branch with the patch?
> >
> > So two fields, one for the repo URL and one for the branch name?
>
> Yeah, I think that's it. It might actually be interesting to pull the
> latest version date and make a note in the cf management stuff
> automagically in case there the git repo has a more updated version
> than the one that was submitted. I think that could be quite useful -
> shouldn't be too hard to do, I think. Probably just a cron job that
> updates a third col in the db?

Can you get git to dynamically generate a tree diff via a URL? That
would be nice. Extra points for a context diff. ;-)

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Sun, Nov 15, 2009 at 8:08 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Magnus Hagander wrote:
>> On Sat, Nov 14, 2009 at 13:35, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> > On Sat, Nov 14, 2009 at 4:11 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>> >> How about we add specific feature(s) about tihs to the commitfest
>> >> management tool? Like the possibility to directly link a git
>> >> repo/branch with the patch?
>> >
>> > So two fields, one for the repo URL and one for the branch name?
>>
>> Yeah, I think that's it. It might actually be interesting to pull the
>> latest version date and make a note in the cf management stuff
>> automagically in case there the git repo has a more updated version
>> than the one that was submitted. I think that could be quite useful -
>> shouldn't be too hard to do, I think. Probably just a cron job that
>> updates a third col in the db?
>
> Can you get git to dynamically generate a tree diff via a URL?  That
> would be nice.  Extra points for a context diff.  ;-)

I dunno about the automated comment generation thing. Seems like it
could generate a lot of comment spam inside the app. Also, I'm not
sure if we really want to move away from the mailing list as the
primary way of sharing patches. One nice thing about having them on
the mailing list is that it is a permanent archive. Another is that
it it is a "push" mechanism - you don't have to go to the CommitFest
app and notice, hey, there are new patches here, or new versions of
existing patches. You just read your email and there they are.

I'm not averse to adding fields for repo and branch; that seems pretty
uncontroversial, since they'll be optional and those who don't want to
use them needn't. But I think the rest of this needs a bit more
thought. Just MHO, of course.

...Robert

On Mon, Nov 16, 2009 at 02:08, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Magnus Hagander wrote:
>> On Sat, Nov 14, 2009 at 13:35, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> > On Sat, Nov 14, 2009 at 4:11 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>> >> How about we add specific feature(s) about tihs to the commitfest
>> >> management tool? Like the possibility to directly link a git
>> >> repo/branch with the patch?
>> >
>> > So two fields, one for the repo URL and one for the branch name?
>>
>> Yeah, I think that's it. It might actually be interesting to pull the
>> latest version date and make a note in the cf management stuff
>> automagically in case there the git repo has a more updated version
>> than the one that was submitted. I think that could be quite useful -
>> shouldn't be too hard to do, I think. Probably just a cron job that
>> updates a third col in the db?
>
> Can you get git to dynamically generate a tree diff via a URL?  That
> would be nice.  Extra points for a context diff.  ;-)

yes, easily. Just pass it the commit id. And unlike cvs, there is one
diff for the patch, not one for every file ;)
For example:
http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=d3359020ef7e0fba02ac552d88ede0c3ce5128cc

turning it into context-diff style will require patching gitweb
though, it doesn't do that by default.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On Mon, Nov 16, 2009 at 02:32, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Sun, Nov 15, 2009 at 8:08 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> Magnus Hagander wrote:
>>> On Sat, Nov 14, 2009 at 13:35, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>> > On Sat, Nov 14, 2009 at 4:11 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>> >> How about we add specific feature(s) about tihs to the commitfest
>>> >> management tool? Like the possibility to directly link a git
>>> >> repo/branch with the patch?
>>> >
>>> > So two fields, one for the repo URL and one for the branch name?
>>>
>>> Yeah, I think that's it. It might actually be interesting to pull the
>>> latest version date and make a note in the cf management stuff
>>> automagically in case there the git repo has a more updated version
>>> than the one that was submitted. I think that could be quite useful -
>>> shouldn't be too hard to do, I think. Probably just a cron job that
>>> updates a third col in the db?
>>
>> Can you get git to dynamically generate a tree diff via a URL?  That
>> would be nice.  Extra points for a context diff.  ;-)
>
> I dunno about the automated comment generation thing.  Seems like it
> could generate a lot of comment spam inside the app.  Also, I'm not
> sure if we really want to move away from the mailing list as the
> primary way of sharing patches.  One nice thing about having them on
> the mailing list is that it is a permanent archive.  Another is that
> it it is a "push" mechanism - you don't have to go to the CommitFest
> app and notice, hey, there are new patches here, or new versions of
> existing patches.  You just read your email and there they are.

Yeah, I agree.

> I'm not averse to adding fields for repo and branch; that seems pretty
> uncontroversial, since they'll be optional and those who don't want to
> use them needn't.  But I think the rest of this needs a bit more
> thought.  Just MHO, of course.

Just to be clear, I wasn't suggesting auto-adding comments. I was just
suggesting adding a flag that'll put some kind of marker on the line
saying "there's something newer in the git repo". Obviously only if
the git repo has been specified, and not mandatory.

The idea being to encourage people to use git, not force them to :-)
And exploiting the extra value we get from git for patches that do
change often.

It's not a replacement - the current policy of posting patches to the
list is still the way they should be submitted. It's just an extra
tool.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Magnus Hagander wrote:
> On Mon, Nov 16, 2009 at 02:08, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> Magnus Hagander wrote:
>>> On Sat, Nov 14, 2009 at 13:35, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>>> On Sat, Nov 14, 2009 at 4:11 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>>>> How about we add specific feature(s) about tihs to the commitfest
>>>>> management tool? Like the possibility to directly link a git
>>>>> repo/branch with the patch?
>>>> So two fields, one for the repo URL and one for the branch name?
>>> Yeah, I think that's it. It might actually be interesting to pull the
>>> latest version date and make a note in the cf management stuff
>>> automagically in case there the git repo has a more updated version
>>> than the one that was submitted. I think that could be quite useful -
>>> shouldn't be too hard to do, I think. Probably just a cron job that
>>> updates a third col in the db?
>> Can you get git to dynamically generate a tree diff via a URL? That
>> would be nice. Extra points for a context diff. ;-)
>
> yes, easily. Just pass it the commit id. And unlike cvs, there is one
> diff for the patch, not one for every file ;)
> For example:
> http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=d3359020ef7e0fba02ac552d88ede0c3ce5128cc
>
> turning it into context-diff style will require patching gitweb
> though, it doesn't do that by default.

Any idea how the get the equivalent of "git diff <branch A> <branch B>"
through the web interface?

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Mon, Nov 16, 2009 at 10:20, Heikki Linnakangas
<heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
> Magnus Hagander wrote:
>> On Mon, Nov 16, 2009 at 02:08, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>>> Magnus Hagander wrote:
>>>> On Sat, Nov 14, 2009 at 13:35, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>>>> On Sat, Nov 14, 2009 at 4:11 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>>>>> How about we add specific feature(s) about tihs to the commitfest
>>>>>> management tool? Like the possibility to directly link a git
>>>>>> repo/branch with the patch?
>>>>> So two fields, one for the repo URL and one for the branch name?
>>>> Yeah, I think that's it. It might actually be interesting to pull the
>>>> latest version date and make a note in the cf management stuff
>>>> automagically in case there the git repo has a more updated version
>>>> than the one that was submitted. I think that could be quite useful -
>>>> shouldn't be too hard to do, I think. Probably just a cron job that
>>>> updates a third col in the db?
>>> Can you get git to dynamically generate a tree diff via a URL?  That
>>> would be nice.  Extra points for a context diff.  ;-)
>>
>> yes, easily. Just pass it the commit id. And unlike cvs, there is one
>> diff for the patch, not one for every file ;)
>> For example:
>> http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=d3359020ef7e0fba02ac552d88ede0c3ce5128cc
>>
>> turning it into context-diff style will require patching gitweb
>> though, it doesn't do that by default.
>
> Any idea how the get the equivalent of "git diff <branch A> <branch B>"
> through the web interface?

I don't think you can - but it's probably not a huge thing to implement it.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On Mon, Nov 16, 2009 at 4:30 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Mon, Nov 16, 2009 at 10:20, Heikki Linnakangas
> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>> Magnus Hagander wrote:
>>> On Mon, Nov 16, 2009 at 02:08, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>>>> Magnus Hagander wrote:
>>>>> On Sat, Nov 14, 2009 at 13:35, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>>>>> On Sat, Nov 14, 2009 at 4:11 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>>>>>> How about we add specific feature(s) about tihs to the commitfest
>>>>>>> management tool? Like the possibility to directly link a git
>>>>>>> repo/branch with the patch?
>>>>>> So two fields, one for the repo URL and one for the branch name?
>>>>> Yeah, I think that's it. It might actually be interesting to pull the
>>>>> latest version date and make a note in the cf management stuff
>>>>> automagically in case there the git repo has a more updated version
>>>>> than the one that was submitted. I think that could be quite useful -
>>>>> shouldn't be too hard to do, I think. Probably just a cron job that
>>>>> updates a third col in the db?
>>>> Can you get git to dynamically generate a tree diff via a URL?  That
>>>> would be nice.  Extra points for a context diff.  ;-)
>>>
>>> yes, easily. Just pass it the commit id. And unlike cvs, there is one
>>> diff for the patch, not one for every file ;)
>>> For example:
>>> http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=d3359020ef7e0fba02ac552d88ede0c3ce5128cc
>>>
>>> turning it into context-diff style will require patching gitweb
>>> though, it doesn't do that by default.
>>
>> Any idea how the get the equivalent of "git diff <branch A> <branch B>"
>> through the web interface?
>
> I don't think you can - but it's probably not a huge thing to implement it.

I think git log <branch A>...<branch B> would also be really useful.
If you update your patches by merging rather than rebasing, the
existing gitweb view is nearly useless. I'm astonished this hasn't
bothered any of the kernel developers enough for them to fix it. But
then maybe they use the same workaround I do: the command-line.

...Robert