Re: Why is CF 2011-11 still listed as "In Progress"?

Shouldn't it have been closed weeks ago?

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On 09.01.2012 20:37, Josh Berkus wrote:
> Shouldn't it have been closed weeks ago?

There are still patches in "Needs Review" and "Ready for Committer"
states...

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On 1/9/12 10:39 AM, Heikki Linnakangas wrote:
> On 09.01.2012 20:37, Josh Berkus wrote:
>> Shouldn't it have been closed weeks ago?
>
> There are still patches in "Needs Review" and "Ready for Committer"
> states...

Well, at this point I think we should bump them to CF4. Certainly
nobody is working on those, and CF4 begins in a week.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On 1/9/12 1:37 PM, Josh Berkus wrote:

> Shouldn't it have been closed weeks ago?

It's still "In Progress" mostly because I flaked out for the holidays
after pushing to get most things ready for commit or returned a few
weeks ago, but not quite nailing it shut. I'm back to mostly full-time
on this starting tomorrow, the remains I can deal with will get sorted
out then.

The main question still lingering about is the viability of pushing out
an 9.2alpha3 at this point. That was originally scheduled for December
20th. There was a whole lot of active code whacking still in progress
that week though. And as soon as that settled (around the 30th), there
was a regular flurry of bug fixes for a solid week there. A quick
review of recent activity suggests right now might finally be a good
time to at least tag alpha3; exactly what to do about releasing the
result I don't have a good suggestion for.

There were 31 things committed during CF 2011-11. It feels to me like
there was a larger balance of refactoring compared to feature changes in
this one compared to most. That seems like something we'd like to get
more regression testing on, but at the same time there's not too many
new things for people to be excited about trying.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

On 01/09/2012 09:56 PM, Greg Smith wrote:
> The main question still lingering about is the viability of pushing
> out an 9.2alpha3 at this point. That was originally scheduled for
> December 20th. There was a whole lot of active code whacking still in
> progress that week though. And as soon as that settled (around the
> 30th), there was a regular flurry of bug fixes for a solid week
> there. A quick review of recent activity suggests right now might
> finally be a good time to at least tag alpha3; exactly what to do
> about releasing the result I don't have a good suggestion for.

I would have sworn I left this next to the bike shed...from the crickets
chirping I guess not. I did complete bumping forward the patches that
slipped through the November CF the other day, and it's properly closed now.

As for CF 2012-01, I had thought Robert Haas was going to run that one.
My saying that is not intended to put him on the hook. Normally we'd
have an official deadline announcement by now too, which as one of the
notable lagging cat herders I'm content to absorb a chunk of blame for.

If someone wants to advocate an early time for the official cut-off
tomorrow, don't let me stop you. But since this last one for 9.2 is
"too big to fail" for me, I'm happy to take care of the announcement
myself as the 15th comes to end relative to PST time tomorrow.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

On Sat, Jan 14, 2012 at 8:58 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> I would have sworn I left this next to the bike shed...from the crickets
> chirping I guess not.  I did complete bumping forward the patches that
> slipped through the November CF the other day, and it's properly closed now.
>
> As for CF 2012-01, I had thought Robert Haas was going to run that one.  My
> saying that is not intended to put him on the hook.

Last year, I tried to talk a fairly active roll in getting the
CommitFest wrapped up in a reasonable period of time, and that didn't
really get a lot of support, so I'm not particularly inclined to do it
again. I have long felt that it's important, especially for
non-committers, not to go too long between CommitFests, because that
means going a long time without much opportunity to get things
committed, or even to get feedback. And even for committers, it's not
particularly productive to sit around for a long time with the tree
closed to new work. Virtually nobody wants to grow a gigantic patch
before seeing any of it go in; the chance of rejection is way too
high. At least, that's my view.

But, I've noticed that nothing good comes of me pressing my own view
too hard. Either we as a community value having the CommitFest wrap
up in a reasonable period of time, or we don't. If we do, then let's
make it happen together. If we don't, then let's resign ourselves now
to the fact that 9.2 will not hit the shelves for a very long time.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 1/16/12 11:42 AM, Robert Haas wrote:
> But, I've noticed that nothing good comes of me pressing my own view
> too hard. Either we as a community value having the CommitFest wrap
> up in a reasonable period of time, or we don't.

Reality is, alas, not nearly so binary as this, and therin lie the delays.

While almost everyone agrees that ending the Commitfests on time is a
good thing, almost everyone has at least one patch they would extend the
CF in order to get done. This is the fundamental scheduling struggle of
every single software project I've ever worked on, so I don't see why we
would expect it to be different on PostgreSQL just because we adopted
the CF model.

The benefit of the CF process is that it makes it *visible* when we're
getting behind. But it doesn't stop us from doing so.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On 01/16/2012 02:42 PM, Robert Haas wrote:
> But, I've noticed that nothing good comes of me pressing my own view
> too hard. Either we as a community value having the CommitFest wrap
> up in a reasonable period of time, or we don't. If we do, then let's
> make it happen together. If we don't, then let's resign ourselves now
> to the fact that 9.2 will not hit the shelves for a very long time.

I think this is getting more predictable simply based on having some
history. The trail blazing you led here for some time didn't know what
was and wasn't possible yet. I feel that the basic shape of things,
while still fuzzy in spots, is a lot more clear now.

We need to have schedule goals. There needs to be a date where we
switch toward a more aggressive "is someone going to commit this soon?"
stance on things that are still open. At that point, someone needs to
be the person not afraid to ramp up pressure toward returning things
that just aren't going to make it to commit quality. That's a thankless
task that rarely leaves anyone involved happy.

But this project won't easily move to "ship on this date" instead of
"ship when it's ready", and I don't want that to change. There's two
sides to that. The quality control on most of the 100% date driven
release software I use is terrible. The way the CF schedule is lined up
now, there's enough margin in the schedule that we can handle some drift
there, while still keeping the quality standards high. The currently
open CF is probably going to take at least 6 weeks. That doesn't change
the fact that hard decisions about return vs. continue toward possible
commit should be accelerating by or before the 4 week mark.

The other side to this is that when some big and/or hard features land
does impact PostgreSQL's adoption. To close some of them, you almost
need the sort of focus that only seems to come from recognizing you're
past the original goal date, this one big thing is holding up forward
progress, and everyone who can should be pushing on that usefully to
wrap it up. Could the last 9.1 CF have closed up 1 to 2 weeks earlier
if Sync Rep had been bumped? Probably. Was it worth going off-schedule
by that much so that it did ship in 9.1? I think so. But with every
day marching past the original goal, the thinking should turn toward
what simplified subset is commit quality. If there's not a scaled down
feature some trimming might extract and get to commit quality--which was
the case with how Sync Rep ended up being committed--that one needs to
close. The couple of weeks over target 9.1 slipped is as bad as we can
let this go now.

I made one big mistake for 2011-11 CF I want to learn how to avoid next
time. When we went past the schedule goal--closing on 12/15--I got most
of the patches closed out. What I should have done at that point is
push toward alpha3 release, even though there were ~5 things still
open. The fact that some patches in that CF are still being tinkered
with shouldn't delay a date-driven alpha drop.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

On Mon, Jan 16, 2012 at 7:01 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> I think this is getting more predictable simply based on having some
> history.  The trail blazing you led here for some time didn't know what was
> and wasn't possible yet.  I feel that the basic shape of things, while still
> fuzzy in spots, is a lot more clear now.
>
> We need to have schedule goals.  There needs to be a date where we switch
> toward a more aggressive "is someone going to commit this soon?" stance on
> things that are still open.  At that point, someone needs to be the person
> not afraid to ramp up pressure toward returning things that just aren't
> going to make it to commit quality.  That's a thankless task that rarely
> leaves anyone involved happy.

There's some value in deciding early that there's no hope for a given
patch, because it frees up resources, of which we do not have an
unlimited supply, to deal with other patches.

I have mixed feelings about the whole 4-weeks vs. 6-weeks thing with
respect to the final CommitFest. If everyone signed up to review as
many patches as they submitted, then on day one of the CommitFest
every patch would have a reviewer, and if anyone who didn't submit
patches signed up as well, some patches would have more than one. In
a week, they'd all have an initial review, and in two weeks they'd all
have had two reviews, and anything that wasn't ready at that point
could be punted while the committers ground through the rest. In
fact, we have a gigantic number of patches that have no reviewer
assigned, and as the CommitFest goes on and on, the number of people
who still have the energy to keep slogging through the pile is going
to steadily diminish. So when we say that we're going to let the
CommitFest go on for 6 weeks, what we're really saying is that we'd
like to reward the few brave souls who will actually keep plugging at
this for 4 weeks by asking them to go another 2 weeks after that. Or
in some cases, what we're saying that we'd like to give patch authors
another two weeks to finish work that should really have been done
before submitting the patch in the first place.

Now, on the flip side, I think that if we get the CommitFest done in 6
weeks, that will be almost as good as getting it done in 4 weeks,
because the last two release cycles I've put huge amounts of energy
into trying to get the release stable enough to release before July
and August roll around and everybody disappears. It didn't work,
either time. If that's not going to happen anyway, then there's not
really much point in getting stressed about another week or two. On
the other hand, speaking only for myself, if I review and/or commit 15
patches in the next month - i.e. three times the number I've
submitted, and note that most of what I've submitted for this
CommitFest is pretty simple stuff - I'm not going to be very
enthusiastic about taking another weeks to pick up 7 or 8 more.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 01/16/2012 08:27 PM, Robert Haas wrote:
> the last two release cycles I've put huge amounts of energy
> into trying to get the release stable enough to release before July
> and August roll around and everybody disappears. It didn't work,
> either time. If that's not going to happen anyway, then there's not
> really much point in getting stressed about another week or two.

Adjusting that expectation is another side to pragmatism based on recent
history I think needs to be acknowledged, but is unlikely to be improved
on. 9.0 shipped on September 20. 9.1 shipped on September 11. If we
say the last CF of each release is unlikely to wrap up before early
March each year, that's 6 months of "settling" time between major
feature freeze and release. So far that seems to result in stable
releases to be proud of, on a predictable enough yearly schedule.
Trying to drop the settling time has been frustrating for you, difficult
to accomplish, and I'm unsure it's even necessary.

Yes, there are some users of PostgreSQL who feel the yearly release
cycle is too slow. As I was saying upthread, I don't see any similarly
complicated projects doing better whose QA hasn't suffered for it. Are
there any examples of serious database software that navigate the new
features vs. low bug count trade-off as well as PostgreSQL, while also
releasing more often?

The one thing that really wasn't acceptable was holding off all new
development during the entire freeze period. Branching 9.2 much
earlier, then adding the June CommitFest last year, seems to have
released a lot of the pressure there. Did it push back the 9.1 release
or drop its quality level? Those two things are not decoupled. I think
we'd need to look at "fixes backported to 9.1 after 9.2 was branched" to
see how much benefit there was to holding off release until September,
instead of the July/August time-frame you were pushing for. Could 9.1
have shipped in July and kept the same quality level? My guess is that
the additional delay had some value for smoking bugs out. Would have to
actually look at the commit history more closely to have an informed
opinion on that.

I find your tone during this thread a bit strange. I see the way you in
particular have pushed on formalizing the CommitFest process the last
few years to be a big success. I've been staring at the approaching
work left on 9.2, finding a successful track record that outlines a game
plan for what's left, even seeing enough data for rough metrics on how
long things should take. That's a huge step forward for everyone
compared to the state of things a few years ago, where the state of the
art was a patch queue in everyone's mailbox, and new submitters had no
idea when they'd get feedback. Your hoping that it was possible to get
releases out in the summer of each year hasn't worked out so far. I
know that was frustrating for you, but I certainly don't see that as a
failure; just something we've now seen enough feedback on to acknowledge
and accept as impractical. If the flood of last minute submissions
right before the freeze submission deadline takes 6 weeks to clear now,
that still seems a whole lot better than what I remember of 8.3 and 8.4
development.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com

On Mon, Jan 16, 2012 at 10:00 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> On 01/16/2012 08:27 PM, Robert Haas wrote:
>> the last two release cycles I've put huge amounts of energy
>> into trying to get the release stable enough to release before July
>> and August roll around and everybody disappears.  It didn't work,
>> either time.  If that's not going to happen anyway, then there's not
>> really much point in getting stressed about another week or two.
>
> Adjusting that expectation is another side to pragmatism based on recent
> history I think needs to be acknowledged, but is unlikely to be improved on.
>  9.0 shipped on September 20.  9.1 shipped on September 11.  If we say the
> last CF of each release is unlikely to wrap up before early March each year,
> that's 6 months of "settling" time between major feature freeze and release.
>  So far that seems to result in stable releases to be proud of, on a
> predictable enough yearly schedule.  Trying to drop the settling time has
> been frustrating for you, difficult to accomplish, and I'm unsure it's even
> necessary.
>
> Yes, there are some users of PostgreSQL who feel the yearly release cycle is
> too slow.  As I was saying upthread, I don't see any similarly complicated
> projects doing better whose QA hasn't suffered for it.  Are there any
> examples of serious database software that navigate the new features vs. low
> bug count trade-off as well as PostgreSQL, while also releasing more often?

Totally agreed, on all of the above.

> The one thing that really wasn't acceptable was holding off all new
> development during the entire freeze period.  Branching 9.2 much earlier,
> then adding the June CommitFest last year, seems to have released a lot of
> the pressure there.

Also agreed.

> Did it push back the 9.1 release or drop its quality
> level?  Those two things are not decoupled.  I think we'd need to look at
> "fixes backported to 9.1 after 9.2 was branched" to see how much benefit
> there was to holding off release until September, instead of the July/August
> time-frame you were pushing for.  Could 9.1 have shipped in July and kept
> the same quality level?  My guess is that the additional delay had some
> value for smoking bugs out.  Would have to actually look at the commit
> history more closely to have an informed opinion on that.

Having looked over the commit history just now and thought about it a
bit, I don't think it either pushed back the 9.1 release or dropped
its quality level. We were still fixing bugs over the summer, and
most of those wouldn't have been found any sooner if we haven't
branched. Actually, I think 9.1 has probably been the most solid
release of the three I've been around long to remember; and maybe the
best feature set, too.

> I find your tone during this thread a bit strange.  I see the way you in
> particular have pushed on formalizing the CommitFest process the last few
> years to be a big success.

Thanks; I appreciate the sentiment.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On mån, 2012-01-16 at 22:00 -0500, Greg Smith wrote:
> Adjusting that expectation is another side to pragmatism based on
> recent history I think needs to be acknowledged, but is unlikely to be
> improved on. 9.0 shipped on September 20. 9.1 shipped on September
> 11. If we say the last CF of each release is unlikely to wrap up
> before early March each year, that's 6 months of "settling" time
> between major feature freeze and release. So far that seems to result
> in stable releases to be proud of, on a predictable enough yearly
> schedule.

Well, has it? I think, it took until version 9.1.2 to have a release
without major issues that you could consider for production. So do we
need 8 or 10 months of settling time? Or should we release earlier,
realizing that we won't get proper testing before the final release
anyway? I don't know.

Another concern is that we are now essentially freezing 9.2 features
with at best about four weeks of production experience and feedback from
9.1. I expect that this will also contribute to dragging out the
finalization of 9.2 once more.

On Tue, Jan 17, 2012 at 3:27 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On mån, 2012-01-16 at 22:00 -0500, Greg Smith wrote:
>> Adjusting that expectation is another side to pragmatism based on
>> recent history I think needs to be acknowledged, but is unlikely to be
>> improved on.  9.0 shipped on September 20.  9.1 shipped on September
>> 11.  If we say the last CF of each release is unlikely to wrap up
>> before early March each year, that's 6 months of "settling" time
>> between major feature freeze and release.  So far that seems to result
>> in stable releases to be proud of, on a predictable enough yearly
>> schedule.
>
> Well, has it?  I think, it took until version 9.1.2 to have a release
> without major issues that you could consider for production.  So do we
> need 8 or 10 months of settling time?  Or should we release earlier,
> realizing that we won't get proper testing before the final release
> anyway?  I don't know.

There's definitely some things that we're not going to catch until
after final release; after a certain point, settling doesn't help
much. But the general pattern for the last few releases is that after
the end of the cycle we've done a sweep for open issues and found many
of them. Early testing also tends to shake out a bunch of bugs in
whatever the new features are. During the 9.0 cycle, it took until
June to clear that backlog; during the 9.1 cycle, it took until
sometime around June-July. In both cases, a large percentage of those
issues were bugs introduced during the final CommitFest, because
that's when nearly all the big features hit. If we froze the tree
today, we likely wouldn't need more than a month to put out a good
beta, but a month from now it'll take four or five months. People
know when the end of the cycle is coming; just look at the number of
patches in the last CommitFest of any given cycle versus the earlier
ones. We've been doing a pretty good job fielding it, but it isn't
perfect.

> Another concern is that we are now essentially freezing 9.2 features
> with at best about four weeks of production experience and feedback from
> 9.1.  I expect that this will also contribute to dragging out the
> finalization of 9.2 once more.

I don't believe that this really matters all that much; a lot of the
things we're fixing have been an issue for years. Even for the
patches that are improving on 9.1 features (e.g. recv and apply sync
rep modes), I don't think this particular consideration is going to
hold things up.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company