Re: CommitFest 2011-11 Update

All of the information on the CommitFest app is as accurate as I could
make it now, I made a pass over every open thread there to look for
major changes that hadn't gotten message ID archive links. Since the
official start 13 patches have been committed and 6 bounced out.
Reminder notes have gone out to most of the people who there's something
waiting for, mostly off-list nagging, and several people have already
gotten back to say they're planning to hash out open issues over this
weekend.

Of the 34 patches still waiting for review or their author, there's a
couple of the usual suspects responsible for a good chunk of them.

Greg Smith: 10 patches that troublemaker is meddling with in some
form. I'm clearly not going anywhere this upcoming week, as I've had
"panic over closing of CommitFest in the middle of December" on my
calendar for a while. General plan of attack is:

-Try and break the deadlock over what to do with
pg_last_xact_insert_timestamp (done with that for now)
-Update "Configuration include directory" to fix all of Noah's suggestions
-Make a similar pass over "includeifexists in configuration file", which
is a little cut and paste from the first one and may have some of the
same errors.
-Revisit the "unite recovery.conf and postgresql.conf" from a new
perspective that presumes those two features are coming. I suspect we
can implement some of the "what I'd like is this" requests people wanted
here with these features, to chop away enough edge cases to make this
more tractable.
-Mixed with the bigger above bits, during smaller chunks of time help
rework "pg_terminate_backend and pg_cancel_backend by not administrator
user", "Measuring relation free space", and "Separate pg_stat_activity
into current_query into state and query columns" features to commit
quality.
-Resume slogging through the controversy around "Core extensions
relocation" and see if that goes anywhere.
-Join in on any benchmarking help I can provide for some of Robert's
patches, starting with "Make pgBufferUsage track dirty buffers"
-Continued work with Peter G on pg_stat_statements normalization. I
consider a major feature in the "Performance Release" theme 9.2 has
taken on.

There's a normal sized plate lined up for Robert since he's been keeping
up better; in no particular order:

-More review on the always a new surprise "Fix Leaky Views Problem, again"
-Extra fun with "Tuplesort comparison overhead reduction"
-His own "avoid taking two snapshots per query" and "FlexLocks" improvements

And then Dimitri is on:

-His "Command Triggers"
-Review on "Prep object creation hooks"
-Another likely pass over Robert's "avoid taking two snapshots per query"

Other people in similarly loaded and overlapping lists with the above
include Fujii Masao and KaiGai Kohei. So about half of the open items
are from contributors who have a track record of just coming back for
more every time. It's not like we're going to wonder off based on
whether our submission goes in now or we have to keep chugging along. I
think most of the above is achievable in 9.2.

What I'm happy about is that I'm not seeing any giant controversial
things here, the sort that tend to hit the last CF and then push its
boundary back too. Last year at this time, patches on the table at this
point were things like Extensions, Sync Rep, Per-column collation, and
KNN-gist. Those had the bad mix of "we want this feature for the
relase" and "this is hard". That pushed some of them into early March
before they got committed. I think only Dimitri's Command Triggers has
that sort of potential this time. And I've been talking with him enough
about the plan there to feel it chunks into useful pieces pretty well if
the target feature set has to scale back. We'll have to see if anyone
tries to sneak one of the more complicated things into the final CF.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us

On Sat, Dec 10, 2011 at 1:07 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> There's a normal sized plate lined up for Robert since he's been keeping up
> better; in no particular order:
>
> -More review on the always a new surprise "Fix Leaky Views Problem, again"
> -Extra fun with "Tuplesort comparison overhead reduction"
> -His own "avoid taking two snapshots per query" and "FlexLocks" improvements

Greg,

Thanks for working through this. I see I still need to look more at
the leaky views stuff. I think that the tuplesort comparison overhead
reduction is in Peter G's hands to rebase at the moment; I will look
at that again when it reemerges. I think "avoid two snapshots per
query" is pretty much ready to go, though I'm vaguely waiting for any
further input from Dimitri. I believe I'm going to punt on FlexLocks
for now, for the reasons set out in the email I just sent on that
topic, and accordingly am marking that patch with feedback.

One more time to make sure I made my main point: thanks for working on
this. :-)

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 15 December 2011 19:07, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> I think that the tuplesort comparison overhead
> reduction is in Peter G's hands to rebase at the moment; I will look
> at that again when it reemerges.

Yes, sorry about that - I was a little bit sidetracked by something
else. I'll work on it tomorrow.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

It's time for another one of these again. We're now just past the
nominal 1 month goal of each CommitFest, and as you can obviously see
from my list traffic I'm trying to close things up.

One of the CF goals is to give everyone who submits something a fair
review. There are a few patches that haven't gotten any full review yet.

"Collect frequency statistics and selectivity estimation for arrays":
this area is tough to find help on. I've been in touch with Nathan
Boley recently and he does intend to get to this one soon. Not chasing
this down earlier is my bad. If anyone else is interested in this area,
please let me know. I hope we get something soon from Nathan, but would
like to have a backup plan in case he continues to be too busy to work
on it.

Neither of the two pg_stat_statements enhancement patches have gotten a
review yet. The one Peter G has been working on has gotten regular
updates during the CF but hasn't been reviewed yet. Daniel Farina has
only had a few days where it was in a state he could do that against the
last update, and there's been of system updates at Heroku keeping away
during most of that. This I intend to keep nudging forward myself.

A fourth patch is better but not treated well yet:

"Allow substitute allocator for PGresult": Due to some mishandling on
my side this didn't make it though a second pass of review yet, so it's
state is a bit unknown still.

Then there are also another 5 patches that have been updated recently
enough they might be ready to commit, but we're still looking for a
final review on them first. They are:

"pgsql_fdw contrib module" and the related "Join push-down for foreign
tables". I don't have a good feel for the order these two need to be
applied in, or exactly where they stand right now.

"Fix Leaky Views Problem, again": Robert is planning to look at this more

"avoid taking two snapshots per query": Dimitri is done with this one
now. I think that moves this toward Ready for Committer, which in
theory Robert could do himself now. Playing around with this area seems
complicated enough that it would be nice to have another committer look
at this though.

"splitting plpython into smaller parts": This was still being worked on
by Peter E as of yesterday, so I'm assuming it's still viable to commit
soon.

On a positive note, I seem to have unstuck two of the long running
arguments this week. "unite recovery.conf and postgresql.conf" has a
potential path forward again. I don't think we can just forget about
executing on that until the last minute before the final CommitFest.
Nailing down who is working on each of those moving parts is something
we should do, but that seems unlikely to move until the other CF work is
closed up though.

pg_last_xact_insert_timestamp thread has taken an odd turn from ready to
commit back to receiving a set of alternate proposals. The "primary
keepalive" patch and "archive_keepalive_command" design proposal from
Simon pair seem to add up to something that addresses the concerns I
raised about computations from a single system and differences between
transaction and WAL timing. I'm finding myself on the hook for this
area, so I'll try myself to help keep it from stalling again.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us