Re: GSoC on WAL-logging hash indexes

From: Greg Stark <stark(at)mit(dot)edu>
To: Peter Geoghegan <pg(at)heroku(dot)com>
Cc: Jeff Janes <jeff(dot)janes(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Tan Tran <tankimtran(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: GSoC on WAL-logging hash indexes
Date: 2014-04-30 19:16:53
Message-ID: CAM-w4HPkFEWCXEcJ_=yhdceedGvRG_vS=mqsKcK+YBLyStQ8wQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-advocacy pgsql-hackers pgsql-students

I think the key question was if someone wanted to develop a good hash
index would they start from our current hash index or would they be
better off starting from a fresh codebase? If the former then we
should add WAL logging and throw GSOC and other people who ask for
projects at it. If the latter then we should throw out the current
codebase and let people develop extensions that add new hash index
code until someone comes up with a good design we want to move to
core.

But imnsho doing nothing is a bad idea. We should have long ago either
added WAL logging or removed the index type. We shouldn't have left a
foot-gun that large lying around for so long.

Incidentally something else I've considered would be having a WAL
record type saying "relation corrupted" and emitting one the first
time a hash index is touched after a checkpoint. That could provide a
general mechanism that might be useful for unlogged operations (and
might be combinable with the infrastructure for unlogged tables). But
it seems like a better tool for other objects than hash indexes.

Another quick fix would be having a GUC allow_unrecoverable_relations
which defaulted to false. Creating a hash index (and presumably
unlogged tables) would error out with a hint to set that to true so at
least users who create them would have to know what they were in for.
Right now we're seeing a lot of users who create hash indexes and then
complain about corrupted standbys.

I could do the legwork on either the GUC or moving hash indexes to an
extension if there's a consensus. But I suspect either will be quite
controversial...

In response to

Responses

Browse pgsql-advocacy by date

  From Date Subject
Next Message Stephen Frost 2014-04-30 19:29:34 Re: GSoC on WAL-logging hash indexes
Previous Message Peter Geoghegan 2014-04-30 18:19:33 Re: GSoC on WAL-logging hash indexes

Browse pgsql-hackers by date

  From Date Subject
Next Message Greg Stark 2014-04-30 19:23:59 Re: pg_get_viewdefs() indentation considered harmful
Previous Message Andrew Dunstan 2014-04-30 19:03:06 Re: Fix initdb for path with whitespace and at char

Browse pgsql-students by date

  From Date Subject
Next Message Stephen Frost 2014-04-30 19:29:34 Re: GSoC on WAL-logging hash indexes
Previous Message Peter Geoghegan 2014-04-30 18:19:33 Re: GSoC on WAL-logging hash indexes