Re: Proposal for CSN based snapshots

Andres Freund wrote:
> On 2014-05-15 15:40:06 -0400, Robert Haas wrote:
> > On Thu, May 15, 2014 at 2:34 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:

> > If the larger clog size is a show-stopper (and I'm not sure I have an
> > intelligent opinion on that just yet), one way to get around the
> > problem would be to summarize CLOG entries after-the-fact. Once an
> > XID precedes the xmin of every snapshot, we don't need to know the
> > commit LSN any more. So we could read the old pg_clog files and write
> > new summary files. Since we don't need to care about subcommitted
> > transactions either, we could get by with just 1 bit per transaction,
> > 1 = committed, 0 = aborted. Once we've written and fsync'd the
> > summary files, we could throw away the original files. That might
> > leave us with a smaller pg_clog than what we have today.
>
> I think the easiest way for now would be to have pg_clog with the same
> format as today and a rangewise much smaller pg_csn storing the lsns
> that are needed. That'll leave us with pg_upgrade'ability without
> needing to rewrite pg_clog during the upgrade.

Err, we're proposing a patch to add timestamps to each commit,
http://www.postgresql.org/message-id/20131022221600.GE4987@eldon.alvh.no-ip.org
which does so in precisely this way.

The idea that pg_csn or pg_committs can be truncated much earlier than
pg_clog has its merit, no doubt. If we can make sure that the atomicity
is sane, +1 from me.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services