Re: WAL format changes

From: Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Andres Freund <andres(at)2ndquadrant(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: WAL format changes
Date: 2012-06-19 08:14:08
Message-ID: 4FE034D0.5070900@enterprisedb.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 18.06.2012 21:08, Heikki Linnakangas wrote:
> On 18.06.2012 21:00, Robert Haas wrote:
>> On Thu, Jun 14, 2012 at 5:58 PM, Andres Freund<andres(at)2ndquadrant(dot)com>
>> wrote:
>>>> 1. Use a 64-bit segment number, instead of the log/seg combination. And
>>>> don't waste the last segment on each logical 4 GB log file. The concept
>>>> of a "logical log file" is now completely gone. XLogRecPtr is
>>>> unchanged,
>>>> but it should now be understood as a plain 64-bit value, just split
>>>> into
>>>> two 32-bit integers for historical reasons. On disk, this means that
>>>> there will be log files ending in FF, those were skipped before.
>>> Whats the reason for keeping that awkward split now? There aren't
>>> that many
>>> users of xlogid/xcrecoff and many of those would be better served by
>>> using
>>> helper macros.
>>
>> I wondered that, too. There may be a good reason for keeping it split
>> up that way, but we at least oughta think about it a bit.
>
> The page header contains an XLogRecPtr (LSN), so if we change it we'll
> have to deal with pg_upgrade. I guess we could still keep XLogRecPtr
> around as the on-disk representation, and convert between the 64-bit
> integer and XLogRecPtr in PageGetLSN/PageSetLSN. I can try that out -
> many xlog calculations would admittedly be simpler if it was an uint64.

Well, that was easier than I thought. Attached is a patch to make
XLogRecPtr a uint64, on top of my other WAL format patches. I think we
should go ahead with this.

The LSNs on pages are still stored in the old format, to avoid changing
the on-disk format and breaking pg_upgrade. The XLogRecPtrs stored the
control file and WAL are changed, however, so an initdb (or at least
pg_resetxlog) is required.

Should we keep the old representation in the replication protocol
messages? That would make it simpler to write a client that works with
different server versions (like pg_receivexlog). Or, while we're at it,
perhaps we should mandate network-byte order for all the integer and
XLogRecPtr fields in the replication protocol. That would make it easier
to write a client that works across different architectures, in >= 9.3.
The contents of the WAL would of course be architecture-dependent, but
it would be nice if pg_receivexlog and similar tools could nevertheless
be architecture-independent.

I kept the %X/%X representation in error messages etc. I'm quite used to
that output, so reluctant to change it, although it's a bit silly now
that it represents just 64-bit value. Using UINT64_FORMAT would also
make the messages harder to translate.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Attachment Content-Type Size
xlogrecptr-uint64-1.patch text/x-diff 72.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro HORIGUCHI 2012-06-19 08:30:46 Re: Skip checkpoint on promoting from streaming replication
Previous Message Andres Freund 2012-06-19 07:24:06 Re: [PATCH 10/16] Introduce the concept that wal has a 'origin' node