Re: WAL format changes

On Monday, June 18, 2012 09:19:48 PM Heikki Linnakangas wrote:
> On 18.06.2012 21:45, Andres Freund wrote:
> > On Monday, June 18, 2012 08:32:54 PM Heikki Linnakangas wrote:
> >> On 18.06.2012 21:13, Andres Freund wrote:
> >>> On Monday, June 18, 2012 08:08:14 PM Heikki Linnakangas wrote:
> >>>> The page header contains an XLogRecPtr (LSN), so if we change it we'll
> >>>> have to deal with pg_upgrade. I guess we could still keep XLogRecPtr
> >>>> around as the on-disk representation, and convert between the 64-bit
> >>>> integer and XLogRecPtr in PageGetLSN/PageSetLSN. I can try that out -
> >>>> many xlog calculations would admittedly be simpler if it was an
> >>>> uint64.
> >>>
> >>> I am out of my depth here, not having read any of the relevant code,
> >>> but couldn't we simply replace the lsn from disk with
> >>> InvalidXLogRecPtr without marking the page dirty?
> >>>
> >>> There is the valid argument that you would loose some information when
> >>> pages with hint bits are written out again, but on the other hand you
> >>> would also gain the information that it was a hint-bit write...
> >>
> >> Sorry, I don't understand that. Where would you "replace the LSN from
> >> disk with InvalidXLogRecPtr" ? (and no, it probably won't work ;-) )
> >
> > In ReadBuffer_common or such, after reading a page from disk and
> > verifying that the page has a valid header it seems to be possible to
> > replace pd_lsn *in memory* by InvalidXLogRecPtr without marking the page
> > dirty.
> > If the page isn't modified the lsn on disk won't be changed so you don't
> > loose debugging information in that case. If will be zero if it has been
> > written by a hint-bit write and thats arguable a win.
>
> We use the LSN to decide whether a full-page image need to be xlogged if
> the page is modified. If you reset LSN every time you read in a page,
> you'll be doing full page writes every time a page is read from disk and
> modified, whether or not it's the first time the page is modified after
> the last checkpoint.
Yea, I somehow disregarded that hint-bit writes would make a problem with full
page writes in that case. Normal writes wouldn't be a problem...
This should be fixable but the result would be too ugly. So its either
converting the on-disk representation or keeping the duplicated
representation.

pd_lsn seems to be well enough abstracted, doing the conversion in
PageSet/GetLSN seems to be easy enough.

Andres
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services