Re: Freezing without write I/O

On 2013-09-16 16:59:28 +0300, Heikki Linnakangas wrote:
> Here's a rebased version of the patch, including the above-mentioned fixes.
> Nothing else new.

* We need some higherlevel description of the algorithm somewhere in the
source. I don't think I've understood the concept from the patch alone
without having read the thread previously.
* why do we need to do the PageUpdateNeedsFreezing() dance in
heap_page_prune? No xids should change during it.
* Why can we do a GetOldestXmin(allDbs = false) in
BeginXidLSNRangeSwitch()?
* Is there any concrete reasoning behind the current values for
XID_LSN_RANGE_INTERVAL and NUM_XID_LSN_RANGES or just gut feeling?
* the lsn ranges file can possibly become bigger than 512bytes (the size
we assume to be written atomically) and you write it inplace. If we
fail halfway through writing, we seem to be able to recover by using
the pageMatureLSN from the last checkpoint, but it seems better to
do the fsync(),rename(),fsync() dance either way.
* Should we preemptively freeze tuples on a page in lazy_scan_heap if we
already have dirtied the page? That would make future modifcations
cheaper.
* lazy_scan_heap now blocks acquiring a cleanup lock on every buffer
that contains dead tuples. Shouldn't we use some kind of cutoff xid
there? That might block progress too heavily. Also the comment above
it still refers to the old logic.
* There's no way to force a full table vacuum anymore, that seems
problematic to me.
* I wonder if CheckPointVarsup() doesn't need to update
minRecoveryPoint. StartupVarsup() should be ok, because we should only
read one from the future during a basebackup?
* xidlsnranges_recently[_dirtied] are not obvious on a first glance. Why
can't we just reset dirty before the WriteXidLSNRangesFile() call?
There's only one process doing the writeout. Just because the
checkpointing process could be killed?
* I think we should either not require consuming an multixactid or use a
function that doesn't need MultiXactIdSetOldestMember(). If the
transaction doing so lives for long it will unnecessarily prevent
truncation of mxacts.
* switchFinishXmin and nextSwitchXid should probably be either volatile
or have a compiler barrier between accessing shared memory and
checking them. The compiler very well could optimize them away and
access shmem all the time which could lead to weird results.
* I wonder whether the fact that we're doing the range switches after
acquiring an xid could be problematic if we're preventing xid
allocation due to the checks earlier in that function?
* I think heap_lock_tuple() needs to unset all-visible, otherwise we
won't vacuum that page again which can lead to problems since we
don't do full-table vacuums again?

So, I think that's enough for a first look. Will think about general
issues a bit more.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services