Re: MMAP Buffers

From: Radosław Smogura <rsmogura(at)softperience(dot)eu>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Peter Eisentraut <peter_e(at)gmx(dot)net>, Greg Stark <gsstark(at)mit(dot)edu>, Greg Smith <greg(at)2ndquadrant(dot)com>, Joshua Berkus <josh(at)agliodbs(dot)com>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>, PG Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: MMAP Buffers
Date: 2011-04-18 05:47:36
Message-ID: 201104180747.36265.rsmogura@softperience.eu
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Robert Haas <robertmhaas(at)gmail(dot)com> Monday 18 April 2011 03:06:17
> On Sun, Apr 17, 2011 at 5:32 PM, Radosław Smogura
>
> <rsmogura(at)softperience(dot)eu> wrote:
> > Each process has simple "mirror" of shared descriptors.
> >
> > I "believe" that modifications to buffer content may be only done when
> > holding exclusive lock (with some simple exceptions) (+ MVCC), actually
> > I saw only two things that can change already loaded data and cause
> > damage, you have described (setting hint bits during scan, and vacuum -
> > 1st may only cause, I think, that two processes will ask for same
> > transaction statuses <except vacuum>, 2nd one is impossible as vacumm
> > requires exclusive pin). When buffer tag is changed the version of
> > buffer is bumped up, and checked against local version - this about
> > reading buffer.
>
> Yes, an exclusive lock is required for substantive content changes.
> But if vacuum cleaning up the buffer is an issue for your patch, then
> it's probably also a problem if someone grabs an exclusive content
> lock and deletes the tuple (by setting XMAX) and some other backend
> later sees the old buffer contents after having in the meanwhile taken
> a new snapshot; or if likewise someone grabs an exclusive-lock, adds a
> tuple, and then your backend takes a new snapshot and then sees the
> old buffer contents. Basically, any time someone grabs an
> exclusive-lock and releases it, it's necessary for all observers to
> see the updated contents by the time the exclusive lock is released.
> > In other cases after obtaining lock check is done if buffer has
> > associated updatable buffer and if local "mirror" has it too, then swap
> > should take place.
>
> I think this check would have to be done every time someone
> share-locks the buffer, which seems rather expensive.
I don't treat as issues, but it's disadvantage.

> > Logic about updatable buffers is similar to "shared buffers", each
> > updatable buffer has pin count, and updatable buffer can't be free if
> > someone uses it, but in contrast to "normal buffers", updatable buffers
> > doesn't have any support for locking etc. Updatable buffers exists only
> > on free list, or when associated with buffer.
>
> I don't see how you're going to get away with removing buffer locks.
> They exist for a reason, and adding mmap() to the mix is going to
> require MORE locking, not less.
>
> > In future, I will change version to shared segment id, something like
> > relation's oid + block, but ids will have continuous numbering 1,2,3...,
> > so I will be able to bypass smgr/md during read, and tag version check -
> > this looks like faster solution.
>
> I don't understand this part at all.
Versioning is witch approach where I thought about really often changes of
mmaped areas, I allocated part of segments, but now the segment is mmaped with
reservation, to it's full possible size, addresses of segments can't change
(problem is only with segment deletion).

Regards,
Radek

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2011-04-18 05:50:22 Re: Formatting Curmudgeons WAS: MMAP Buffers
Previous Message Tom Lane 2011-04-18 05:43:50 Re: Formatting Curmudgeons WAS: MMAP Buffers