Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...

Lists: pgsql-hackers
From: Sean Chittenden <sean(at)chittenden(dot)org>
To: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-08-12 16:42:30
Message-ID: sig.030123e089.53EA43F6.2040108@chittenden.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

One of the patches that I've been sitting on and am derelict in punting
upstream is the attached mmap(2) flags patch for the BSDs. Is there any
chance this can be squeezed in to the PostreSQL 9.4 release?

The patch is trivial in size and is used to add one flag to mmap(2)
calls in dsm_impl.c. Alan Cox (FreeBSD alc, not Linux) and I went back
and forth regarding PostgreSQL's use of mmap(2) and determined that the
following is correct and will prevent a likely performance regression in
PostgreSQL 9.4. In PostgreSQL 9.3, all mmap(2) calls were called with
the flags MAP_ANON | MAP_SHARED, whereas in PostgreSQL 9.4 this is not
the case.

Digging in to the patch, in reviewing
src/backend/storage/ipc/dsm_impl.c, it's clear that rhaas@ understood
the consequences of mmap(2), and the possible consequences of having
dirty pages gratuitously flushed to disk:

src/backend/storage/ipc/dsm_impl.c:781
* Operating system primitives to support mmap-based shared memory.
*
* Calling this "shared memory" is somewhat of a misnomer, because what
* we're really doing is creating a bunch of files and mapping them into
* our address space. The operating system may feel obliged to
* synchronize the contents to disk even if nothing is being paged out,
* which will not serve us well. The user can relocate the pg_dynshmem
* directory to a ramdisk to avoid this problem, if available.

In order for the above comment to be true for FreeBSD, an extra flag
needs to be passed to mmap(2). From FreeBSD 10's mmap(2) page[2]:

MAP_NOSYNC Causes data dirtied via this VM map to be
flushed to
physical media only when necessary (usually by the
pager) rather than gratuitously. Typically
this pre-
vents the update daemons from flushing pages
dirtied
through such maps and thus allows efficient
sharing of
memory across unassociated processes using a file-
backed shared memory map. Without this option
any VM
pages you dirty may be flushed to disk every so
often
(every 30-60 seconds usually) which can create
perfor-
mance problems if you do not need that to occur
(such
as when you are using shared file-backed mmap
regions
for IPC purposes). Note that VM/file system
coherency
is maintained whether you use MAP_NOSYNC or
not. This
option is not portable across UNIX platforms (yet),
though some may implement the same behavior by
default.

WARNING! Extending a file with ftruncate(2), thus
creating a big hole, and then filling the hole
by mod-
ifying a shared mmap() can lead to severe file
frag-
mentation. In order to avoid such
fragmentation you
should always pre-allocate the file's backing
store by
write()ing zero's into the newly extended area
prior
to modifying the area via your mmap(). The
fragmenta-
tion problem is especially sensitive to MAP_NOSYNC
pages, because pages may be flushed to disk in a
totally random order.

The same applies when using MAP_NOSYNC to
implement a
file-based shared memory store. It is recommended
that you create the backing store by write()ing
zero's
to the backing file rather than ftruncate()ing it.
You can test file fragmentation by observing
the KB/t
(kilobytes per transfer) results from an
``iostat 1''
while reading a large file sequentially, e.g. using
``dd if=filename of=/dev/null bs=32k''.

The fsync(2) system call will flush all dirty
data and
metadata associated with a file, including dirty
NOSYNC VM data, to physical media. The sync(8)
com-
mand and sync(2) system call generally do not flush
dirty NOSYNC VM data. The msync(2) system call is
usually not needed since BSD implements a coherent
file system buffer cache. However, it may be
used to
associate dirty VM pages with file system
buffers and
thus cause them to be flushed to physical media
sooner
rather than later.

The man page for madvise(2) has more pointed advise[3]:

MADV_NOSYNC Request that the system not flush the data associated
with this map to physical backing store unless it
needs
to. Typically this prevents the file system
update dae-
mon from gratuitously writing pages dirtied by the VM
system to physical disk. Note that VM/file system
coherency is always maintained, this feature simply
ensures that the mapped data is only flush when
it needs
to be, usually by the system pager.

This feature is typically used when you want to use a
file-backed shared memory area to communicate between
processes (IPC) and do not particularly need the data
being stored in that area to be physically written to
disk. With this feature you get the equivalent
perfor-
mance with mmap that you would expect to get with
SysV
shared memory calls, but in a more controllable
and less
restrictive manner. However, note that this
feature is
not portable across UNIX platforms (though some
may do
the right thing by default). For more
information see
the MAP_NOSYNC section of mmap(2)

Anyway, could you give this a quick review and apply the patch in time
so the build farm can get a full build completed before the release?

Thanks in advance. -sc

[1] https://kib.kiev.ua/kib/pgsql_perf.pdf
[2]
http://www.freebsd.org/cgi/man.cgi?query=mmap&apropos=0&sektion=0&manpath=FreeBSD+10.0-stable&arch=default&format=html
[3]
http://www.freebsd.org/cgi/man.cgi?query=madvise&sektion=2&apropos=0&manpath=FreeBSD+10.0-stable

--
Sean Chittenden

Attachment Content-Type Size
postgresql-94-mmap.patch text/plain 1.4 KB

From: Andres Freund <andres(at)2ndquadrant(dot)com>
To: Sean Chittenden <sean(at)chittenden(dot)org>
Cc: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-08-12 16:59:49
Message-ID: 20140812165949.GA26489@awork2.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 2014-08-12 09:42:30 -0700, Sean Chittenden wrote:
> One of the patches that I've been sitting on and am derelict in punting
> upstream is the attached mmap(2) flags patch for the BSDs. Is there any
> chance this can be squeezed in to the PostreSQL 9.4 release?
>
> The patch is trivial in size and is used to add one flag to mmap(2) calls in
> dsm_impl.c. Alan Cox (FreeBSD alc, not Linux) and I went back and forth
> regarding PostgreSQL's use of mmap(2) and determined that the following is
> correct and will prevent a likely performance regression in PostgreSQL 9.4.
> In PostgreSQL 9.3, all mmap(2) calls were called with the flags MAP_ANON |
> MAP_SHARED, whereas in PostgreSQL 9.4 this is not the case.

The performancewise important call to mmap will still use that set of
flags, no? That's the one backing shared_buffers.

The mmap backend for *dynamic* shared memory (aka dsm) is *NOT* supposed
to be used on common platforms. Both posix and sysv shared memory will
be used before falling back to the mmap() backend.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Andres Freund <andres(at)2ndquadrant(dot)com>
Cc: Sean Chittenden <sean(at)chittenden(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-08-12 23:08:06
Message-ID: CA+TgmoZfS46N7X7Bu_keO0Vv06N7vkS8CbMpfsirZ4-J84RiYQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Tue, Aug 12, 2014 at 12:59 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2014-08-12 09:42:30 -0700, Sean Chittenden wrote:
>> One of the patches that I've been sitting on and am derelict in punting
>> upstream is the attached mmap(2) flags patch for the BSDs. Is there any
>> chance this can be squeezed in to the PostreSQL 9.4 release?
>>
>> The patch is trivial in size and is used to add one flag to mmap(2) calls in
>> dsm_impl.c. Alan Cox (FreeBSD alc, not Linux) and I went back and forth
>> regarding PostgreSQL's use of mmap(2) and determined that the following is
>> correct and will prevent a likely performance regression in PostgreSQL 9.4.
>> In PostgreSQL 9.3, all mmap(2) calls were called with the flags MAP_ANON |
>> MAP_SHARED, whereas in PostgreSQL 9.4 this is not the case.
>
> The performancewise important call to mmap will still use that set of
> flags, no? That's the one backing shared_buffers.
>
> The mmap backend for *dynamic* shared memory (aka dsm) is *NOT* supposed
> to be used on common platforms. Both posix and sysv shared memory will
> be used before falling back to the mmap() backend.

Hmm, yeah. This might still be a good thing to do (because what do we
lose?) but it shouldn't really be an issue in practice.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Andres Freund <andres(at)2ndquadrant(dot)com>, Sean Chittenden <sean(at)chittenden(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-10-12 00:33:57
Message-ID: 20141012003357.GS21267@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Tue, Aug 12, 2014 at 07:08:06PM -0400, Robert Haas wrote:
> On Tue, Aug 12, 2014 at 12:59 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > On 2014-08-12 09:42:30 -0700, Sean Chittenden wrote:
> >> One of the patches that I've been sitting on and am derelict in punting
> >> upstream is the attached mmap(2) flags patch for the BSDs. Is there any
> >> chance this can be squeezed in to the PostreSQL 9.4 release?
> >>
> >> The patch is trivial in size and is used to add one flag to mmap(2) calls in
> >> dsm_impl.c. Alan Cox (FreeBSD alc, not Linux) and I went back and forth
> >> regarding PostgreSQL's use of mmap(2) and determined that the following is
> >> correct and will prevent a likely performance regression in PostgreSQL 9.4.
> >> In PostgreSQL 9.3, all mmap(2) calls were called with the flags MAP_ANON |
> >> MAP_SHARED, whereas in PostgreSQL 9.4 this is not the case.
> >
> > The performancewise important call to mmap will still use that set of
> > flags, no? That's the one backing shared_buffers.
> >
> > The mmap backend for *dynamic* shared memory (aka dsm) is *NOT* supposed
> > to be used on common platforms. Both posix and sysv shared memory will
> > be used before falling back to the mmap() backend.
>
> Hmm, yeah. This might still be a good thing to do (because what do we
> lose?) but it shouldn't really be an issue in practice.

Is there a reason this was not applied?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +


From: Andres Freund <andres(at)2ndquadrant(dot)com>
To: Bruce Momjian <bruce(at)momjian(dot)us>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Sean Chittenden <sean(at)chittenden(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-10-12 09:36:53
Message-ID: 20141012093653.GI18020@awork2.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 2014-10-11 20:33:57 -0400, Bruce Momjian wrote:
> On Tue, Aug 12, 2014 at 07:08:06PM -0400, Robert Haas wrote:
> > On Tue, Aug 12, 2014 at 12:59 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > > On 2014-08-12 09:42:30 -0700, Sean Chittenden wrote:
> > >> One of the patches that I've been sitting on and am derelict in punting
> > >> upstream is the attached mmap(2) flags patch for the BSDs. Is there any
> > >> chance this can be squeezed in to the PostreSQL 9.4 release?
> > >>
> > >> The patch is trivial in size and is used to add one flag to mmap(2) calls in
> > >> dsm_impl.c. Alan Cox (FreeBSD alc, not Linux) and I went back and forth
> > >> regarding PostgreSQL's use of mmap(2) and determined that the following is
> > >> correct and will prevent a likely performance regression in PostgreSQL 9.4.
> > >> In PostgreSQL 9.3, all mmap(2) calls were called with the flags MAP_ANON |
> > >> MAP_SHARED, whereas in PostgreSQL 9.4 this is not the case.
> > >
> > > The performancewise important call to mmap will still use that set of
> > > flags, no? That's the one backing shared_buffers.
> > >
> > > The mmap backend for *dynamic* shared memory (aka dsm) is *NOT* supposed
> > > to be used on common platforms. Both posix and sysv shared memory will
> > > be used before falling back to the mmap() backend.
> >
> > Hmm, yeah. This might still be a good thing to do (because what do we
> > lose?) but it shouldn't really be an issue in practice.
>
> Is there a reason this was not applied?

IIRC, as pointed out above, it's primarily based on a misunderstanding
about when mmap is used for in dsm. I.e. that it's essentially just a
fallback/toy implementation and that posix or sysv should rather be
used.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Andres Freund <andres(at)2ndquadrant(dot)com>
Cc: Bruce Momjian <bruce(at)momjian(dot)us>, Sean Chittenden <sean(at)chittenden(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-10-13 14:15:29
Message-ID: CA+TgmoagNfK0Pa+pDNXAhBoYeBLqUBcEQ7GqoJ0eyZ9rUTG+QQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Sun, Oct 12, 2014 at 5:36 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2014-10-11 20:33:57 -0400, Bruce Momjian wrote:
>> On Tue, Aug 12, 2014 at 07:08:06PM -0400, Robert Haas wrote:
>> > On Tue, Aug 12, 2014 at 12:59 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> > > On 2014-08-12 09:42:30 -0700, Sean Chittenden wrote:
>> > >> One of the patches that I've been sitting on and am derelict in punting
>> > >> upstream is the attached mmap(2) flags patch for the BSDs. Is there any
>> > >> chance this can be squeezed in to the PostreSQL 9.4 release?
>> > >>
>> > >> The patch is trivial in size and is used to add one flag to mmap(2) calls in
>> > >> dsm_impl.c. Alan Cox (FreeBSD alc, not Linux) and I went back and forth
>> > >> regarding PostgreSQL's use of mmap(2) and determined that the following is
>> > >> correct and will prevent a likely performance regression in PostgreSQL 9.4.
>> > >> In PostgreSQL 9.3, all mmap(2) calls were called with the flags MAP_ANON |
>> > >> MAP_SHARED, whereas in PostgreSQL 9.4 this is not the case.
>> > >
>> > > The performancewise important call to mmap will still use that set of
>> > > flags, no? That's the one backing shared_buffers.
>> > >
>> > > The mmap backend for *dynamic* shared memory (aka dsm) is *NOT* supposed
>> > > to be used on common platforms. Both posix and sysv shared memory will
>> > > be used before falling back to the mmap() backend.
>> >
>> > Hmm, yeah. This might still be a good thing to do (because what do we
>> > lose?) but it shouldn't really be an issue in practice.
>>
>> Is there a reason this was not applied?
>
> IIRC, as pointed out above, it's primarily based on a misunderstanding
> about when mmap is used for in dsm. I.e. that it's essentially just a
> fallback/toy implementation and that posix or sysv should rather be
> used.

Perhaps, but I still see no reason not to apply it. It may not help
many people, but it won't hurt anything, either. So why not?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: Andres Freund <andres(at)2ndquadrant(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Bruce Momjian <bruce(at)momjian(dot)us>, Sean Chittenden <sean(at)chittenden(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-10-13 14:19:39
Message-ID: 20141013141939.GL18020@awork2.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 2014-10-13 10:15:29 -0400, Robert Haas wrote:
> On Sun, Oct 12, 2014 at 5:36 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > IIRC, as pointed out above, it's primarily based on a misunderstanding
> > about when mmap is used for in dsm. I.e. that it's essentially just a
> > fallback/toy implementation and that posix or sysv should rather be
> > used.
>
> Perhaps, but I still see no reason not to apply it. It may not help
> many people, but it won't hurt anything, either. So why not?

More complicated, less tested code. For no practial benefit, it'll still
be slower than posix shm if there's any memmory pressure. But if you
want to apply it, go ahead, I won't cry louder than this email.

I still think the mmap dsm implementation is a bad idea. We shouldn't
put additional effort into it. If anything we should remove it.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


From: Sean Chittenden <sean(at)chittenden(dot)org>
To: Andres Freund <andres(at)2ndquadrant(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-10-13 14:49:44
Message-ID: sig.0363a724bd.098A57F3-AE53-4597-AA69-34C3AC5E356A@chittenden.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

>> Perhaps, but I still see no reason not to apply it. It may not help
>> many people, but it won't hurt anything, either. So why not?
>
> More complicated, less tested code. For no practial benefit, it'll still
> be slower than posix shm if there's any memmory pressure. But if you
> want to apply it, go ahead, I won't cry louder than this email.
>
> I still think the mmap dsm implementation is a bad idea. We shouldn't
> put additional effort into it. If anything we should remove it.

While you're not wrong in that use of mmap(2) here is potentially a bad idea, much of that is mitigated through the correct use of flags to mmap(2) (i.e. prevent mmap(2) pages from hooking in to the syncer). In the same breath, it would also be nice if the following were committed:

> --- src/template/freebsd.orig 2014-05-26 23:54:53.854165855 +0300
> +++ src/template/freebsd 2014-05-26 23:55:12.307880900 +0300
> @@ -3,3 +3,4 @@
> case $host_cpu in
> alpha*) CFLAGS="-O";; # alpha has problems with -O2
> esac
> +USE_NAMED_POSIX_SEMAPHORES=1

-sc

--
Sean Chittenden
sean(at)chittenden(dot)org


From: Andres Freund <andres(at)2ndquadrant(dot)com>
To: Sean Chittenden <sean(at)chittenden(dot)org>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-10-13 14:55:16
Message-ID: 20141013145516.GM18020@awork2.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 2014-10-13 07:49:44 -0700, Sean Chittenden wrote:
> >> Perhaps, but I still see no reason not to apply it. It may not help
> >> many people, but it won't hurt anything, either. So why not?
> >
> > More complicated, less tested code. For no practial benefit, it'll still
> > be slower than posix shm if there's any memmory pressure. But if you
> > want to apply it, go ahead, I won't cry louder than this email.
> >
> > I still think the mmap dsm implementation is a bad idea. We shouldn't
> > put additional effort into it. If anything we should remove it.
>
> While you're not wrong in that use of mmap(2) here is potentially a
> bad idea, much of that is mitigated through the correct use of flags
> to mmap(2) (i.e. prevent mmap(2) pages from hooking in to the syncer).
> In the same breath, it would also be nice if the following were
> committed:

Unless I'm mistaken the pages will still be written back to disk (and
not just swap, the actual backing file) if there's memory pressure, no?

> > --- src/template/freebsd.orig 2014-05-26 23:54:53.854165855 +0300
> > +++ src/template/freebsd 2014-05-26 23:55:12.307880900 +0300
> > @@ -3,3 +3,4 @@
> > case $host_cpu in
> > alpha*) CFLAGS="-O";; # alpha has problems with -O2
> > esac
> > +USE_NAMED_POSIX_SEMAPHORES=1

If so, that should be a separate change. But why?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Sean Chittenden <sean(at)chittenden(dot)org>
Cc: Andres Freund <andres(at)2ndquadrant(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-10-13 15:01:27
Message-ID: 477.1413212487@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Sean Chittenden <sean(at)chittenden(dot)org> writes:
> In the same breath, it would also be nice if the following were committed:
> [ use named POSIX semaphores on FreeBSD ]

Really? Why? According to the notes in our code, named POSIX semaphores
are the least attractive of the three Unixoid semaphore APIs we support,
because they require eating a file descriptor per backend per
max_connection slot. That's a lot of FDs in any large configuration.
FreeBSD's support for SysV semaphores would have to be pretty darn awful
to make me think this was a good change, and I've not heard complaints
in that direction before.

If you meant to propose using *unnamed* POSIX semaphores, that might be
a reasonable change, but it would still need some supporting evidence.

regards, tom lane


From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Andres Freund <andres(at)2ndquadrant(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Sean Chittenden <sean(at)chittenden(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-10-13 15:18:26
Message-ID: 20141013151826.GW21267@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Mon, Oct 13, 2014 at 04:19:39PM +0200, Andres Freund wrote:
> On 2014-10-13 10:15:29 -0400, Robert Haas wrote:
> > On Sun, Oct 12, 2014 at 5:36 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > > IIRC, as pointed out above, it's primarily based on a misunderstanding
> > > about when mmap is used for in dsm. I.e. that it's essentially just a
> > > fallback/toy implementation and that posix or sysv should rather be
> > > used.
> >
> > Perhaps, but I still see no reason not to apply it. It may not help
> > many people, but it won't hurt anything, either. So why not?
>
> More complicated, less tested code. For no practical benefit, it'll still
> be slower than posix shm if there's any memmory pressure. But if you
> want to apply it, go ahead, I won't cry louder than this email.
>
> I still think the mmap dsm implementation is a bad idea. We shouldn't
> put additional effort into it. If anything we should remove it.

If we have it, we should improve it, or remove it. We might want to use
this code for something else in the future, so it should be improved
where feasible.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +


From: Andres Freund <andres(at)2ndquadrant(dot)com>
To: Bruce Momjian <bruce(at)momjian(dot)us>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Sean Chittenden <sean(at)chittenden(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-10-13 15:21:32
Message-ID: 20141013152132.GC9267@awork2.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 2014-10-13 11:18:26 -0400, Bruce Momjian wrote:
> On Mon, Oct 13, 2014 at 04:19:39PM +0200, Andres Freund wrote:
> > On 2014-10-13 10:15:29 -0400, Robert Haas wrote:
> > > On Sun, Oct 12, 2014 at 5:36 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > > > IIRC, as pointed out above, it's primarily based on a misunderstanding
> > > > about when mmap is used for in dsm. I.e. that it's essentially just a
> > > > fallback/toy implementation and that posix or sysv should rather be
> > > > used.
> > >
> > > Perhaps, but I still see no reason not to apply it. It may not help
> > > many people, but it won't hurt anything, either. So why not?
> >
> > More complicated, less tested code. For no practical benefit, it'll still
> > be slower than posix shm if there's any memmory pressure. But if you
> > want to apply it, go ahead, I won't cry louder than this email.
> >
> > I still think the mmap dsm implementation is a bad idea. We shouldn't
> > put additional effort into it. If anything we should remove it.
>
> If we have it, we should improve it, or remove it. We might want to use
> this code for something else in the future, so it should be improved
> where feasible.

Meh. We don't put in effort into code that doesn't matter just because
it might get used elsewhere some day. By that argument we'd need to
performance optimize a lot of code. And actually, using that code
somewhere else is more of a counter indication than a pro
argument. MAP_NOSYNC isn't a general purpose flag.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Andres Freund <andres(at)2ndquadrant(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Sean Chittenden <sean(at)chittenden(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-10-13 15:35:18
Message-ID: 20141013153518.GY21267@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Mon, Oct 13, 2014 at 05:21:32PM +0200, Andres Freund wrote:
> > If we have it, we should improve it, or remove it. We might want to use
> > this code for something else in the future, so it should be improved
> > where feasible.
>
> Meh. We don't put in effort into code that doesn't matter just because
> it might get used elsewhere some day. By that argument we'd need to
> performance optimize a lot of code. And actually, using that code
> somewhere else is more of a counter indication than a pro
> argument. MAP_NOSYNC isn't a general purpose flag.

The key is that this is platform-specific behavior, so if we should use
a flag to use it right, we should. You are right that optimizing
rarely used code with generic calls isn't a good use of time.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +


From: Sean Chittenden <sean(at)chittenden(dot)org>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Andres Freund <andres(at)2ndquadrant(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2014-10-14 05:28:16
Message-ID: sig.0364cdbb18.8309646D-CD50-46AB-926E-73E88FAA5945@chittenden.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

> Really? Why?

Because it was found to be lighter weight. See §5 (bottom of ppg 8).

https://kib.kiev.ua/kib/pgsql_perf.pdf

> According to the notes in our code, named POSIX semaphores
> are the least attractive of the three Unixoid semaphore APIs we support,
> because they require eating a file descriptor per backend per
> max_connection slot. That's a lot of FDs in any large configuration.
> FreeBSD's support for SysV semaphores would have to be pretty darn awful
> to make me think this was a good change, and I've not heard complaints
> in that direction before.
>
> If you meant to propose using *unnamed* POSIX semaphores, that might be
> a reasonable change, but it would still need some supporting evidence.

https://lists.freebsd.org/pipermail/svn-src-stable-10/2014-October/003515.html

-sc

--
Sean Chittenden
sean(at)chittenden(dot)org


From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Andres Freund <andres(at)2ndquadrant(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Sean Chittenden <sean(at)chittenden(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2015-03-19 16:16:07
Message-ID: 20150319161607.GE6061@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Mon, Oct 13, 2014 at 11:35:18AM -0400, Bruce Momjian wrote:
> On Mon, Oct 13, 2014 at 05:21:32PM +0200, Andres Freund wrote:
> > > If we have it, we should improve it, or remove it. We might want to use
> > > this code for something else in the future, so it should be improved
> > > where feasible.
> >
> > Meh. We don't put in effort into code that doesn't matter just because
> > it might get used elsewhere some day. By that argument we'd need to
> > performance optimize a lot of code. And actually, using that code
> > somewhere else is more of a counter indication than a pro
> > argument. MAP_NOSYNC isn't a general purpose flag.
>
> The key is that this is platform-specific behavior, so if we should use
> a flag to use it right, we should. You are right that optimizing
> rarely used code with generic calls isn't a good use of time.

I have adjusted Sean's mmap() options patch to match our C layout and
plan to apply this to head, as it is from August.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

Attachment Content-Type Size
mmap.diff text/x-diff 1.7 KB

From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Bruce Momjian <bruce(at)momjian(dot)us>
Cc: Andres Freund <andres(at)2ndquadrant(dot)com>, Sean Chittenden <sean(at)chittenden(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2015-03-19 18:06:14
Message-ID: CA+TgmoZpgn6g6n3KD8mcG5qpwQVSh5odqUWgbmWb4YHHygyRLw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Thu, Mar 19, 2015 at 12:16 PM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> On Mon, Oct 13, 2014 at 11:35:18AM -0400, Bruce Momjian wrote:
>> On Mon, Oct 13, 2014 at 05:21:32PM +0200, Andres Freund wrote:
>> > > If we have it, we should improve it, or remove it. We might want to use
>> > > this code for something else in the future, so it should be improved
>> > > where feasible.
>> >
>> > Meh. We don't put in effort into code that doesn't matter just because
>> > it might get used elsewhere some day. By that argument we'd need to
>> > performance optimize a lot of code. And actually, using that code
>> > somewhere else is more of a counter indication than a pro
>> > argument. MAP_NOSYNC isn't a general purpose flag.
>>
>> The key is that this is platform-specific behavior, so if we should use
>> a flag to use it right, we should. You are right that optimizing
>> rarely used code with generic calls isn't a good use of time.
>
> I have adjusted Sean's mmap() options patch to match our C layout and
> plan to apply this to head, as it is from August.

Looks great, thanks for taking care of that.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Andres Freund <andres(at)2ndquadrant(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Sean Chittenden <sean(at)chittenden(dot)org>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] PostgreSQL 9.4 mmap(2) performance regression on FreeBSD...
Date: 2015-03-22 02:06:39
Message-ID: 20150322020639.GF10795@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Thu, Mar 19, 2015 at 12:16:07PM -0400, Bruce Momjian wrote:
> On Mon, Oct 13, 2014 at 11:35:18AM -0400, Bruce Momjian wrote:
> > On Mon, Oct 13, 2014 at 05:21:32PM +0200, Andres Freund wrote:
> > > > If we have it, we should improve it, or remove it. We might want to use
> > > > this code for something else in the future, so it should be improved
> > > > where feasible.
> > >
> > > Meh. We don't put in effort into code that doesn't matter just because
> > > it might get used elsewhere some day. By that argument we'd need to
> > > performance optimize a lot of code. And actually, using that code
> > > somewhere else is more of a counter indication than a pro
> > > argument. MAP_NOSYNC isn't a general purpose flag.
> >
> > The key is that this is platform-specific behavior, so if we should use
> > a flag to use it right, we should. You are right that optimizing
> > rarely used code with generic calls isn't a good use of time.
>
> I have adjusted Sean's mmap() options patch to match our C layout and
> plan to apply this to head, as it is from August.

Patch applied.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +