Re: Fw: [vorbis-dev] ogg123: shared memory by mmap()

For those interested in the topic, this is something that went through
the Vorbis-dev mailing list not that long ago which implements the
above topic into the vorbis decoder. Might be useful to see what
systems it works on, and where it breaks as well as a reference
implementation. (patch included for files affected)

--
Rod Taylor

There are always four sides to every story: your side, their side, the
truth, and what really happened.
----- Original Message -----
From: "Christian Weisgerber" <naddy(at)mips(dot)inka(dot)de>
Newsgroups: list.vorbis.dev
To: <vorbis-dev(at)xiph(dot)org>
Sent: Saturday, March 17, 2001 12:01 PM
Subject: [vorbis-dev] ogg123: shared memory by mmap()

> The patch below adds:
>
> - acinclude.m4: A new macro A_FUNC_SMMAP to check that sharing
pages
> through mmap() works. This is taken from Joerg Schilling's star.
> - configure.in: A_FUNC_SMMAP
> - ogg123/buffer.c: If we have a working mmap(), use it to create
> a region of shared memory instead of using System V IPC.
>
> Works on BSD. Should also work on SVR4 and offspring (Solaris),
> and Linux.
>
>
> --- acinclude.m4.orig Wed Feb 28 03:36:50 2001
> +++ acinclude.m4 Sat Mar 17 17:39:58 2001
> @@ -300,3 +300,65 @@
> AC_SUBST(AO_LIBS)
> rm -f conf.aotest
> ])
<-- SNIP -->
> +if test $ac_cv_func_smmap = yes; then
> + AC_DEFINE(HAVE_SMMAP)
> +fi])
> --- configure.in.orig Mon Feb 26 06:56:46 2001
> +++ configure.in Sat Mar 17 17:39:45 2001
> @@ -67,7 +67,7 @@
> dnl Check for library functions
> dnl --------------------------------------------------
>
> -dnl none
> +AC_FUNC_SMMAP
>
> dnl --------------------------------------------------
> dnl Work around FHS stupidity
> --- ogg123/buffer.c.old Sat Mar 17 15:37:07 2001
> +++ ogg123/buffer.c Sat Mar 17 17:40:16 2001
> @@ -6,16 +6,16 @@
<-- SNIP -->
> buffer_init (buf, size);
>
> --
> Christian "naddy" Weisgerber
naddy(at)mips(dot)inka(dot)de
>
> --- >8 ----
> List archives: http://www.xiph.org/archives/
> Ogg project homepage: http://www.xiph.org/ogg/
> To unsubscribe from this list, send a message to
'vorbis-dev-request(at)xiph(dot)org'
> containing only the word 'unsubscribe' in the body. No subject is
needed.
> Unsubscribe messages sent to the list will be ignored/filtered.

WOOT WOOT! DANGER WILL ROBINSON!

> ----- Original Message -----
> From: "Christian Weisgerber" <naddy(at)mips(dot)inka(dot)de>
> Newsgroups: list.vorbis.dev
> To: <vorbis-dev(at)xiph(dot)org>
> Sent: Saturday, March 17, 2001 12:01 PM
> Subject: [vorbis-dev] ogg123: shared memory by mmap()
>
>
> > The patch below adds:
> >
> > - acinclude.m4: A new macro A_FUNC_SMMAP to check that sharing
> pages
> > through mmap() works. This is taken from Joerg Schilling's star.
> > - configure.in: A_FUNC_SMMAP
> > - ogg123/buffer.c: If we have a working mmap(), use it to create
> > a region of shared memory instead of using System V IPC.
> >
> > Works on BSD. Should also work on SVR4 and offspring (Solaris),
> > and Linux.

This is a really bad idea performance wise. Solaris has a special
code path for SYSV shared memory that doesn't require tons of swap
tracking structures per-page/per-process. FreeBSD also has this
optimization (it's off by default, but should work since FreeBSD
4.2 via the sysctl kern.ipc.shm_use_phys=1)

Both OS's use a trick of making the pages non-pageable, this allows
signifigant savings in kernel space required for each attached
process, as well as the use of large pages which reduce the amount
of TLB faults your processes will incurr.

Anyhow, if you could make this a runtime option it wouldn't be so
evil, but as a compile time option, it's a really bad idea for
Solaris and FreeBSD.

--
-Alfred Perlstein - [bright(at)wintelcom(dot)net|alfred(at)freebsd(dot)org]

> > > The patch below adds:
> > >
> > > - acinclude.m4: A new macro A_FUNC_SMMAP to check that sharing
> > pages
> > > through mmap() works. This is taken from Joerg Schilling's star.
> > > - configure.in: A_FUNC_SMMAP
> > > - ogg123/buffer.c: If we have a working mmap(), use it to create
> > > a region of shared memory instead of using System V IPC.
> > >
> > > Works on BSD. Should also work on SVR4 and offspring (Solaris),
> > > and Linux.
>
> This is a really bad idea performance wise. Solaris has a special
> code path for SYSV shared memory that doesn't require tons of swap
> tracking structures per-page/per-process. FreeBSD also has this
> optimization (it's off by default, but should work since FreeBSD
> 4.2 via the sysctl kern.ipc.shm_use_phys=1)

>
> Both OS's use a trick of making the pages non-pageable, this allows
> signifigant savings in kernel space required for each attached
> process, as well as the use of large pages which reduce the amount
> of TLB faults your processes will incurr.

That is interesting. BSDi has SysV shared memory as non-pagable, and I
always thought of that as a bug. Seems you are saying that having it
pagable has a significant performance penalty. Interesting.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

* Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> [010320 14:10] wrote:
> > > > The patch below adds:
> > > >
> > > > - acinclude.m4: A new macro A_FUNC_SMMAP to check that sharing
> > > pages
> > > > through mmap() works. This is taken from Joerg Schilling's star.
> > > > - configure.in: A_FUNC_SMMAP
> > > > - ogg123/buffer.c: If we have a working mmap(), use it to create
> > > > a region of shared memory instead of using System V IPC.
> > > >
> > > > Works on BSD. Should also work on SVR4 and offspring (Solaris),
> > > > and Linux.
> >
> > This is a really bad idea performance wise. Solaris has a special
> > code path for SYSV shared memory that doesn't require tons of swap
> > tracking structures per-page/per-process. FreeBSD also has this
> > optimization (it's off by default, but should work since FreeBSD
> > 4.2 via the sysctl kern.ipc.shm_use_phys=1)
>
> >
> > Both OS's use a trick of making the pages non-pageable, this allows
> > signifigant savings in kernel space required for each attached
> > process, as well as the use of large pages which reduce the amount
> > of TLB faults your processes will incurr.
>
> That is interesting. BSDi has SysV shared memory as non-pagable, and I
> always thought of that as a bug. Seems you are saying that having it
> pagable has a significant performance penalty. Interesting.

Yes, having it pageable is actually sort of bad.

It doesn't allow you to do several important optimizations.

--
-Alfred Perlstein - [bright(at)wintelcom(dot)net|alfred(at)freebsd(dot)org]