2Q implementaion for PostgreSQL buffer replacement.

Lists: pgsql-hackers
From: Yutaka tanida <yutaka(at)nonsensecorner(dot)com>
To: <pgsql-hackers(at)postgresql(dot)org>
Subject: 2Q implementaion for PostgreSQL buffer replacement.
Date: 2003-06-23 05:41:59
Message-ID: 20030623144157.B16C.YUTAKA@nonsensecorner.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hi.

I implement 2Q algorithm to PostgreSQL for buffer management , instead
of LRU.
It's known as low overhead and high performance than LRU. If you have
some interests , see following URL.

http://www.vldb.org/conf/1994/P439.PDF

In my test (pgbench -S) , it improves 4% cache hit rate and 2% up
performance comparing from LRU.

Do you have any interest about this patch?

--
Yutaka tanida <yutaka(at)nonsensecorner(dot)com>
http://www.nonsensecorner.com/

Attachment Content-Type Size
qq_1.patch application/octet-stream 527 bytes
buf_init.c.diff application/octet-stream 2.5 KB
freelist.c.diff application/octet-stream 10.6 KB

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Yutaka tanida <yutaka(at)nonsensecorner(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: 2Q implementaion for PostgreSQL buffer replacement.
Date: 2003-06-24 03:49:17
Message-ID: 200306240349.h5O3nHn17430@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


Looks good to me --- we will include it in 7.4.

Your patch has been added to the PostgreSQL unapplied patches list at:

http://momjian.postgresql.org/cgi-bin/pgpatches

I will try to apply it within the next 48 hours.

---------------------------------------------------------------------------

Yutaka tanida wrote:
> Hi.
>
> I implement 2Q algorithm to PostgreSQL for buffer management , instead
> of LRU.
> It's known as low overhead and high performance than LRU. If you have
> some interests , see following URL.
>
> http://www.vldb.org/conf/1994/P439.PDF
>
> In my test (pgbench -S) , it improves 4% cache hit rate and 2% up
> performance comparing from LRU.
>
> Do you have any interest about this patch?
>
> --
> Yutaka tanida <yutaka(at)nonsensecorner(dot)com>
> http://www.nonsensecorner.com/

[ Attachment, skipping... ]

[ Attachment, skipping... ]

[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073


From: Yutaka tanida <yutaka(at)nonsensecorner(dot)com>
To: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: 2Q implementaion for PostgreSQL buffer replacement.
Date: 2003-06-24 12:12:08
Message-ID: 20030624210833.91BE.YUTAKA@nonsensecorner.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


On Mon, 23 Jun 2003 23:49:17 -0400 (EDT)
Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> wrote:

>
> Looks good to me --- we will include it in 7.4.

Thanks.But please note it is not completed yet. I must implement more ,
and move configurable parameter to postgresql.conf file.

--
Yutaka tanida <yutaka(at)nonsensecorner(dot)com>
http://www.nonsensecorner.com/


From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Yutaka tanida <yutaka(at)nonsensecorner(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: 2Q implementaion for PostgreSQL buffer replacement.
Date: 2003-06-24 13:54:04
Message-ID: 200306241354.h5ODs4W18937@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


OK, thanks. I will remove it from the queue, and someone suggested a
different algorithm today:

> I was researching on cache replacement strategy as well. 2Q has one
> disadvantage see this exellent paper:
> http://www.almaden.ibm.com/cs/people/dmodha/#ARC see the paper
> "ARC: A Self-Tuning, Low Overhead Replacement Cache" for theory and "One
> Up on LRU" for implementation details. ARC requires no tuning and can
> switch fast between chaging patterns. Best of all is it is resistant to a
> "sequential scan" pattern. and i think it's even easier to implement then
> 2q :)

---------------------------------------------------------------------------

Yutaka tanida wrote:
>
> On Mon, 23 Jun 2003 23:49:17 -0400 (EDT)
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> wrote:
>
> >
> > Looks good to me --- we will include it in 7.4.
>
> Thanks.But please note it is not completed yet. I must implement more ,
> and move configurable parameter to postgresql.conf file.
>
> --
> Yutaka tanida <yutaka(at)nonsensecorner(dot)com>
> http://www.nonsensecorner.com/
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: the planner will ignore your desire to choose an index scan if your
> joining column's datatypes do not match
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073


From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Yutaka tanida <yutaka(at)nonsensecorner(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: 2Q implementaion for PostgreSQL buffer replacement.
Date: 2003-06-24 13:54:32
Message-ID: 200306241354.h5ODsWj19025@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


Patch removed at author's request.

---------------------------------------------------------------------------

Yutaka tanida wrote:
> Hi.
>
> I implement 2Q algorithm to PostgreSQL for buffer management , instead
> of LRU.
> It's known as low overhead and high performance than LRU. If you have
> some interests , see following URL.
>
> http://www.vldb.org/conf/1994/P439.PDF
>
> In my test (pgbench -S) , it improves 4% cache hit rate and 2% up
> performance comparing from LRU.
>
> Do you have any interest about this patch?
>
> --
> Yutaka tanida <yutaka(at)nonsensecorner(dot)com>
> http://www.nonsensecorner.com/

[ Attachment, skipping... ]

[ Attachment, skipping... ]

[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073