WIP: buffer manager rewrite (take 2)

Second iteration of buffer manager rewrite. This uses the idea of a
usage counter instead of just a recently-used flag bit. I allowed the
counter to go up to 5, but some playing around with that value would
be interesting. (Tweak BM_MAX_USAGE_COUNT in
src/include/storage/buf_internals.h, then recompile the files in
src/backend/storage/buffer/.) Also there are more GUC variables now
for controlling the bgwriter.

regards, tom lane

On Wed, Feb 16, 2005 at 07:50:28PM -0500, Tom Lane wrote:
> Second iteration of buffer manager rewrite. This uses the idea of a
> usage counter instead of just a recently-used flag bit. I allowed the
> counter to go up to 5, but some playing around with that value would
> be interesting. (Tweak BM_MAX_USAGE_COUNT in
> src/include/storage/buf_internals.h, then recompile the files in
> src/backend/storage/buffer/.) Also there are more GUC variables now
> for controlling the bgwriter.
>

I see a huge performance increase, when applied to CVS from 20050301.

Baseline against 8.0.1:
http://www.osdl.org/projects/dbt2dev/results/dev4-010/309/
throughput 3639.97

CVS from 20050301:
http://www.osdl.org/projects/dbt2dev/results/dev4-010/314/
throughput 5483.01
I only ran this for 30 minutes, as opposed to 60, but it looks
promising.

So about a 50% increase in throughput for my test. Not to shabby. ;)

Mark

Mark Wong <markw(at)osdl(dot)org> writes:
> CVS from 20050301 [ plus clock-sweep buffer manager ] :
> http://www.osdl.org/projects/dbt2dev/results/dev4-010/314/
> throughput 5483.01
> I only ran this for 30 minutes, as opposed to 60, but it looks
> promising.

> So about a 50% increase in throughput for my test. Not to shabby. ;)

Sweet ... and the response-time improvement is just stunning. I think
we may have a winner. Is there a reason you didn't run the test longer
though?

regards, tom lane

On Wed, Mar 02, 2005 at 08:48:35PM -0500, Tom Lane wrote:
> Mark Wong <markw(at)osdl(dot)org> writes:
> > CVS from 20050301 [ plus clock-sweep buffer manager ] :
> > http://www.osdl.org/projects/dbt2dev/results/dev4-010/314/
> > throughput 5483.01
> > I only ran this for 30 minutes, as opposed to 60, but it looks
> > promising.
>
> > So about a 50% increase in throughput for my test. Not to shabby. ;)
>
> Sweet ... and the response-time improvement is just stunning. I think
> we may have a winner. Is there a reason you didn't run the test longer
> though?

I would normally run for 1 hour, but I guess my fingers were thinking
something different at the time. =P

Mark

Tom Lane wrote:
> Mark Wong <markw(at)osdl(dot)org> writes:
> > CVS from 20050301 [ plus clock-sweep buffer manager ] :
> > http://www.osdl.org/projects/dbt2dev/results/dev4-010/314/
> > throughput 5483.01
> > I only ran this for 30 minutes, as opposed to 60, but it looks
> > promising.
>
> > So about a 50% increase in throughput for my test. Not to shabby. ;)
>
> Sweet ... and the response-time improvement is just stunning. I think
> we may have a winner. Is there a reason you didn't run the test longer
> though?

This is HUGE. Was it because of the clock sweep?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> This is HUGE. Was it because of the clock sweep?

No, it was because of the elimination of the global BufMgrLock.
Clock sweep is the price we had to pay to get that ...

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > This is HUGE. Was it because of the clock sweep?
>
> No, it was because of the elimination of the global BufMgrLock.
> Clock sweep is the price we had to pay to get that ...

Wow, certainly stunning. Was it a big benefit only for SMP machines?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Was it a big benefit only for SMP machines?

Probably. I did some simple pgbench checks and found that the new code
seemed no better than the old on a one-processor machine. However, as
long as it's not worse for a uniprocessor, who cares?

It would be good to try to get some more measurements to verify this.
Never did trust pgbench ...

regards, tom lane