Re: Synchrnonized Scan test

From: Jeff Davis <pgsql(at)j-davis(dot)com>
To: Luke Lonergan <llonergan(at)greenplum(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Synchrnonized Scan test
Date: 2006-12-10 22:36:21
Message-ID: 457C8BE5.30309@j-davis.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Luke Lonergan wrote:
> I've attached the results of a test with the first patch and the test. It
> uses five simultaneous scans as before, but this time the table is 120GB on
> a machine with 8GB of RAM. The data is stored on one non-raid disk, so a
> single scan should take about 33 minutes.
>
> The scans are started 5 minutes apart, so the last scan would end in about 3
> hours if they were independent (they're not). 8.2 unmodified runs the test
> in about 4 hours and 20 minutes. With the first patch, it runs in about 5
> hours and 30 minutes.
>
> The results show that the first patch actually slowed things down quite a
> bit. I'll test the last patch you sent and see what we get.
>
> I think this is worthwhile, you can definitely see a slowdown by having the
> competing scans. The OS (Linux 2.6.18) is doing a pretty good job of
> keeping the scans scheduled well, but we could still speed things up by
> about 30% if we got syncscan working.
>

Thanks for running the tests.

I don't know exactly what's happening in your case, but it may have to
do with some unbalanced scheduling. What if we actually throttled each
"follower" to read no more than N pages before it task-switched? Might
that keep the scans closer together?

If you have any way of tracing what is actually happening, that would be
very helpful.

It would be nice to know:
(1) Are the scans diverging because some processes are running forward
with fast shared_buffers page hits, and maybe other processes
task-switch when they need to read() (even if it's in the OS buffer cache)?
(2) How much difference is there really between a shared_buffer cache
hit and an OS buffer cache hit?
(3) Does an OS buffer cache hit always cause a context switch? How often?
(4) Is read-ahead per-process (like Tom suggested), or is it system wide?

Regards,
Jeff Davis

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Josh Berkus 2006-12-10 22:37:45 Re: Synchronized Scan preliminary results
Previous Message Jeff Davis 2006-12-10 22:18:49 Synchronized Scan preliminary results