Re: Scaling shared buffer eviction

On Thu, Aug 28, 2014 at 4:41 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
wrote:
>
> I have yet to collect data under varying loads, however I have
> collected performance data for 8GB shared buffers which shows
> reasonably good performance and scalability.
>
> I think the main part left for this patch is more data for various loads
> which I will share in next few days, however I think patch is ready for
> next round of review, so I will mark it as Needs Review.

I have collected more data with the patch. I understand that you
have given more review comments due to which patch require
changes, however I think it will not effect the performance data
to a great extent and I have anyway taken the data, so sharing the
same.

> Performance Data:
> -------------------------------
>
> Configuration and Db Details
> IBM POWER-7 16 cores, 64 hardware threads
> RAM = 64GB
> Database Locale =C
> checkpoint_segments=256
> checkpoint_timeout =15min
> scale factor = 3000
> Client Count = number of concurrent sessions and threads (ex. -c 8 -j 8)
> Duration of each individual run = 5mins
>
> All the data is in tps and taken using pgbench read-only load

Common configuration remains same as above.

Shared_Buffers = 500MB
Client Count/Patch_Ver 8 16 32 64 128 HEAD 56248 100112 121341 81128
56552 Patch 59389 112483 157034 185740 166725

Shared_Buffers = 1GB
Client Count/Patch_Ver 8 16 32 64 128 HEAD 56401 102557 121643 81686
57091 Patch 59361 114813 157528 188209 167752

Shared_Buffers = 14GB
Client Count/Patch_Ver 8 16 32 64 128 HEAD 60059 110582 152051 130718
97014 Patch 61462 117377 169767 225803 229083

Shared_Buffers = 15GB
Client Count/Patch_Ver 8 16 32 64 128 HEAD 60005 112928 153650 135203
36343 Patch 61345 115569 168767 226150 36985

Observations
---------------------
1. Performance improvement is upto 2~3 times for higher client
counts (64, 128).
2. For lower client count (8), we can see 2~5 % performance
improvement.
3. Overall, this improves the read scalability.
4. For lower number of shared buffers, we see that there is a minor
dip in tps even after patch (it might be that we can improve it by
tuning higher water mark for the number of buffers on freelist, I will
try this by varying high water mark).
5. For larger shared buffers (15GB), we can see that there is still a
dip at large client count, although situation is not bad as compare to
HEAD. The reason is that at such high shared buffers and client count,
I/O starts happening because all the data can't be contained in RAM.

I will try to take some data for tpc-b load as well. Kindly let me know
if you want to see data for some other configuration.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com