Re: contrib/cache_scan (Re: What's needed for cache-only table scan?)

Hello,

The attached patches are revised ones according to the latest custom-plan
interface patch (v11).
The cache-scan module was re-implemented on the newer interface, and also
I noticed the extension does not handle the tuples being redirected correctly,
So, I revised the logic in ccache_vacuum_page() totally. It now becomes to
synchronize the cached tuples per page, not per tuple, basic and tries to
merge t-tree chunks per page basis also.

Also, I split the patches again because *demonstration* part is much larger
than the patches to the core backend. It will help reviewing.
* pgsql-v9.4-vacuum_page_hook.v11.patch
-> It adds a hook for each page being vacuumed; that needs to synchronize
the status of in-memory cache managed by extension.
* pgsql-v9.4-mvcc_allows_cache.v11.patch
-> It allows to run HeapTupleSatisfiesVisibility() towards the tuples
on the in-memory cache, not on the heap.
* pgsql-v9.4-example-cache_scan.v11.patch
-> It demonstrates the usage of above two patches. It allows to scan
a relation without storage access if possible.

Thanks,
--
NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>

> -----Original Message-----
> From: pgsql-hackers-owner(at)postgresql(dot)org
> [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Haribabu Kommi
> Sent: Wednesday, March 12, 2014 3:43 PM
> To: Kaigai Kouhei(海外 浩平)
> Cc: Kohei KaiGai; Tom Lane; PgHacker; Robert Haas
> Subject: Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only
> table scan?)
>
> On Wed, Mar 12, 2014 at 5:26 PM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> > Thanks for your efforts!
> >> Head patched
> >> Diff
> >> Select - 500K 772ms 2659ms -200%
> >> Insert - 400K 3429ms 1948ms 43% (I am
> >> not sure how it improved in this case)
> >> delete - 200K 2066ms 3978ms -92%
> >> update - 200K 3915ms 5899ms -50%
> >>
> >> This patch shown how the custom scan can be used very well but coming
> >> to patch as It is having some performance problem which needs to be
> >> investigated.
> >>
> >> I attached the test script file used for the performance test.
> >>
> > First of all, it seems to me your test case has too small data set
> > that allows to hold all the data in memory - briefly 500K of 200bytes
> > record will consume about 100MB. Your configuration allocates 512MB of
> > shared_buffer, and about 3GB of OS-level page cache is available.
> > (Note that Linux uses free memory as disk cache adaptively.)
>
> Thanks for the information and a small correction. The Total number of
> records are 5 million.
> The select operation is selecting 500K records. The total table size is
> around 1GB.
>
> Once I get your new patch re-based on the custom scan patch, I will test
> the performance again by increasing my database size more than the RAM size.
> And also I will make sure that memory available for disk cache is less.
>
> Regards,
> Hari Babu
> Fujitsu Australia
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org) To make
> changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers