From: | KONDO Mitsumasa <kondo(dot)mitsumasa(at)lab(dot)ntt(dot)co(dot)jp> |
---|---|
To: | PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Optimize kernel readahead using buffer access strategy |
Date: | 2013-11-14 12:09:27 |
Message-ID: | 5284BD77.20503@lab.ntt.co.jp |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi,
I create a patch that is improvement of disk-read and OS file caches. It can
optimize kernel readahead parameter using buffer access strategy and
posix_fadvice() in various disk-read situations.
In general OS, readahead parameter was dynamically decided by disk-read
situations. If long time disk-read was happened, readahead parameter becomes big.
However it is based on experienced or heuristic algorithm, it causes waste
disk-read and throws out useful OS file caches in some case. It is bad for
disk-read performance a lot.
My proposed method is controlling OS readahead parameter by using buffer access
strategy in PostgreSQL and posix_fadvice() system call which can control OS
readahead parameter. Though, it is a general method in database.
For your information of effect of this patch, I got results of pgbench which are
in-memory-size database and out-memory-size database, and postgresql.conf
settings are always used by us. It seems to improve performance to a better. And
I think that this feature is going to be necessary for business intelligence
which will be realized at PostgreSQL version 10. I seriously believe Simon's
presentation in PostgreSQL conference Europe 2013! It was very exciting!!!
PostgreSQL have a lot of kind of disk-read method that are selected by planner,
however. I think that we need to discuss more other situations except pgbench,
and other cache cold situations. I think that optimizing kernel readahead
parameter with considering planner in PostgreSQL seems to be quite difficult, so
I seriously recruit co-author in this patch:-)
Regards,
--
Mitsumasa KONDO
NTT Open Source Software Center
Attachment | Content-Type | Size |
---|---|---|
image/jpeg | 27.4 KB | |
image/jpeg | 27.6 KB | |
optimize_kernel-readahead_using_buffer-access-strategy_v1.patch | text/x-diff | 8.2 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | KONDO Mitsumasa | 2013-11-14 12:25:12 | Re: Add min and max execute statement time in pg_stat_statement |
Previous Message | Haribabu kommi | 2013-11-14 12:08:29 | New option for pg_basebackup, to specify a different directory for pg_xlog |