Re: how to demonstrate the effect of direct I/O ?

Hi all,

I've been running a lot of benchmarks recently (I'll publish the results
once I properly analyze them). One thing I'd like to demonstrate is the
effect of direct I/O when the wal_fsync_method is set to
open_sync/open_datasync.

I.e. I'd like to see cases when this improves/hurts performance
(compared to fsync/fdatasync) and if/how this works on SSD compared to
old-fashioned HDD. But no matter what, I see no significant differences
in performance.

This is what pg_test_fsync gives on the SSD (Intel 320):

open_datasync 12492.192 ops/sec
fdatasync 11646.257 ops/sec
fsync 9839.101 ops/sec
fsync_writethrough n/a
open_sync 10420.971 ops/sec

and this is what I get on the HDD (7.2k SATA)

open_datasync 120.041 ops/sec
fdatasync 120.042 ops/sec
fsync 48.000 ops/sec
fsync_writethrough n/a
open_sync 48.116 ops/sec

I can post the rest of the pg_test_fsync output if needed.

What should I do to see the effect of direct I/O? I'm wondering if I
need something like a RAID array or a controller with write cache to see
the difference.

All this was run on a kernel 3.1.5 using an ext4 filesystem.

thanks
Tomas

On 5.2.2012 00:25, Tomas Vondra wrote:
> Hi all,
>
> I've been running a lot of benchmarks recently (I'll publish the results
> once I properly analyze them). One thing I'd like to demonstrate is the
> effect of direct I/O when the wal_fsync_method is set to
> open_sync/open_datasync.
>
> I.e. I'd like to see cases when this improves/hurts performance
> (compared to fsync/fdatasync) and if/how this works on SSD compared to
> old-fashioned HDD. But no matter what, I see no significant differences
> in performance.

BTW the benchmark suite I run consists of two parts:

(a) read-write pgbench
(b) TPC-H-like benchmark that loads a few GBs of data (and then
queries them)

I'd expect to see the effect on the TPC-H load part, and maybe on the
pgbench (not sure if positive or negative).

> All this was run on a kernel 3.1.5 using an ext4 filesystem.

And the Pg versions tested were 9.1.2 and the current 9.2dev snapshot.

Tomas

On 02/04/2012 06:25 PM, Tomas Vondra wrote:
> What should I do to see the effect of direct I/O?

Test something other than a mainstream Linux filesystem. The two times
I've either measured an improvement myself for direct I/O were a)
Veritas VxFS on Linux, which has some documented acceleration here and
b) on Solaris. You won't find a compelling performance improvement
listed at
https://ext4.wiki.kernel.org/articles/c/l/a/Clarifying_Direct_IO%27s_Semantics_fd79.html
and Linux has generally ignored direct I/O as something important to
optimize for.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com