Re: Defaulting wal_sync_method to fdatasync on Linux for 9.1?

From: Scott Carey <scott(at)richrelevance(dot)com>
To: Marti Raudsepp <marti(at)juffo(dot)org>
Cc: Greg Smith <greg(at)2ndquadrant(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>
Subject: Re: Defaulting wal_sync_method to fdatasync on Linux for 9.1?
Date: 2010-11-08 18:13:43
Message-ID: 9D7DB2F2-683B-480C-B781-8D282A280E35@richrelevance.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance


On Nov 7, 2010, at 6:35 PM, Marti Raudsepp wrote:

> On Mon, Nov 8, 2010 at 01:35, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
>> Yes; it's supposed to, and that logic works fine on some other platforms.
>
> No, the logic was broken to begin with. Linux technically supported
> O_DSYNC all along. PostgreSQL used fdatasync as the default. Now,
> because Linux added proper O_SYNC support, PostgreSQL suddenly prefers
> O_DSYNC over fdatasync?
>
>> Until you've
>> quantified which of the cases do that--which is required for reliable
>> operation of PostgreSQL--and which don't, you don't have any data that can
>> be used to draw a conclusion from. If some setups are faster because they
>> write less reliably, that doesn't automatically make them the better choice.
>
> I don't see your point. If fdatasync worked on Linux, AS THE DEFAULT,
> all the time until recently, then how does it all of a sudden need
> proof NOW?
>
> If anything, the new open_datasync should be scrutinized because it
> WASN'T the default before and it hasn't gotten as much testing on
> Linux.
>

I agree. Im my opinion, the burden of proof lies with those contending that the default value should _change_ from fdatasync to O_DSYNC on linux. If the default changes, all power-fail testing and other reliability tests done prior on a hardware configuration may become invalid without users even knowing.

Unfortunately, a code change in postgres is required to _prevent_ the default from changing when compiled and run against the latest kernels.

Summary:
Until recently, there was code with a code comment in the Linux kernel that said "For now, when the user asks for O_SYNC, we'll actually give O_DSYNC". Linux has had O_DSYNC forever and ever, but not O_SYNC.
If O_DSYNC is preferred over fdatasync for Postgres xlog (as the code indicates), it should have been the preferred for years on Linux as well. If fdatasync has been the preferred method on Linux, and the O_SYNC = O_DSYNC test was for that, then the purpose behind the test has broken.

No matter how you slice it, the default on Linux is implicitly changing and the choice is to either:
* Return the default to fdatasync
* Let it implicitly change to O_DSYNC

The latter choice is the one that requires testing to prove that it is the proper and preferred default from the performance and data reliability POV. The former is the status quo -- but requires a code change.

> Regards,
> Marti
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Tom Lane 2010-11-08 18:40:34 Re: Defaulting wal_sync_method to fdatasync on Linux for 9.1?
Previous Message Kevin Grittner 2010-11-08 17:15:56 Re: Select * is very slow