Re: WAL documentation changes

Hi,

the comment WRT WAL recovery and FS journals [1] is a bit misleading in
it's current form.

First, none of the general purpose filesystems I've seen so far do data
journalling per default, since it's a huge performance penalty, even for
non-RDBMS workloads. The feature you talk about is ext3 specific (and
should be pointed out as such) and only disables write ordering, meaning
that metadata and file content updates are not synchronized.

best regards,
Michael

[1] 64b3d98baaf96afea815b0c37ff918f02fda11c9

Michael Renner wrote:
> Hi,
>
> the comment WRT WAL recovery and FS journals [1] is a bit misleading in
> it's current form.
>
> First, none of the general purpose filesystems I've seen so far do data
> journalling per default, since it's a huge performance penalty, even for
> non-RDBMS workloads. The feature you talk about is ext3 specific (and
> should be pointed out as such) and only disables write ordering, meaning
> that metadata and file content updates are not synchronized.

You are right that my docs were misleading. I have improved them by
mentioning that it is _data_ flush that as part of journalling that can
be a problem, and documented that the mount option listed is
ext3-specific, not linux-specific.

Updated docs attached. Please let me know if I can improve it some
more.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

>> First, none of the general purpose filesystems I've seen so far do data
>> journalling per default, since it's a huge performance penalty, even for
>> non-RDBMS workloads. The feature you talk about is ext3 specific (and
>> should be pointed out as such) and only disables write ordering, meaning
>> that metadata and file content updates are not synchronized.
>
> You are right that my docs were misleading. I have improved them by
> mentioning that it is _data_ flush that as part of journalling that can
> be a problem, and documented that the mount option listed is
> ext3-specific, not linux-specific.

Actually, I think that some of the other journalling filesystems allow
data journalling (I know ReiserFS does), they just don't default to it.
For that matter, a few (ZFS in particular) have data journalling which
can't be turned off. While it's not a tuning parameter, users should be
warned that they'll take a performance hit from it.

--Josh

Josh Berkus wrote:
>
> >> First, none of the general purpose filesystems I've seen so far do data
> >> journalling per default, since it's a huge performance penalty, even for
> >> non-RDBMS workloads. The feature you talk about is ext3 specific (and
> >> should be pointed out as such) and only disables write ordering, meaning
> >> that metadata and file content updates are not synchronized.
> >
> > You are right that my docs were misleading. I have improved them by
> > mentioning that it is _data_ flush that as part of journalling that can
> > be a problem, and documented that the mount option listed is
> > ext3-specific, not linux-specific.
>
> Actually, I think that some of the other journalling filesystems allow
> data journalling (I know ReiserFS does), they just don't default to it.
> For that matter, a few (ZFS in particular) have data journalling which
> can't be turned off. While it's not a tuning parameter, users should be
> warned that they'll take a performance hit from it.

So I assume you are saying the docs are fine now.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce,

In your document change which one can be placed on non-journalling
file system? data? wal? or both?

For me it seems it's not clear.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

> Josh Berkus wrote:
> >
> > >> First, none of the general purpose filesystems I've seen so far do data
> > >> journalling per default, since it's a huge performance penalty, even for
> > >> non-RDBMS workloads. The feature you talk about is ext3 specific (and
> > >> should be pointed out as such) and only disables write ordering, meaning
> > >> that metadata and file content updates are not synchronized.
> > >
> > > You are right that my docs were misleading. I have improved them by
> > > mentioning that it is _data_ flush that as part of journalling that can
> > > be a problem, and documented that the mount option listed is
> > > ext3-specific, not linux-specific.
> >
> > Actually, I think that some of the other journalling filesystems allow
> > data journalling (I know ReiserFS does), they just don't default to it.
> > For that matter, a few (ZFS in particular) have data journalling which
> > can't be turned off. While it's not a tuning parameter, users should be
> > warned that they'll take a performance hit from it.
>
> So I assume you are saying the docs are fine now.
>
> --
> Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
> EnterpriseDB http://enterprisedb.com
>
> + If your life is a hard drive, Christ can be your backup. +
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

Tatsuo Ishii wrote:
> Bruce,
>
> In your document change which one can be placed on non-journalling
> file system? data? wal? or both?

Both. I have updated the docs to mention this, patch attached.
--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

>>> Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Tatsuo Ishii wrote:
>> In your document change which one can be placed on non-journalling
>> file system? data? wal? or both?
>
> Both. I have updated the docs to mention this, patch attached.

Did you mean to say that journaled file systems are *not* necessary?

-Kevin

Kevin Grittner wrote:
> >>> Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > Tatsuo Ishii wrote:
> >> In your document change which one can be placed on non-journalling
> >> file system? data? wal? or both?
> >
> > Both. I have updated the docs to mention this, patch attached.
>
> Did you mean to say that journaled file systems are *not* necessary?

Yes, not needed for database reliablity. The patch text was attached;
was it unclear?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

>>> Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> Kevin Grittner wrote:
>> Did you mean to say that journaled file systems are *not*
necessary?
>
> Yes, not needed for database reliablity. The patch text was
attached;
> was it unclear?

I think you accidentally left out the word "not".

-Kevin

Kevin Grittner wrote:
> >>> Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> > Kevin Grittner wrote:
> >> Did you mean to say that journaled file systems are *not*
> necessary?
> >
> > Yes, not needed for database reliablity. The patch text was
> attached;
> > was it unclear?
>
> I think you accidentally left out the word "not".

Oops, right, added. Good catch. Warping that sentence into something
that allowed the mention of WAL and data files was obviously too much
for me. ;-)

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +