Re: BUG #6372: Error while creating database with fsync parameter as on incase of CIFS

The following bug has been logged on the website:

Bug reference: 6372
Logged by: Anjali Arora
Email address: anjali_524(at)yahoo(dot)co(dot)in
PostgreSQL version: 9.0.4
Operating system: Cent OS
Description:

The 8.2.2 postgres version works well with default fsync value on CIFS
2.6.32.

This problem seems to be specific to postgres 9.0.4 release

Error Message:

PST ERROR: could not fsync file "base/16409": Invalid argument Dec 30
03:00:26 devok64-8 postgres_cifs_kaz_1[15812]: [2-2] [local] 15812
2011-12-30 03:00:26.511 PST STATEMENT: CREATE DATABASE "KazDB

Please help to make it work.

On Mon, Jan 2, 2012 at 17:27, <anjali_524(at)yahoo(dot)co(dot)in> wrote:
> The following bug has been logged on the website:
>
> Bug reference:      6372
> Logged by:          Anjali Arora
> Email address:      anjali_524(at)yahoo(dot)co(dot)in
> PostgreSQL version: 9.0.4
> Operating system:   Cent OS
> Description:
>
> The 8.2.2 postgres version works well with default fsync value on CIFS
> 2.6.32.

It does not, really. It may appear to, but it does not.

> This problem seems to be specific to postgres 9.0.4 release
>
> Error Message:
>
> PST ERROR:  could not fsync file "base/16409": Invalid argument Dec 30
> 03:00:26 devok64-8 postgres_cifs_kaz_1[15812]: [2-2] [local] 15812
> 2011-12-30 03:00:26.511 PST STATEMENT:  CREATE DATABASE "KazDB
>
> Please help to make it work.

PostgreSQL does not support data directory over CIFS.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> On Mon, Jan 2, 2012 at 17:27, <anjali_524(at)yahoo(dot)co(dot)in> wrote:
>> PST ERROR: could not fsync file "base/16409": Invalid argument Dec 30
>> 03:00:26 devok64-8 postgres_cifs_kaz_1[15812]: [2-2] [local] 15812
>> 2011-12-30 03:00:26.511 PST STATEMENT: CREATE DATABASE "KazDB

The specific error seems to be coming from copydir.c's attempt to fsync
a directory. We are already ignoring EBADF there, and could presumably
fix at least this symptom if we ignored EINVAL.

> PostgreSQL does not support data directory over CIFS.

I'm wondering what's your basis for asserting we don't support CIFS in
general? It's probably not terribly bulletproof, but any worse than NFS?

regards, tom lane

On Mon, Jan 2, 2012 at 21:14, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> On Mon, Jan 2, 2012 at 17:27,  <anjali_524(at)yahoo(dot)co(dot)in> wrote:
>>> PST ERROR:  could not fsync file "base/16409": Invalid argument Dec 30
>>> 03:00:26 devok64-8 postgres_cifs_kaz_1[15812]: [2-2] [local] 15812
>>> 2011-12-30 03:00:26.511 PST STATEMENT:  CREATE DATABASE "KazDB
>
> The specific error seems to be coming from copydir.c's attempt to fsync
> a directory.  We are already ignoring EBADF there, and could presumably
> fix at least this symptom if we ignored EINVAL.

Sure, we could - and I guess if you're running over CIFS, reliability
might not be the biggest concern in the first place...

>> PostgreSQL does not support data directory over CIFS.
>
> I'm wondering what's your basis for asserting we don't support CIFS in
> general?  It's probably not terribly bulletproof, but any worse than NFS?

Yes, it is a lot worse than NFS from experience. I can't find a
reference to it anywhere now, but IIRC there are bigger issues - with
blocksizes, with syncing not properly, with write ordering.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> On Mon, Jan 2, 2012 at 21:14, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> I'm wondering what's your basis for asserting we don't support CIFS in
>> general? It's probably not terribly bulletproof, but any worse than NFS?

> Yes, it is a lot worse than NFS from experience. I can't find a
> reference to it anywhere now, but IIRC there are bigger issues - with
> blocksizes, with syncing not properly, with write ordering.

Hmm. I searched the list archives and couldn't find any previous
discussion of such things, but that may just prove that no one thinks
it's worth attempting.

Anyway the immediate question is which errnos are reasonable for copydir
to ignore. Just looking at the standard's description of fsync's error
conditions:

The fsync() function shall fail if:
[EBADF]
The fildes argument is not a valid descriptor.
[EINTR]
The fsync() function was interrupted by a signal.
[EINVAL]
The fildes argument does not refer to a file on which this operation is possible.
[EIO]
An I/O error occurred while reading from or writing to the file system.

it seems like EINVAL is a considerably more reasonable thing to return
than EBADF, if the filesystem is trying to tell you that it won't fsync
a directory. So I'm a bit surprised this question hasn't come up for
other filesystems.

regards, tom lane

On Mon, Jan 2, 2012 at 21:28, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> On Mon, Jan 2, 2012 at 21:14, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> I'm wondering what's your basis for asserting we don't support CIFS in
>>> general?  It's probably not terribly bulletproof, but any worse than NFS?
>
>> Yes, it is a lot worse than NFS from experience. I can't find a
>> reference to it anywhere now, but IIRC there are bigger issues - with
>> blocksizes, with syncing not properly, with write ordering.
>
> Hmm.  I searched the list archives and couldn't find any previous
> discussion of such things, but that may just prove that no one thinks
> it's worth attempting.

Yeah, I don't think it was in our archives, it was somewhere else.

And as a disclaime r- it may have been about the win32 cifs *client*.
It was at the time just talking windows cifs client -> windows cifs
server.

> Anyway the immediate question is which errnos are reasonable for copydir
> to ignore.  Just looking at the standard's description of fsync's error
> conditions:
>
>        The fsync() function shall fail if:
>        [EBADF]
>        The fildes argument is not a valid descriptor.
>        [EINTR]
>        The fsync() function was interrupted by a signal.
>        [EINVAL]
>        The fildes argument does not refer to a file on which this operation is possible.
>        [EIO]
>        An I/O error occurred while reading from or writing to the file system.
>
> it seems like EINVAL is a considerably more reasonable thing to return
> than EBADF, if the filesystem is trying to tell you that it won't fsync
> a directory.  So I'm a bit surprised this question hasn't come up for
> other filesystems.

Agreed. But do we really want to accept this with fsync=on? It
basically means fsync=maybe, no?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> On Mon, Jan 2, 2012 at 21:28, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> it seems like EINVAL is a considerably more reasonable thing to return
>> than EBADF, if the filesystem is trying to tell you that it won't fsync
>> a directory. So I'm a bit surprised this question hasn't come up for
>> other filesystems.

> Agreed. But do we really want to accept this with fsync=on? It
> basically means fsync=maybe, no?

Well, given the number of cases that the code already ignores when
isdir is true, I don't think that argument holds much water at all.

However, I'm not real eager to change this just on the basis of the CIFS
case. If we find another filesystem that returns the same errno,
though, I would vote to change it.

regards, tom lane

Excerpts from Tom Lane's message of lun ene 02 17:28:33 -0300 2012:

> Anyway the immediate question is which errnos are reasonable for copydir
> to ignore. Just looking at the standard's description of fsync's error
> conditions:
>
> The fsync() function shall fail if:
> [EBADF]
> The fildes argument is not a valid descriptor.
> [EINTR]
> The fsync() function was interrupted by a signal.
> [EINVAL]
> The fildes argument does not refer to a file on which this operation is possible.
> [EIO]
> An I/O error occurred while reading from or writing to the file system.
>
> it seems like EINVAL is a considerably more reasonable thing to return
> than EBADF, if the filesystem is trying to tell you that it won't fsync
> a directory. So I'm a bit surprised this question hasn't come up for
> other filesystems.

Probably because other filesystems do allow you to fsync directories.
In fact for some cases they _require_ it ... remember the fiasco when
MTA writers were told that they needed to fsync their queue dirs in
order for all queued email to persist?

--
Álvaro Herrera <alvherre(at)commandprompt(dot)com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> Excerpts from Tom Lane's message of lun ene 02 17:28:33 -0300 2012:
>> it seems like EINVAL is a considerably more reasonable thing to return
>> than EBADF, if the filesystem is trying to tell you that it won't fsync
>> a directory. So I'm a bit surprised this question hasn't come up for
>> other filesystems.

> Probably because other filesystems do allow you to fsync directories.
> In fact for some cases they _require_ it ... remember the fiasco when
> MTA writers were told that they needed to fsync their queue dirs in
> order for all queued email to persist?

Yeah, the long and the short of it is that if the filesystem won't
accept an fsync on a directory, we have to assume that it doesn't need
it and will manage metadata persistence safely without prodding.

The only real question here is whether an EINVAL could mean something
besides "fsync on directory is not accepted". If there are any
scenarios where it represents a transient/fixable error, then we'd
want to report it. It's far from clear to me that there are any
though. What it could mean in general is not at issue, because we
know the target is a directory that we just created moments before.

regards, tom lane

Hi Tom,
 
Thanks for the solution. CIFS worked with fsync flag by ingnoring EINVAL in copydir.c.
 
I tested fsync with 8.2.2 version of PostgreSQL, it worked fine without EINVAL patch. I wanted to know is something changed in version 9.0.4 of postgreSQL.
 
As fsync flag was not working with PostgreSQL version 9.0.4 without applying the patch.
 
Regards,
Anjali

________________________________
From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: anjali_524(at)yahoo(dot)co(dot)in; pgsql-bugs(at)postgresql(dot)org
Sent: Tuesday, 3 January 2012 2:00 AM
Subject: Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS

On Mon, Jan 2, 2012 at 21:28, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> On Mon, Jan 2, 2012 at 21:14, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> I'm wondering what's your basis for asserting we don't support CIFS in
>>> general?  It's probably not terribly bulletproof, but any worse than NFS?
>
>> Yes, it is a lot worse than NFS from experience. I can't find a
>> reference to it anywhere now, but IIRC there are bigger issues - with
>> blocksizes, with syncing not properly, with write ordering.
>
> Hmm.  I searched the list archives and couldn't find any previous
> discussion of such things, but that may just prove that no one thinks
> it's worth attempting.

Yeah, I don't think it was in our archives, it was somewhere else.

And as a disclaime r- it may have been about the win32 cifs *client*.
It was at the time just talking windows cifs client -> windows cifs
server.

> Anyway the immediate question is which errnos are reasonable for copydir
> to ignore.  Just looking at the standard's description of fsync's error
> conditions:
>
>        The fsync() function shall fail if:
>        [EBADF]
>        The fildes argument is not a valid descriptor.
>        [EINTR]
>        The fsync() function was interrupted by a signal.
>        [EINVAL]
>        The fildes argument does not refer to a file on which this operation is possible.
>        [EIO]
>        An I/O error occurred while reading from or writing to the file system.
>
> it seems like EINVAL is a considerably more reasonable thing to return
> than EBADF, if the filesystem is trying to tell you that it won't fsync
> a directory.  So I'm a bit surprised this question hasn't come up for
> other filesystems.

Agreed. But do we really want to accept this with fsync=on? It
basically means fsync=maybe, no?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Hi Tom,
 
Thanks for the solution. CIFS worked with fsync flag by ingnoring EINVAL in copydir.c.
 
I tested fsync with 8.2.2 version of PostgreSQL, it worked fine without EINVAL patch. I wanted to know is something changed in version 9.0.4 of postgreSQL.
 
As fsync flag was not working with PostgreSQL version 9.0.4 without applying the patch.
 
Regards,
Anjali

________________________________
From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: anjali_524(at)yahoo(dot)co(dot)in; pgsql-bugs(at)postgresql(dot)org
Sent: Tuesday, 3 January 2012 2:00 AM
Subject: Re: [BUGS] BUG #6372: Error while creating database with fsync parameter as on incase of CIFS

On Mon, Jan 2, 2012 at 21:28, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> On Mon, Jan 2, 2012 at 21:14, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> I'm wondering what's your basis for asserting we don't support CIFS in
>>> general?  It's probably not terribly bulletproof, but any worse than NFS?
>
>> Yes, it is a lot worse than NFS from experience. I can't find a
>> reference to it anywhere now, but IIRC there are bigger issues - with
>> blocksizes, with syncing not properly, with write ordering.
>
> Hmm.  I searched the list archives and couldn't find any previous
> discussion of such things, but that may just prove that no one thinks
> it's worth attempting.

Yeah, I don't think it was in our archives, it was somewhere else.

And as a disclaime r- it may have been about the win32 cifs *client*.
It was at the time just talking windows cifs client -> windows cifs
server.

> Anyway the immediate question is which errnos are reasonable for copydir
> to ignore.  Just looking at the standard's description of fsync's error
> conditions:
>
>        The fsync() function shall fail if:
>        [EBADF]
>        The fildes argument is not a valid descriptor.
>        [EINTR]
>        The fsync() function was interrupted by a signal.
>        [EINVAL]
>        The fildes argument does not refer to a file on which this operation is possible.
>        [EIO]
>        An I/O error occurred while reading from or writing to the file system.
>
> it seems like EINVAL is a considerably more reasonable thing to return
> than EBADF, if the filesystem is trying to tell you that it won't fsync
> a directory.  So I'm a bit surprised this question hasn't come up for
> other filesystems.

Agreed. But do we really want to accept this with fsync=on? It
basically means fsync=maybe, no?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Mon, Jan 2, 2012 at 04:00:19PM -0500, Tom Lane wrote:
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> > Excerpts from Tom Lane's message of lun ene 02 17:28:33 -0300 2012:
> >> it seems like EINVAL is a considerably more reasonable thing to return
> >> than EBADF, if the filesystem is trying to tell you that it won't fsync
> >> a directory. So I'm a bit surprised this question hasn't come up for
> >> other filesystems.
>
> > Probably because other filesystems do allow you to fsync directories.
> > In fact for some cases they _require_ it ... remember the fiasco when
> > MTA writers were told that they needed to fsync their queue dirs in
> > order for all queued email to persist?
>
> Yeah, the long and the short of it is that if the filesystem won't
> accept an fsync on a directory, we have to assume that it doesn't need
> it and will manage metadata persistence safely without prodding.
>
> The only real question here is whether an EINVAL could mean something
> besides "fsync on directory is not accepted". If there are any
> scenarios where it represents a transient/fixable error, then we'd
> want to report it. It's far from clear to me that there are any
> though. What it could mean in general is not at issue, because we
> know the target is a directory that we just created moments before.

I assume this never got resolved. Should it be changed to ignore
EINVAL?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +