Re: [Re] [Re] Re: PANIC: could not write to log file

On 21/11/12 18:10:12, mailto:jeff(dot)janes(at)gmail(dot)com wrote:
> On Wed, Nov 21, 2012 at 8:51 AM, Cyril VELTER <cyril(dot)velter(at)metadys(dot)com> wrote:
> >
> > After upgrading a pretty big database (serveral hundred gig) from 8.2 to 9.2 I'm getting some "PANIC: could not write to log file" messages. Actually I
> > got two of them. One yesterday and one today.
>
> How was the upgrade done?

The upgrade was done with the following steps :

* create a new cluster using 9.2.1 initdb
* run pg_dump | psql using the 9.2.1 binairies on a linux machines (both servers the old 8.2 and the new 9.2.1 running on windows machines).

Thanks,

cyril

On 21/11/12 20:39:01 mailto:cyril(dot)velter(at)metadys(dot)com wrote:
> On 21/11/12 18:10:12, mailto:jeff(dot)janes(at)gmail(dot)com wrote:
> > On Wed, Nov 21, 2012 at 8:51 AM, Cyril VELTER <cyril(dot)velter(at)metadys(dot)com> wrote:
> > >
> > > After upgrading a pretty big database (serveral hundred gig) from 8.2 to 9.2 I'm getting some "PANIC: could not write to log file" messages. Actually I
> > > got two of them. One yesterday and one today.
> >
> > How was the upgrade done?
>
> The upgrade was done with the following steps :
>
> * create a new cluster using 9.2.1 initdb
> * run pg_dump | psql using the 9.2.1 binairies on a linux machines (both
> servers the old 8.2 and the new 9.2.1 running on windows machines).

I follow up on my previous message. Just got two more crash today very similar to the first ones :

PANIC: could not write to log file 118, segment 237 at offset 2686976, length 5578752: Invalid argument
STATEMENT: COMMIT
PANIC: could not write to log file 118, segment 227 at offset 9764864, length 57344: Invalid argument
STATEMENT: COMMIT

for memory the first ones are

PANIC: could not write to log file 117, segment 117 at offset 5660672, length 4096000: Invalid argument
STATEMENT: COMMIT
PANIC: could not write to log file 118, segment 74 at offset 12189696, length 475136: Invalid argument
STATEMENT: COMMIT

I dug a bit in the source code and the error is in XLogWrite in xlog.c. MSDN states that an EINVAL return code from write mean that the buffer is NULL which seem pretty strange. Any help on what I can do to solve this situation would be greatly apreciated.

One thing I forgot to mention on my first message. I'm running the 32 bits binaries on a win2003 64 bits and no antivirus is running on the machine.

Regards,

Cyril VELTER

"Cyril VELTER" <cyril(dot)velter(at)metadys(dot)com> writes:
> I follow up on my previous message. Just got two more crash today very similar to the first ones :

> PANIC: could not write to log file 118, segment 237 at offset 2686976, length 5578752: Invalid argument
> STATEMENT: COMMIT
> PANIC: could not write to log file 118, segment 227 at offset 9764864, length 57344: Invalid argument
> STATEMENT: COMMIT

> for memory the first ones are

> PANIC: could not write to log file 117, segment 117 at offset 5660672, length 4096000: Invalid argument
> STATEMENT: COMMIT
> PANIC: could not write to log file 118, segment 74 at offset 12189696, length 475136: Invalid argument
> STATEMENT: COMMIT

Hm, those write lengths seem rather large. What have you got
wal_buffers set to? Does it help if you back it off to whatever you
were using in the 8.2 installation? (The default back in 8.2 days
seems to have been 64kB, if that helps.)

regards, tom lane

On 23/11/12 17:24:48 mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us wrote :
> "Cyril VELTER" <cyril(dot)velter(at)metadys(dot)com> writes:
> > I follow up on my previous message. Just got two more crash today very similar to the first ones :
>
> > PANIC: could not write to log file 118, segment 237 at offset 2686976, length 5578752: Invalid argument
> > STATEMENT: COMMIT
> > PANIC: could not write to log file 118, segment 227 at offset 9764864, length 57344: Invalid argument
> > STATEMENT: COMMIT
>
> Hm, those write lengths seem rather large. What have you got
> wal_buffers set to? Does it help if you back it off to whatever you
> were using in the 8.2 installation? (The default back in 8.2 days
> seems to have been 64kB, if that helps.)

I changed wal_buffer to 64kb friday evening. It worked flawlessly this week end (but with very litlle load). I got another PANIC this morning with the same kind of message, but just before there is a FATAL :

FATAL: could not create signal handler thread
PANIC: could not write to log file 121, segment 168 at offset 9592832, length 8192: Invalid argument

Unfortunatly there is no error code in the FATAL message.

Best regards,

cyril