Re: PostgreSQL crashes

Hi,

We're using postgresql-8.3.5-1.

Lately, once in a while, the PostgreSQL service crashes and the
following appears in the log. Any idea what could cause that?

I believe there are two problems here: one is the process 2604 exiting
with code 128, and the other one is the re-initialization failure with
the following fatal message: "BST FATAL: pre-existing shared memory
block is still in use".

2009-05-27 10:39:41 BST LOG: server process (PID 2604) exited with exit
code 128

2009-05-27 10:39:41 BST LOG: terminating any other active server
processes

2009-05-27 10:39:41 BST WARNING: terminating connection because of
crash of another server process

2009-05-27 10:39:41 BST DETAIL: The postmaster has commanded this
server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.

2009-05-27 10:39:41 BST HINT: In a moment you should be able to
reconnect to the database and repeat your command.

2009-05-27 10:39:41 BST WARNING: terminating connection because of
crash of another server process

2009-05-27 10:39:41 BST DETAIL: The postmaster has commanded this
server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.

2009-05-27 10:39:41 BST HINT: In a moment you should be able to
reconnect to the database and repeat your command.

2009-05-27 10:39:41 BST WARNING: terminating connection because of
crash of another server process

2009-05-27 10:39:41 BST DETAIL: The postmaster has commanded this
server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.

2009-05-27 10:39:41 BST HINT: In a moment you should be able to
reconnect to the database and repeat your command.

2009-05-27 10:39:41 BST WARNING: terminating connection because of
crash of another server process

2009-05-27 10:39:41 BST DETAIL: The postmaster has commanded this
server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.

2009-05-27 10:39:41 BST HINT: In a moment you should be able to
reconnect to the database and repeat your command.

2009-05-27 10:39:41 BST WARNING: terminating connection because of
crash of another server process

2009-05-27 10:39:41 BST DETAIL: The postmaster has commanded this
server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.

2009-05-27 10:39:41 BST HINT: In a moment you should be able to
reconnect to the database and repeat your command.

2009-05-27 10:39:41 BST WARNING: terminating connection because of
crash of another server process

2009-05-27 10:39:41 BST DETAIL: The postmaster has commanded this
server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.

2009-05-27 10:39:41 BST HINT: In a moment you should be able to
reconnect to the database and repeat your command.

2009-05-27 10:39:41 BST LOG: all server processes terminated;
reinitializing

2009-05-27 10:39:42 BST FATAL: pre-existing shared memory block is
still in use

2009-05-27 10:39:42 BST HINT: Check if there are any old server
processes still running, and terminate them.

Thanks,

Moshe.

The information contained in this message is proprietary to the sender, protected from disclosure, and may be privileged. The information is intended to be conveyed only to the designated recipient(s) of the message. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, use, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you.

************************************************************************************
This footnote confirms that this email message has been scanned by
PineApp Mail-SeCure for the presence of malicious code, vandals & computer viruses.
************************************************************************************

"Moshe Ben-Shoham" <mosheb(at)nexperience(dot)com> writes:
> 2009-05-27 10:39:41 BST LOG: server process (PID 2604) exited with exit
> code 128

What platform is this on?

regards, tom lane

It's on Windows Server 2003

Thanks,
Moshe.

-----Original Message-----
From: Tom Lane [mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us]
Sent: Wednesday, May 27, 2009 5:33 PM
To: Moshe Ben-Shoham
Cc: pgsql-admin(at)postgresql(dot)org
Subject: Re: [ADMIN] PostgreSQL crashes

"Moshe Ben-Shoham" <mosheb(at)nexperience(dot)com> writes:
> 2009-05-27 10:39:41 BST LOG: server process (PID 2604) exited with
exit
> code 128

What platform is this on?

regards, tom lane

************************************************************************
************
This footnote confirms that this email message has been scanned by
PineApp Mail-SeCure for the presence of malicious code, vandals &
computer viruses.
************************************************************************
************

The information contained in this message is proprietary to the sender, protected from disclosure, and may be privileged. The information is intended to be conveyed only to the designated recipient(s) of the message. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, use, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you.

************************************************************************************
This footnote confirms that this email message has been scanned by
PineApp Mail-SeCure for the presence of malicious code, vandals & computer viruses.
************************************************************************************

"Moshe Ben-Shoham" <mosheb(at)nexperience(dot)com> writes:
> I believe there are two problems here: one is the process 2604 exiting
> with code 128, and the other one is the re-initialization failure with
> the following fatal message: "BST FATAL: pre-existing shared memory
> block is still in use".

The second problem is most likely going to be resolved by this recent fix:
http://archives.postgresql.org/pgsql-committers/2009-05/msg00085.php

I have no idea what is provoking an exit(128) though.

regards, tom lane

Hi,

It just happened again, now with 8.4, so it seems like the fix mentioned
below didn't really help.

Does anyone have an idea what could cause it?

As a reminder, the log is below.

Thanks,
Moshe.

2009-09-07 12:15:49 IDTLOG: server process (PID 60524) exited with exit
code 128
2009-09-07 12:15:50 IDTLOG: terminating any other active server
processes
2009-09-07 12:15:50 IDTWARNING: terminating connection because of crash
of another server process
2009-09-07 12:15:50 IDTDETAIL: The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2009-09-07 12:15:50 IDTHINT: In a moment you should be able to
reconnect to the database and repeat your command.
2009-09-07 12:15:50 IDTWARNING: terminating connection because of crash
of another server process
2009-09-07 12:15:50 IDTDETAIL: The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2009-09-07 12:15:50 IDTHINT: In a moment you should be able to
reconnect to the database and repeat your command.
2009-09-07 12:15:50 IDTWARNING: terminating connection because of crash
of another server process
2009-09-07 12:15:50 IDTDETAIL: The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2009-09-07 12:15:50 IDTHINT: In a moment you should be able to
reconnect to the database and repeat your command.
2009-09-07 12:15:50 IDTWARNING: terminating connection because of crash
of another server process
2009-09-07 12:15:50 IDTDETAIL: The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2009-09-07 12:15:50 IDTHINT: In a moment you should be able to
reconnect to the database and repeat your command.
2009-09-07 12:15:50 IDTWARNING: terminating connection because of crash
of another server process
2009-09-07 12:15:50 IDTDETAIL: The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2009-09-07 12:15:50 IDTHINT: In a moment you should be able to
reconnect to the database and repeat your command.
2009-09-07 12:15:50 IDTWARNING: terminating connection because of crash
of another server process
2009-09-07 12:15:50 IDTDETAIL: The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2009-09-07 12:15:50 IDTHINT: In a moment you should be able to
reconnect to the database and repeat your command.
2009-09-07 12:15:50 IDTWARNING: terminating connection because of crash
of another server process
2009-09-07 12:15:50 IDTDETAIL: The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2009-09-07 12:15:50 IDTHINT: In a moment you should be able to
reconnect to the database and repeat your command.
2009-09-07 12:15:50 IDTWARNING: terminating connection because of crash
of another server process
2009-09-07 12:15:50 IDTDETAIL: The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2009-09-07 12:15:50 IDTHINT: In a moment you should be able to
reconnect to the database and repeat your command.
2009-09-07 12:15:50 IDTWARNING: terminating connection because of crash
of another server process
2009-09-07 12:15:50 IDTDETAIL: The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2009-09-07 12:15:50 IDTHINT: In a moment you should be able to
reconnect to the database and repeat your command.
2009-09-07 12:15:50 IDTWARNING: terminating connection because of crash
of another server process
2009-09-07 12:15:50 IDTDETAIL: The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2009-09-07 12:15:50 IDTHINT: In a moment you should be able to
reconnect to the database and repeat your command.
2009-09-07 12:15:51 IDTWARNING: terminating connection because of crash
of another server process
2009-09-07 12:15:51 IDTDETAIL: The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2009-09-07 12:15:51 IDTHINT: In a moment you should be able to
reconnect to the database and repeat your command.
2009-09-07 12:15:51 IDTWARNING: terminating connection because of crash
of another server process
2009-09-07 12:15:51 IDTDETAIL: The postmaster has commanded this server
process to roll back the current transaction and exit, because another
server process exited abnormally and possibly corrupted shared memory.
2009-09-07 12:15:51 IDTHINT: In a moment you should be able to
reconnect to the database and repeat your command.
2009-09-07 12:15:51 IDTLOG: archiver process (PID 94132) exited with
exit code 1
2009-09-07 12:15:52 IDTLOG: all server processes terminated;
reinitializing
2009-09-07 12:16:02 IDTFATAL: pre-existing shared memory block is still
in use
2009-09-07 12:16:02 IDTHINT: Check if there are any old server
processes still running, and terminate them.

-----Original Message-----
From: Tom Lane [mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us]
Sent: Sunday, May 31, 2009 7:39 PM
To: Moshe Ben-Shoham
Cc: pgsql-admin(at)postgresql(dot)org
Subject: Re: [ADMIN] PostgreSQL crashes

"Moshe Ben-Shoham" <mosheb(at)nexperience(dot)com> writes:
> I believe there are two problems here: one is the process 2604 exiting
> with code 128, and the other one is the re-initialization failure with
> the following fatal message: "BST FATAL: pre-existing shared memory
> block is still in use".

The second problem is most likely going to be resolved by this recent
fix:
http://archives.postgresql.org/pgsql-committers/2009-05/msg00085.php

I have no idea what is provoking an exit(128) though.

regards, tom lane

************************************************************************
************
This footnote confirms that this email message has been scanned by
PineApp Mail-SeCure for the presence of malicious code, vandals &
computer viruses.
************************************************************************
************

The information contained in this message is proprietary to the sender, protected from disclosure, and may be privileged. The information is intended to be conveyed only to the designated recipient(s) of the message. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, use, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you.

************************************************************************************
This footnote confirms that this email message has been scanned by
PineApp Mail-SeCure for the presence of malicious code, vandals & computer viruses.
************************************************************************************