data file corruption

Hi All,

We are facing one strange problem about data file corruptions.

We have many postgres databases. At some point, one simple query on one
database started crashing back-end.

The query is

select count(*), col1 from tab1 group by col1;

After using pg_filedump (http://pgfoundry.org/projects/pgfiledump/) on data
files for tab1 (relnodeid in pg_class), we found that the number of
attributes per tuple is different for few tuples in data file than the rest.

pg_filedump utility prints the contents of each tuple along with block
header, data header.

In our case, the same data file has the following two data headers.

valid header

============

<Data> ------

Item 1 – Length: 114 Offset: 32648 (0x7f88) Flags: NORMAL

XMIN: 8849668 XMAX: 0 CID|XVAC: 0

Block Id: 0 linp Index: 1 Attributes: 10 Size: 24

infomask: 0x0902 (HASVARWIDTH|XMIN_COMMITTED|XMAX_INVALID)

Invalid header

==========

<Data> ------

Item 1 – Length: 234 Offset: 32528 (0x7f10) Flags: NORMAL

XMIN: 2959623 XMAX: 0 CID|XVAC: 0

Block Id: 155 linp Index: 1 *Attributes: 92* Size: 40

infomask: 0x0903 (HASNULL|HASVARWIDTH|XMIN_COMMITTED|XMAX_INVALID)

As you can see, # of attributes and XMIN are different.

I have few questions regarding this.

1. When such case can occur?

2. Later we found that invalid header is actually valid header for other
table's data file. So somehow data of one table got inserted into another
and both tables have different # of attributes. Can this be possible? Any
hardware issue can cause this?

3. Has anybody seen this problem?

Thanking you.

- Nachiket