Re: Database file compatability

"Qingqing Zhou" <zhouqq(at)cs(dot)toronto(dot)edu> writes:
> ""Jim C. Nasby"" <jnasby(at)pervasive(dot)com> wrote
>> If a database is created with a 64 bit version of initdb, would a 32bit
>> backend be able to talk to it? Likewise, would a backend compiled by a
>> different compiler be able to?

> The key problem I believe is the serials of ALIGNOF macros. Especially for
> MAX_ALIGNOF. Different Hardware/OS/compiler will have different
> understanding of it.

Yeah. It might be worth adding MAX_ALIGNOF to the set of configuration
data stored in pg_control, just to be sure you couldn't shoot yourself
in the foot that way.

regards, tom lane

""Jim C. Nasby"" <jnasby(at)pervasive(dot)com> wrote
> If a database is created with a 64 bit version of initdb, would a 32bit
> backend be able to talk to it? Likewise, would a backend compiled by a
> different compiler be able to?
>

The key problem I believe is the serials of ALIGNOF macros. Especially for
MAX_ALIGNOF. Different Hardware/OS/compiler will have different
understanding of it. Compare your two versions PG, if they match, then with
big chance, you can exchange their data.

> If there was some kind of incompatability, would the backend just refuse
> to start, or would it start and start silently trashing data?
> --

Undefined. Mostly core dump.

Regards,
Qingqing

On Mon, Sep 26, 2005 at 07:05:28PM -0400, Tom Lane wrote:
> "Qingqing Zhou" <zhouqq(at)cs(dot)toronto(dot)edu> writes:
> > ""Jim C. Nasby"" <jnasby(at)pervasive(dot)com> wrote
> >> If a database is created with a 64 bit version of initdb, would a 32bit
> >> backend be able to talk to it? Likewise, would a backend compiled by a
> >> different compiler be able to?
>
> > The key problem I believe is the serials of ALIGNOF macros. Especially for
> > MAX_ALIGNOF. Different Hardware/OS/compiler will have different
> > understanding of it.
>
> Yeah. It might be worth adding MAX_ALIGNOF to the set of configuration
> data stored in pg_control, just to be sure you couldn't shoot yourself
> in the foot that way.

PLEASE. :)

ISTM that 64 bit is becomming much more common, so I think the odds of
someone going from a 32 to 64 bit (or vice-versa) version of PostgreSQL
on the same machine is much larger now than it has been in the past. I
think we really need to protect this as much as possible. This isn't so
much a foot-gun as a foot-nuclear-weapon.

Would I be correct in assuming that doing this for 8.1 would require
another initdb? :/ For something as minor as this, would it be
reasonable to ship a utility to avoid the initdb? I'd very much like to
see this in 8.1...
--
Jim C. Nasby, Sr. Engineering Consultant jnasby(at)pervasive(dot)com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461

"Qingqing Zhou" <zhouqq(at)cs(dot)toronto(dot)edu> writes:
> I think all ALIGNOF macros should be checked.

There are no platforms for which ALIGNOF_SHORT is different from 2.
I don't think there are any platforms we care about where ALIGNOF_INT
is different from 4. The cases of interest are ALIGNOF_DOUBLE,
ALIGNOF_LONG, ALIGNOF_LONG_LONG_INT (note that MAXIMUM_ALIGNOF is
just the largest of these). In practice "long int" is the same type
as either "int" or "long long int", so ALIGNOF_LONG isn't a distinct
case either. What it comes down to is that MAXIMUM_ALIGNOF is
sufficient to tell the difference between the platforms we need to
deal with. If you have a counterexample, tell us about it.

regards, tom lane

""Jim C. Nasby"" <jnasby(at)pervasive(dot)com> wrote
>> Yeah. It might be worth adding MAX_ALIGNOF to the set of configuration
>> data stored in pg_control, just to be sure you couldn't shoot yourself
>> in the foot that way.
>
> PLEASE. :)
>

I am coming up with a patch of it. I think all ALIGNOF macros should be
checked.

Regards,
Qingqing

"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote in message
news:1223(dot)1127878072(at)sss(dot)pgh(dot)pa(dot)us(dot)(dot)(dot)
>
> There are no platforms for which ALIGNOF_SHORT is different from 2.
> I don't think there are any platforms we care about where ALIGNOF_INT
> is different from 4. The cases of interest are ALIGNOF_DOUBLE,
> ALIGNOF_LONG, ALIGNOF_LONG_LONG_INT (note that MAXIMUM_ALIGNOF is
> just the largest of these). In practice "long int" is the same type
> as either "int" or "long long int", so ALIGNOF_LONG isn't a distinct
> case either. What it comes down to is that MAXIMUM_ALIGNOF is
> sufficient to tell the difference between the platforms we need to
> deal with. If you have a counterexample, tell us about it.
>
(1)
Yes, ALIGNOF_SHORT is always 2.

(2)
There is a possible sequence like this:

ALIGNOF_LONG 4
ALIGNOF_DOUBLE 8
MAXIMUM_ALIGNOF 8

vs.

ALIGNOF_LONG 8
ALIGNOF_DOUBLE 8
MAXIMUM_ALIGNOF 8

Eg.
http://developers.sun.com/prodtech/cc/articles/about_amd64_abi.html
http://devrsrc1.external.hp.com/STK/wellbehavedrestrict.html

So we should at least check ALIGNOF_LONG as well.

(3)
There are some machines with sizeof(int) equals to 64, if my memory saves,
which might imply that ALIGNOF_INT equals to 8.

So conservatively, we'd better check ALIGNOF_INT, ALIGNOF_LONG and
MAXIMUM_ALIGNOF.

Regards,
Qingqing

"Qingqing Zhou" <zhouqq(at)cs(dot)toronto(dot)edu> writes:
> "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote in message
> There is a possible sequence like this:

> ALIGNOF_LONG 4
> ALIGNOF_DOUBLE 8
> MAXIMUM_ALIGNOF 8

> vs.

> ALIGNOF_LONG 8
> ALIGNOF_DOUBLE 8
> MAXIMUM_ALIGNOF 8

> So we should at least check ALIGNOF_LONG as well.

No, we don't need to, because we do not really care about ALIGNOF_LONG
per se. We don't use "long" as an on-disk datatype, precisely because
we don't know what size it is. We use int32 and int64. The former has
align 4 on all machines AFAIK, and the latter has MAXIMUM_ALIGNOF.

> There are some machines with sizeof(int) equals to 64, if my memory saves,
> which might imply that ALIGNOF_INT equals to 8.

If there were such a machine, Postgres wouldn't run on it anyway, and
a lot of other software too. There'd be no way to have both int16 and
int32 types ("short" could cover only one of them).

regards, tom lane

"Qingqing Zhou" <zhouqq(at)cs(dot)toronto(dot)edu> wrote
>
> "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote in message
> news:1223(dot)1127878072(at)sss(dot)pgh(dot)pa(dot)us(dot)(dot)(dot)
> >
> > There are no platforms for which ALIGNOF_SHORT is different from 2.
> > I don't think there are any platforms we care about where ALIGNOF_INT
> > is different from 4. The cases of interest are ALIGNOF_DOUBLE,
> > ALIGNOF_LONG, ALIGNOF_LONG_LONG_INT (note that MAXIMUM_ALIGNOF is
> > just the largest of these). In practice "long int" is the same type
> > as either "int" or "long long int", so ALIGNOF_LONG isn't a distinct
> > case either. What it comes down to is that MAXIMUM_ALIGNOF is
> > sufficient to tell the difference between the platforms we need to
> > deal with. If you have a counterexample, tell us about it.
> >
> (1)
> Yes, ALIGNOF_SHORT is always 2.
>
> (2)
> There is a possible sequence like this:
>
> ALIGNOF_LONG 4
> ALIGNOF_DOUBLE 8
> MAXIMUM_ALIGNOF 8
>
> vs.
>
> ALIGNOF_LONG 8
> ALIGNOF_DOUBLE 8
> MAXIMUM_ALIGNOF 8
>
> Eg.
> http://developers.sun.com/prodtech/cc/articles/about_amd64_abi.html
> http://devrsrc1.external.hp.com/STK/wellbehavedrestrict.html
>
> So we should at least check ALIGNOF_LONG as well.
>
> (3)
> There are some machines with sizeof(int) equals to 64, if my memory saves,
> which might imply that ALIGNOF_INT equals to 8.

sizeof(int) maybe 8, but not 64.
And the configure option `--enable-integer-datetimes' may affect the data
layout.

>
> So conservatively, we'd better check ALIGNOF_INT, ALIGNOF_LONG and
> MAXIMUM_ALIGNOF.
>
> Regards,
> Qingqing
>
>

"William ZHANG" <uniware(at)zedware(dot)org> wrote >
> sizeof(int) maybe 8, but not 64.
> And the configure option `--enable-integer-datetimes' may affect the data
> layout.
>

Yes, typo. This has been checked by ControlFileData.enableIntTimes.

Regards,
Qingqing

On Wed, Sep 28, 2005 at 10:22:51AM -0400, Tom Lane wrote:
> "Qingqing Zhou" <zhouqq(at)cs(dot)toronto(dot)edu> writes:
> > "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote in message
> > There is a possible sequence like this:
>
> > ALIGNOF_LONG 4
> > ALIGNOF_DOUBLE 8
> > MAXIMUM_ALIGNOF 8
>
> > vs.
>
> > ALIGNOF_LONG 8
> > ALIGNOF_DOUBLE 8
> > MAXIMUM_ALIGNOF 8
>
> > So we should at least check ALIGNOF_LONG as well.
>
> No, we don't need to, because we do not really care about ALIGNOF_LONG
> per se. We don't use "long" as an on-disk datatype, precisely because
> we don't know what size it is. We use int32 and int64. The former has
> align 4 on all machines AFAIK, and the latter has MAXIMUM_ALIGNOF.

Is there a serious penalty associated with just checking them all? Seems
like better safe than sorry...

On a related note, are checks for endianness made as well?
--
Jim C. Nasby, Sr. Engineering Consultant jnasby(at)pervasive(dot)com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461