Re: [HACKERS] Binary cursors, numerics and other types

Theo Kramer <theo(at)flame(dot)co(dot)za> writes:
> I am using the libpq interface with binary cursors and am using numeric
> fields. There seems to be no conversion routines available in the
> front end library for numeric types. Am I missing something or do
> I have to roll my own from numeric.c as per the backend?
> I also can't find anything in libpq on dates.
> Should appropriate conversion routines exist in libpq?

It is not libpq's job to try to deal with binary data from the server
--- for one thing, libpq may be compiled on a different architecture
with a different representation than the server is (wrong endianness,
different floating point format, etc). libpq doesn't even have any
way of finding out whether a conversion is needed, let alone doing it.

In the current scheme of things, binary cursors are of very limited use,
and you are *really* foolish if you try to use them for anything except
the most primitive data types like "int4". Your code will break without
warning whenever Jan feels like changing the internal representation of
numeric, as I believe he intends to do soon. We have never guaranteed
that the internal representation of date/time types is frozen, either
--- Thomas has been heard muttering about replacing timestamp with
datetime, for example.

There has been some talk of creating a CORBA interface to Postgres,
which would make use of binary representations for the basic data types
safer, since I believe CORBA offers facilities for cross-platform
transfer of binary integers (floats too? not sure). But I don't think
that would extend to nonstandard Postgres datatypes.

regards, tom lane

Tom Lane wrote:
> It is not libpq's job to try to deal with binary data from the server
> --- for one thing, libpq may be compiled on a different architecture
> with a different representation than the server is (wrong endianness,
> different floating point format, etc). libpq doesn't even have any
> way of finding out whether a conversion is needed, let alone doing it.
>
> In the current scheme of things, binary cursors are of very limited use,
> and you are *really* foolish if you try to use them for anything except
> the most primitive data types like "int4". Your code will break without
> warning whenever Jan feels like changing the internal representation of
> numeric, as I believe he intends to do soon. We have never guaranteed
> that the internal representation of date/time types is frozen, either
> --- Thomas has been heard muttering about replacing timestamp with
> datetime, for example.

Hmm, a real pity. Always thought that it was the responsibility of the
persistant store to provide a heterogenous interface. Oh well back
to messy error prone conversions to and from strings.

--------
Regards
Theo

> There has been some talk of creating a CORBA interface to Postgres,
> which would make use of binary representations for the basic data types
> safer, since I believe CORBA offers facilities for cross-platform
> transfer of binary integers (floats too? not sure). But I don't think
> that would extend to nonstandard Postgres datatypes.

All Postgres data types could be described easily using Corba
constructs. And all of those types would then have complete binary
compatibility across platforms. There are also facilities to
dynamically map declarations from clients to servers and vica versa,
though one takes a performance hit to do so.

We've been using Corba (the ACE/TAO toolset) to build realtime systems
at work. We're just getting far enough to start deploying some testbed
systems in a month or two. And in a lot of ways Corba is really great.
One thing to worry about is that afaik there is no single ORB which
supports as many platforms as we do. So we'd have to support multiple
ORBS to get at all of our targets, which will be a royal pain.

- Thomas

--
Thomas Lockhart lockhart(at)alumni(dot)caltech(dot)edu
South Pasadena, California