Re: Reducing NUMERIC size for 8.3

We previously discussed compressing the numeric data type for small values:

http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php

We didn't do this for 8.3 but in any case Tom did suggest we ought to reverse
the weight and sign/dscale so we could do this sometime without introducing
another incompatibility.

I think we also should move the NumericData and declaration to numeric.c and
make the Numeric type an opaque pointer for the rest of the source tree. That
will catch any contrib modules or third-party modules which would be broken by
any new data representation.

--- numeric.h 27 Feb 2007 23:48:10 +0000 1.24
+++ numeric.h 24 Sep 2007 16:07:24 +0100
@@ -63,8 +63,8 @@
typedef struct NumericData
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int16 n_weight; /* Weight of 1st digit */
uint16 n_sign_dscale; /* Sign + display scale */
+ int16 n_weight; /* Weight of 1st digit */
char n_data[1]; /* Digits (really array of NumericDigit) */
} NumericData;

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com

Gregory Stark <stark(at)enterprisedb(dot)com> writes:
> We previously discussed compressing the numeric data type for small values:
> http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php

> We didn't do this for 8.3 but in any case Tom did suggest we ought to reverse
> the weight and sign/dscale so we could do this sometime without introducing
> another incompatibility.

I had forgotten about that, but it does seem like a good idea to do it now.
Any objections?

> I think we also should move the NumericData and declaration to numeric.c and
> make the Numeric type an opaque pointer for the rest of the source
> tree.

I don't agree with that; we are not in the habit of doing it that way
for any other on-disk data type. All it will accomplish is to force
people to make private copies of the struct declaration, thereby
entirely guaranteeing that they fail to track changes. There will
always be legitimate reasons for external code to want to look at
on-disk bits.

regards, tom lane

"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
>
>> I think we also should move the NumericData and declaration to numeric.c and
>> make the Numeric type an opaque pointer for the rest of the source
>> tree.
>
> I don't agree with that; we are not in the habit of doing it that way
> for any other on-disk data type. All it will accomplish is to force
> people to make private copies of the struct declaration, thereby
> entirely guaranteeing that they fail to track changes. There will
> always be legitimate reasons for external code to want to look at
> on-disk bits.

Well the macros to do so would become quite a bit more complex. I imagine they
would become functions instead. I suppose a reasonable simple interface could
be ginned up. But anyone currently accessing the data directly would have to
go through the functions to access the bits.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com

Tom Lane wrote:
> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
>> We previously discussed compressing the numeric data type for small values:
>> http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php
>
>> We didn't do this for 8.3 but in any case Tom did suggest we ought to reverse
>> the weight and sign/dscale so we could do this sometime without introducing
>> another incompatibility.
>
> I had forgotten about that, but it does seem like a good idea to do it now.
> Any objections?

For in-place upgrade purpose It would be good change also OID for
numeric type and preserve current OID for current implementation on
updated system.

Zdenek

Zdenek Kotala wrote:
> Tom Lane wrote:
>> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
>>> We previously discussed compressing the numeric data type for small
>>> values:
>>> http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php
>>
>>> We didn't do this for 8.3 but in any case Tom did suggest we ought
>>> to reverse
>>> the weight and sign/dscale so we could do this sometime without
>>> introducing
>>> another incompatibility.
>>
>> I had forgotten about that, but it does seem like a good idea to do
>> it now.
>> Any objections?
>
> For in-place upgrade purpose It would be good change also OID for
> numeric type and preserve current OID for current implementation on
> updated system.
>
>
>

If we want to get into that game we need a better way of allocating
Oids. Right now anything not currently used is liable to be grabbed, so
there's a high risk of reuse.

cheers

andrew

Andrew Dunstan wrote:

>
>
> Zdenek Kotala wrote:
>
>> Tom Lane wrote:
>>
>>> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
>>>
>>>> We previously discussed compressing the numeric data type for small
>>>> values:
>>>> http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php
>>>
>>>
>>>> We didn't do this for 8.3 but in any case Tom did suggest we ought
>>>> to reverse
>>>> the weight and sign/dscale so we could do this sometime without
>>>> introducing
>>>> another incompatibility.
>>>
>>>
>>> I had forgotten about that, but it does seem like a good idea to do
>>> it now.
>>> Any objections?
>>
>>
>> For in-place upgrade purpose It would be good change also OID for
>> numeric type and preserve current OID for current implementation on
>> updated system.
>>
>>
>>
>
>
> If we want to get into that game we need a better way of allocating
> Oids. Right now anything not currently used is liable to be grabbed,
> so there's a high risk of reuse.

Yes, it will be necessary. Or maybe second way is create only really
base types (name, int, bool ...) on bootstrap and others types will be
created in standard manner by CREATE TYPE, CREATE OPERATOR ...
commands. Or third way is not remove old datatypes and only rename them
to e.g. numeric_old1 ...

Zdenek

On 9/24/07, Gregory Stark <stark(at)enterprisedb(dot)com> wrote:
>
>
> We previously discussed compressing the numeric data type for small values:
>
> http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php
>
> We didn't do this for 8.3 but in any case Tom did suggest we ought to reverse
> the weight and sign/dscale so we could do this sometime without introducing
> another incompatibility.
>
> I think we also should move the NumericData and declaration to numeric.c and
> make the Numeric type an opaque pointer for the rest of the source tree. That
> will catch any contrib modules or third-party modules which would be broken by
> any new data representation.
>
>
> --- numeric.h 27 Feb 2007 23:48:10 +0000 1.24
> +++ numeric.h 24 Sep 2007 16:07:24 +0100
> @@ -63,8 +63,8 @@
> typedef struct NumericData
> {
> int32 vl_len_; /* varlena header (do not touch directly!) */
> - int16 n_weight; /* Weight of 1st digit */
> uint16 n_sign_dscale; /* Sign + display scale */
> + int16 n_weight; /* Weight of 1st digit */
> char n_data[1]; /* Digits (really array of NumericDigit) */
> } NumericData;

would this break any application pulling a numeric field as binary
over the protocol?

merlin

"Merlin Moncure" <mmoncure(at)gmail(dot)com> writes:
> would this break any application pulling a numeric field as binary
> over the protocol?

No.

regards, tom lane