Writing values to relation using bytearray ...

Hi,

I am trying to write values of different types to relation using following
code.

if(typbyval)
{
min_ba = (bytea *) palloc(len+1+VARHDRSZ);
memcpy(VARDATA(min_ba), &min_datum, len);
SET_VARSIZE(min_ba, len+VARHDRSZ);
VARDATA(min_ba)[len] = '\0';
values[Anum_pg_partition_minval -1]= (Datum)min_ba ;

max_ba = (bytea *) palloc(len+1+VARHDRSZ);
memcpy(VARDATA(max_ba), &max_datum, len);
SET_VARSIZE(max_ba, len+VARHDRSZ);
VARDATA(max_ba)[len] = '\0';
values[Anum_pg_partition_maxval -1]=(Datum)max_ba;
}
else
{
values[Anum_pg_partition_minval -1]=min_datum;
values[Anum_pg_partition_maxval -1]=max_datum;
}

These values are then written to relation using heap_form_tuple() and
simple_heap_insert() functions.

I am using following code to read the values from relation.

part_attr = heap_getattr (pg_parttup,Anum_pg_partition_maxval,
pg_partrel->rd_att,&isnull);
if ( typbyval )
{
short_datum = 0;
memcpy(&short_datum, VARDATA_ANY(part_attr), len);
part_attr = short_datum;
}
else if (len != -1 )
part_attr = (Datum)VARDATA_ANY(part_attr);

The aforementioned code works fine for types like int, data, text and I can
read values from the relation correctly. The problem arises for type
"float8" which is not "by value" type and it has fixed length (8) where I
can't read the values written to relation correctly.

Am i missing something here?

Thanking you in anticipation.

With warm regards,
--
Kedar.

On Fri, Mar 6, 2009 at 10:03 AM, Kedar Potdar <kedar(dot)potdar(at)gmail(dot)com> wrote:
>
> The aforementioned code works fine for types like int, data, text and I can
> read values from the relation correctly. The problem arises for type
> "float8" which is not "by value" type and it has fixed length (8) where I
> can't read the values written to relation correctly.
>
> Am i missing something here?

Well as you've correctly diagnosed, not all byvalue data types are
variable-length.

This code all seems unnecessary. The whole point of heap_form_datum
and heap_deform_datum/heap_getattr is that you don't have to worry
about all this. there are also functions like datumCopy() but you
probably don't even need them here, you can just put the datums you
have handy into the values[] array and pass that to heap_form_tuple --
it'll copy them into the resulting tuple so once you've formed the
tuple you don't have to worry about the lifetime of the original
datums. heap_deform_tuple() and heap_getattr can return pointers into
the original tuple so you do have to be careful to copy them if you
need them to survive the original tuple -- but you might not be
anyways.

--
greg

Thanks Greg, for showing interest.

The problem here is I need to store values of different types into bytearray
column of relation.

On Fri, Mar 6, 2009 at 4:33 PM, Greg Stark <stark(at)enterprisedb(dot)com> wrote:

> On Fri, Mar 6, 2009 at 10:03 AM, Kedar Potdar <kedar(dot)potdar(at)gmail(dot)com>
> wrote:
> >
> > The aforementioned code works fine for types like int, data, text and I
> can
> > read values from the relation correctly. The problem arises for type
> > "float8" which is not "by value" type and it has fixed length (8) where I
> > can't read the values written to relation correctly.
> >
> > Am i missing something here?
>
> Well as you've correctly diagnosed, not all byvalue data types are
> variable-length.
>
> This code all seems unnecessary. The whole point of heap_form_datum
> and heap_deform_datum/heap_getattr is that you don't have to worry
> about all this. there are also functions like datumCopy() but you
> probably don't even need them here, you can just put the datums you
> have handy into the values[] array and pass that to heap_form_tuple --
> it'll copy them into the resulting tuple so once you've formed the
> tuple you don't have to worry about the lifetime of the original
> datums. heap_deform_tuple() and heap_getattr can return pointers into
> the original tuple so you do have to be careful to copy them if you
> need them to survive the original tuple -- but you might not be
> anyways.
>
>
> --
> greg
>

On Fri, Mar 6, 2009 at 11:41 AM, Kedar Potdar <kedar(dot)potdar(at)gmail(dot)com> wrote:
> Thanks Greg, for showing interest.
>
> The problem here is I need to store values of different types into bytearray
> column of relation.

Oh, hm. I think you need to look at typlen instead of typbyval.

But you have an additional problem for typbyval types: the pointer to
Datum isn't necessarily pointing at the right bytes. I think you have
to use the GET_[1248]_BYTES macros depending on typlen. There may be a
helper function for this but I don't know of one.

--
greg

On Fri, Mar 6, 2009 at 12:01 PM, Greg Stark <stark(at)enterprisedb(dot)com> wrote:
> But you have an additional problem for typbyval types: the pointer to
> Datum isn't necessarily pointing at the right bytes. I think you have
> to use the GET_[1248]_BYTES macros depending on typlen. There may be a
> helper function for this but I don't know of one.

Actually on further thought I think I would suggest just storing the
whole datum for all typbyval types setting your bytea length to
SIZEOF_DATUM.

And use datumGetSize() for non-typbyval datums. Assuming you have
typbyval and typlen handy.

--
greg

Kedar Potdar <kedar(dot)potdar(at)gmail(dot)com> writes:
> The problem here is I need to store values of different types into bytearray
> column of relation.

Perhaps you should study the ANALYZE code. AFAICS your requirements are
not different from those of the pg_statistic data store. You should do
things the same way they are done there, if only to reduce the surprise
factor for readers of the code.

regards, tom lane

Thanks Tom for your interest.

I could find a workaround for the issue wherein the value of type
which is not stored "by value" and has fixed data length, is being
stored in string format to the relation.

While retrieving from relation, this value is converted by using
"coerce_to_specific_type()"
to its native type datum represation as required.

This is a bit of overhead and I'd like to find more efficient solution
and will look pg_statistics data store.

Regards,
-
Kedar

- sent from a mobile device.

On 3/6/09, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Kedar Potdar <kedar(dot)potdar(at)gmail(dot)com> writes:
>> The problem here is I need to store values of different types into
>> bytearray
>> column of relation.
>
> Perhaps you should study the ANALYZE code. AFAICS your requirements are
> not different from those of the pg_statistic data store. You should do
> things the same way they are done there, if only to reduce the surprise
> factor for readers of the code.
>
> regards, tom lane
>

Kedar Potdar <kedar(dot)potdar(at)gmail(dot)com> writes:
> I could find a workaround for the issue wherein the value of type
> which is not stored "by value" and has fixed data length, is being
> stored in string format to the relation.

The answer is simple: don't do that. You do not need to, and should
not, convert to string format. Again, please look at how ANALYZE
does it.

regards, tom lane