Re: SOLVED Re: _penalty gist method invoked with one key

As in subject.
What it does, it gets through picksplit, I return good values, valid unions,
etc. Than (I guess) postgres is trying to insert another value in tree, hence
penalty is called. Why one of the values penalty is called with is NULL, and
I have no idea if that's valid.
From all the examples available in contrib dir, I can deduct that it shouldn't
happend.
Now question, how shall I debug postgres in this case?
can I just run postgres, in non forking mode, on gdb?
I am running psql with all commands required to create new type, functions,
operators, and gist. So there must be separate connection available for that.

Thanks.

--
GJ

Binary system, you're either 1 or 0...
dead or alive ;)

Grzegorz Piotr Jaskiewicz <gj(at)pointblue(dot)com(dot)pl> writes:
> can I just run postgres, in non forking mode, on gdb?

Just start a normal session and then attach to the backend process with
gdb in a separate window. There's no reason to fool with a standalone
backend for 99.99% of debugging problems. With a regular session, you
can use psql to input the SQL commands, which is MUCH friendlier than a
standalone backend, and you can run additional sessions in case you'd
like to examine the state of the database while you've got your debug
backend pinned down someplace.

I tend to use the attached script so that I don't even need to manually
discover the PID to attach to, assuming there's only one backend running
(else it shows me all the plausible candidates to attach to).

Dunno about the gist-specific issue, Oleg or Teodor will have to answer
that.

regards, tom lane

#!/bin/sh

# Usage: gdblive

# tee /dev/tty is for user to see the set of procs considered
PROCS=`ps auxww | \
grep postgres: | \
grep -v -e 'grep postgres:' -e 'postgres: stats' -e 'postgres: writer' -e 'postgres: archiver' -e 'postgres: logger' | \
tee /dev/tty | \
awk '{print $2}'`

if [ `echo "$PROCS" | wc -w` -eq 1 ]
then
exec gdb $PGINSTROOT/bin/postgres -silent "$PROCS"
else
exec gdb $PGINSTROOT/bin/postgres -silent
fi

Ok, Thanks for that. Script works great.

Here's bt I get:
#0 0xb7ef26a4 in ?? ()
#1 0xb7cae460 in _IO_list_all () from /lib/tls/libc.so.6
#2 0xb7ef2ea8 in ?? ()
#3 0x00000000 in ?? ()
#4 0x0844bebc in ?? ()
#5 0x00000000 in ?? ()
#6 0xb7cadff4 in ?? () from /lib/tls/libc.so.6
#7 0xb7cadff4 in ?? () from /lib/tls/libc.so.6
#8 0x08432746 in ?? ()
#9 0xbfc24eb0 in ?? ()
#10 0xbfc252d4 in ?? ()
#11 0xbfc24fd4 in ?? ()
#12 0xbfc24fb0 in ?? ()
#13 0x0844bebc in ?? ()
#14 0x00000000 in ?? ()
#15 0xbfc24cd0 in ?? ()
#16 0xb7cadff4 in ?? () from /lib/tls/libc.so.6
#17 0xbfc24fb0 in ?? ()
#18 0xbfc24fd4 in ?? ()
#19 0xbfc24ef8 in ?? ()
#20 0x082c0b1d in FunctionCall3 (flinfo=0xbfc24cf0, arg1=3083469667,
arg2=3083469667, arg3=3083469667) at fmgr.c:1179
#21 0x082c0b1d in FunctionCall3 (flinfo=0xbfc260e4, arg1=3217183444,
arg2=3217182676, arg3=3217182640) at fmgr.c:1179
#22 0x0808730d in gistpenalty (giststate=0xbfc254e4, attno=0, key1=0xbfc252d4,
isNull1=0 '\0', key2=0xbfc24fd4, isNull2=0 '\0',
penalty=0xbfc24fb0) at gistutil.c:821
#23 0x080868fd in gistchoose (r=0xb710039c, p=0xb739b0a0 "", it=0x844beb4,
giststate=0xbfc254e4) at gistutil.c:688


I guess, what's important is #22 with two keys, both NOT nulls.
It's not a secret how my _penalty function is defined:

CREATE OR REPLACE FUNCTION enum_penalty( internal, internal, internal)
RETURNS internal
AS 'enum2916', 'enum_penalty'
LANGUAGE C STRICT;

and in C it looks like that:

PG_FUNCTION_INFO_V1(enum_penalty);

Datum enum_penalty(PG_FUNCTION_ARGS)
{
GISTENTRY *origentry = (GISTENTRY *) PG_GETARG_POINTER(0);
GISTENTRY *newentry = (GISTENTRY *) PG_GETARG_POINTER(1);
float *result = (float *) PG_GETARG_POINTER(2);
struct enumNumber *key;
struct enumNumber* ud;
float tmp1, tmp2;
ud = (struct enumNumber*) DatumGetPointer(origentry->key);
key = (struct enumNumber*) DatumGetPointer(newentry->key);

tmp2 = ((float)ud->eStart)*(1.5*(float)ud->percent);
tmp1 = ((float)key->eStart)*(1.5*(float)key->percent);

*result = tmp1 - tmp2;

PG_RETURN_POINTER(result);
}

for whatever reason here origentry->key is NULL.

--
GJ

Breakpoint 1, gistpenalty (giststate=0xbfc254e4, attno=0, key1=0xbfc252d4,
isNull1=0 '\0', key2=0xbfc24fd4, isNull2=0 '\0',
penalty=0xbfc24fb0) at gistutil.c:821
821 FunctionCall3(&giststate->penaltyFn[attno],
(gdb) p key1
$1 = (GISTENTRY *) 0xbfc252d4
(gdb) p key1->key
$2 = 0
(gdb) p key2->key
$3 = 138721324


Sorry, key1 and key2 are just GISTENTRY pointers.
so, as I can see this is NULL indeed, so my _penalty function is ok.

Seems like decompress and compress were offending here, simply
removing them from gist index create helped.
But, I still get on explain analyze that seqscan was used, rather
than gist. Even tho ~ operator is defined for gist, and that seqscan
is set to false.

On 2005-10-28, at 00:24, gj wrote:

>
>
> Breakpoint 1, gistpenalty (giststate=0xbfc254e4, attno=0,
> key1=0xbfc252d4,
> isNull1=0 '\0', key2=0xbfc24fd4, isNull2=0 '\0',
> penalty=0xbfc24fb0) at gistutil.c:821
> 821 FunctionCall3(&giststate->penaltyFn[attno],
> (gdb) p key1
> $1 = (GISTENTRY *) 0xbfc252d4
> (gdb) p key1->key
> $2 = 0
> (gdb) p key2->key
> $3 = 138721324
>
>
> Sorry, key1 and key2 are just GISTENTRY pointers.
> so, as I can see this is NULL indeed, so my _penalty function is ok.
>
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo(at)postgresql(dot)org so that
> your
> message can get through to the mailing list cleanly
>

Sorry for all this crap, this is bullocks.
reason was, one of internal functions didn't filled out length value,
and since the type is variable length, we had trouble.
Postgres wasn't copying anything, since length was 0, hence the NULL -
>key.
So here's little request. Could someone please put ASSERT somewhere
that would make it visible to ppl that are as stupid as me, and
forgot to fill that field out?
Would be great, me myself I have no idea where it should go really.

Thanks again.

p.s.. yeah, good docs on gist are still missing, I mean tutorial, or
really really good docs.
I have to thank here to AndrewSN for all the help, and have enough
patience to help me, even through I sometimes ask too many stupid
questions, or simply try to do stuff my way, rather than use his very
good suggestions. Thank You.

--
GJ

Grzegorz - it'd be great if you submitted documentation improvements :)

Grzegorz Jaskiewicz wrote:
> Sorry for all this crap, this is bullocks.
> reason was, one of internal functions didn't filled out length value,
> and since the type is variable length, we had trouble.
> Postgres wasn't copying anything, since length was 0, hence the NULL -
> >key.
> So here's little request. Could someone please put ASSERT somewhere
> that would make it visible to ppl that are as stupid as me, and forgot
> to fill that field out?
> Would be great, me myself I have no idea where it should go really.
>
> Thanks again.
>
> p.s.. yeah, good docs on gist are still missing, I mean tutorial, or
> really really good docs.
> I have to thank here to AndrewSN for all the help, and have enough
> patience to help me, even through I sometimes ask too many stupid
> questions, or simply try to do stuff my way, rather than use his very
> good suggestions. Thank You.
>

On Fri, 28 Oct 2005, Christopher Kings-Lynne wrote:

> Grzegorz - it'd be great if you submitted documentation improvements :)

I don't see any GiST specific problem in Grzegorz's case.

>
> Grzegorz Jaskiewicz wrote:
>> Sorry for all this crap, this is bullocks.
>> reason was, one of internal functions didn't filled out length value, and
>> since the type is variable length, we had trouble.
>> Postgres wasn't copying anything, since length was 0, hence the NULL -
>> >key.
>> So here's little request. Could someone please put ASSERT somewhere that
>> would make it visible to ppl that are as stupid as me, and forgot to fill
>> that field out?
>> Would be great, me myself I have no idea where it should go really.
>>
>> Thanks again.
>>
>> p.s.. yeah, good docs on gist are still missing, I mean tutorial, or
>> really really good docs.
>> I have to thank here to AndrewSN for all the help, and have enough
>> patience to help me, even through I sometimes ask too many stupid
>> questions, or simply try to do stuff my way, rather than use his very good
>> suggestions. Thank You.
>>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
>

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

On 2005-10-28, at 07:37, Oleg Bartunov wrote:

> On Fri, 28 Oct 2005, Christopher Kings-Lynne wrote:
>
>
>> Grzegorz - it'd be great if you submitted documentation
>> improvements :)
>>
>
> I don't see any GiST specific problem in Grzegorz's case.
>
Other than just stupid bug, I know. It was just hard to find.
Luckily, I have test cases for internal functions. This is separate
program, that I can valgrind (me hugs valgrind).
Valgrind showed me that there is branch based on not initialized
value. So I thought, maybe also length isn't initialized there, and I
was right.
So, here's a tip from me: if your type has variable length, create
separate function to locate all memory, and to fill out all fields.
In my case it was possible, and helped.

Where is about docs, yes, I do plan to put out some tut based on my
experiences. I will definitely give it to you guys to review here.

Thanks.