Re: pl/python custom exceptions for SPI

Lists: pgsql-hackers
From: Jan Urbański <wulczer(at)wulczer(dot)org>
To: Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: pl/python custom exceptions for SPI
Date: 2010-12-23 14:40:43
Message-ID: 4D135F6B.6020809@wulczer.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Here's a patch implementing custom Python exceptions for SPI errors
mentioned in
http://archives.postgresql.org/pgsql-hackers/2010-12/msg01991.php. It's
an incremental patch on top of the explicit-subxacts patch sent eariler.

Git branch for this patch:
https://github.com/wulczer/postgres/tree/custom-spi-exceptions.

What the patch does is provide a Python exception per each error defined
in utils/errcodes.h, and then raise the corresponding exception when a
SPI call fails. The parsing of errcodes.h in the Makefile is a little
grotty and would probably have to be ported to the Windows build system,
which I have no idea about.

With this patch you can do:

from plpy import spiexceptions

try:
plpy.execute("insert into t values (4)")
catch spiexceptions.UniqueViolation:
plpy.notice("unique constraint violation")
catch spiexceptions.NotNullViolation:
plpy.notice("not null constraint violation")

All exceptions inherint from plpy.SPIError, so code thta just catches a
blanket SPIErorr will continue to work.

The patch lacks user-facing docs, if it gets accepted I'll write some.
Not sure if we should provide a table similar to
http://www.postgresql.org/docs/current/static/errcodes-appendix.html, or
just refer to that table and state that the rule is changing underscores
to camel case...

Also, I just realised that this patch does not really depend on the
explicit-subxacts change, but rather only on the spi-in-subxacts, so if
need be I can generate it as an incremental changeset ofer the latter
and not the former.

Cheers,
Jan

Attachment Content-Type Size
plpython-custom-spi-exceptions.diff text/x-patch 18.0 KB

From: Jan Urbański <wulczer(at)wulczer(dot)org>
To: Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-01-10 16:20:44
Message-ID: 4D2B31DC.4010307@wulczer.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 23/12/10 15:40, Jan Urbański wrote:
> Here's a patch implementing custom Python exceptions for SPI errors
> mentioned in
> http://archives.postgresql.org/pgsql-hackers/2010-12/msg01991.php. It's
> an incremental patch on top of the explicit-subxacts patch sent eariler.

I changed that patch to use Perl instead of sed to generate the
exceptions, which should be a more portable. It's still not nice, and I
think the way forward is to have a common format for SQLSTATE
conditions, as proposed in
http://archives.postgresql.org/message-id/4D19C93C.5000703@wulczer.org.

I failed to follow on with that patch because I couldn't figure out how
to persuade the buildsystem to generate errcodes.h early enough for the
rest of the system to compile, not to mention doing it for the MSVC
build system.

Cheers,
Jan

Attachment Content-Type Size
plpython-custom-spi-exceptions.diff text/x-patch 18.4 KB

From: Hannu Krosing <hannu(at)2ndquadrant(dot)com>
To: Jan Urbański <wulczer(at)wulczer(dot)org>
Cc: Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-01-10 23:21:28
Message-ID: 4D2B9478.3080405@2ndquadrant.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 10.1.2011 17:20, Jan Urbański wrote:
> On 23/12/10 15:40, Jan Urbański wrote:
>> Here's a patch implementing custom Python exceptions for SPI errors
>> mentioned in
>> http://archives.postgresql.org/pgsql-hackers/2010-12/msg01991.php. It's
>> an incremental patch on top of the explicit-subxacts patch sent eariler.
> I changed that patch to use Perl instead of sed to generate the
> exceptions, which should be a more portable.
Why not python ?
> It's still not nice, and I
> think the way forward is to have a common format for SQLSTATE
> conditions, as proposed in
> http://archives.postgresql.org/message-id/4D19C93C.5000703@wulczer.org.
>
> I failed to follow on with that patch because I couldn't figure out how
> to persuade the buildsystem to generate errcodes.h early enough for the
> rest of the system to compile, not to mention doing it for the MSVC
> build system.
>
> Cheers,
> Jan
>
>
>

--
--------------------------------------------
Hannu Krosing
Senior Consultant,
Infinite Scalability& Performance
http://www.2ndQuadrant.com/books/


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Hannu Krosing <hannu(at)2ndquadrant(dot)com>
Cc: Jan Urbański <wulczer(at)wulczer(dot)org>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-01-11 00:27:44
Message-ID: 3044.1294705664@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hannu Krosing <hannu(at)2ndquadrant(dot)com> writes:
> On 10.1.2011 17:20, Jan Urbaski wrote:
>> I changed that patch to use Perl instead of sed to generate the
>> exceptions, which should be a more portable.

> Why not python ?

Because we're not adding even more different tool requirements to the
build process. Perl is what we've chosen to depend on, and there is no
reason to use a different tool here.

regards, tom lane


From: Jan Urbański <wulczer(at)wulczer(dot)org>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Hannu Krosing <hannu(at)2ndquadrant(dot)com>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-01-11 11:20:09
Message-ID: 4D2C3CE9.8010805@wulczer.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 11/01/11 01:27, Tom Lane wrote:
> Hannu Krosing <hannu(at)2ndquadrant(dot)com> writes:
>> On 10.1.2011 17:20, Jan Urbański wrote:
>>> I changed that patch to use Perl instead of sed to generate the
>>> exceptions, which should be a more portable.
>
>> Why not python ?
>
> Because we're not adding even more different tool requirements to the
> build process. Perl is what we've chosen to depend on, and there is no
> reason to use a different tool here.

Yep, exactly.

While looking at it again I fixed a problem with spiexceptions.h not
being distributed (and IIRC we don't require Perl for tarball builds)
and added generating it to the MSVC build system.

Updated patch attached.

Cheers,
Jan

Attachment Content-Type Size
plpython-custom-spi-exceptions.diff text/x-patch 19.6 KB

From: Jan Urbański <wulczer(at)wulczer(dot)org>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Hannu Krosing <hannu(at)2ndquadrant(dot)com>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-01-27 22:24:28
Message-ID: 4D41F09C.3050901@wulczer.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 11/01/11 12:20, Jan Urbański wrote:
> On 11/01/11 01:27, Tom Lane wrote:
>> Hannu Krosing <hannu(at)2ndquadrant(dot)com> writes:
>>> On 10.1.2011 17:20, Jan Urbański wrote:
>>>> I changed that patch to use Perl instead of sed to generate the
>>>> exceptions, which should be a more portable.

Updated as an incremental patch on to of the recently sent version of
explicit-subxacts.

Attachment Content-Type Size
plpython-custom-spi-exceptions.diff text/x-patch 21.7 KB

From: Jan Urbański <wulczer(at)wulczer(dot)org>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Hannu Krosing <hannu(at)2ndquadrant(dot)com>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-02-06 19:44:33
Message-ID: 4D4EFA21.1010605@wulczer.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 27/01/11 23:24, Jan Urbański wrote:
> On 11/01/11 12:20, Jan Urbański wrote:
>> On 11/01/11 01:27, Tom Lane wrote:
>>> Hannu Krosing <hannu(at)2ndquadrant(dot)com> writes:
>>>> On 10.1.2011 17:20, Jan Urbański wrote:
>>>>> I changed that patch to use Perl instead of sed to generate the
>>>>> exceptions, which should be a more portable.
>
> Updated as an incremental patch on to of the recently sent version of
> explicit-subxacts.

Updated again.

Attachment Content-Type Size
plpython-custom-spi-exceptions.patch text/x-patch 21.3 KB

From: Peter Eisentraut <peter_e(at)gmx(dot)net>
To: Jan Urbański <wulczer(at)wulczer(dot)org>
Cc: Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-02-10 19:24:16
Message-ID: 1297365856.27142.8.camel@vanquo.pezone.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On sön, 2011-02-06 at 20:44 +0100, Jan Urbański wrote:
> On 27/01/11 23:24, Jan Urbański wrote:
> > On 11/01/11 12:20, Jan Urbański wrote:
> >> On 11/01/11 01:27, Tom Lane wrote:
> >>> Hannu Krosing <hannu(at)2ndquadrant(dot)com> writes:
> >>>> On 10.1.2011 17:20, Jan Urbański wrote:
> >>>>> I changed that patch to use Perl instead of sed to generate the
> >>>>> exceptions, which should be a more portable.
> >
> > Updated as an incremental patch on to of the recently sent version of
> > explicit-subxacts.
>
> Updated again.

Why do the error messages print spiexceptions.SyntaxError instead of
plpy.spiexceptions.SyntaxError? Is this intentional or just the way it
comes out of Python?

Please add some documentation. Not a list of all exceptions, but at
least a paragraph that various kinds of specific exceptions may be
generated, what package and module they are in, and how they relate.


From: Jan Urbański <wulczer(at)wulczer(dot)org>
To: Peter Eisentraut <peter_e(at)gmx(dot)net>
Cc: Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-02-10 20:13:55
Message-ID: 4D544703.7070206@wulczer.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 10/02/11 20:24, Peter Eisentraut wrote:
> On sön, 2011-02-06 at 20:44 +0100, Jan Urbański wrote:
>> On 27/01/11 23:24, Jan Urbański wrote:
>>> On 11/01/11 12:20, Jan Urbański wrote:
>>>> On 11/01/11 01:27, Tom Lane wrote:
>>>>> Hannu Krosing <hannu(at)2ndquadrant(dot)com> writes:
>>>>>> On 10.1.2011 17:20, Jan Urbański wrote:
>>>>>>> I changed that patch to use Perl instead of sed to generate the
>>>>>>> exceptions, which should be a more portable.
>>>
>>> Updated as an incremental patch on to of the recently sent version of
>>> explicit-subxacts.
>>
>> Updated again.
>
> Why do the error messages print spiexceptions.SyntaxError instead of
> plpy.spiexceptions.SyntaxError? Is this intentional or just the way it
> comes out of Python?

That's how traceback.format_exception() works IIRC, which is what the
Python interpreter uses and what PL/Python mimicks in PLy_traceback.

> Please add some documentation. Not a list of all exceptions, but at
> least a paragraph that various kinds of specific exceptions may be
> generated, what package and module they are in, and how they relate.

Sure, Steve already asked for docs in another thread, and I'm writing them.

Jan


From: Steve Singer <ssinger_pg(at)sympatico(dot)ca>
To: Jan Urbański <wulczer(at)wulczer(dot)org>
Cc: Peter Eisentraut <peter_e(at)gmx(dot)net>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-02-10 21:26:13
Message-ID: BLU0-SMTP40AAF26FCFBBEA47005A5F8EEC0@phx.gbl
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 11-02-10 03:13 PM, Jan Urbański wrote:
> On 10/02/11 20:24, Peter Eisentraut wrote:

Here is the rest of my review.

Submission Review
-------------------
Patch applies cleanly.
Documentation is still outstanding but Jan has promised it soon.

Usability Review
-------------------
We don't have this for plpython, that we have a similar idea with
plpgsql. I think this feature is useful and worth having.

The CamelCase naming of the exceptions is consistent with how the
built-in python exceptions are named (camel case).

Feature Test
---------------
I did basic testing of the feature (catching a few exception types
thrown from both direct SQL and prepared statements) and the feature
worked as expected.

Performance Impact
--------------------
The impact of mapping error codes to exception types shouldn't come into
play unless an SPI error is returned and with the hash it should still
be minimal.

Code Review
-------------

Ideally char * members of ExceptionMap would be const, but since many
versions of python take a non-const value to PyErr_NewException that
won't work :(

After you search the for an exception in the hash you have:

/* We really should find it, but just in case have a fallback */
Assert(entry != NULL);
exc = entry ? entry->exc : PLy_exc_spi_error;

I'm not sure the assert is needed here. Just falling back to the
exception type seems reasonable and more desirable than an assert if
showhow a new exception gets missed from the list. I don't feel that
strongly on this.

line 3575: PLy_elog(ERROR, "Failed to add the spiexceptions module");
"Failed" should be "failed"

Other than that the patch looks fine to me.

>>>
>>> Updated again.
>>
>> Why do the error messages print spiexceptions.SyntaxError instead of
>> plpy.spiexceptions.SyntaxError? Is this intentional or just the way it
>> comes out of Python?
>
> That's how traceback.format_exception() works IIRC, which is what the
> Python interpreter uses and what PL/Python mimicks in PLy_traceback.
>
>> Please add some documentation. Not a list of all exceptions, but at
>> least a paragraph that various kinds of specific exceptions may be
>> generated, what package and module they are in, and how they relate.
>
> Sure, Steve already asked for docs in another thread, and I'm writing them.
>
> Jan
>


From: Jan Urbański <wulczer(at)wulczer(dot)org>
To: Steve Singer <ssinger_pg(at)sympatico(dot)ca>
Cc: Peter Eisentraut <peter_e(at)gmx(dot)net>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-02-11 09:53:01
Message-ID: 4D5506FD.8050604@wulczer.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 10/02/11 22:26, Steve Singer wrote:
> On 11-02-10 03:13 PM, Jan Urbański wrote:
>> On 10/02/11 20:24, Peter Eisentraut wrote:
>
> Here is the rest of my review.

Thanks!

> Ideally char * members of ExceptionMap would be const, but since many
> versions of python take a non-const value to PyErr_NewException that
> won't work :(

Yeah, I got "discards qualifier" warnings when I tried to declare it as
const :(

> After you search the for an exception in the hash you have:
>
> /* We really should find it, but just in case have a fallback */
> Assert(entry != NULL);
> exc = entry ? entry->exc : PLy_exc_spi_error;
>
> I'm not sure the assert is needed here. Just falling back to the
> exception type seems reasonable and more desirable than an assert if
> showhow a new exception gets missed from the list. I don't feel that
> strongly on this.

Maybe the comment doesn't explain this enough. My intention was that in
regular builds you have a fallback and if you're missing an entry in the
exceptions hash, you just get SPIError. But in assert-enabled builds you
get an error, so you can detect it and fix the exceptions hash.

> line 3575: PLy_elog(ERROR, "Failed to add the spiexceptions module");
> "Failed" should be "failed"

Gah, I'll never learn. Will fix.

> Other than that the patch looks fine to me.

Thanks, I'll have the docs ready by today (and I should've have them by
yesterday :/).

Jan


From: Jan Urbański <wulczer(at)wulczer(dot)org>
To: Steve Singer <ssinger_pg(at)sympatico(dot)ca>
Cc: Peter Eisentraut <peter_e(at)gmx(dot)net>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-02-12 10:58:50
Message-ID: 4D5667EA.4040304@wulczer.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 11/02/11 10:53, Jan Urbański wrote:
> On 10/02/11 22:26, Steve Singer wrote:

Here's an updated patch with documentation. It's an incremental patch on
top of the latest explicit-subxacts version.

Cheers,
Jan

Attachment Content-Type Size
plpython-custom-spi-exceptions.patch text/x-patch 24.6 KB

From: Steve Singer <ssinger_pg(at)sympatico(dot)ca>
To: Jan Urbański <wulczer(at)wulczer(dot)org>
Cc: Peter Eisentraut <peter_e(at)gmx(dot)net>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-02-13 17:49:14
Message-ID: BLU0-SMTP103FBE80215C64D6A193808ED10@phx.gbl
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 11-02-12 05:58 AM, Jan Urbański wrote:
> On 11/02/11 10:53, Jan Urbański wrote:
>> On 10/02/11 22:26, Steve Singer wrote:
> Here's an updated patch with documentation. It's an incremental patch on
> top of the latest explicit-subxacts version.
>

This looks fine. I've attached a one word documentation change to go o
top of the patch.

I'll let Peter decide if he likes those assert's or not. I don't have a
good enough sense of if we often use asserts in that fashion elsewhere.

> Cheers,
> Jan
>
>
>

Attachment Content-Type Size
plpython_custom_exceptions_docpatch.diff text/x-patch 898 bytes

From: Peter Eisentraut <peter_e(at)gmx(dot)net>
To: Jan Urbański <wulczer(at)wulczer(dot)org>
Cc: Steve Singer <ssinger_pg(at)sympatico(dot)ca>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-02-28 16:52:31
Message-ID: 1298911951.26119.1.camel@vanquo.pezone.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On lör, 2011-02-12 at 11:58 +0100, Jan Urbański wrote:
> On 11/02/11 10:53, Jan Urbański wrote:
> > On 10/02/11 22:26, Steve Singer wrote:
>
> Here's an updated patch with documentation. It's an incremental patch on
> top of the latest explicit-subxacts version.

Committed.


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Peter Eisentraut <peter_e(at)gmx(dot)net>
Cc: Jan Urbański <wulczer(at)wulczer(dot)org>, Steve Singer <ssinger_pg(at)sympatico(dot)ca>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-02-28 17:08:48
Message-ID: 18694.1298912928@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> On lr, 2011-02-12 at 11:58 +0100, Jan Urbaski wrote:
>> Here's an updated patch with documentation. It's an incremental patch on
>> top of the latest explicit-subxacts version.

> Committed.

I'm seeing a core dump as well as multiple inconsistencies in error
message spelling in the plpython regression tests on a Fedora 13 box
(python 2.6.4). Several buildfarm critters don't look too happy either.

regards, tom lane


From: Peter Eisentraut <peter_e(at)gmx(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Jan Urbański <wulczer(at)wulczer(dot)org>, Steve Singer <ssinger_pg(at)sympatico(dot)ca>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-02-28 17:45:45
Message-ID: 1298915145.26119.12.camel@vanquo.pezone.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On mån, 2011-02-28 at 12:08 -0500, Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > On lör, 2011-02-12 at 11:58 +0100, Jan Urbański wrote:
> >> Here's an updated patch with documentation. It's an incremental patch on
> >> top of the latest explicit-subxacts version.
>
> > Committed.
>
> I'm seeing a core dump as well as multiple inconsistencies in error
> message spelling in the plpython regression tests on a Fedora 13 box
> (python 2.6.4). Several buildfarm critters don't look too happy either.

Fixed. (Well, some of it. We'll see ...)


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Peter Eisentraut <peter_e(at)gmx(dot)net>
Cc: Jan Urbański <wulczer(at)wulczer(dot)org>, Steve Singer <ssinger_pg(at)sympatico(dot)ca>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-02-28 18:38:58
Message-ID: 26971.1298918338@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> On mn, 2011-02-28 at 12:08 -0500, Tom Lane wrote:
>> I'm seeing a core dump as well as multiple inconsistencies in error
>> message spelling in the plpython regression tests on a Fedora 13 box
>> (python 2.6.4). Several buildfarm critters don't look too happy either.

> Fixed. (Well, some of it. We'll see ...)

Core dump is still there. It appears to be a python assertion failure.
I installed python's debuginfo and got this backtrace:

Program terminated with signal 6, Aborted.
#0 0x00000032a36328f5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
Missing separate debuginfos, use: debuginfo-install keyutils-libs-1.2-6.fc12.x86_64 krb5-libs-1.7.1-17.fc13.x86_64 libcom_err-1.41.10-7.fc13.x86_64 libselinux-2.0.94-2.fc13.x86_64 openssl-1.0.0c-1.fc13.x86_64 zlib-1.2.3-23.fc12.x86_64
(gdb) bt
#0 0x00000032a36328f5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x00000032a36340d5 in abort () at abort.c:92
#2 0x00000032a362b8b5 in __assert_fail (assertion=0x32a5b46391 "gc->gc.gc_refs != 0", file=<value optimized out>, line=277, function=<value optimized out>)
at assert.c:81
#3 0x00000032a5b0853e in visit_decref (op=<module at remote 0x7f11c3666d38>, data=<value optimized out>) at Modules/gcmodule.c:277
#4 0x00000032a5a7cbd9 in dict_traverse (op=
{'info': <built-in function info>, 'notice': <built-in function notice>, 'Fatal': <type at remote 0x1bba7e0>, 'log': <built-in function log>, 'prepare': <built-in function prepare>, 'spiexceptions': <module at remote 0x7f11c3666d38>, 'SPIError': <type at remote 0x1bbacc0>, 'Error': <type at remote 0x1bba300>, 'execute': <built-in function execute>, '__package__': None, 'quote_ident': <built-in function quote_ident>, 'warning': <built-in function warning>, 'subtransaction': <built-in function subtransaction>, 'quote_literal': <built-in function quote_literal>, 'quote_nullable': <built-in function quote_nullable>, 'error': <built-in function error>, 'debug': <built-in function debug>, '__name__': 'plpy', 'fatal': <built-in function fatal>, '__doc__': None}, visit=0x32a5b084c0 <visit_decref>, arg=0x0)
at Objects/dictobject.c:2003
#5 0x00000032a5b08c9f in subtract_refs (generation=1) at Modules/gcmodule.c:296
#6 collect (generation=1) at Modules/gcmodule.c:817
#7 0x00000032a5b096fa in collect_generations (basicsize=<value optimized out>) at Modules/gcmodule.c:924
#8 _PyObject_GC_Malloc (basicsize=<value optimized out>) at Modules/gcmodule.c:1363
#9 0x00000032a5b0972e in _PyObject_GC_NewVar (tp=0x32a5d899a0, nitems=1) at Modules/gcmodule.c:1383
#10 0x00000032a5a9703f in PyTuple_New (size=1) at Objects/tupleobject.c:69
#11 0x00000032a5af3697 in r_object (p=0x7fffe1f5f330) at Python/marshal.c:788
#12 0x00000032a5af389c in r_object (p=0x7fffe1f5f330) at Python/marshal.c:927
#13 0x00000032a5af36dd in r_object (p=0x7fffe1f5f330) at Python/marshal.c:794
#14 0x00000032a5af389c in r_object (p=0x7fffe1f5f330) at Python/marshal.c:927
#15 0x00000032a5af36dd in r_object (p=0x7fffe1f5f330) at Python/marshal.c:794
#16 0x00000032a5af389c in r_object (p=0x7fffe1f5f330) at Python/marshal.c:927
#17 0x00000032a5af3fd8 in PyMarshal_ReadObjectFromString (str=<value optimized out>, len=<value optimized out>) at Python/marshal.c:1107
#18 0x00000032a5af5952 in PyMarshal_ReadLastObjectFromFile (fp=0x1c66e50) at Python/marshal.c:1066
#19 0x00000032a5aedb39 in read_compiled_module (cpathname=0x7fffe1f63540 "/usr/lib64/python2.6/string.pyc", fp=<value optimized out>) at Python/import.c:767
#20 0x00000032a5aef69d in load_source_module (name=0x7fffe1f656a0 "string", pathname=0x7fffe1f645c0 "/usr/lib64/python2.6/string.py", fp=0x1c29b30)
at Python/import.c:991
#21 0x00000032a5af0cb5 in import_submodule (mod=None, subname=0x7fffe1f656a0 "string", fullname=0x7fffe1f656a0 "string") at Python/import.c:2589
#22 0x00000032a5af0f34 in load_next (mod=None, altmod=None, p_name=<value optimized out>, buf=0x7fffe1f656a0 "string", p_buflen=0x7fffe1f65698)
at Python/import.c:2409
#23 0x00000032a5af1582 in import_module_level (name=0x0, globals=<value optimized out>, locals=<value optimized out>, fromlist=None, level=<value optimized out>)
at Python/import.c:2131
#24 0x00000032a5af22c4 in PyImport_ImportModuleLevel (name=0x7f11c40c2084 "string", globals=
{'plpy': <module at remote 0x7f11c3666ad0>, 'GD': {}, 'args': [], '__builtins__': <module at remote 0x7f11c4158830>, '__name__': '__main__', 'SD': {}, '__doc__': None, '__plpython_procedure_import_succeed_41194': <function at remote 0x7f11c3652aa0>, '__package__': None}, locals=None, fromlist=None,
level=<value optimized out>) at Python/import.c:2182
#25 0x00000032a5ad762f in builtin___import__ (self=<value optimized out>, args=<value optimized out>, kwds=<value optimized out>) at Python/bltinmodule.c:48
#26 0x00000032a5a43db3 in PyObject_Call (func=<built-in function __import__>, arg=<value optimized out>, kw=<value optimized out>) at Objects/abstract.c:2492
#27 0x00000032a5ad7ba3 in PyEval_CallObjectWithKeywords (func=<built-in function __import__>, arg=
('string', {'plpy': <module at remote 0x7f11c3666ad0>, 'GD': {}, 'args': [], '__builtins__': <module at remote 0x7f11c4158830>, '__name__': '__main__', 'SD': {}, '__doc__': None, '__plpython_procedure_import_succeed_41194': <function at remote 0x7f11c3652aa0>, '__package__': None}, None, None), kw=<value optimized out>)
at Python/ceval.c:3633
#28 0x00000032a5ada844 in PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2192
#29 0x00000032a5ade312 in fast_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:3850
#30 call_function (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:3785
#31 PyEval_EvalFrameEx (f=<value optimized out>, throwflag=<value optimized out>) at Python/ceval.c:2445
#32 0x00000032a5adf4e9 in PyEval_EvalCodeEx (co=0x7f11c36526c0, globals=<value optimized out>, locals=<value optimized out>, args=<value optimized out>,
argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3026
#33 0x00000032a5adf5b2 in PyEval_EvalCode (co=<value optimized out>, globals=<value optimized out>, locals=<value optimized out>) at Python/ceval.c:526
#34 0x00007f11c418cab2 in PLy_procedure_call (proc=0x1c2c9d8, kargs=<value optimized out>, vargs=<value optimized out>) at plpython.c:1317
#35 0x00007f11c418cdfe in PLy_function_handler (fcinfo=0x1c3fd30, proc=0x1c2c9d8) at plpython.c:1091
#36 0x00007f11c418fb77 in plpython_call_handler (fcinfo=0x1c3fd30) at plpython.c:569
#37 0x0000000000588100 in ExecMakeFunctionResult (fcache=0x1c3fcc0, econtext=0x1c3fa98, isNull=0x1c40670 "\177~\177\177\177\177\177\177\270\004\253\001",
isDone=<value optimized out>) at execQual.c:1826
#38 0x00000000005831b6 in ExecTargetList (projInfo=<value optimized out>, isDone=0x7fffe1f6703c) at execQual.c:5123
#39 ExecProject (projInfo=<value optimized out>, isDone=0x7fffe1f6703c) at execQual.c:5338
#40 0x000000000059b65b in ExecResult (node=0x1c3f980) at nodeResult.c:155
#41 0x0000000000582738 in ExecProcNode (node=0x1c3f980) at execProcnode.c:367
#42 0x0000000000581496 in ExecutePlan (queryDesc=0x19e4df8, direction=<value optimized out>, count=0) at execMain.c:1385
#43 standard_ExecutorRun (queryDesc=0x19e4df8, direction=<value optimized out>, count=0) at execMain.c:311
#44 0x000000000065d5d7 in PortalRunSelect (portal=0x19e2de8, forward=<value optimized out>, count=0, dest=0x1c3de28) at pquery.c:950
#45 0x000000000065e9f0 in PortalRun (portal=0x19e2de8, count=9223372036854775807, isTopLevel=1 '\001', dest=0x1c3de28, altdest=0x1c3de28, completionTag=
0x7fffe1f672a0 "") at pquery.c:794
#46 0x000000000065ae39 in exec_simple_query (query_string=0x1a70ac8 "SELECT import_succeed();") at postgres.c:1059
#47 0x000000000065bdf0 in PostgresMain (argc=<value optimized out>, argv=<value optimized out>, username=<value optimized out>) at postgres.c:3943
#48 0x0000000000615081 in BackendRun () at postmaster.c:3590
#49 BackendStartup () at postmaster.c:3275
#50 ServerLoop () at postmaster.c:1449
#51 0x000000000061795f in PostmasterMain (argc=<value optimized out>, argv=<value optimized out>) at postmaster.c:1110
#52 0x00000000005b2e70 in main (argc=3, argv=0x19c3910) at main.c:199

Don't know python enough to do anything useful with this, but the
reference to "gc_refs" sure makes it look like something is dropping the
ball on when to do INCREF/DECREF.

regards, tom lane


From: Jan Urbański <wulczer(at)wulczer(dot)org>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Peter Eisentraut <peter_e(at)gmx(dot)net>, Steve Singer <ssinger_pg(at)sympatico(dot)ca>, Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pl/python custom exceptions for SPI
Date: 2011-03-01 10:59:38
Message-ID: 4D6CD19A.9030802@wulczer.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 28/02/11 19:38, Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
>> On mån, 2011-02-28 at 12:08 -0500, Tom Lane wrote:
>>> I'm seeing a core dump as well as multiple inconsistencies in error
>>> message spelling in the plpython regression tests on a Fedora 13 box
>>> (python 2.6.4). Several buildfarm critters don't look too happy either.
>
>> Fixed. (Well, some of it. We'll see ...)
>
> Core dump is still there. It appears to be a python assertion failure.
> I installed python's debuginfo and got this backtrace:
>
> Program terminated with signal 6, Aborted.
> #0 0x00000032a36328f5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> 64 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
> Missing separate debuginfos, use: debuginfo-install keyutils-libs-1.2-6.fc12.x86_64 krb5-libs-1.7.1-17.fc13.x86_64 libcom_err-1.41.10-7.fc13.x86_64 libselinux-2.0.94-2.fc13.x86_64 openssl-1.0.0c-1.fc13.x86_64 zlib-1.2.3-23.fc12.x86_64
> (gdb) bt
> #0 0x00000032a36328f5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> #1 0x00000032a36340d5 in abort () at abort.c:92
> #2 0x00000032a362b8b5 in __assert_fail (assertion=0x32a5b46391 "gc->gc.gc_refs != 0", file=<value optimized out>, line=277, function=<value optimized out>)
> at assert.c:81
> #3 0x00000032a5b0853e in visit_decref (op=<module at remote 0x7f11c3666d38>, data=<value optimized out>) at Modules/gcmodule.c:277
> #4 0x00000032a5a7cbd9 in dict_traverse (op=
> {'info': <built-in function info>, 'notice': <built-in function notice>, 'Fatal': <type at remote 0x1bba7e0>, 'log': <built-in function log>, 'prepare': <built-in function prepare>, 'spiexceptions': <module at remote 0x7f11c3666d38>, 'SPIError': <type at remote 0x1bbacc0>, 'Error': <type at remote 0x1bba300>, 'execute': <built-in function execute>, '__package__': None, 'quote_ident': <built-in function quote_ident>, 'warning': <built-in function warning>, 'subtransaction': <built-in function subtransaction>, 'quote_literal': <built-in function quote_literal>, 'quote_nullable': <built-in function quote_nullable>, 'error': <built-in function error>, 'debug': <built-in function debug>, '__name__': 'plpy', 'fatal': <built-in function fatal>, '__doc__': None}, visit=0x32a5b084c0 <visit_decref>, arg=0x0)
> at Objects/dictobject.c:2003
> [...]
> #24 0x00000032a5af22c4 in PyImport_ImportModuleLevel (name=0x7f11c40c2084 "string", globals=
>
> Don't know python enough to do anything useful with this, but the
> reference to "gc_refs" sure makes it look like something is dropping the
> ball on when to do INCREF/DECREF.

That's strange, the error occurs while trying to import the "string"
module. But the error itself seems to be caused by trying to unref the
spiexceptions module (showing up here as <module at remote
0x7f11c3666d38>). Apparently adding spiexceptions as an object to the
plpy module is not done exactly right.

I'll try to reproduce it in my environment.

Cheers,
Jan