Re: PL/Python result set slicing broken in Python 3

This doesn't work anymore with Python 3:

rv = plpy.execute(...)
do_something(rv[0:1])

Apparently, they changed the C API for doing slicing, or rather made one
of the two APIs for it silently do nothing. Details are difficult to
find, but this email message seems to contain something:
<http://mail.python.org/pipermail/python-3000/2007-August/009851.html>.

I'll try to sort this out sometime, but if someone wants to take a shot
at it, go ahead.

On 02/05/12 20:18, Peter Eisentraut wrote:
> This doesn't work anymore with Python 3:
>
> rv = plpy.execute(...)
> do_something(rv[0:1])
>
> Apparently, they changed the C API for doing slicing, or rather made one
> of the two APIs for it silently do nothing. Details are difficult to
> find, but this email message seems to contain something:
> <http://mail.python.org/pipermail/python-3000/2007-August/009851.html>.
>
> I'll try to sort this out sometime, but if someone wants to take a shot
> at it, go ahead.

Sounds ugly. I'll take a look.

Cheers,
Jan

On 03/05/12 11:04, Jan Urbański wrote:
> On 02/05/12 20:18, Peter Eisentraut wrote:
>> This doesn't work anymore with Python 3:
>>
>> rv = plpy.execute(...)
>> do_something(rv[0:1])
>>
>> Apparently, they changed the C API for doing slicing, or rather made one
>> of the two APIs for it silently do nothing. Details are difficult to
>> find, but this email message seems to contain something:
>> <http://mail.python.org/pipermail/python-3000/2007-August/009851.html>.
>>
>> I'll try to sort this out sometime, but if someone wants to take a shot
>> at it, go ahead.
>
> Sounds ugly. I'll take a look.

I found some instructions on how to deal with the Python 2/Python 3
slicing mess:

http://renesd.blogspot.com/2009/07/python3-c-api-simple-slicing-sqslice.html

Apparently you need that egregious hack in order to avoid code
duplication. I'll try to produce a patch over the weekend.

Cheers,
Jan

On 04/05/12 20:00, Jan Urbański wrote:
> On 03/05/12 11:04, Jan Urbański wrote:
>> On 02/05/12 20:18, Peter Eisentraut wrote:
>>> This doesn't work anymore with Python 3:
>>>
>>> rv = plpy.execute(...)
>>> do_something(rv[0:1])
>>>
>> Sounds ugly. I'll take a look.
>
> I found some instructions on how to deal with the Python 2/Python 3
> slicing mess:
>
> http://renesd.blogspot.com/2009/07/python3-c-api-simple-slicing-sqslice.html

Thanks to the helpful folk at #python I found out that the fix is much
easier. Attached is a patch that fixes the bug and passes regression
tests on Pythons 2.3 through 3.2.

Apparently once you implement PyMappingMethods.mp_subscript you can drop
PySequenceMethods.sq_slice, but I guess there's no harm in keeping it
(and I'm not sure it'd work on Python 2.3 with only mp_subscript
implemented).

Do we want to backpatch this? If so, I'd need to produce a version that
applies to the monolithic plpython.c file from the previous releases.

Cheers,
Jan

On Sat, May 5, 2012 at 4:45 PM, Jan Urbański <wulczer(at)wulczer(dot)org> wrote:
>> I found some instructions on how to deal with the Python 2/Python 3
>> slicing mess:
>>
>>
>> http://renesd.blogspot.com/2009/07/python3-c-api-simple-slicing-sqslice.html
>
>
> Thanks to the helpful folk at #python I found out that the fix is much
> easier. Attached is a patch that fixes the bug and passes regression tests
> on Pythons 2.3 through 3.2.
>
> Apparently once you implement PyMappingMethods.mp_subscript you can drop
> PySequenceMethods.sq_slice, but I guess there's no harm in keeping it (and
> I'm not sure it'd work on Python 2.3 with only mp_subscript implemented).
>
> Do we want to backpatch this? If so, I'd need to produce a version that
> applies to the monolithic plpython.c file from the previous releases.

Did this get forgotten about?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On tor, 2012-05-10 at 12:37 -0400, Robert Haas wrote:
> On Sat, May 5, 2012 at 4:45 PM, Jan Urbański <wulczer(at)wulczer(dot)org> wrote:
> >> I found some instructions on how to deal with the Python 2/Python 3
> >> slicing mess:
> >>
> >>
> >> http://renesd.blogspot.com/2009/07/python3-c-api-simple-slicing-sqslice.html
> >
> >
> > Thanks to the helpful folk at #python I found out that the fix is much
> > easier. Attached is a patch that fixes the bug and passes regression tests
> > on Pythons 2.3 through 3.2.
> >
> > Apparently once you implement PyMappingMethods.mp_subscript you can drop
> > PySequenceMethods.sq_slice, but I guess there's no harm in keeping it (and
> > I'm not sure it'd work on Python 2.3 with only mp_subscript implemented).
> >
> > Do we want to backpatch this? If so, I'd need to produce a version that
> > applies to the monolithic plpython.c file from the previous releases.
>
> Did this get forgotten about?

I'm working on it.

On lör, 2012-05-05 at 22:45 +0200, Jan Urbański wrote:
> Apparently once you implement PyMappingMethods.mp_subscript you can
> drop PySequenceMethods.sq_slice, but I guess there's no harm in
> keeping it (and I'm not sure it'd work on Python 2.3 with only
> mp_subscript implemented).

Committed this now.

From test coverage reports, I now see that PLy_result_ass_item() is no
longer called. That's probably OK, if assignments are now handled
through the mapping methods. But should we remove the function then?

>
> Do we want to backpatch this? If so, I'd need to produce a version
> that applies to the monolithic plpython.c file from the previous
> releases.

I don't think this should be backpatched.

On 10/05/12 19:45, Peter Eisentraut wrote:
> On lör, 2012-05-05 at 22:45 +0200, Jan Urbański wrote:
>> Apparently once you implement PyMappingMethods.mp_subscript you can
>> drop PySequenceMethods.sq_slice, but I guess there's no harm in
>> keeping it (and I'm not sure it'd work on Python 2.3 with only
>> mp_subscript implemented).
>
> Committed this now.
>
> From test coverage reports, I now see that PLy_result_ass_item() is no
> longer called. That's probably OK, if assignments are now handled
> through the mapping methods. But should we remove the function then?

Have you tried on Python 2.3 as well? People on #python said that if you
implement the mapping functions, the sequence slicing functions are no
longer used, but maybe we should revisit for the next release, rather
than risk introducing a regression for the benefit of removing a few
dead lines.

Cheers,
Jan

On fre, 2012-05-11 at 11:28 +0200, Jan Urbański wrote:
> On 10/05/12 19:45, Peter Eisentraut wrote:
> > On lör, 2012-05-05 at 22:45 +0200, Jan Urbański wrote:
> >> Apparently once you implement PyMappingMethods.mp_subscript you can
> >> drop PySequenceMethods.sq_slice, but I guess there's no harm in
> >> keeping it (and I'm not sure it'd work on Python 2.3 with only
> >> mp_subscript implemented).
> >
> > Committed this now.
> >
> > From test coverage reports, I now see that PLy_result_ass_item() is no
> > longer called. That's probably OK, if assignments are now handled
> > through the mapping methods. But should we remove the function then?
>
> Have you tried on Python 2.3 as well? People on #python said that if you
> implement the mapping functions, the sequence slicing functions are no
> longer used, but maybe we should revisit for the next release, rather
> than risk introducing a regression for the benefit of removing a few
> dead lines.

I did test Python 2.3, but you're right, we should leave this alone
during beta.