Unexpected plperl difference between 8.4 and 9.1

After upgrading from 8.4 to 9.1, one of my plperl functions stopped working
properly.

For some reason, when matching a string using a regex, the $1 variable
cannot be returned directly using return_next() but must be
set to a variable first.
If returned directly, it appears to be cached in some strange way,
returning the same value for all 10 rows in the example below.

In 8.4, these two functions returns the same thing, 10 rows of random
numbers.

Is this a feature or a bug?

CREATE OR REPLACE FUNCTION test1() RETURNS SETOF NUMERIC AS $BODY$
use strict;
use warnings;
for(my $i=0 ; $i<10; $i++) {
my $rand = rand();
$rand =~ m/(.*)/;
return_next($1);
}
return;
$BODY$ LANGUAGE plperl;

joel=# select * from test1();
0.482287904847535
0.482287904847535
0.482287904847535
0.482287904847535
0.482287904847535
0.482287904847535
0.482287904847535
0.482287904847535
0.482287904847535
0.482287904847535
(10 rows)

CREATE OR REPLACE FUNCTION test2() RETURNS SETOF NUMERIC AS $BODY$
use strict;
use warnings;
for(my $i=0 ; $i<10; $i++) {
my $rand = rand();
$rand =~ m/(.*)/;
my $val = $1;
return_next($val);
}
return;
$BODY$ LANGUAGE plperl;

joel=# select * from test2();
test2
--------------------
0.504361211998972
0.015757483449562
0.154422531777254
0.383329383899088
0.578318997407354
0.0022126436077059
0.970868502733449
0.465566753133679
0.215372148522395
0.390036490131536
(10 rows)

On Sun, Aug 19, 2012 at 2:26 PM, Joel Jacobson <joel(at)trustly(dot)com> wrote:

> After upgrading from 8.4 to 9.1, one of my plperl functions stopped
> working properly.
>
> For some reason, when matching a string using a regex, the $1 variable
> cannot be returned directly using return_next() but must be
> set to a variable first.
> If returned directly, it appears to be cached in some strange way,
> returning the same value for all 10 rows in the example below.
>
>
Hrm seems to work for me. What version of perl is this?
$ perl -V
Summary of my perl5 (revision 5 version 16 subversion 0) configuration:
[snip]
Characteristics of this binary (from libperl):
Compile-time options: HAS_TIMES MYMALLOC PERLIO_LAYERS
PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP
PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT
USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE
USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_PERLIO
USE_PERL_ATOF
$!psql
baroque=> SELECT version();
version

---------------------------------------------------------------------------------------------------------
PostgreSQL 9.1.5 on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.7.1
20120721 (prerelease), 64-bit
(1 row)

baroque=> CREATE OR REPLACE FUNCTION test1() RETURNS SETOF NUMERIC AS $BODY$
baroque$> use strict;
baroque$> use warnings;
baroque$> for(my $i=0 ; $i<10; $i++) {
baroque$> my $rand = rand();
baroque$> $rand =~ m/(.*)/;
baroque$> return_next($1);
baroque$> }
baroque$> return;
baroque$> $BODY$ LANGUAGE plperl;
CREATE FUNCTION
baroque=>
baroque=> select * from test1();
test1
-------------------
0.284491935120062
0.213769321886019
0.758221121077565
0.810816779589864
0.649781285447791
0.630792307420037
0.17897035660857
0.876314955338863
0.899575315174307
0.225134707347706
(10 rows)

Excerpts from Alex Hunsaker's message of lun ago 20 12:03:11 -0400 2012:
> On Sun, Aug 19, 2012 at 2:26 PM, Joel Jacobson <joel(at)trustly(dot)com> wrote:
>
> > After upgrading from 8.4 to 9.1, one of my plperl functions stopped
> > working properly.
> >
> > For some reason, when matching a string using a regex, the $1 variable
> > cannot be returned directly using return_next() but must be
> > set to a variable first.
> > If returned directly, it appears to be cached in some strange way,
> > returning the same value for all 10 rows in the example below.
> >
> >
> Hrm seems to work for me. What version of perl is this?
> $ perl -V
> Summary of my perl5 (revision 5 version 16 subversion 0) configuration:
> [snip]
> Characteristics of this binary (from libperl):
> Compile-time options: HAS_TIMES MYMALLOC PERLIO_LAYERS
> PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP
> PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT
> USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE
> USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_PERLIO
> USE_PERL_ATOF

I can reproduce the failure with 5.14.2

alvherre=# CREATE OR REPLACE FUNCTION test1() RETURNS SETOF NUMERIC AS $BODY$
alvherre$# use strict;
alvherre$# use warnings;
alvherre$# for(my $i=0 ; $i<10; $i++) {
alvherre$# my $rand = rand();
alvherre$# $rand =~ m/(.*)/;
alvherre$# return_next($1);
alvherre$# }
alvherre$# return;
alvherre$# $BODY$ LANGUAGE plperl;
CREATE FUNCTION
alvherre=# select * from test1();
test1
-------------------
0.396088311522366
0.396088311522366
0.396088311522366
0.396088311522366
0.396088311522366
0.396088311522366
0.396088311522366
0.396088311522366
0.396088311522366
0.396088311522366
(10 filas)

It works fine if I assign $1 to another variable before return_next'ing
it:

alvherre=# CREATE OR REPLACE FUNCTION test1() RETURNS SETOF NUMERIC AS $BODY$
use strict;
use warnings;
for(my $i=0 ; $i<10; $i++) {
my $rand = rand();
$rand =~ m/(.*)/;
my $a=$1; return_next($a);
}
return;
$BODY$ LANGUAGE plperl;
CREATE FUNCTION
alvherre=# select * from test1();
test1
-------------------
0.693569484473304
0.757589839023666
0.477233897467283
0.572963701418253
0.189924114046409
0.20155773007717
0.624452321926892
0.134135086596039
0.417606638502921
0.95250325772281
(10 filas)

(In short, same as Joel).

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Alvaro Herrera <alvherre(at)2ndquadrant(dot)com> writes:
> Excerpts from Alex Hunsaker's message of lun ago 20 12:03:11 -0400 2012:
>> Hrm seems to work for me. What version of perl is this?
>> $ perl -V
>> Summary of my perl5 (revision 5 version 16 subversion 0) configuration:

> I can reproduce the failure with 5.14.2

Smells like a Perl bug to me. Has anybody attempted to reproduce this
just in Perl itself, not PL/Perl?

regards, tom lane

On 2012-08-20 18:36, Tom Lane wrote:
> Alvaro Herrera<alvherre(at)2ndquadrant(dot)com> writes:
>> Excerpts from Alex Hunsaker's message of lun ago 20 12:03:11 -0400 2012:
>>> Hrm seems to work for me. What version of perl is this?
>>> $ perl -V
>>> Summary of my perl5 (revision 5 version 16 subversion 0) configuration:
>> I can reproduce the failure with 5.14.2
> Smells like a Perl bug to me. Has anybody attempted to reproduce this
> just in Perl itself, not PL/Perl?
>
> regards, tom lane
>
>
I can't reproduce it in perl, but it's in PL/perl from 9.2 beta built
with perl 5.14.2. Currently I don't have another perl with libperl
installed.

This produces the error:

CREATE OR REPLACE FUNCTION test1() RETURNS SETOF NUMERIC AS $BODY$
use strict;
use warnings;
for (0..9) {
my $rand = rand();
$rand =~ m/(.*)/;
return_next($1);
}
return;
$BODY$ LANGUAGE plperl;

Adding 'elog(NOTICE, "rand:$rand, :$1");' after the capture maked the
error go away. Do does changining the return_next statemtnt to 'my
$retvalue=$1;return_next($retvalue);'

On Mon, Aug 20, 2012 at 10:14 AM, Alvaro Herrera
<alvherre(at)2ndquadrant(dot)com>wrote:

> Excerpts from Alex Hunsaker's message of lun ago 20 12:03:11 -0400 2012:
> > On Sun, Aug 19, 2012 at 2:26 PM, Joel Jacobson <joel(at)trustly(dot)com> wrote:
> >
> > > After upgrading from 8.4 to 9.1, one of my plperl functions stopped
> > > working properly.
>
> I can reproduce the failure with 5.14.2
>
>
Me too, however it works for me with 5.14.1, looking more like a strange
perl bug.

I've tried reproducing this in straight perl but I don't think I can
without using some C code, specifically the call to SvPVutf8 in sv2cstr()
seems to be the culprit. If I change that to SvPV() it seems to work. Im
wondering if there is some strange caching of utf8 strings going on that =~
m// is not clearing.

Ill keep digging and hopefully be able to narrow it down to a commit
between 5.14.1 and 5.14.2 so we can understand more whats going here.

Alex Hunsaker <badalex(at)gmail(dot)com> writes:
>> I can reproduce the failure with 5.14.2

> Me too, however it works for me with 5.14.1, looking more like a strange
> perl bug.

Curiously, I do *not* see the bug on my Fedora 16 machine, running
perl-5.14.2-198.fc16.x86_64

I wondered if Fedora is carrying a patch that fixes it, but none of
their dozen or so patches look related:
http://pkgs.fedoraproject.org/cgit/perl.git/tree/?h=f16

Baffled...

regards, tom lane

On 08/22/2012 11:08 PM, Tom Lane wrote:
> Alex Hunsaker <badalex(at)gmail(dot)com> writes:
>>> I can reproduce the failure with 5.14.2
>> Me too, however it works for me with 5.14.1, looking more like a strange
>> perl bug.
> Curiously, I do *not* see the bug on my Fedora 16 machine, running
> perl-5.14.2-198.fc16.x86_64
>
> I wondered if Fedora is carrying a patch that fixes it, but none of
> their dozen or so patches look related:
> http://pkgs.fedoraproject.org/cgit/perl.git/tree/?h=f16
>

Possibly we need to look at the output of perl -V to see if there's a
difference.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> On 08/22/2012 11:08 PM, Tom Lane wrote:
>> Curiously, I do *not* see the bug on my Fedora 16 machine, running
>> perl-5.14.2-198.fc16.x86_64

> Possibly we need to look at the output of perl -V to see if there's a
> difference.

Mine sez

Summary of my perl5 (revision 5 version 14 subversion 2) configuration:

Platform:
osname=linux, osvers=2.6.32-220.4.1.el6.x86_64, archname=x86_64-linux-thread-multi
uname='linux x86-17.phx2.fedoraproject.org 2.6.32-220.4.1.el6.x86_64 #1 smp thu jan 19 14:50:54 est 2012 x86_64 x86_64 x86_64 gnulinux '
config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Dccdlflags=-Wl,--enable-new-dtags -DDEBUGGING=-g -Dversion=5.14.2 -Dmyhostname=localhost -Dperladmin=root(at)localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl -Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl -Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads -Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_prot!
o -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.6.3 20120306 (Red Hat 4.6.3-2)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='gcc', ldflags =' -fstack-protector'
libpth=/usr/local/lib64 /lib64 /usr/lib64
libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.14.90'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,--enable-new-dtags -Wl,-rpath,/usr/lib64/perl5/CORE'
cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic'

Characteristics of this binary (from libperl):
Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT
USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_PERL_ATOF
USE_REENTRANT_API
Built under linux
Compiled at May 30 2012 15:41:26
@INC:
/usr/local/lib64/perl5
/usr/local/share/perl5
/usr/lib64/perl5/vendor_perl
/usr/share/perl5/vendor_perl
/usr/lib64/perl5
/usr/share/perl5
.

regards, tom lane