Re: plperl/plperlu interaction

Recently while doing a little research on how we could do perl module
preloading nicely, I constructed the following:

create function loadmods() returns void language plperlu as $$
use LWP::UserAgent;
$$;
select loadmods();
create function loadurl() returns text language plperl as $$
my $ua = LWP::UserAgent->new;
my $response = $ua->get('http://search.cpan.org/');
return $response->as_string;
$$;
select loadurl();

This works because plperl and plperlu share a common interpreter. I have
thought some about whether or not it is a security risk, and decided it
probably isn't, because only a superuser could construct the plperlu
function to load the external module - if an ordinary user tried it in
trusted plperl code there would be a perl error generated. It remains
true that a plperl function cannot on its own get access to an external
module, and to that extent we haven't broken the trust criteria. The
only way I know of in which we could actually prevent this effect would
be to run separate interpreters for plperl and plperlu. That wouldn't be
a great tragedy on its own, as perl interpreters aren't hugely heavy
objects, but we would probably break some legacy code, and it would take
a not insignificant coding effort. So we'd want to be very sure we
wanted to do that - personally I can live with this easily enough - the
superuser just has to be careful what they do. In cases of paranoia they
could use Symbol::delete_package() when they were done with the module,
although constantly loading and unloading a module won't perform very
nicely.

Anyway, it is probably not expected by many users that loading a module
in plperlu makes it available to plperl - I was slightly surprised
myself to see it work and I am probably more aware than most of perl and
plperl subtleties. I think therefore that at least this should be
documented.

thoughts?

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Anyway, it is probably not expected by many users that loading a module
> in plperlu makes it available to plperl - I was slightly surprised
> myself to see it work and I am probably more aware than most of perl and
> plperl subtleties.

I think that is a bug and needs to be fixed. We have the precedent of
pltcl, which uses separate interpreters for pltcl and pltclu for exactly
this reason.

regards, tom lane

Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>
>> Anyway, it is probably not expected by many users that loading a module
>> in plperlu makes it available to plperl - I was slightly surprised
>> myself to see it work and I am probably more aware than most of perl and
>> plperl subtleties.
>>
>
> I think that is a bug and needs to be fixed. We have the precedent of
> pltcl, which uses separate interpreters for pltcl and pltclu for exactly
> this reason.
>
>

Fair enough.

I am not sure what our release timetable is - and presumably this should
also be backpatched if we regard it as a bug. I won't be able to do much
on this front for the next 2 weeks at least.

cheers

andrew

Andrew Dunstan wrote:
> Tom Lane wrote:
>> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>>
>>> Anyway, it is probably not expected by many users that loading a
>>> module in plperlu makes it available to plperl - I was slightly
>>> surprised myself to see it work and I am probably more aware than
>>> most of perl and plperl subtleties.
>>>
>>
>> I think that is a bug and needs to be fixed. We have the precedent of
>> pltcl, which uses separate interpreters for pltcl and pltclu for exactly
>> this reason.
>>
>>
>
> Fair enough.
>
> I am not sure what our release timetable is - and presumably this
> should also be backpatched if we regard it as a bug. I won't be able
> to do much on this front for the next 2 weeks at least.
>

There is one other wrinkle, that has just come to my attention courtesy
of Andrew(at)SuperNews(dot) This is what the perlembed man page says:

Now suppose we have more than one interpreter instance running at the
same time. This is feasible, but only if you used the Configure
option
"-Dusemultiplicity" or the options "-Dusethreads -Duseithreads" when
building perl.

Now my local perl (FC5/ia64) has usemultiplicity defined. I am not sure
how common this is.

Perhaps people who use other platforms could look for these flags in the
output of
perl -e 'use Config qw(myconfig config_sh config_vars config_re);
print config_sh();'

cheers

andrew

On Thu, Oct 26, 2006 at 03:15:00PM -0400, Andrew Dunstan wrote:
> Perhaps people who use other platforms could look for these flags in the
> output of
> perl -e 'use Config qw(myconfig config_sh config_vars config_re);
> print config_sh();'

My Debian Sarge (i386) has:

useithreads='define'
usethreads='define'
usemultiplicity='define'

Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Now suppose we have more than one interpreter instance running at the
> same time. This is feasible, but only if you used the Configure
> option
> "-Dusemultiplicity" or the options "-Dusethreads -Duseithreads" when
> building perl.

> Now my local perl (FC5/ia64) has usemultiplicity defined. I am not sure
> how common this is.

Ouch. It's certainly not the default configuration :-(

regards, tom lane

On Oct 26, 2006, at 3:23 PM, Martijn van Oosterhout wrote:

> On Thu, Oct 26, 2006 at 03:15:00PM -0400, Andrew Dunstan wrote:
>> Perhaps people who use other platforms could look for these flags
>> in the
>> output of
>> perl -e 'use Config qw(myconfig config_sh config_vars config_re);
>> print config_sh();'
>

OSX 10.4.8:

usemultiplicity='define'
usethreads='define'
useithreads='define'

--
Jeff Trout <jeff(at)jefftrout(dot)com>
http://www.dellsmartexitin.com/
http://www.stuarthamm.net/

Jeff Trout wrote:
>
> On Oct 26, 2006, at 3:23 PM, Martijn van Oosterhout wrote:
>
> >On Thu, Oct 26, 2006 at 03:15:00PM -0400, Andrew Dunstan wrote:
> >>Perhaps people who use other platforms could look for these flags
> >>in the
> >>output of
> >> perl -e 'use Config qw(myconfig config_sh config_vars config_re);
> >>print config_sh();'
> >
>
> OSX 10.4.8:
>
> usemultiplicity='define'
> usethreads='define'
> useithreads='define'

Same here on Debian unstable (stock Perl packages).

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

On Thu, Oct 26, 2006 at 03:35:11PM -0400, Jeff Trout wrote:
>
> On Oct 26, 2006, at 3:23 PM, Martijn van Oosterhout wrote:
>
> >On Thu, Oct 26, 2006 at 03:15:00PM -0400, Andrew Dunstan wrote:
> >>Perhaps people who use other platforms could look for these flags
> >>in the
> >>output of
> >> perl -e 'use Config qw(myconfig config_sh config_vars config_re);
> >>print config_sh();'
> >
>
> OSX 10.4.8:
>
> usemultiplicity='define'
> usethreads='define'
> useithreads='define'

All 3 are undef on FreeBSD 6.1.
--
Jim Nasby jim(at)nasby(dot)net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

On Thu, 26 Oct 2006, Alvaro Herrera wrote:

> Jeff Trout wrote:
> >
> > On Oct 26, 2006, at 3:23 PM, Martijn van Oosterhout wrote:
> >
> > >On Thu, Oct 26, 2006 at 03:15:00PM -0400, Andrew Dunstan wrote:
> > >>Perhaps people who use other platforms could look for these flags
> > >>in the
> > >>output of
> > >> perl -e 'use Config qw(myconfig config_sh config_vars config_re);
> > >>print config_sh();'
> > >
> >
> > OSX 10.4.8:
> >
> > usemultiplicity='define'
> > usethreads='define'
> > useithreads='define'
>
> Same here on Debian unstable (stock Perl packages).

On my current Gentoo box:
useithreads='undef'
usemultiplicity='undef'
usethreads='undef'

My USE flags have ithreads disabled, since the description of the feature
is "Enable Perl threads, has some compatibility problems"

--
Whether you can hear it or not
The Universe is laughing behind your back
-- National Lampoon, "Deteriorata"

Jeremy Drake wrote:
> On Thu, 26 Oct 2006, Alvaro Herrera wrote:
>
>
>> Jeff Trout wrote:
>>
>>> On Oct 26, 2006, at 3:23 PM, Martijn van Oosterhout wrote:
>>>
>>>
>>>> On Thu, Oct 26, 2006 at 03:15:00PM -0400, Andrew Dunstan wrote:
>>>>
>>>>> Perhaps people who use other platforms could look for these flags
>>>>> in the
>>>>> output of
>>>>> perl -e 'use Config qw(myconfig config_sh config_vars config_re);
>>>>> print config_sh();'
>>>>>
>>> OSX 10.4.8:
>>>
>>> usemultiplicity='define'
>>> usethreads='define'
>>> useithreads='define'
>>>
>> Same here on Debian unstable (stock Perl packages).
>>
>
> On my current Gentoo box:
> useithreads='undef'
> usemultiplicity='undef'
> usethreads='undef'
>
> My USE flags have ithreads disabled, since the description of the feature
> is "Enable Perl threads, has some compatibility problems"
>
>
>
On my Ubuntu 'Dapper' system:
useithreads='define'
usemultiplicity='define'
usethreads='define'

And I'm getting 'undef' for each of these flags on both Gentoo 2006.1
and Gentoo 1.4 systems using the default perl installation.

Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>
>> Now suppose we have more than one interpreter instance running at the
>> same time. This is feasible, but only if you used the Configure
>> option
>> "-Dusemultiplicity" or the options "-Dusethreads -Duseithreads" when
>> building perl.
>>
>
>
>> Now my local perl (FC5/ia64) has usemultiplicity defined. I am not sure
>> how common this is.
>>
>
> Ouch. It's certainly not the default configuration :-(
>
>
>

Well, so far many Linux platforms look OK, but FBSD does not.

This could be ugly ;-(

cheers

andrew

Andrew Dunstan wrote:
> Tom Lane wrote:
>> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>>
>>> Now suppose we have more than one interpreter instance running
>>> at the
>>> same time. This is feasible, but only if you used the
>>> Configure option
>>> "-Dusemultiplicity" or the options "-Dusethreads
>>> -Duseithreads" when
>>> building perl.
>>>
>>
>>
>>> Now my local perl (FC5/ia64) has usemultiplicity defined. I am not
>>> sure how common this is.
>>>
>>
>> Ouch. It's certainly not the default configuration :-(
>>
>>
>>
>
> Well, so far many Linux platforms look OK, but FBSD does not.

OpenBSD (which has perl in base) also has those 3 NOT defined ...

>
> This could be ugly ;-(

yeah ...

Stefan

Andrew,

> My Debian Sarge (i386) has:
>
> useithreads='define'
> usethreads='define'
> usemultiplicity='define'

I get the same on Ubuntu and SuSE 9.3, so I think those are pervasive
settings for Linux.

Solaris 10update1:

useithreads='undef'
usethreads='undef'
usemultiplicity='undef'

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

On 10/27/06, Jim C. Nasby <jim(at)nasby(dot)net> wrote:
Undef in Slackware 10.2
Def in Ubuntu 6.06
Undef in Mandriva 2006
Undef in Solaris 10 06
Def in SLES 9.2
Perl 5.8 in SLES 8.1 throws a fit:
"Array found where operator expected at
/usr/lib/perl5/5.8.0/warnings.pm line 294, at end of line
(Missing operator before ?)
Undefined subroutine &main::config_sh called at -e line 2."

Perl 5.004 in solaris 6&7 does't doesn't do config_re,
neither does the perl 5.6 in Solaris 9

Andrej Ricnik-Bay wrote:
> On 10/27/06, Jim C. Nasby <jim(at)nasby(dot)net> wrote:
> Undef in Slackware 10.2
> Def in Ubuntu 6.06
> Undef in Mandriva 2006
> Undef in Solaris 10 06
> Def in SLES 9.2
> Perl 5.8 in SLES 8.1 throws a fit:
> "Array found where operator expected at
> /usr/lib/perl5/5.8.0/warnings.pm line 294, at end of line
> (Missing operator before ?)
> Undefined subroutine &main::config_sh called at -e line 2."
>
>
> Perl 5.004 in solaris 6&7 does't doesn't do config_re,
> neither does the perl 5.6 in Solaris 9
>

You can also examine the output from perl -V

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> You can also examine the output from perl -V

I think we've already established that we won't be able to ignore the
case of not having support for multiple perl interpreters :-(

So it seems we have these choices:

1. Do nothing (document it as a feature not a bug)

2. Support separate interpreters if possible, do nothing if not
(still needs documentation)

3. Support separate interpreters if possible, refuse to run both plperl
and plperlu functions in the same backend if not.

Any other compromises possible?

regards, tom lane

Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>
>> You can also examine the output from perl -V
>>
>
> I think we've already established that we won't be able to ignore the
> case of not having support for multiple perl interpreters :-(
>
> So it seems we have these choices:
>
> 1. Do nothing (document it as a feature not a bug)
>
> 2. Support separate interpreters if possible, do nothing if not
> (still needs documentation)
>
> 3. Support separate interpreters if possible, refuse to run both plperl
> and plperlu functions in the same backend if not.
>
> Any other compromises possible?
>
>

How would we decide which wins in the third case? "first in" seems
rather arbitrary. If we went that way I'd probably plump for just
plperlu to be allowed. The the worst effect would be that the functions
would have to be created by the superuser. It would be a great pity, of
course - this threatens to do horrible things to portability ;-(

I guess another possibility would be to allow 3 to be overridden by a
switch to become 2.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Tom Lane wrote:
>> 3. Support separate interpreters if possible, refuse to run both plperl
>> and plperlu functions in the same backend if not.

> How would we decide which wins in the third case? "first in" seems
> rather arbitrary. If we went that way I'd probably plump for just
> plperlu to be allowed.

"First used in a given backend" was exactly what I had in mind.
Certainly it wouldn't be perfect, but your proposal seems to be
"disable plperl altogether if no separate-interpreter support",
which seems overly harsh. Especially for someone who doesn't
even want to install plperlu.

regards, tom lane

Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> Anyway, it is probably not expected by many users that loading a module
>> in plperlu makes it available to plperl - I was slightly surprised
>> myself to see it work and I am probably more aware than most of perl and
>> plperl subtleties.
>
> I think that is a bug and needs to be fixed. We have the precedent of
> pltcl, which uses separate interpreters for pltcl and pltclu for exactly
> this reason.

If this is fixed, what becomes the mechanism for an administrator to make a perl
module available to plperl functions? I didn't see any other way to do this
documented. Thanks,

mark

Mark Dilger wrote:
> Tom Lane wrote:
>> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>>> Anyway, it is probably not expected by many users that loading a
module
>>> in plperlu makes it available to plperl - I was slightly surprised
myself to see it work and I am probably more aware than most of perl
and
>>> plperl subtleties.
>> I think that is a bug and needs to be fixed. We have the precedent of
pltcl, which uses separate interpreters for pltcl and pltclu for
exactly
>> this reason.
>
> If this is fixed, what becomes the mechanism for an administrator to
make
> a perl
> module available to plperl functions? I didn't see any other way to do
this
> documented. Thanks,
>

This isn't documented either :-)

I discovered this when I was working on a way of doing this nicely and
safely. I hope to have that for 8.3.

cheers

andrew