Re: replace plugins directory with GUC

About that plugins directory ($libdir/plugins) ... I don't think we ever
really got that to work sensibly. I don't remember the original design
discussion, but I have seen a number of explanations offered over the
years. It's not clear who decides what to put in there (plugin author,
packager, DBA?), how to put it there (move it, copy it, symlink it? --
no support in pgxs), and based on what criteria.

It would seem to be much more in the spirit of things to simply list the
allowed plugins in a GUC variable, like

some_clever_name_here = $libdir/this, $libdir/that

but there is probably a reason why this wasn't done that way in the
first place.

Anyway, the current setup stinks. Can we come up with something better?

On 10.10.2012 03:45, Peter Eisentraut wrote:
> About that plugins directory ($libdir/plugins) ... I don't think we ever
> really got that to work sensibly. I don't remember the original design
> discussion, but I have seen a number of explanations offered over the
> years. It's not clear who decides what to put in there (plugin author,
> packager, DBA?), how to put it there (move it, copy it, symlink it? --
> no support in pgxs), and based on what criteria.

Yeah, it would be good to clarify that. It was originally added for the
pldebugger module
(http://archives.postgresql.org/pgsql-hackers/2006-07/msg00803.php), but
I couldn't find any discussion on the decision to create a new 'plugins'
directory.

It was never quite enough for pldebugger, as you still needed to add it
to shared_preload_libraries for it to work, at least if you wanted to
get the full functionality. The installation procedure in the README
clearly instructs to add it to shared_preload_libraries, it doesn't say
anything about local_preload_libraries. I recently refactored pldebugger
to not install in the plugins directory anymore, it now just drops it in
$libdir.

> It would seem to be much more in the spirit of things to simply list the
> allowed plugins in a GUC variable, like
>
> some_clever_name_here = $libdir/this, $libdir/that
>
> but there is probably a reason why this wasn't done that way in the
> first place.

I think the idea was that plugins directory would be easier for
users/admins. I agree that a GUC like above feels more natural.

Now that we support include-directories in postgresql.conf, you could
put a "mylib.conf" file in the include directory that contains the above
line, if you want to enable/disable a module just by moving things
around in the filesystem (after configuring an include directory in
postgresql.conf). But actually, you can't, because there's no way to
append to a setting, you can only override. That's an obvious missing
feature in the include mechanism. Even ignoring the plugins directory,
it would be nice to be able to append libraries to shared_preload_libraries.

- Heikki

Heikki Linnakangas <hlinnakangas(at)vmware(dot)com> writes:
> Now that we support include-directories in postgresql.conf, you could put a
> "mylib.conf" file in the include directory that contains the above line, if
> you want to enable/disable a module just by moving things around in the
> filesystem (after configuring an include directory in postgresql.conf). But
> actually, you can't, because there's no way to append to a setting, you can
> only override. That's an obvious missing feature in the include mechanism.
> Even ignoring the plugins directory, it would be nice to be able to append
> libraries to shared_preload_libraries.

I think we need a += operator in the configuration file, complementing
the = we have now, and defined so that when the setting is not yet
defined it's a bare =.

And I think this plugins thing needs to be revisited, yes.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On Tue, 2012-10-09 at 20:45 -0400, Peter Eisentraut wrote:
> About that plugins directory ($libdir/plugins) ... I don't think we
> ever
> really got that to work sensibly. I don't remember the original
> design
> discussion, but I have seen a number of explanations offered over the
> years. It's not clear who decides what to put in there (plugin
> author,
> packager, DBA?), how to put it there (move it, copy it, symlink it? --
> no support in pgxs), and based on what criteria.
>
> It would seem to be much more in the spirit of things to simply list
> the
> allowed plugins in a GUC variable, like
>
> some_clever_name_here = $libdir/this, $libdir/that

Here is a patch, with some_clever_name = user_loadable_libraries.

There are obviously some conflict/transition issues with using
user_loadable_libraries vs the plugins directory. I have tried to
explain the mechanisms in the documentation, but there are other choices
possible in some situations.

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Here is a patch, with some_clever_name = user_loadable_libraries.

Shouldn't user_loadable_libraries be PGC_SUSET? PGC_SIGHUP is typically
used when the same setting must apply in all active sessions, but it's
not clear why such a restriction must be made for this.

regards, tom lane

2013/1/15 Peter Eisentraut <peter_e(at)gmx(dot)net>:
> On Tue, 2012-10-09 at 20:45 -0400, Peter Eisentraut wrote:
>> About that plugins directory ($libdir/plugins) ... I don't think we
>> ever
>> really got that to work sensibly. I don't remember the original
>> design
>> discussion, but I have seen a number of explanations offered over the
>> years. It's not clear who decides what to put in there (plugin
>> author,
>> packager, DBA?), how to put it there (move it, copy it, symlink it? --
>> no support in pgxs), and based on what criteria.
>>
>> It would seem to be much more in the spirit of things to simply list
>> the
>> allowed plugins in a GUC variable, like
>>
>> some_clever_name_here = $libdir/this, $libdir/that
>
> Here is a patch, with some_clever_name = user_loadable_libraries.
>
> There are obviously some conflict/transition issues with using
> user_loadable_libraries vs the plugins directory. I have tried to
> explain the mechanisms in the documentation, but there are other choices
> possible in some situations.
>
Do we still need to continue hardwired "$libdir/plugins/" ?
If user_loadable_libraries allows to specify directories, not only libraries
themselves, and its default value is "$libdir/plugins/", it seems to me
this enhancement can offer more flexibility without losing backward
compatibility.

On the other hand, I'd like to see your opinion about fine-grained
superuser capabilities for module loading, that we have discussed
in the thread of untrusted language privilege.
It might be a situation where a capability to load module make sense.

Thanks,
--
KaiGai Kohei <kaigai(at)kaigai(dot)gr(dot)jp>

On 1/15/13 7:04 AM, Peter Eisentraut wrote:
>> It would seem to be much more in the spirit of things to simply list
>> > the
>> > allowed plugins in a GUC variable, like
>> >
>> > some_clever_name_here = $libdir/this, $libdir/that
> Here is a patch, with some_clever_name = user_loadable_libraries.
>
> There are obviously some conflict/transition issues with using
> user_loadable_libraries vs the plugins directory. I have tried to
> explain the mechanisms in the documentation, but there are other choices
> possible in some situations.

After thinking about this some more, this is not actually the mechanism
I need for my particular application. What I actually needed was
something in between shared_preload_libraries and
local_preload_libraries, namely that it is loaded into backend processes
only (not postmaster), but only by superusers, and typically configured
in postgresql.conf.

I'll postpone actually implementing that to the next release.

I'm not aware of anything that actually uses the "plugins" mechanism
(seeing that the debugger no longer does), so I don't know if this
present change would actually be an improvement or not for those
applications. So I'd be OK with withdrawing this patch for now.