Where to load modules from?

Hi,

This topic gets back at every release, more often now that we have
proper Extensions with ability to dump&restore. Lately the guys from
Open Shift project (a Red Hat team) have asked for a way to load DSO
module files from user-owned directory.

The way they make that safe is by using cgroups and SELinux, IIUC.

We can attack the problem in several ways:

- have an initdb switch to tweak the library path per cluster,

- have a superuser-only GUC to tweak the library path,

- consider on-disk extension as templates and move their module files
somewhere private in $PGDATA and load the code from there

That would allow OS upgrades not to impact running instances until
they do ALTER EXTENSION UPDATE; and allowing co-existence of
different versions of the same extension in different clusters of
the same major version, and maybe in separate databases of the same
cluster in some cases (depends on the extension's module specifics),

- do nothing even though the current solution is clearly broken, as in
not allowing to answer several user needs and preventing us to
implement full support (e.g. base backups, hot standby) for
extensions.

This proposal comes with no patch because I think we are able to
understand it without that, so that it would only be a waste of
everybody's time to attach code for a random solution on the list here
to that email. Or consider that the fourth point is currently the only
one addressed in this very proposal…

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On 2013-09-14 22:15:58 +0200, Dimitri Fontaine wrote:
> The way they make that safe is by using cgroups and SELinux, IIUC.
>
> We can attack the problem in several ways:
>
> - have an initdb switch to tweak the library path per cluster,

That sounds like an utterly horrible idea without any advantages.

> - have a superuser-only GUC to tweak the library path,

Hm. I think we might want to make it a PGC_POSTMASTER/postgresql.conf
variable instead. Is that stopping usecases of yours?

That's what I vote for.

> - consider on-disk extension as templates and move their module files
> somewhere private in $PGDATA and load the code from there

I don't understand what that does to address the security concerns.

> That would allow OS upgrades not to impact running instances until
> they do ALTER EXTENSION UPDATE; and allowing co-existence of
> different versions of the same extension in different clusters of
> the same major version, and maybe in separate databases of the same
> cluster in some cases (depends on the extension's module specifics),

And it would be an upgrade nightmare.

> This proposal comes with no patch because I think we are able to
> understand it without that, so that it would only be a waste of
> everybody's time to attach code for a random solution on the list here
> to that email. Or consider that the fourth point is currently the only
> one addressed in this very proposal…

Yea, the code issue seem to be small here.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Sat, 2013-09-14 at 22:15 +0200, Dimitri Fontaine wrote:
>
> This proposal comes with no patch because I think we are able to
> understand it without that, so that it would only be a waste of
> everybody's time to attach code for a random solution on the list here
> to that email.

It shouldn't be in the commit fest if it has no patch.

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> It shouldn't be in the commit fest if it has no patch.

What should I do if my goal is to get community consensus on the best
way to solve a problem, and want to start the discussion with some
proposals?

My understanding is that a Commit Fest is mainly about Reviewing, that's
why I still added an entry for two designs that I need feedback on
before actually coding a solution.

Writing the code is the easiest part of those proposals, but that's only
true as soon as we decide what code we should be writing.

Regards,
--
Dimitri Fontaine 06 63 07 10 78
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On Sun, 2013-09-15 at 16:09 +0200, Dimitri Fontaine wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > It shouldn't be in the commit fest if it has no patch.
>
> What should I do if my goal is to get community consensus on the best
> way to solve a problem, and want to start the discussion with some
> proposals?

Post it to the pgsql-hackers list.

> My understanding is that a Commit Fest is mainly about Reviewing, that's
> why I still added an entry for two designs that I need feedback on
> before actually coding a solution.
>
> Writing the code is the easiest part of those proposals, but that's only
> true as soon as we decide what code we should be writing.

I understand why using the commit fest process is attractive for this,
because it enables you to force the issue. But the point of the commit
fest is to highlight patches whose discussion has mostly concluded and
get them committed. If we add general discussion to the commit fest,
it'll just become a mirror of the mailing list, and then we'll need yet
another level of process to isolate the ready patches from that.

On 2013-09-15 16:51, Peter Eisentraut wrote:
> On Sun, 2013-09-15 at 16:09 +0200, Dimitri Fontaine wrote:
>> My understanding is that a Commit Fest is mainly about Reviewing, that's
>> why I still added an entry for two designs that I need feedback on
>> before actually coding a solution.
>>
>> Writing the code is the easiest part of those proposals, but that's only
>> true as soon as we decide what code we should be writing.
>
> I understand why using the commit fest process is attractive for this,
> because it enables you to force the issue. But the point of the commit
> fest is to highlight patches whose discussion has mostly concluded and
> get them committed. If we add general discussion to the commit fest,
> it'll just become a mirror of the mailing list, and then we'll need yet
> another level of process to isolate the ready patches from that.

I have one item like this in the current commit fest. I wrote a PoC
patch, but that's just a bad excuse to get around the issue that we
don't really want just RFCs on there.

The problem is when you post an idea requesting comments on -HACKERS,
and nobody or only one person answers despite efforts to try and keep
the discussion alive and/or revive it. What should one do in that case?
Writing a patch just to throw it away later because something's
fundamentally broken (or unnacceptable) seems silly if people could have
just looked at the original -HACKERS post and said "this can't possibly
work with your current design".

Regards,
Marko Tiikkaja

On 2013-09-15 17:03:10 +0200, Marko Tiikkaja wrote:
> On 2013-09-15 16:51, Peter Eisentraut wrote:
> >On Sun, 2013-09-15 at 16:09 +0200, Dimitri Fontaine wrote:
> >>My understanding is that a Commit Fest is mainly about Reviewing, that's
> >>why I still added an entry for two designs that I need feedback on
> >>before actually coding a solution.
> >>
> >>Writing the code is the easiest part of those proposals, but that's only
> >>true as soon as we decide what code we should be writing.
> >
> >I understand why using the commit fest process is attractive for this,
> >because it enables you to force the issue. But the point of the commit
> >fest is to highlight patches whose discussion has mostly concluded and
> >get them committed. If we add general discussion to the commit fest,
> >it'll just become a mirror of the mailing list, and then we'll need yet
> >another level of process to isolate the ready patches from that.
>
> I have one item like this in the current commit fest. I wrote a PoC patch,
> but that's just a bad excuse to get around the issue that we don't really
> want just RFCs on there.
>
> The problem is when you post an idea requesting comments on -HACKERS, and
> nobody or only one person answers despite efforts to try and keep the
> discussion alive and/or revive it. What should one do in that case?

Adding it to the CF in that case seeems like a acceptable emergency
measure in the case that nobody has replied to a proposal in a couple of
days. But afaics, that's not the case with the patches that Peter is
complaining about. This issue certainly hasn't had a lack of comments
and the archive proposal is completely new, so I see where Peter is
coming from and I tend to agree.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Post it to the pgsql-hackers list.

Well. I think I just did ;-)

> I understand why using the commit fest process is attractive for this,
> because it enables you to force the issue. But the point of the commit

It enables me to have a slight chance of seeing the effort happen within
the current release development time frame, 9.4. The next commit fest is
going to happen in november, if I don't have code to submit at this time
the next release will still have no solution to offer.

> fest is to highlight patches whose discussion has mostly concluded and
> get them committed. If we add general discussion to the commit fest,
> it'll just become a mirror of the mailing list, and then we'll need yet
> another level of process to isolate the ready patches from that.

Well, even with what I said before, you're completely right in this
phrasing of the situation, so I've just removed my two entries with no
patch from the current CF.

With some luck I will be able to revive the discussions and reach some
consensus in time to have code written for the Open CF 2013-11.

Regards,
--
Dimitri Fontaine 06 63 07 10 78
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On Sun, Sep 15, 2013 at 6:51 AM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:

> On Sat, 2013-09-14 at 22:15 +0200, Dimitri Fontaine wrote:
> >
> > This proposal comes with no patch because I think we are able to
> > understand it without that, so that it would only be a waste of
> > everybody's time to attach code for a random solution on the list here
> > to that email.
>
> It shouldn't be in the commit fest if it has no patch.
>
>
I thought the general recommendation was the opposite, that planning and
road maps should be submitted for review before non-trivial coding is
started; and that despite the name the commitfest is the best way that this
is done. Of course now I can't find the hackers thread where this
recommendation was made...

Cheers,

Jeff

On 09/15/2013 05:52 PM, Jeff Janes wrote:
> On Sun, Sep 15, 2013 at 6:51 AM, Peter Eisentraut <peter_e(at)gmx(dot)net
> <mailto:peter_e(at)gmx(dot)net>> wrote:
>
> On Sat, 2013-09-14 at 22:15 +0200, Dimitri Fontaine wrote:
> >
> > This proposal comes with no patch because I think we are able to
> > understand it without that, so that it would only be a waste of
> > everybody's time to attach code for a random solution on the
> list here
> > to that email.
>
> It shouldn't be in the commit fest if it has no patch.
>
>
> I thought the general recommendation was the opposite, that planning
> and road maps should be submitted for review before non-trivial coding
> is started; and that despite the name the commitfest is the best way
> that this is done. Of course now I can't find the hackers thread where
> this recommendation was made...
>
>

It is unquestionably correct that roadmaps and planning should be made
available for review and discussion. But the assertion that this should
be done via the commitfest is not. The commitfest app has never been for
anything other than code, that I am aware of, and I am quite sure you
will find fierce resistance to any notion that design discussions should
take place anywhere but on this mailing list.

cheers

andrew

On 15 Sep 2013 18:55, "Andrew Dunstan" <andrew(at)dunslane(dot)net> wrote:
>
>
> On 09/15/2013 05:52 PM, Jeff Janes wrote:
>
>> On Sun, Sep 15, 2013 at 6:51 AM, Peter Eisentraut <peter_e(at)gmx(dot)net<mailto:
peter_e(at)gmx(dot)net>> wrote:
>>
>> On Sat, 2013-09-14 at 22:15 +0200, Dimitri Fontaine wrote:
>> >
>> > This proposal comes with no patch because I think we are able to
>> > understand it without that, so that it would only be a waste of
>> > everybody's time to attach code for a random solution on the
>> list here
>> > to that email.
>>
>> It shouldn't be in the commit fest if it has no patch.
>>
>>
>> I thought the general recommendation was the opposite, that planning and
road maps should be submitted for review before non-trivial coding is
started; and that despite the name the commitfest is the best way that this
is done. Of course now I can't find the hackers thread where this
recommendation was made...
>>
>>
>
> It is unquestionably correct that roadmaps and planning should be made
available for review and discussion. But the assertion that this should be
done via the commitfest is not. The commitfest app has never been for
anything other than code, that I am aware of, and I am quite sure you will
find fierce resistance to any notion that design discussions should take
place anywhere but on this mailing list.

Well the code reviews should also go via the list so that's neither here
nor there.

One of the original problems the commitfest was aiming to solve was Tay
people would had be a project, make some tentative progress, ask if they're
on the right track or how to tackle some problem, hear nothing until
feature freeze at which point the original author had moved on and dropped
the project.

In other words, "forcing the issue" is one of the original design goals of
commitfests.

On Sun, Sep 15, 2013 at 10:51 AM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On Sun, 2013-09-15 at 16:09 +0200, Dimitri Fontaine wrote:
>> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
>> > It shouldn't be in the commit fest if it has no patch.
>>
>> What should I do if my goal is to get community consensus on the best
>> way to solve a problem, and want to start the discussion with some
>> proposals?
>
> Post it to the pgsql-hackers list.

The idea of using the CommitFest process to request design review was
floated at one of the last couple of developer meetings in Ottawa.
Personally, I'm for it.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Sat, Sep 14, 2013 at 4:15 PM, Dimitri Fontaine
<dimitri(at)2ndquadrant(dot)fr> wrote:
> We can attack the problem in several ways:
>
> - have an initdb switch to tweak the library path per cluster,

I see no advantage to making this impossible to change after initdb time.

> - have a superuser-only GUC to tweak the library path,

I could live with a GUC. Like Andres, I think it should be PGC_POSTMASTER.

> - consider on-disk extension as templates and move their module files
> somewhere private in $PGDATA and load the code from there

I think this will be horrid mess of security vulnerabilities and upgrade woes.

Here's another idea. At initdb time, create an empty directory called
called pg_you_can_load_stuff_from_here (pick a better name) inside
$PGDATA. Allow it to be replaced with a symlink. This would be
similar to what we do today with pg_xlog. In fact, you can imagine an
equivalent of initdb -X that does something precisely analogous. This
feels a bit more natural to me than a GUC.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> - consider on-disk extension as templates and move their module files
>> somewhere private in $PGDATA and load the code from there
>
> I think this will be horrid mess of security vulnerabilities and upgrade woes.

I think it's a solution to that horrid mess. Care to expand on your thoughs?

> Here's another idea. At initdb time, create an empty directory called
> called pg_you_can_load_stuff_from_here (pick a better name) inside
> $PGDATA. Allow it to be replaced with a symlink. This would be
> similar to what we do today with pg_xlog. In fact, you can imagine an
> equivalent of initdb -X that does something precisely analogous. This
> feels a bit more natural to me than a GUC.

I like that too.

Now the thing I've heard the most about our extension infrastructure is
how to make it support loading different versions of the same module in
different databases in the same cluster. The second one on that topic
might be why are extensions managed per-database except for their binary
module parts?

For that reason I proposed have the same mechanism as what you say here,
but with a per-database directory layout.

Thinking about it more, I think we would actually need both: some
extensions need shared_preload_libraries and cluster-wide settings is
all we can offer here. For local_preload_libraries compatible ones, then
per-database setting would be best.

Maybe adding a system view listing all the currently loaded modules,
which extension made the system load them (if any) and where it was
loaded from, superuser only, would then be appropriate.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On Wed, Sep 18, 2013 at 9:26 AM, Dimitri Fontaine
<dimitri(at)2ndquadrant(dot)fr> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>>> - consider on-disk extension as templates and move their module files
>>> somewhere private in $PGDATA and load the code from there
>>
>> I think this will be horrid mess of security vulnerabilities and upgrade woes.
>
> I think it's a solution to that horrid mess. Care to expand on your thoughs?

I think that would largely be rehashing previous discussions, in which
it's already been established that we don't see eye to eye on this
issue. But briefly, I think that replacing shared libraries ought to
be the job of the OS packaging system. System administrators are
already familiar with the notion that they need to run 'yum update' or
whatever the comparable command is from time to time, or they're going
to be vulnerable. But such packaging systems aren't going to update
copies of shared libraries we've placed inside $PGDATA.

And, I can't see what advantage it offers. If you want to be able to
have more than one version of the extension installed, you can solve
that at the OS level by choosing distinct names for their shared
libraries. If you want to be able to download and install C code into
your PostgreSQL database via a libpq connection, someone can write an
extension to do that today, without any core changes at all. If it's
nifty enough, it might even be worth shipping in contrib as an example
of how you can make deployment really, really easy at some price in
security. If your concern is that someone might modify the
OS-installed versions of those extensions in a way that renders them
incompatible with your running PostgreSQL binaries, well, they
shouldn't be doing that *anyway*. If your OS distributor is in the
habit of shipping new versions of shared libraries that break backward
compatibility in minor OS releases, you need a new OS distributor.

If you upgrade your PostgreSQL distribution, and it breaks ABI
compatibility, then using the shared libraries baked into the old
cluster is exactly the wrong thing to do. You need to use the NEW
shared libraries compiled against the new ABI, or your code may do
anything from "not load" to "crash he server". You had better hope
that both the old and new libraries have the same user-facing API, or
there's going to be trouble, but we've taken care to do that with the
things we ship in contrib and there's no reason to assume other module
authors can't do likewise.

>> Here's another idea. At initdb time, create an empty directory called
>> called pg_you_can_load_stuff_from_here (pick a better name) inside
>> $PGDATA. Allow it to be replaced with a symlink. This would be
>> similar to what we do today with pg_xlog. In fact, you can imagine an
>> equivalent of initdb -X that does something precisely analogous. This
>> feels a bit more natural to me than a GUC.
>
> I like that too.
>
> Now the thing I've heard the most about our extension infrastructure is
> how to make it support loading different versions of the same module in
> different databases in the same cluster. The second one on that topic
> might be why are extensions managed per-database except for their binary
> module parts?

The basic problem here is that extensions are allowed to do anything,
including cluster-wide stuff like "allocate and use chunks of shared
memory". If you try to load and use two different and incompatible
versions of pg_stat_statements in two different databases, there will
not be a happy ending. Also, if you load a version of some library in
shared_preload_libraries, you won't subsequently be able to decide to
load some other version in a backend bound to some particular
database. Maybe these problems are solvable, but as things stand
today I think that trying to use identically-named .so files in
different databases at the same time should be regarded as dangerously
unsupported. In any event, any kind of infrastructure intended to
support such use cases is clearly a different project from simply
allowing modules to be loaded from a different location.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> I think that would largely be rehashing previous discussions, in which
> it's already been established that we don't see eye to eye on this
> issue. But briefly, I think that replacing shared libraries ought to

Partly yes, but as I'm feeling that we are getting closer than ever to a
consensus of a first move to be made, I want to be sure to understand
the limits of that very move so that I'm able to implement the list
consensus and nothing else.

While I don't agree with all you said in your answer, I clearly
understand that part: per-database management of modules is its own can
of worms and another discussion and patch altogether.

> Maybe these problems are solvable, but as things stand
> today I think that trying to use identically-named .so files in
> different databases at the same time should be regarded as dangerously
> unsupported. In any event, any kind of infrastructure intended to
> support such use cases is clearly a different project from simply
> allowing modules to be loaded from a different location.

Baring objections, I'll then implement a patch for allowing the core
code to load modules from a per-cluster location within $PGDATA.

The patch should also have a new initdb option much alike -X for pg_xlog
so that it's possible to relocate that new per-cluster modules directory
anywhere on the file system at initdb time.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On 2013-09-18 08:46:08 -0400, Robert Haas wrote:
> Here's another idea. At initdb time, create an empty directory called
> called pg_you_can_load_stuff_from_here (pick a better name) inside
> $PGDATA. Allow it to be replaced with a symlink. This would be
> similar to what we do today with pg_xlog. In fact, you can imagine an
> equivalent of initdb -X that does something precisely analogous. This
> feels a bit more natural to me than a GUC.

I think I'd prefer a GUC that allows specifying multiple directories
that are searched in order to a single symlinked directory.

Also, aren't "symlinks" an absolute PITA to manipulate by hand on
windows?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Wed, Sep 18, 2013 at 12:53 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2013-09-18 08:46:08 -0400, Robert Haas wrote:
>> Here's another idea. At initdb time, create an empty directory called
>> called pg_you_can_load_stuff_from_here (pick a better name) inside
>> $PGDATA. Allow it to be replaced with a symlink. This would be
>> similar to what we do today with pg_xlog. In fact, you can imagine an
>> equivalent of initdb -X that does something precisely analogous. This
>> feels a bit more natural to me than a GUC.
>
> I think I'd prefer a GUC that allows specifying multiple directories
> that are searched in order to a single symlinked directory.

Why?

I ask because I have the opposite preference, based on the precedent of pg_xlog.

> Also, aren't "symlinks" an absolute PITA to manipulate by hand on
> windows?

Maybe so, but if that's an issue here it's a preexisting issue also.
I think we shouldn't burden this patch with fixing it.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> I think I'd prefer a GUC that allows specifying multiple directories
>> that are searched in order to a single symlinked directory.
>
> Why?
>
> I ask because I have the opposite preference, based on the precedent of pg_xlog.

I understand Andres preference, as it would allow a management somewhat
comparable to PATH or LD_LIBRARY_PATH here.

In an effort to reach consensus, what about having both, with the GUC
being empty by default? That way you have a default per-cluster place
where to stuff binaries to be loaded, and a GUC to manage finer settings
if needs be.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On 2013-09-19 22:56:52 +0200, Dimitri Fontaine wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> >> I think I'd prefer a GUC that allows specifying multiple directories
> >> that are searched in order to a single symlinked directory.
> >
> > Why?
> >
> > I ask because I have the opposite preference, based on the precedent
> > of pg_xlog.

Because I want to specify multiple paths. E.g. one with modules for a
specific postgres version, one for the cluster and one for my
development directory.
Now we could recursively search a directory that contains symlinks to
directories, but that seems ugly.

> I understand Andres preference, as it would allow a management somewhat
> comparable to PATH or LD_LIBRARY_PATH here.

> In an effort to reach consensus, what about having both, with the GUC
> being empty by default? That way you have a default per-cluster place
> where to stuff binaries to be loaded, and a GUC to manage finer settings
> if needs be.

Well, we can have the guc have a default value of $datadir/pg_lib or
such. But using two independent mechanisms seems like a bad idea to me.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Thu, Sep 19, 2013 at 5:54 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2013-09-19 22:56:52 +0200, Dimitri Fontaine wrote:
>> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> >> I think I'd prefer a GUC that allows specifying multiple directories
>> >> that are searched in order to a single symlinked directory.
>> >
>> > Why?
>> >
>> > I ask because I have the opposite preference, based on the precedent
>> > of pg_xlog.
>
> Because I want to specify multiple paths. E.g. one with modules for a
> specific postgres version, one for the cluster and one for my
> development directory.
> Now we could recursively search a directory that contains symlinks to
> directories, but that seems ugly.

I see. My main hesitation is around security. I feel somehow that
changing a GUC to trojan the system would be easier for a remote user
to accomplish than having to replace a directory with a symlink.

>> In an effort to reach consensus, what about having both, with the GUC
>> being empty by default? That way you have a default per-cluster place
>> where to stuff binaries to be loaded, and a GUC to manage finer settings
>> if needs be.
>
> Well, we can have the guc have a default value of $datadir/pg_lib or
> such. But using two independent mechanisms seems like a bad idea to me.

Heartily agreed.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2013-09-20 08:06:56 -0400, Robert Haas wrote:
> On Thu, Sep 19, 2013 at 5:54 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > Because I want to specify multiple paths. E.g. one with modules for a
> > specific postgres version, one for the cluster and one for my
> > development directory.
> > Now we could recursively search a directory that contains symlinks to
> > directories, but that seems ugly.

> I see. My main hesitation is around security. I feel somehow that
> changing a GUC to trojan the system would be easier for a remote user
> to accomplish than having to replace a directory with a symlink.

If they can change a PGC_POSTMASTER GUC, they already can easily enough
do:
shared_preload_libraries='/path/to/my/bad/so.so'

that's already allowed.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Fri, Sep 20, 2013 at 8:10 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2013-09-20 08:06:56 -0400, Robert Haas wrote:
>> On Thu, Sep 19, 2013 at 5:54 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> > Because I want to specify multiple paths. E.g. one with modules for a
>> > specific postgres version, one for the cluster and one for my
>> > development directory.
>> > Now we could recursively search a directory that contains symlinks to
>> > directories, but that seems ugly.
>
>> I see. My main hesitation is around security. I feel somehow that
>> changing a GUC to trojan the system would be easier for a remote user
>> to accomplish than having to replace a directory with a symlink.
>
> If they can change a PGC_POSTMASTER GUC, they already can easily enough
> do:
> shared_preload_libraries='/path/to/my/bad/so.so'
>
> that's already allowed.

OK. Well, in that case, it seems we wouldn't be opening any new doors.

So... our usual comma-separated GUC syntax? Empty means no extra
places to search.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> So... our usual comma-separated GUC syntax? Empty means no extra
> places to search.

Sounds pretty good to me.

The only advantage of using an initdb place would have been the
opportunity to actually register modules and WAL log that step so that
the standby pg_modules directory gets filled automatically.

I realise that might be another discussion and patch entirely.

I'll prepare a patch using GUCs just doing the bare minimum for now,
allowing to load modules from GUC directed places.

Regards,
--
Dimitri Fontaine 06 63 07 10 78
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On 2013-09-20 14:35:31 +0200, Dimitri Fontaine wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> > So... our usual comma-separated GUC syntax? Empty means no extra
> > places to search.

+1.

> The only advantage of using an initdb place would have been the
> opportunity to actually register modules and WAL log that step so that
> the standby pg_modules directory gets filled automatically.

-many

> I realise that might be another discussion and patch entirely.

Yes.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Robert Haas escribió:
> On Sun, Sep 15, 2013 at 10:51 AM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> > On Sun, 2013-09-15 at 16:09 +0200, Dimitri Fontaine wrote:
> >> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> >> > It shouldn't be in the commit fest if it has no patch.
> >>
> >> What should I do if my goal is to get community consensus on the best
> >> way to solve a problem, and want to start the discussion with some
> >> proposals?
> >
> > Post it to the pgsql-hackers list.
>
> The idea of using the CommitFest process to request design review was
> floated at one of the last couple of developer meetings in Ottawa.
> Personally, I'm for it.

I did it for minmax indexes on CF1 and nobody complained.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services