Re: [PATCH] Add session_preload_libraries configuration parameter

Lists: pgsql-hackers
From: Peter Eisentraut <peter_e(at)gmx(dot)net>
To: pgsql-hackers(at)postgresql(dot)org
Subject: [PATCH] Add session_preload_libraries configuration parameter
Date: 2013-06-13 03:12:16
Message-ID: 1371093136.309.3.camel@vanquo.pezone.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

This is like shared_preload_libraries except that it takes effect at
backend start and can be changed without a full postmaster restart. It
is like local_preload_libraries except that it is still only settable by
a superuser. This can be a better way to load modules such as
auto_explain.

Since there are now three preload parameters, regroup the documentation
a bit. Put all parameters into one section, explain common
functionality only once, update the descriptions to reflect current and
future realities.
---
This is a reboot of the ideas discussed in
http://www.postgresql.org/message-id/1358251492.401.5.camel@vanquo.pezone.net
and http://www.postgresql.org/message-id/515357F4.6000307@gmx.net
during the 2013-01 commit fest.

doc/src/sgml/auto-explain.sgml | 5 +-
doc/src/sgml/config.sgml | 281 ++++++++++++++++++++++---------------
src/backend/tcop/postgres.c | 2 +-
src/backend/utils/init/miscinit.c | 6 +-
src/backend/utils/misc/guc.c | 19 ++-
src/include/miscadmin.h | 3 +-
src/include/utils/guc_tables.h | 1 +
7 files changed, 193 insertions(+), 124 deletions(-)

diff --git a/doc/src/sgml/auto-explain.sgml b/doc/src/sgml/auto-explain.sgml
index 03b2309..cd0d6d8 100644
--- a/doc/src/sgml/auto-explain.sgml
+++ b/doc/src/sgml/auto-explain.sgml
@@ -24,7 +24,8 @@ <title>auto_explain</title>
</programlisting>

(You must be superuser to do that.) More typical usage is to preload
- it into all sessions by including <literal>auto_explain</> in
+ it into some or all sessions by including <literal>auto_explain</> in
+ <xref linkend="guc-session-preload-libraries"> or
<xref linkend="guc-shared-preload-libraries"> in
<filename>postgresql.conf</>. Then you can track unexpectedly slow queries
no matter when they happen. Of course there is a price in overhead for
@@ -185,7 +186,7 @@ <title>Configuration Parameters</title>

<programlisting>
# postgresql.conf
-shared_preload_libraries = 'auto_explain'
+session_preload_libraries = 'auto_explain'

auto_explain.log_min_duration = '3s'
</programlisting>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index c7d84b5..b8e766d 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -1252,66 +1252,6 @@ <title>Kernel Resource Usage</title>
</para>
</listitem>
</varlistentry>
-
- <varlistentry id="guc-shared-preload-libraries" xreflabel="shared_preload_libraries">
- <term><varname>shared_preload_libraries</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>shared_preload_libraries</> configuration parameter</primary>
- </indexterm>
- <listitem>
- <para>
- This variable specifies one or more shared libraries
- to be preloaded at server start. For example,
- <literal>'$libdir/mylib'</literal> would cause
- <literal>mylib.so</> (or on some platforms,
- <literal>mylib.sl</>) to be preloaded from the installation's
- standard library directory.
- All library names are converted to lower case unless double-quoted.
- If more than one library is to be loaded, separate their names
- with commas. This parameter can only be set at server start.
- </para>
-
- <para>
- <productname>PostgreSQL</productname> procedural language
- libraries can be preloaded in this way, typically by using the
- syntax <literal>'$libdir/plXXX'</literal> where
- <literal>XXX</literal> is <literal>pgsql</>, <literal>perl</>,
- <literal>tcl</>, or <literal>python</>.
- </para>
-
- <para>
- By preloading a shared library, the library startup time is avoided
- when the library is first used. However, the time to start each new
- server process might increase slightly, even if that process never
- uses the library. So this parameter is recommended only for
- libraries that will be used in most sessions.
- </para>
-
- <note>
- <para>
- On Windows hosts, preloading a library at server start will not reduce
- the time required to start each new server process; each server process
- will re-load all preload libraries. However, <varname>shared_preload_libraries
- </varname> is still useful on Windows hosts because some shared libraries may
- need to perform certain operations that only take place at postmaster start
- (for example, a shared library may need to reserve lightweight locks
- or shared memory and you can't do that after the postmaster has started).
- </para>
- </note>
- <para>
- If a specified library is not found,
- the server will fail to start.
- </para>
-
- <para>
- Every PostgreSQL-supported library has a <quote>magic
- block</> that is checked to guarantee compatibility.
- For this reason, non-PostgreSQL libraries cannot be
- loaded in this way.
- </para>
- </listitem>
- </varlistentry>
-
</variablelist>
</sect2>

@@ -5494,6 +5434,171 @@ <title>Locale and Formatting</title>
</variablelist>

</sect2>
+
+ <sect2 id="runtime-config-client-preload">
+ <title>Shared Library Preloading</title>
+
+ <para>
+ Several settings are available for preloading shared libraries into the
+ server, in order to load additional functionality or achieve performance
+ benefits. For example, a setting of
+ <literal>'$libdir/mylib'</literal> would cause
+ <literal>mylib.so</> (or on some platforms,
+ <literal>mylib.sl</>) to be preloaded from the installation's standard
+ library directory. The differences between the settings are when they
+ take effect and what privileges are required to change them.
+ </para>
+
+ <para>
+ <productname>PostgreSQL</productname> procedural language libraries can
+ be preloaded in this way, typically by using the
+ syntax <literal>'$libdir/plXXX'</literal> where
+ <literal>XXX</literal> is <literal>pgsql</>, <literal>perl</>,
+ <literal>tcl</>, or <literal>python</>.
+ </para>
+
+ <para>
+ For each parameter, if more than one library is to be loaded, separate
+ their names with commas. All library names are converted to lower case
+ unless double-quoted.
+ </para>
+
+ <para>
+ Only shared libraries specifically intended to be used with PostgreSQL
+ can be loaded this way. Every PostgreSQL-supported library has
+ a <quote>magic block</> that is checked to guarantee compatibility. For
+ this reason, non-PostgreSQL libraries cannot be loaded in this way. You
+ might be able to use operating-system facilities such
+ as <envar>LD_PRELOAD</envar> for that.
+ </para>
+
+ <para>
+ In general, refer to the documentation of a specific module for the
+ recommended way to load that module.
+ </para>
+
+ <variablelist>
+ <varlistentry id="guc-local-preload-libraries" xreflabel="local_preload_libraries">
+ <term><varname>local_preload_libraries</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>local_preload_libraries</> configuration parameter</primary>
+ </indexterm>
+ <indexterm>
+ <primary><filename>$libdir/plugins</></primary>
+ </indexterm>
+ <listitem>
+ <para>
+ This variable specifies one or more shared libraries that are to be
+ preloaded at connection start. This parameter cannot be changed after
+ the start of a particular session. If a specified library is not
+ found, the connection attempt will fail.
+ </para>
+
+ <para>
+ This option can be set by any user. Because of that, the libraries
+ that can be loaded are restricted to those appearing in the
+ <filename>plugins</> subdirectory of the installation's
+ standard library directory. (It is the database administrator's
+ responsibility to ensure that only <quote>safe</> libraries
+ are installed there.) Entries in <varname>local_preload_libraries</>
+ can specify this directory explicitly, for example
+ <literal>$libdir/plugins/mylib</literal>, or just specify
+ the library name &mdash; <literal>mylib</literal> would have
+ the same effect as <literal>$libdir/plugins/mylib</literal>.
+ </para>
+
+ <para>
+ Unless a module is specifically designed to be used in this way by
+ non-superusers, this is usually not the right setting to use. Look
+ at <xref linkend="guc-session-preload-libraries"> instead.
+ </para>
+ </listitem>
+ </varlistentry>
+
+
+ <varlistentry id="guc-session-preload-libraries" xreflabel="session_preload_libraries">
+ <term><varname>session_preload_libraries</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>session_preload_libraries</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ This variable specifies one or more shared libraries that are to be
+ preloaded at connection start. Only superusers can change this setting.
+ The parameter value only takes effect at the start of the connection.
+ Subsequent changes have no effect. If a specified library is not
+ found, the connection attempt will fail.
+ </para>
+
+ <para>
+ The intent of this feature is to allow debugging or
+ performance-measurement libraries to be loaded into specific sessions
+ without an explicit
+ <command>LOAD</> command being given. For
+ example, <xref linkend="auto-explain"> could be enabled for all
+ sessions under a given user name by setting this parameter
+ with <command>ALTER ROLE SET</>. Also, this parameter can be changed
+ without restarting the server (but changes only take effect when a new
+ session is started), so it is easier to add new modules this way, even
+ if they should apply to all sessions.
+ </para>
+
+ <para>
+ Unlike <xref linkend="guc-shared-preload-libraries">, there is no large
+ performance advantage to loading a library at session start rather than
+ when it is first used. There is some advantage, however, when
+ connection pooling is used.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-shared-preload-libraries" xreflabel="shared_preload_libraries">
+ <term><varname>shared_preload_libraries</varname> (<type>string</type>)</term>
+ <indexterm>
+ <primary><varname>shared_preload_libraries</> configuration parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ This variable specifies one or more shared libraries to be preloaded at
+ server start. with commas. This parameter can only be set at server
+ start. If a specified library is not found, the server will fail to
+ start.
+ </para>
+
+ <para>
+ Some libraries need to perform certain operations that can only take
+ place at postmaster start, such as allocating shared memory, reserving
+ light-weight locks, or starting background workers. Those libraries
+ must be loaded at server start through this parameter. See the
+ documentation of each library for details.
+ </para>
+
+ <para>
+ Other libraries can also be preloaded. By preloading a shared library,
+ the library startup time is avoided when the library is first used.
+ However, the time to start each new server process might increase
+ slightly, even if that process never uses the library. So this
+ parameter is recommended only for libraries that will be used in most
+ sessions. Also, changing this parameter requires a server restart, so
+ this is not the right setting to use for short-term debugging tasks,
+ say. Use <xref linkend="guc-session-preload-libraries"> for that
+ instead.
+ </para>
+
+ <note>
+ <para>
+ On Windows hosts, preloading a library at server start will not reduce
+ the time required to start each new server process; each server process
+ will re-load all preload libraries. However, <varname>shared_preload_libraries
+ </varname> is still useful on Windows hosts for libraries that need to
+ perform operations at postmaster start time..
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
<sect2 id="runtime-config-client-other">
<title>Other Defaults</title>

@@ -5565,62 +5670,6 @@ <title>Other Defaults</title>
</listitem>
</varlistentry>

- <varlistentry id="guc-local-preload-libraries" xreflabel="local_preload_libraries">
- <term><varname>local_preload_libraries</varname> (<type>string</type>)</term>
- <indexterm>
- <primary><varname>local_preload_libraries</> configuration parameter</primary>
- </indexterm>
- <indexterm>
- <primary><filename>$libdir/plugins</></primary>
- </indexterm>
- <listitem>
- <para>
- This variable specifies one or more shared libraries that are
- to be preloaded at connection start. If more than one library
- is to be loaded, separate their names with commas. All library
- names are converted to lower case unless double-quoted.
- This parameter cannot be changed after the start of a particular
- session.
- </para>
-
- <para>
- Because this is not a superuser-only option, the libraries
- that can be loaded are restricted to those appearing in the
- <filename>plugins</> subdirectory of the installation's
- standard library directory. (It is the database administrator's
- responsibility to ensure that only <quote>safe</> libraries
- are installed there.) Entries in <varname>local_preload_libraries</>
- can specify this directory explicitly, for example
- <literal>$libdir/plugins/mylib</literal>, or just specify
- the library name &mdash; <literal>mylib</literal> would have
- the same effect as <literal>$libdir/plugins/mylib</literal>.
- </para>
-
- <para>
- Unlike <xref linkend="guc-shared-preload-libraries">, there is no
- performance advantage to loading a library at session
- start rather than when it is first used. Rather, the intent of
- this feature is to allow debugging or performance-measurement
- libraries to be loaded into specific sessions without an explicit
- <command>LOAD</> command being given. For example, debugging could
- be enabled for all sessions under a given user name by setting
- this parameter with <command>ALTER ROLE SET</>.
- </para>
-
- <para>
- If a specified library is not found,
- the connection attempt will fail.
- </para>
-
- <para>
- Every PostgreSQL-supported library has a <quote>magic
- block</> that is checked to guarantee compatibility.
- For this reason, non-PostgreSQL libraries cannot be
- loaded in this way.
- </para>
- </listitem>
- </varlistentry>
-
</variablelist>
</sect2>
</sect1>
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 31ea313..da0c5c9 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3738,7 +3738,7 @@
* process any libraries that should be preloaded at backend start (this
* likewise can't be done until GUC settings are complete)
*/
- process_local_preload_libraries();
+ process_session_preload_libraries();

/*
* Send this backend's cancellation info to the frontend.
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index cb78caf..ed514f6 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -1222,6 +1222,7 @@
* GUC variables: lists of library names to be preloaded at postmaster
* start and at backend start
*/
+char *session_preload_libraries_string = NULL;
char *shared_preload_libraries_string = NULL;
char *local_preload_libraries_string = NULL;

@@ -1318,8 +1319,11 @@
* process any libraries that should be preloaded at backend start
*/
void
-process_local_preload_libraries(void)
+process_session_preload_libraries(void)
{
+ load_libraries(session_preload_libraries_string,
+ "session_preload_libraries",
+ false);
load_libraries(local_preload_libraries_string,
"local_preload_libraries",
true);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index ea16c64..f05b801 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -589,6 +589,8 @@ static char *config_enum_get_options(struct config_enum * record,
gettext_noop("Client Connection Defaults / Statement Behavior"),
/* CLIENT_CONN_LOCALE */
gettext_noop("Client Connection Defaults / Locale and Formatting"),
+ /* CLIENT_CONN_PRELOAD */
+ gettext_noop("Client Connection Defaults / Shared Library Preloading"),
/* CLIENT_CONN_OTHER */
gettext_noop("Client Connection Defaults / Other Defaults"),
/* LOCK_MANAGEMENT */
@@ -2745,7 +2747,18 @@ static char *config_enum_get_options(struct config_enum * record,
},

{
- {"shared_preload_libraries", PGC_POSTMASTER, RESOURCES_KERNEL,
+ {"session_preload_libraries", PGC_SUSET, CLIENT_CONN_PRELOAD,
+ gettext_noop("Lists shared libraries to preload into each backend."),
+ NULL,
+ GUC_LIST_INPUT | GUC_LIST_QUOTE | GUC_SUPERUSER_ONLY
+ },
+ &session_preload_libraries_string,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"shared_preload_libraries", PGC_POSTMASTER, CLIENT_CONN_PRELOAD,
gettext_noop("Lists shared libraries to preload into server."),
NULL,
GUC_LIST_INPUT | GUC_LIST_QUOTE | GUC_SUPERUSER_ONLY
@@ -2756,8 +2769,8 @@ static char *config_enum_get_options(struct config_enum * record,
},

{
- {"local_preload_libraries", PGC_BACKEND, CLIENT_CONN_OTHER,
- gettext_noop("Lists shared libraries to preload into each backend."),
+ {"local_preload_libraries", PGC_BACKEND, CLIENT_CONN_PRELOAD,
+ gettext_noop("Lists unprivileged shared libraries to preload into each backend."),
NULL,
GUC_LIST_INPUT | GUC_LIST_QUOTE
},
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index be3add9..77861ea 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -402,6 +402,7 @@ extern void BaseInit(void);
/* in utils/init/miscinit.c */
extern bool IgnoreSystemIndexes;
extern PGDLLIMPORT bool process_shared_preload_libraries_in_progress;
+extern char *session_preload_libraries_string;
extern char *shared_preload_libraries_string;
extern char *local_preload_libraries_string;

@@ -437,7 +438,7 @@ extern void TouchSocketLockFiles(void);
extern void AddToDataDirLockFile(int target_line, const char *str);
extern void ValidatePgVersion(const char *path);
extern void process_shared_preload_libraries(void);
-extern void process_local_preload_libraries(void);
+extern void process_session_preload_libraries(void);
extern void pg_bindtextdomain(const char *domain);
extern bool has_rolreplication(Oid roleid);

diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
index 8dcdd4b..4f1f6e0 100644
--- a/src/include/utils/guc_tables.h
+++ b/src/include/utils/guc_tables.h
@@ -88,6 +88,7 @@ enum config_group
CLIENT_CONN,
CLIENT_CONN_STATEMENT,
CLIENT_CONN_LOCALE,
+ CLIENT_CONN_PRELOAD,
CLIENT_CONN_OTHER,
LOCK_MANAGEMENT,
COMPAT_OPTIONS,
--
1.7.10.4


From: Dimitri Fontaine <dimitri(at)2ndQuadrant(dot)fr>
To: Peter Eisentraut <peter_e(at)gmx(dot)net>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PATCH] Add session_preload_libraries configuration parameter
Date: 2013-06-21 09:51:52
Message-ID: m2y5a37pvr.fsf@2ndQuadrant.fr
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hi,

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> This is like shared_preload_libraries except that it takes effect at
> backend start and can be changed without a full postmaster restart. It
> is like local_preload_libraries except that it is still only settable by
> a superuser. This can be a better way to load modules such as
> auto_explain.

I had a pretty hard time to get my head around that new one, and I'm not
sure I totally did. The important things to me are:

- No need to manually copy the lib into the plugin directory,
- ALTER ROLE support.

So basically it's a very good solution for auto_explain and any other
module you want to load eagerly but not for everyone and when not using
shared memory.

I have a feeling that something simpler could be made, but I will have
to continue thinking about it.

I found it strange that those two paras read differently for saying the
same thing?

> + preloaded at connection start. This parameter cannot be changed after
> + the start of a particular session. If a specified library is not

> + The parameter value only takes effect at the start of the connection.
> + Subsequent changes have no effect. If a specified library is not

Will review the code in more details, wanted to get back on the context
where this patch is useful first, and try to understand better the trade
offs involved.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support


From: Robins <robins(at)pobox(dot)com>
To: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] Add session_preload_libraries configuration parameter
Date: 2013-06-29 13:51:42
Message-ID: CAEP4nAw8AzZ_36yK0T=4WAp9Z_jmGTxosnzsAVeu26BYLW=19w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 12 June 2013 22:12, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:

> This is like shared_preload_libraries except that it takes effect at
> backend start and can be changed without a full postmaster restart. It
> is like local_preload_libraries except that it is still only settable by
> a superuser. This can be a better way to load modules such as
> auto_explain.
>
> Since there are now three preload parameters, regroup the documentation
> a bit. Put all parameters into one section, explain common
> functionality only once, update the descriptions to reflect current and
> future realities.
> ---
>

Hi,

Did some basic checks on this patch. List-wise feedback below.

- Cleanly applies to Git-Head: Yes (Minor 1 line offset in guc.c, but
that's probably because of the delay in reviewing)
- Documentation Updated: Yes
- All tests pass: Yes
- Removed unnecessary extra-lines: Yes

- Do we want it?: ???

- Any visible issues: No
- Any compiler warnings: No

- Others:
Number of new lines added not covered by tests: -107 (Wierd but, it seems
to have reduced line coverage). But correct me if am not seeing the
elephant in the room.

--
Robins Tharakan


From: Robins <robins(at)pobox(dot)com>
To: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] Add session_preload_libraries configuration parameter
Date: 2013-06-29 15:30:53
Message-ID: CAEP4nAy5_QSAMvwAdzVzTkDUObB6JXA=RWAYAT+GR-nxrOsR_A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

> Number of new lines added not covered by tests: -107 (Wierd but, it seems
> to have reduced line coverage). But correct me if am not seeing the
> elephant in the room.
>
>
My apologies.

Number of new untested lines: 108

Robins


From: Peter Eisentraut <peter_e(at)gmx(dot)net>
To: Dimitri Fontaine <dimitri(at)2ndQuadrant(dot)fr>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PATCH] Add session_preload_libraries configuration parameter
Date: 2013-07-07 19:40:10
Message-ID: 1373226010.12837.25.camel@vanquo.pezone.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, 2013-06-21 at 11:51 +0200, Dimitri Fontaine wrote:
> I found it strange that those two paras read differently for saying
> the
> same thing?
>
> > + preloaded at connection start. This parameter cannot be
> changed after
> > + the start of a particular session. If a specified library
> is not
>
> > + The parameter value only takes effect at the start of the
> connection.
> > + Subsequent changes have no effect. If a specified library
> is not

The first says that changing the parameter after connection start will
result in an error. The second says that you can change the parameter
after connection start, but it won't have any effect.

This is just a result of the various GUC context mechanisms.


From: Dimitri Fontaine <dimitri(at)2ndQuadrant(dot)fr>
To: Peter Eisentraut <peter_e(at)gmx(dot)net>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PATCH] Add session_preload_libraries configuration parameter
Date: 2013-07-07 20:50:22
Message-ID: m2r4fap01d.fsf@2ndQuadrant.fr
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> This is like shared_preload_libraries except that it takes effect at
> backend start and can be changed without a full postmaster restart. It
> is like local_preload_libraries except that it is still only settable by
> a superuser. This can be a better way to load modules such as
> auto_explain.

I finally took enough time to actually understand (I think) your
proposal. The key point seems to be around ALTER ROLE SET and such like
dynamic setting of a local_preload_libraries.

Would it make sense to review the local_preload_libraries semantics
instead? It looks like session_preload_libraries only adds flexibility,
and that current usages of local_preload_libraries would be served as
well by the new setting.

Did you figure out a case where a local_preload_libraries setting would
stop working if made into a session_preload_libraries setting?

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support


From: Peter Eisentraut <peter_e(at)gmx(dot)net>
To: Dimitri Fontaine <dimitri(at)2ndQuadrant(dot)fr>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PATCH] Add session_preload_libraries configuration parameter
Date: 2013-07-08 01:03:24
Message-ID: 1373245404.12837.33.camel@vanquo.pezone.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Sun, 2013-07-07 at 22:50 +0200, Dimitri Fontaine wrote:
> Would it make sense to review the local_preload_libraries semantics
> instead? It looks like session_preload_libraries only adds
> flexibility,
> and that current usages of local_preload_libraries would be served as
> well by the new setting.
>
> Did you figure out a case where a local_preload_libraries setting
> would
> stop working if made into a session_preload_libraries setting?

local_preload_libraries allows unprivileged users to preload whitelisted
libraries. session_preload_libraries allows superusers to preload any
library. It's hard to see how to consolidate those, at least without
adding another setting that whitelists the libraries somehow, at which
point you haven't really gained anything in terms of complexity.

I don't know of any actual legitimate uses of local_preload_libraries.
I recall that the plpgsql debugger was meant to use it, but doesn't
anymore. So it's hard to judge what to do about this, without any
actual use cases.


From: Dimitri Fontaine <dimitri(at)2ndQuadrant(dot)fr>
To: Peter Eisentraut <peter_e(at)gmx(dot)net>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PATCH] Add session_preload_libraries configuration parameter
Date: 2013-07-08 08:37:42
Message-ID: m27gh1o3ah.fsf@2ndQuadrant.fr
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> local_preload_libraries allows unprivileged users to preload whitelisted
> libraries. session_preload_libraries allows superusers to preload any
> library. It's hard to see how to consolidate those, at least without
> adding another setting that whitelists the libraries somehow, at which
> point you haven't really gained anything in terms of complexity.

Good point.

> I don't know of any actual legitimate uses of local_preload_libraries.
> I recall that the plpgsql debugger was meant to use it, but doesn't
> anymore. So it's hard to judge what to do about this, without any
> actual use cases.

Well there's my preprepare thing at

https://github.com/dimitri/preprepare

I don't think that the whitelisting is actually used in a way to allow
for non superusers to load modules in the field, because the only way to
do that with local_preload_libraries that I know of is to edit the
postgresql.conf file and reload.

alter role dim set local_preload_libraries = 'auto_explain';
ERROR: 55P02: parameter "local_preload_libraries" cannot be set after connection start

That said, I think we could go with your current patch and revise the
local_preload_libraries down the road in the 9.4 development cycle.
Having both for a while is a good way to better understand if they are
useful to have together.

Going to mark your patch ready for commiter, thanks!

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support


From: Peter Eisentraut <peter_e(at)gmx(dot)net>
To: Dimitri Fontaine <dimitri(at)2ndQuadrant(dot)fr>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PATCH] Add session_preload_libraries configuration parameter
Date: 2013-07-09 12:07:23
Message-ID: 51DBFCFB.3080000@gmx.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 7/8/13 4:37 AM, Dimitri Fontaine wrote:
>> I don't know of any actual legitimate uses of local_preload_libraries.
>> I recall that the plpgsql debugger was meant to use it, but doesn't
>> anymore. So it's hard to judge what to do about this, without any
>> actual use cases.
>
> Well there's my preprepare thing at
>
> https://github.com/dimitri/preprepare

preprepare has an SQL function as entry point, so you don't need to
preload it.

> I don't think that the whitelisting is actually used in a way to allow
> for non superusers to load modules in the field, because the only way to
> do that with local_preload_libraries that I know of is to edit the
> postgresql.conf file and reload.
>
> alter role dim set local_preload_libraries = 'auto_explain';
> ERROR: 55P02: parameter "local_preload_libraries" cannot be set after connection start

I think the idea was to use PGOPTIONS to load it, controlled by the
client side.


From: Dimitri Fontaine <dimitri(at)2ndQuadrant(dot)fr>
To: Peter Eisentraut <peter_e(at)gmx(dot)net>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PATCH] Add session_preload_libraries configuration parameter
Date: 2013-07-09 12:37:58
Message-ID: m2d2qrc3ix.fsf@2ndQuadrant.fr
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> preprepare has an SQL function as entry point, so you don't need to
> preload it.

Well the whole goal of that extension is to autoload itself entirely
server side at connection time. The function API had been made to cope
with 8.3 where it wasn't possible to get a snapshot from within the init
function of a module.

> I think the idea was to use PGOPTIONS to load it, controlled by the
> client side.

Oh. Never done that, I don't know if that's common in the field. If it's
in use, then the new session_preload_libraries GUC looks the only way
forward then.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support