shared_preload_libraries support on Win32?

(working on the PL debugger...)

It appears that the libraries listed in shared_preload_libraries will
*not* be inherited by spawned backends on Win32 platforms.

Do we have to do something special to make that work?

Using ProcessExplorer (from sysinternals.com), I can see that my plugins
are loaded into the postmaster, but not into the individual backends.

If I set local_preload_libraries equal to shared_preload_libraries, the
plugins are loaded into the backends as expected (so it seems unlikely
that I have a pathname or permissions problem).

Should we just call process_shared_preload_libraries() after calling
read_nondefault_variables() (in SubPostmasterMain())?

-- Korry

--
Korry Douglas korryd(at)enterprisedb(dot)com
EnterpriseDB http://www.enterprisedb.com

<korryd(at)enterprisedb(dot)com> writes:
> It appears that the libraries listed in shared_preload_libraries will
> *not* be inherited by spawned backends on Win32 platforms.

Well, yeah, because it's a fork/exec on that platform.

> Should we just call process_shared_preload_libraries() after calling
> read_nondefault_variables() (in SubPostmasterMain())?

I don't entirely see the point. The value of shared_preload_libraries
is to avoid paying per-process overhead to load the libraries, and that
benefit is already lost in a fork/exec world. Might as well just let
the libraries be loaded when and if needed.

I think we've failed to document that shared_preload_libraries doesn't
work on Windows, which is an oversight.

regards, tom lane

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Tom Lane wrote:
>> I don't entirely see the point. The value of shared_preload_libraries
>> is to avoid paying per-process overhead to load the libraries, and that
>> benefit is already lost in a fork/exec world. Might as well just let
>> the libraries be loaded when and if needed.

> The only benefit I can see is that it would assist in having a common
> configuration.

Actually ... I take that back. I was thinking of the original purpose
of preload_libraries, which was strictly performance optimization.
But in the new world of plugins there may be functional reasons for
wanting libraries to be loaded into backends --- and
shared_preload_libraries is not isomorphic to local_preload_libraries.
The permissions situation is different.

Korry's right, we should force re-loading of shared_preload_libraries
in the EXEC_BACKEND case. The needed documentation change is to point
out that on Windows this is not a performance win, but it might still
be wanted for instrumentation or debugging plugins.

regards, tom lane

Tom Lane wrote:
> <korryd(at)enterprisedb(dot)com> writes:
>
>> It appears that the libraries listed in shared_preload_libraries will
>> *not* be inherited by spawned backends on Win32 platforms.
>>
>
> Well, yeah, because it's a fork/exec on that platform.
>
>
>> Should we just call process_shared_preload_libraries() after calling
>> read_nondefault_variables() (in SubPostmasterMain())?
>>
>
> I don't entirely see the point. The value of shared_preload_libraries
> is to avoid paying per-process overhead to load the libraries, and that
> benefit is already lost in a fork/exec world. Might as well just let
> the libraries be loaded when and if needed.
>
>

The only benefit I can see is that it would assist in having a common
configuration.

> I think we've failed to document that shared_preload_libraries doesn't
> work on Windows, which is an oversight.
>
>

Maybe postmaster should also log a warning if it detects this.

cheers

andrew

> Actually ... I take that back. I was thinking of the original purpose
> of preload_libraries, which was strictly performance optimization.
> But in the new world of plugins there may be functional reasons for
> wanting libraries to be loaded into backends --- and
> shared_preload_libraries is not isomorphic to local_preload_libraries.
> The permissions situation is different.

And, shared_preload_libraries is processed (in the postmaster) before
the shared-memory segment is created, so a shared_preload_library can
call RequestAddinShmemSpace() and RequestAddinLWLocks(), but a
local_preload_library cannot.

-- Korry

<korryd(at)enterprisedb(dot)com> writes:
>> But in the new world of plugins there may be functional reasons for
>> wanting libraries to be loaded into backends --- and
>> shared_preload_libraries is not isomorphic to local_preload_libraries.
>> The permissions situation is different.

> And, shared_preload_libraries is processed (in the postmaster) before
> the shared-memory segment is created, so a shared_preload_library can
> call RequestAddinShmemSpace() and RequestAddinLWLocks(), but a
> local_preload_library cannot.

That doesn't seem like an issue though, since the copy in the postmaster
will have done that anyway.

regards, tom lane

korryd(at)enterprisedb(dot)com wrote:
> > > And, shared_preload_libraries is processed (in the postmaster) before
> > > the shared-memory segment is created, so a shared_preload_library can
> > > call RequestAddinShmemSpace() and RequestAddinLWLocks(), but a
> > > local_preload_library cannot.
> >
> > That doesn't seem like an issue though, since the copy in the postmaster
> > will have done that anyway.
>
>
> You're right - we need the copy in the postmaster (to setup shared
> memory and LW locks), and we need them in the backends too. I just want
> to avoid having to set both shared_preload_libraries and
> local_preload_libraries (to the same thing). Adding a call to
> process_shared_preload_libraries() in SubPostmasterMain() seems to fix
> the problem for me.

Just make sure you don't load the libraries in bgwriter et al ...

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

> > And, shared_preload_libraries is processed (in the postmaster) before
> > the shared-memory segment is created, so a shared_preload_library can
> > call RequestAddinShmemSpace() and RequestAddinLWLocks(), but a
> > local_preload_library cannot.
>
> That doesn't seem like an issue though, since the copy in the postmaster
> will have done that anyway.

You're right - we need the copy in the postmaster (to setup shared
memory and LW locks), and we need them in the backends too. I just want
to avoid having to set both shared_preload_libraries and
local_preload_libraries (to the same thing). Adding a call to
process_shared_preload_libraries() in SubPostmasterMain() seems to fix
the problem for me.

Thanks for your input. I'll submit a patch.

-- Korry

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> korryd(at)enterprisedb(dot)com wrote:
>> You're right - we need the copy in the postmaster (to setup shared
>> memory and LW locks), and we need them in the backends too.

> Just make sure you don't load the libraries in bgwriter et al ...

I see that Korry's patch doesn't do that, but I'm wondering why exactly.
In a Unix environment such libraries *would* be propagated into bgwriter
and every other postmaster child; is there a reason for the setup on
Windows to be different? In particular, what about autovacuum, which
ISTM should be as close to a standard backend as possible?

Either way we do it, authors of plugins used this way will have to test
both cases (I'm glad I insisted on EXEC_BACKEND mode being testable under
Unix ...)

regards, tom lane

> >> You're right - we need the copy in the postmaster (to setup shared
> >> memory and LW locks), and we need them in the backends too.
>
> > Just make sure you don't load the libraries in bgwriter et al ...
>
> I see that Korry's patch doesn't do that, but I'm wondering why exactly.
> In a Unix environment such libraries *would* be propagated into bgwriter
> and every other postmaster child; is there a reason for the setup on
> Windows to be different? In particular, what about autovacuum, which
> ISTM should be as close to a standard backend as possible?

I thought about that too... does autovacuum ever need to re-index? If
so, it seems that it would need access to any index functions (for
function-based indexes) that might reside in a shared_preload_library.

> Either way we do it, authors of plugins used this way will have to test
> both cases (I'm glad I insisted on EXEC_BACKEND mode being testable under
> Unix ...)

And I'm glad that RequestAddinShmemSpace() and RequestAddinLWLocks()
don't complain if called after postmaster start :-)

-- Korry

--
Korry Douglas korryd(at)enterprisedb(dot)com
EnterpriseDB http://www.enterprisedb.com

<korryd(at)enterprisedb(dot)com> writes:
>> I see that Korry's patch doesn't do that, but I'm wondering why exactly.
>> In a Unix environment such libraries *would* be propagated into bgwriter
>> and every other postmaster child; is there a reason for the setup on
>> Windows to be different? In particular, what about autovacuum, which
>> ISTM should be as close to a standard backend as possible?

> I thought about that too... does autovacuum ever need to re-index? If
> so, it seems that it would need access to any index functions (for
> function-based indexes) that might reside in a shared_preload_library.

Any ordinary C-language function is not an issue, because its library
will get autoloaded upon use. AFAICS what we have to think about here
is instrumentation or debugging plugins that someone might wish to have
running in the postmaster's special children. Maybe there's no such
animal; I'm not sure. But in the Unix environment they'd be active in
those processes.

regards, tom lane