LD_LIBRARY_PATH versus rpath

Over at
http://archives.postgresql.org/pgsql-general/2010-05/msg00091.php
we have a complaint about "make check" failing when the install is
intended to overwrite existing libraries (in particular, replacing
8.4 with 9.0 libpq). I've done some off-list investigation and
found that this appears to be a generic issue on Linux. pg_regress
invokes psql, which depends on libpq.so, and if psql fails due to
picking up the wrong libpq.so then you get behavior as described.

Now, pg_regress tries to ensure that the temporary installation
will work as desired by setting LD_LIBRARY_PATH to point at the
temp installation's lib/ directory. However, the psql executable
will by default get built with a DT_RPATH entry pointing at the
intended final installation lib/. And DT_RPATH overrides
LD_LIBRARY_PATH, in the Linux dynamic loader. man ld.so says:

The shared libraries needed by the program are searched for in the fol-
lowing order:

o (ELF only) Using the directories specified in the DT_RPATH dynamic
section attribute of the binary if present and DT_RUNPATH attribute
does not exist. Use of DT_RPATH is deprecated.

o Using the environment variable LD_LIBRARY_PATH. Except if the exe-
cutable is a set-user-ID/set-group-ID binary, in which case it is
ignored.

o (ELF only) Using the directories specified in the DT_RUNPATH dynamic
section attribute of the binary if present.

o (etc etc)

Given that deprecation note, and the fact that what we're doing entirely
fails to work as desired, it seems like what we need to do is set
DT_RUNPATH instead of DT_RPATH. Further reading discloses that the
way to do that is to add "--enable-new-dtags" to the linker switches.

So the question is, should we modify Makefile.linux along the lines of

-rpath = -Wl,-rpath,'$(rpathdir)'
+rpath = -Wl,-rpath,'$(rpathdir)',--enable-new-dtags

I asked around at Red Hat and was told that this would be unlikely
to have any negative side-effects, but I'm not sure how thoroughly
those guys thought about the consequences for non-mainstream Linux
machines. (In particular, I'm worried about really old distros
possibly not having this switch.)

My inclination is to try this in HEAD only and see if any problems
emerge during the beta cycle.

Comments?

regards, tom lane

On Thu, May 6, 2010 at 12:20 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Now, pg_regress tries to ensure that the temporary installation
> will work as desired by setting LD_LIBRARY_PATH to point at the
> temp installation's lib/ directory.  However, the psql executable
> will by default get built with a DT_RPATH entry pointing at the
> intended final installation lib/.  And DT_RPATH overrides
> LD_LIBRARY_PATH, in the Linux dynamic loader.  man ld.so says:
>

We only set RPATH if the install location isn't part of the default ld
library path specified by /etc/ld.so.conf right? Setting it if it is
in the default path would be antisocial.

--
greg

On ons, 2010-05-05 at 19:20 -0400, Tom Lane wrote:
> Over at
> http://archives.postgresql.org/pgsql-general/2010-05/msg00091.php
> we have a complaint about "make check" failing when the install is
> intended to overwrite existing libraries (in particular, replacing
> 8.4 with 9.0 libpq). I've done some off-list investigation and
> found that this appears to be a generic issue on Linux. pg_regress
> invokes psql, which depends on libpq.so, and if psql fails due to
> picking up the wrong libpq.so then you get behavior as described.

Yeah, that's been broken since forever.

> The shared libraries needed by the program are searched for in the fol-
> lowing order:
>
> o (ELF only) Using the directories specified in the DT_RPATH dynamic
> section attribute of the binary if present and DT_RUNPATH attribute
> does not exist. Use of DT_RPATH is deprecated.
>
> o Using the environment variable LD_LIBRARY_PATH. Except if the exe-
> cutable is a set-user-ID/set-group-ID binary, in which case it is
> ignored.
>
> o (ELF only) Using the directories specified in the DT_RUNPATH dynamic
> section attribute of the binary if present.

Ah, that sounds good.

> So the question is, should we modify Makefile.linux along the lines of
>
> -rpath = -Wl,-rpath,'$(rpathdir)'
> +rpath = -Wl,-rpath,'$(rpathdir)',--enable-new-dtags

I see this feature was added in 2001, so it should be OK to use.

> My inclination is to try this in HEAD only and see if any problems
> emerge during the beta cycle.

I wouldn't consider backpatching it at all.

On tor, 2010-05-06 at 11:20 +0100, Greg Stark wrote:
> We only set RPATH if the install location isn't part of the default ld
> library path specified by /etc/ld.so.conf right?

No. How would you determine that?

> Setting it if it is
> in the default path would be antisocial.

That's why there is --disable-rpath.

Greg Stark wrote
>
> We only set RPATH if the install location isn't part of the default ld
> library path specified by /etc/ld.so.conf right? Setting it if it is
> in the default path would be antisocial.
>
>

How are we going to know at build time what is in the ld.soconf of the
installation machine?

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Greg Stark wrote
>> We only set RPATH if the install location isn't part of the default ld
>> library path specified by /etc/ld.so.conf right? Setting it if it is
>> in the default path would be antisocial.

> How are we going to know at build time what is in the ld.soconf of the
> installation machine?

Exactly. In practice, it's on the packager's head to specify
--disable-rpath if he intends to install into the platform's
regular library search path.

Funny point here: in the Fedora/RHEL RPMs, I use --disable-rpath
because "don't use RPATH" is part of the standard packaging guidelines
for that platform. However, pl/perl has to double back and use rpath
anyway because libperl.so doesn't exist in the ldconfig path; it's in
some version-numbered directory and they don't provide any link or
ldconfig entry so you could find it otherwise. Annoying as heck.
I've always wondered how many other packagers have to carry patches
similar to
http://cvs.fedoraproject.org/viewvc/rpms/postgresql/devel/postgresql-perl-rpath.patch

regards, tom lane

On tor, 2010-05-06 at 09:38 -0400, Tom Lane wrote:
> Funny point here: in the Fedora/RHEL RPMs, I use --disable-rpath
> because "don't use RPATH" is part of the standard packaging guidelines
> for that platform. However, pl/perl has to double back and use rpath
> anyway because libperl.so doesn't exist in the ldconfig path; it's in
> some version-numbered directory and they don't provide any link or
> ldconfig entry so you could find it otherwise. Annoying as heck.
> I've always wondered how many other packagers have to carry patches
> similar to
> http://cvs.fedoraproject.org/viewvc/rpms/postgresql/devel/postgresql-perl-rpath.patch

Debian has libperl in /usr/lib, so there is no issue. But if there
were, there is a relatively new policy that you can should use rpath if
you need a library that is installed in a nonstandard path. (Should
actually use this new runpath thing, perhaps.) The same new policy
prohibits packages from modifying /etc/ld.so.conf.