Re: find libxml2 using pkg-config

In multi-arch OS installations, using a single foo-config script to find
libraries is problematic, because you don't know which architecture it
will point to, and you can't choose which one you want. Using
pkg-config is better in that situation, because you can use its
environment variables to point to your preferred version
of /usr/lib*/pkgconfig or similar.

In configure, we use xml2-config and pthread-config. The latter doesn't
exist on my system, so I'm just dealing with xml2-config now.

The attached patch looks for pkg-config first, and finds libxml2 using
that if available. Otherwise it falls back to using xml2-config.

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> The attached patch looks for pkg-config first, and finds libxml2 using
> that if available. Otherwise it falls back to using xml2-config.

What happens if pkg-config is installed but doesn't know anything about
xml2? I'd expect the code to fall back to the old method, but this
patch doesn't appear to have any sanity check whatsoever on pkg-config's
output.

regards, tom lane

On Mon, 2013-01-14 at 10:25 -0500, Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > The attached patch looks for pkg-config first, and finds libxml2 using
> > that if available. Otherwise it falls back to using xml2-config.
>
> What happens if pkg-config is installed but doesn't know anything about
> xml2? I'd expect the code to fall back to the old method, but this
> patch doesn't appear to have any sanity check whatsoever on pkg-config's
> output.

Updated patch to that effect

On Mon, Jan 14, 2013 at 06:51:05AM -0500, Peter Eisentraut wrote:
> In multi-arch OS installations, using a single foo-config script to find
> libraries is problematic, because you don't know which architecture it
> will point to, and you can't choose which one you want. Using
> pkg-config is better in that situation, because you can use its
> environment variables to point to your preferred version
> of /usr/lib*/pkgconfig or similar.

"./configure XML2_CONFIG=/my/preferred/xml2-config" achieves this today.

> In configure, we use xml2-config and pthread-config. The latter doesn't
> exist on my system, so I'm just dealing with xml2-config now.

pthread-config is now quite deep into legacy territory; no concern there.

> The attached patch looks for pkg-config first, and finds libxml2 using
> that if available. Otherwise it falls back to using xml2-config.

There's a risk to making anything configurable in two places, with one taking
precedence. Some user unaware of $PLACE_1 will scratch his head after
changing $PLACE_2 to no effect. For example, I've been using an override
libxml2 by changing my PATH. With this patch, I will need to also change
PKG_CONFIG_PATH; otherwise, my system libxml2 will quietly take precedence. I
can adapt easily enough, but I wonder whether the patch will be a net win
generally for folks building PostgreSQL.

I'd like this change more if it could be construed as starting us on the road
to use pkg-config, or any one particular configuration discovery method, for
all dependencies. But many dependencies don't ship .pc files at all (perl,
ldap, tcl, pam). Others ship a .pc, but we use neither it nor a -config
script (openssl, gssapi, xslt). I see this patch as adding one more thing for
builders to comprehend without making things notably easier for complex
situations. Adopting this patch wouldn't appal me, but I would rather not.

> + AC_CHECK_PROGS(PKG_CONFIG, pkg-config)

This deserves an AC_ARG_VAR(PKG_CONFIG) somewhere. On the other hand,
AC_ARG_VAR(XML2_CONFIG) is already missing, so perhaps cleaning all that is a
separate change.

> + if test -n "$PKG_CONFIG" && $PKG_CONFIG --exists libxml-2.0; then
> + CPPFLAGS="$CPPFLAGS "`$PKG_CONFIG libxml-2.0 --cflags-only-I`

Under pkg-config, we'll get -I options only, but ...

> + for pgac_option in `$XML2_CONFIG --cflags`; do
> + case $pgac_option in
> + -I*|-D*) CPPFLAGS="$CPPFLAGS $pgac_option";;

... we'll convey both -I and -D options from xml2-config.

--
Noah Misch
EnterpriseDB http://www.enterprisedb.com

On Fri, 2013-03-01 at 14:25 -0500, Noah Misch wrote:
> On Mon, Jan 14, 2013 at 06:51:05AM -0500, Peter Eisentraut wrote:
> > In multi-arch OS installations, using a single foo-config script to
> find
> > libraries is problematic, because you don't know which architecture
> it
> > will point to, and you can't choose which one you want. Using
> > pkg-config is better in that situation, because you can use its
> > environment variables to point to your preferred version
> > of /usr/lib*/pkgconfig or similar.
>
> "./configure XML2_CONFIG=/my/preferred/xml2-config" achieves this
> today.

No.

The way multi-arch installations work is that you have the libraries
under /usr/lib and /usr/lib64, or /usr/lib/$arch1 and /usr/lib/$arch2,
but only one /usr/bin. So there cannot be two foo-config scripts to
cover this. There is, however, /usr/lib/pkgconfig
and /usr/lib64/pkgconfig (or analogous), so pkg-config can handle this.

On Sun, Mar 03, 2013 at 10:30:11PM -0500, Peter Eisentraut wrote:
> On Fri, 2013-03-01 at 14:25 -0500, Noah Misch wrote:
> > On Mon, Jan 14, 2013 at 06:51:05AM -0500, Peter Eisentraut wrote:
> > > In multi-arch OS installations, using a single foo-config script to
> > find
> > > libraries is problematic, because you don't know which architecture
> > it
> > > will point to, and you can't choose which one you want. Using
> > > pkg-config is better in that situation, because you can use its
> > > environment variables to point to your preferred version
> > > of /usr/lib*/pkgconfig or similar.
> >
> > "./configure XML2_CONFIG=/my/preferred/xml2-config" achieves this
> > today.
>
> No.
>
> The way multi-arch installations work is that you have the libraries
> under /usr/lib and /usr/lib64, or /usr/lib/$arch1 and /usr/lib/$arch2,
> but only one /usr/bin. So there cannot be two foo-config scripts to
> cover this. There is, however, /usr/lib/pkgconfig
> and /usr/lib64/pkgconfig (or analogous), so pkg-config can handle this.

Do you have in mind a target system exhibiting a problem? CentOS 6 ships a
single xml2-config, but its --cflags --libs output is the same regardless of
the installed combination of libxml2-dev packages. Ubuntu 13.04 does not ship
32-bit libxml2, so it avoids the question.

--
Noah Misch
EnterpriseDB http://www.enterprisedb.com

On 3/4/13 1:36 PM, Noah Misch wrote:
> Do you have in mind a target system exhibiting a problem? CentOS 6 ships a
> single xml2-config, but its --cflags --libs output is the same regardless of
> the installed combination of libxml2-dev packages. Ubuntu 13.04 does not ship
> 32-bit libxml2, so it avoids the question.

It does, because you can just install the libxml2 package from the
32-bit distribution. (So there will no longer be packages in the 64-bit
distribution that actually contain 32-bit code, at least in the long run.)

But pack to the main question: Stock systems probably won't exhibit the
problem, because they just dodge the problem by omitting the -L option
from the xml2-config output and rely on the default linker paths to do
the right thing. But if you use a nondefault libxml2 install or a
nondefault compiler, interesting things might start to happen.

I think at this point, the issue is probably too obscure, and the people
affected by it hopefully know what they are doing, so it might not be
important in practice. In light of the other flaws that you have
pointed out, I'd be fine with withdrawing this patch for now. But we
should keep an eye on the situation.

On Wed, Mar 20, 2013 at 05:17:11PM -0400, Peter Eisentraut wrote:
> On 3/4/13 1:36 PM, Noah Misch wrote:
> > Do you have in mind a target system exhibiting a problem? CentOS 6 ships a
> > single xml2-config, but its --cflags --libs output is the same regardless of
> > the installed combination of libxml2-dev packages. Ubuntu 13.04 does not ship
> > 32-bit libxml2, so it avoids the question.
>
> It does, because you can just install the libxml2 package from the
> 32-bit distribution. (So there will no longer be packages in the 64-bit
> distribution that actually contain 32-bit code, at least in the long run.)

Ah, interesting. Is there a plan or existing provision for arbitrating the
resulting /usr/bin contents when mixing packages that way?

> But pack to the main question: Stock systems probably won't exhibit the
> problem, because they just dodge the problem by omitting the -L option
> from the xml2-config output and rely on the default linker paths to do
> the right thing. But if you use a nondefault libxml2 install or a
> nondefault compiler, interesting things might start to happen.
>
> I think at this point, the issue is probably too obscure, and the people
> affected by it hopefully know what they are doing, so it might not be
> important in practice. In light of the other flaws that you have
> pointed out, I'd be fine with withdrawing this patch for now. But we
> should keep an eye on the situation.

Agreed. Convincing a package to properly attach to its dependencies is no
fun. I wanted to like this patch.

--
Noah Misch
EnterpriseDB http://www.enterprisedb.com

Noah Misch <noah(at)leadboat(dot)com> writes:
> On Wed, Mar 20, 2013 at 05:17:11PM -0400, Peter Eisentraut wrote:
>> On 3/4/13 1:36 PM, Noah Misch wrote:
>>> Do you have in mind a target system exhibiting a problem? CentOS 6 ships a
>>> single xml2-config, but its --cflags --libs output is the same regardless of
>>> the installed combination of libxml2-dev packages. Ubuntu 13.04 does not ship
>>> 32-bit libxml2, so it avoids the question.

>> It does, because you can just install the libxml2 package from the
>> 32-bit distribution. (So there will no longer be packages in the 64-bit
>> distribution that actually contain 32-bit code, at least in the long run.)

> Ah, interesting. Is there a plan or existing provision for arbitrating the
> resulting /usr/bin contents when mixing packages that way?

There is not. With my other hat on (the red fedora), this annoys me
tremendously. I believe the rationale is that users shouldn't really
care whether /usr/bin/foo is a 32-bit or 64-bit executable as long as it
gets the job done ... but that's a pretty thin rationale IMHO. In any
case, the existing design for multilib only takes care to allow parallel
installation of 32-bit and 64-bit *libraries*, not executables. For the
latter, I think what happens is you get the executables from whichever
package you installed last. At least on Red Hat-based systems.
Possibly other distros have a saner design here.

>> I think at this point, the issue is probably too obscure, and the people
>> affected by it hopefully know what they are doing, so it might not be
>> important in practice. In light of the other flaws that you have
>> pointed out, I'd be fine with withdrawing this patch for now. But we
>> should keep an eye on the situation.

> Agreed. Convincing a package to properly attach to its dependencies is no
> fun. I wanted to like this patch.

In the end, I think this is mostly the territory of packagers. We can't
force the right result as a platform-agnostic upstream source, and I'm
not sure we should try. I would suggest that any changes in this area
be driven by actual complaints from actual packagers.

regards, tom lane