Re: [PATCHES] HEAD doesn't cope with libraries in non-default

CVS tip fails with
./configure --with-openssl \
--with-includes=/usr/local/ssl/include --with-libs=/usr/local/ssl/lib

...
make[3]: Entering directory `/home/postgres/pgsql/src/interfaces/libpq'
...
/usr/ccs/bin/ld +h libpq.sl.4 -b +b /home/postgres/testversion/lib fe-auth.o fe-connect.o fe-exec.o fe-misc.o fe-print.o fe-lobj.o fe-protocol2.o fe-protocol3.o pqexpbuffer.o pqsignal.o fe-secure.o md5.o ip.o wchar.o encnames.o noblock.o pgstrcasecmp.o thread.o getaddrinfo.o -lssl -lcrypto `gcc -L../../../src/port -L/usr/local/ssl/lib -Wl,-z -Wl,+b -Wl,/home/postgres/testversion/lib -print-libgcc-file-name` -L../../../src/port -L/usr/local/ssl/lib -o libpq.sl.4
/usr/ccs/bin/ld: Can't find library for -lssl
make[3]: *** [libpq.sl.4] Error 1

It appears that somebody has changed things so that the -L switches
appear after the -l switches (ie, too late). I'm too tired to
investigate now, but my money is on Autoconf 2.59 being the problem ...

regards, tom lane

Tom Lane wrote:
> CVS tip fails with
> ./configure --with-openssl \
> --with-includes=/usr/local/ssl/include --with-libs=/usr/local/ssl/lib
>
> ...
> make[3]: Entering directory `/home/postgres/pgsql/src/interfaces/libpq'
> ...
> /usr/ccs/bin/ld +h libpq.sl.4 -b +b /home/postgres/testversion/lib fe-auth.o fe-connect.o fe-exec.o fe-misc.o fe-print.o fe-lobj.o fe-protocol2.o fe-protocol3.o pqexpbuffer.o pqsignal.o fe-secure.o md5.o ip.o wchar.o encnames.o noblock.o pgstrcasecmp.o thread.o getaddrinfo.o -lssl -lcrypto `gcc -L../../../src/port -L/usr/local/ssl/lib -Wl,-z -Wl,+b -Wl,/home/postgres/testversion/lib -print-libgcc-file-name` -L../../../src/port -L/usr/local/ssl/lib -o libpq.sl.4
> /usr/ccs/bin/ld: Can't find library for -lssl
> make[3]: *** [libpq.sl.4] Error 1
>
> It appears that somebody has changed things so that the -L switches
> appear after the -l switches (ie, too late). I'm too tired to
> investigate now, but my money is on Autoconf 2.59 being the problem ...

I wonder if it was this commit. I am attaching the patch so you can
test to see if it fixes it. If it does, please let us know.

---------------------------------------------------------------------------

revision 1.91
date: 2005/07/02 23:28:22; author: momjian; state: Exp; lines: +2 -2
> A quick look shows that when you use --with-libraries=/foo/bar the
> generated link line for libraries says
>
> -L/foo/bar -lpq
>
> and it should probably be the other way around (as it is for the
> executables).
>
> So I suspect we need some makefile tuning.

You were correct. This patch fixes it.

Jim C. Nasby

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Tom Lane wrote:
>> It appears that somebody has changed things so that the -L switches
>> appear after the -l switches (ie, too late). I'm too tired to
>> investigate now, but my money is on Autoconf 2.59 being the problem ...

> I wonder if it was this commit. I am attaching the patch so you can
> test to see if it fixes it. If it does, please let us know.

> ! SHLIB_LINK := $(filter -L%, $(LDFLAGS)) $(SHLIB_LINK)
> ...
> ! SHLIB_LINK := $(SHLIB_LINK) $(filter -L%, $(LDFLAGS))

Urgh. This was considered a good idea why exactly?

regards, tom lane

Tom Lane wrote:

> It appears that somebody has changed things so that the -L switches
> appear after the -l switches (ie, too late). I'm too tired to
> investigate now, but my money is on Autoconf 2.59 being the problem ...

Perhaps this:
http://archives.postgresql.org/pgsql-hackers/2005-07/msg00085.php

-O

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom Lane wrote:
> >> It appears that somebody has changed things so that the -L switches
> >> appear after the -l switches (ie, too late). I'm too tired to
> >> investigate now, but my money is on Autoconf 2.59 being the problem ...
>
> > I wonder if it was this commit. I am attaching the patch so you can
> > test to see if it fixes it. If it does, please let us know.
>
> > ! SHLIB_LINK := $(filter -L%, $(LDFLAGS)) $(SHLIB_LINK)
> > ...
> > ! SHLIB_LINK := $(SHLIB_LINK) $(filter -L%, $(LDFLAGS))
>
> Urgh. This was considered a good idea why exactly?

I didn't like it myself but it fixed a problem with a build farm
machine, and no one objected, so I applied it, but it seemed pretty
strange to be reversing those. I will reverse the patch.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian said:
> Tom Lane wrote:
>> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
>> > Tom Lane wrote:
>> >> It appears that somebody has changed things so that the -L switches
>> >> appear after the -l switches (ie, too late). I'm too tired to
>> >> investigate now, but my money is on Autoconf 2.59 being the problem
>> >> ...
>>
>> > I wonder if it was this commit. I am attaching the patch so you can
>> > test to see if it fixes it. If it does, please let us know.
>>
>> > ! SHLIB_LINK := $(filter -L%, $(LDFLAGS)) $(SHLIB_LINK)
>> > ...
>> > ! SHLIB_LINK := $(SHLIB_LINK) $(filter -L%, $(LDFLAGS))
>>
>> Urgh. This was considered a good idea why exactly?
>
> I didn't like it myself but it fixed a problem with a build farm
> machine, and no one objected, so I applied it, but it seemed pretty
> strange to be reversing those. I will reverse the patch.
>

I was also slightly dubious about it. However, we do still need to solve the
problem that the patch addressed. Buildfarm members platypus and cuckoo are
currently failing because dblink is picking up the wrong libpq (and it
appears that incorrect libraries are also being picked up in the ecpg
libraries, although this isn't causing a buildfarm failure.)

Alternatively, if we can't say --with-libraries=/foo/bar when /foo/bar
contains possibly conflicting libraries, that should be tested for at
configure time.

cheers

andrew

Andrew Dunstan wrote:
> I was also slightly dubious about it. However, we do still need to
> solve the problem that the patch addressed. Buildfarm members
> platypus and cuckoo are currently failing because dblink is picking
> up the wrong libpq (and it appears that incorrect libraries are also
> being picked up in the ecpg libraries, although this isn't causing a
> buildfarm failure.)

We have four pieces of information when linking a shared library:

B: in-tree libraries that we might need (in case of ecpglib: libpq)
A: path to those in-tree libraries
D: external libraries that we might need (in case of ecpglib in my case:
-lcrypt -lm)
C: path to those external libraries (e.g., /usr/local/lib)

On the linker command line, we need this information in one of the
following two orders:

A B C D
A C B D

The Makefile.shlib receives from the respective main makefile "A B D" in
SHLIB_LINK and would have to insert "C" in the middle somewhere.
Currently, the actual behavior is "C A B D" and the failed patch wanted
to do "A B D C", both of which are wrong.

So either we code up some intelligence to put the "C" in the right
position or we have to pass down "A B" and "D" separately from the main
makefile.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Mon, Jul 04, 2005 at 04:14:51PM +1200, Oliver Jowett wrote:
> Tom Lane wrote:
>
> >It appears that somebody has changed things so that the -L switches
> >appear after the -l switches (ie, too late). I'm too tired to
> >investigate now, but my money is on Autoconf 2.59 being the problem ...
>
> Perhaps this:
> http://archives.postgresql.org/pgsql-hackers/2005-07/msg00085.php

Would be my guess as well...

So should includes/linking for PostgreSQL-built libs be handled
seperately? The original problem was that if --with-libraries was
specified then libpq etc would be linked from the system locations
instead of from within the build.
--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

Jim C. Nasby wrote:

>On Mon, Jul 04, 2005 at 04:14:51PM +1200, Oliver Jowett wrote:
>
>
>>Tom Lane wrote:
>>
>>
>>
>>>It appears that somebody has changed things so that the -L switches
>>>appear after the -l switches (ie, too late). I'm too tired to
>>>investigate now, but my money is on Autoconf 2.59 being the problem ...
>>>
>>>
>>Perhaps this:
>>http://archives.postgresql.org/pgsql-hackers/2005-07/msg00085.php
>>
>>
>
>Would be my guess as well...
>
>So should includes/linking for PostgreSQL-built libs be handled
>seperately? The original problem was that if --with-libraries was
>specified then libpq etc would be linked from the system locations
>instead of from within the build.
>
>

Yes, I think so. Peter Eisentraut's analysis strikes me as being correct.

cheers

andrew

On Mon, Jul 04, 2005 at 05:58:27PM +0200, Peter Eisentraut wrote:
> Andrew Dunstan wrote:
> > I was also slightly dubious about it. However, we do still need to
> > solve the problem that the patch addressed. Buildfarm members
> > platypus and cuckoo are currently failing because dblink is picking
> > up the wrong libpq (and it appears that incorrect libraries are also
> > being picked up in the ecpg libraries, although this isn't causing a
> > buildfarm failure.)
>
> We have four pieces of information when linking a shared library:
>
> B: in-tree libraries that we might need (in case of ecpglib: libpq)
> A: path to those in-tree libraries

Is A even represented in the build at all right now? ISTM it's not, so
simply adding it in front of C might suffice. What would be a reasonable
way to add that to the makefiles?

> D: external libraries that we might need (in case of ecpglib in my case:
> -lcrypt -lm)
> C: path to those external libraries (e.g., /usr/local/lib)
>
> On the linker command line, we need this information in one of the
> following two orders:
>
> A B C D
> A C B D
>
> The Makefile.shlib receives from the respective main makefile "A B D" in
> SHLIB_LINK and would have to insert "C" in the middle somewhere.
> Currently, the actual behavior is "C A B D" and the failed patch wanted
> to do "A B D C", both of which are wrong.
>
> So either we code up some intelligence to put the "C" in the right
> position or we have to pass down "A B" and "D" separately from the main
> makefile.
>
> --
> Peter Eisentraut
> http://developer.postgresql.org/~petere/
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: don't forget to increase your free space map settings
>

--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

Jim C. Nasby wrote:
> > B: in-tree libraries that we might need (in case of ecpglib: libpq)
> > A: path to those in-tree libraries
>
> Is A even represented in the build at all right now? ISTM it's not,

Sure it is. How else would anything like psql and pg_dump find libpq?

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

I wrote:
> So either we code up some intelligence to put the "C" in the right
> position or we have to pass down "A B" and "D" separately from the
> main makefile.

The following patch might just do the former. Please try it out.

diff -ur ../cvs-pgsql/src/Makefile.shlib ./src/Makefile.shlib
--- ../cvs-pgsql/src/Makefile.shlib 2005-07-04 16:32:57.000000000 +0200
+++ ./src/Makefile.shlib 2005-07-05 22:02:10.556162778 +0200
@@ -240,7 +240,7 @@
SHLIB_LINK += -ltermcap -lstdc++.r4 -lbind -lsocket -L/boot/develop/lib/x86
endif

-SHLIB_LINK := $(filter -L%, $(LDFLAGS)) $(SHLIB_LINK)
+SHLIB_LINK := $(filter -L%, $(SHLIB_LINK)) $(filter -L%, $(LDFLAGS)) $(filter-out -L%, $(SHLIB_LINK))
ifeq ($(enable_rpath), yes)
SHLIB_LINK += $(rpath)
endif

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Tue, Jul 05, 2005 at 10:09:19PM +0200, Peter Eisentraut wrote:
> I wrote:
> > So either we code up some intelligence to put the "C" in the right
> > position or we have to pass down "A B" and "D" separately from the
> > main makefile.
>
> The following patch might just do the former. Please try it out.
>
>
> diff -ur ../cvs-pgsql/src/Makefile.shlib ./src/Makefile.shlib
> --- ../cvs-pgsql/src/Makefile.shlib 2005-07-04 16:32:57.000000000 +0200
> +++ ./src/Makefile.shlib 2005-07-05 22:02:10.556162778 +0200
> @@ -240,7 +240,7 @@
> SHLIB_LINK += -ltermcap -lstdc++.r4 -lbind -lsocket -L/boot/develop/lib/x86
> endif
>
> -SHLIB_LINK := $(filter -L%, $(LDFLAGS)) $(SHLIB_LINK)
> +SHLIB_LINK := $(filter -L%, $(SHLIB_LINK)) $(filter -L%, $(LDFLAGS)) $(filter-out -L%, $(SHLIB_LINK))
> ifeq ($(enable_rpath), yes)
> SHLIB_LINK += $(rpath)
> endif

Worked on platypus:
http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=platypus&dt=2005-07-05%2022:03:35
--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

Could we please get this patch applied? It seems like the right thing to do.

cheers

andrew

Jim C. Nasby wrote:

>On Tue, Jul 05, 2005 at 10:09:19PM +0200, Peter Eisentraut wrote:
>
>
>>I wrote:
>>
>>
>>>So either we code up some intelligence to put the "C" in the right
>>>position or we have to pass down "A B" and "D" separately from the
>>>main makefile.
>>>
>>>
>>The following patch might just do the former. Please try it out.
>>
>>
>>diff -ur ../cvs-pgsql/src/Makefile.shlib ./src/Makefile.shlib
>>--- ../cvs-pgsql/src/Makefile.shlib 2005-07-04 16:32:57.000000000 +0200
>>+++ ./src/Makefile.shlib 2005-07-05 22:02:10.556162778 +0200
>>@@ -240,7 +240,7 @@
>> SHLIB_LINK += -ltermcap -lstdc++.r4 -lbind -lsocket -L/boot/develop/lib/x86
>> endif
>>
>>-SHLIB_LINK := $(filter -L%, $(LDFLAGS)) $(SHLIB_LINK)
>>+SHLIB_LINK := $(filter -L%, $(SHLIB_LINK)) $(filter -L%, $(LDFLAGS)) $(filter-out -L%, $(SHLIB_LINK))
>> ifeq ($(enable_rpath), yes)
>> SHLIB_LINK += $(rpath)
>> endif
>>
>>
>
>Worked on platypus:
>http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=platypus&dt=2005-07-05%2022:03:35
>
>

Patch applied. Thanks.

---------------------------------------------------------------------------

Peter Eisentraut wrote:
> I wrote:
> > So either we code up some intelligence to put the "C" in the right
> > position or we have to pass down "A B" and "D" separately from the
> > main makefile.
>
> The following patch might just do the former. Please try it out.
>
>
> diff -ur ../cvs-pgsql/src/Makefile.shlib ./src/Makefile.shlib
> --- ../cvs-pgsql/src/Makefile.shlib 2005-07-04 16:32:57.000000000 +0200
> +++ ./src/Makefile.shlib 2005-07-05 22:02:10.556162778 +0200
> @@ -240,7 +240,7 @@
> SHLIB_LINK += -ltermcap -lstdc++.r4 -lbind -lsocket -L/boot/develop/lib/x86
> endif
>
> -SHLIB_LINK := $(filter -L%, $(LDFLAGS)) $(SHLIB_LINK)
> +SHLIB_LINK := $(filter -L%, $(SHLIB_LINK)) $(filter -L%, $(LDFLAGS)) $(filter-out -L%, $(SHLIB_LINK))
> ifeq ($(enable_rpath), yes)
> SHLIB_LINK += $(rpath)
> endif
>
>
> --
> Peter Eisentraut
> http://developer.postgresql.org/~petere/
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo(at)postgresql(dot)org
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

This patch seems to have broken builds on Windows and other boxes (e.g.
buildfarm's octopus, a FreeBSD box). Maybe this should be reverted until
we find a more robust solution :-(

cheers

andrew

Bruce Momjian wrote:

>Patch applied. Thanks.
>
>---------------------------------------------------------------------------
>
>
>Peter Eisentraut wrote:
>
>
>>I wrote:
>>
>>
>>>So either we code up some intelligence to put the "C" in the right
>>>position or we have to pass down "A B" and "D" separately from the
>>>main makefile.
>>>
>>>
>>The following patch might just do the former. Please try it out.
>>
>>
>>diff -ur ../cvs-pgsql/src/Makefile.shlib ./src/Makefile.shlib
>>--- ../cvs-pgsql/src/Makefile.shlib 2005-07-04 16:32:57.000000000 +0200
>>+++ ./src/Makefile.shlib 2005-07-05 22:02:10.556162778 +0200
>>@@ -240,7 +240,7 @@
>> SHLIB_LINK += -ltermcap -lstdc++.r4 -lbind -lsocket -L/boot/develop/lib/x86
>> endif
>>
>>-SHLIB_LINK := $(filter -L%, $(LDFLAGS)) $(SHLIB_LINK)
>>+SHLIB_LINK := $(filter -L%, $(SHLIB_LINK)) $(filter -L%, $(LDFLAGS)) $(filter-out -L%, $(SHLIB_LINK))
>> ifeq ($(enable_rpath), yes)
>> SHLIB_LINK += $(rpath)
>> endif
>>
>>

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> This patch seems to have broken builds on Windows and other boxes (e.g.
> buildfarm's octopus, a FreeBSD box). Maybe this should be reverted until
> we find a more robust solution :-(

The only thing I see any evidence for is a broken version of gmake on
octopus.

gmake[3]: Entering directory `/raid0/buildfarm/buildfarm/HEAD/pgsql.54583/src/backend/utils/mb/conversion_procs/ascii_and_mic'
../../../../../../src/Makefile.shlib:250: *** missing separator. Stop.
gmake[3]: Leaving directory `/raid0/buildfarm/buildfarm/HEAD/pgsql.54583/src/backend/utils/mb/conversion_procs/ascii_and_mic'
gmake[2]: *** [all] Error 2

If there were a genuine syntax error in that command, we'd all be seeing
this.

What gmake version is octopus using, anyway?

regards, tom lane

Tom Lane wrote:

>Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>
>
>>This patch seems to have broken builds on Windows and other boxes (e.g.
>>buildfarm's octopus, a FreeBSD box). Maybe this should be reverted until
>>we find a more robust solution :-(
>>
>>
>
>The only thing I see any evidence for is a broken version of gmake on
>octopus.
>
>gmake[3]: Entering directory `/raid0/buildfarm/buildfarm/HEAD/pgsql.54583/src/backend/utils/mb/conversion_procs/ascii_and_mic'
>../../../../../../src/Makefile.shlib:250: *** missing separator. Stop.
>gmake[3]: Leaving directory `/raid0/buildfarm/buildfarm/HEAD/pgsql.54583/src/backend/utils/mb/conversion_procs/ascii_and_mic'
>gmake[2]: *** [all] Error 2
>
>If there were a genuine syntax error in that command, we'd all be seeing
>this.
>
>What gmake version is octopus using, anyway?
>
>

I wondered about that. Certainly the compiler is very old indeed.

Jim?

Meanwhile, we are now choking on building plperl for Windows, at least
with the ActiveState perl port, where we were not before.
Makefile.global gets these settings:

PERL = "/c/perl/bin//perl"
perl_archlibexp = C:\Perl\lib
perl_privlibexp = C:\Perl\lib
perl_useshrplib = yes
perl_embed_ldflags = -nologo -nodefaultlib -debug -opt:ref,icf
-libpath:"C:\Perl\lib\CORE" -machine:x86 C:\Perl\lib\CORE\perl58.lib

and we see this error:

dllwrap -o libplperl.dll --dllname libplperl.dll --def plperl.def plperl.o spi_internal.o SPI.o -L -L../../../src/backend -L../../../src/port -L/c/tcl/lib C:/Perl/lib/CORE -lperl58 -lpostgres
c:\mingw\bin\..\lib\gcc-lib\mingw32\3.2.3\..\..\..\..\mingw32\bin\ld.exe: cannot open C:/Perl/lib/CORE: Permission denied
c:\mingw\bin\dllwrap.exe: c:\mingw\bin\gcc exited with status 1

which looks very odd indeed, especially:

-L -L../../../src/backend -L../../../src/port -L/c/tcl/lib C:/Perl/lib/CORE -lperl58

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> which looks very odd indeed, especially:

> -L -L../../../src/backend -L../../../src/port -L/c/tcl/lib C:/Perl/lib/CORE -lperl58

Ah, I see the problem:

ifeq ($(PORTNAME), win32)
perl_archlibexp := $(subst \,/,$(perl_archlibexp))
perl_privlibexp := $(subst \,/,$(perl_privlibexp))
perl_embed_ldflags := -L $(perl_archlibexp)/CORE -lperl58
^^^^^^^^^^^^^^^^^^^^^^^^^^
override CPPFLAGS += -DPLPERL_HAVE_UID_GID
endif

The filter hack depends on -L and the following argument to not be
space-separated. We made the no-space assumption before for -L in
LDFLAGS, but not for -L in SHLIB_LINK.

I've removed the space in CVS tip, we'll see where that takes us.

regards, tom lane

Jim C. Nasby wrote:

>Turns out there was a cvs conflict. Doh!
>
>

Ouch. I have repeatedly warned buildfarm owners not to make any changes
or run builds in buildfarm's local CVS repo. Use a copy if necessary.

>Hmm... would probably be a good idea to have the script check for
>conflicts and throw a CVS error if it finds any. Here's what CVS does on
>a conflict:
>
>RCS file: /projects/cvsroot/pgsql/src/Makefile.shlib,v
>retrieving revision 1.93
>retrieving revision 1.94
>Merging differences between 1.93 and 1.94 into Makefile.shlib
>rcsmerge: warning: conflicts during merge
>cvs update: conflicts found in Makefile.shlib
>C Makefile.shlib
>
>It also creates a .# file. I'm not really sure what the best way to test
>for this would be.
>
>

I can look for it. It's a pity that cvs doesn't exit with a non-zero
status when it does this. I am assuming we don't ever have such files in
CVS.

>BTW, it would probably be handy to have logs available for all steps of
>the build, even if they succeed.
>
>

They are available locally in the directory lastrun-logs. I made a
deliberate decision not to clog up the server with them. At roughly 1Mb
per build it seemed quite unnecessary - I wanted to keep traffic down
and server storage requirements modest.

>In any case, I've cleared the conflict and I'm running a build right
>now.
>
>
>
>

OK, cool.

cheers

andrew

On Wed, Jul 13, 2005 at 01:24:17PM -0400, Andrew Dunstan wrote:
> >In any case, I've cleared the conflict and I'm running a build right
> >now.

octopus is building again, and is back to the behavior I mentioned in
http://archives.postgresql.org/pgsql-bugs/2005-07/msg00096.php. Is this
something that should be fixed in code? There are two patches in the
FreeBSD ports tree for postgresql 8:

decibel(at)flake(dot)2[14:18]/usr/ports/databases/postgresql80-server/files:47>cat patch-plpython-Makefile patch-src-makefiles-Makefile.freebsd
--- src/pl/plpython/Makefile.orig Fri Nov 19 20:23:01 2004
+++ src/pl/plpython/Makefile Tue Dec 28 23:32:16 2004
@@ -9,7 +9,7 @@
# shared library. Since there is no official way to determine this
# (at least not in pre-2.3 Python), we see if there is a file that is
# named like a shared library.
-ifneq (,$(wildcard $(python_libdir)/libpython*$(DLSUFFIX)*))
+ifneq (,$(wildcard $(python_libdir)/../../libpython*$(DLSUFFIX)*))
shared_libpython = yes
endif

--- src/makefiles/Makefile.freebsd.orig Fri Nov 19 01:41:39 2004
+++ src/makefiles/Makefile.freebsd Tue Dec 21 02:44:09 2004
@@ -11,7 +11,7 @@
ifeq ($(findstring sparc,$(host_cpu)), sparc)
CFLAGS_SL = -fPIC -DPIC
else
-CFLAGS_SL = -fpic -DPIC
+CFLAGS_SL = -fPIC -DPIC
endif


@@ -29,3 +29,5 @@
endif

sqlmansect = 7
+
+allow_nonpic_in_shlib = yes

The first of these patches makes me think that octopus might actually be
finding the wrong library, though things are fine on platypus with
python 2.3 (octopus is running 2.4).

I'm upgrading platypus to 2.4 right now to see what that changes, if
anything.
--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

Tom Lane wrote:

>Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>
>
>>which looks very odd indeed, especially:
>>
>>
>
>
>
>> -L -L../../../src/backend -L../../../src/port -L/c/tcl/lib C:/Perl/lib/CORE -lperl58
>>
>>
>
>Ah, I see the problem:
>
>ifeq ($(PORTNAME), win32)
>perl_archlibexp := $(subst \,/,$(perl_archlibexp))
>perl_privlibexp := $(subst \,/,$(perl_privlibexp))
>perl_embed_ldflags := -L $(perl_archlibexp)/CORE -lperl58
> ^^^^^^^^^^^^^^^^^^^^^^^^^^
>override CPPFLAGS += -DPLPERL_HAVE_UID_GID
>endif
>
>The filter hack depends on -L and the following argument to not be
>space-separated. We made the no-space assumption before for -L in
>LDFLAGS, but not for -L in SHLIB_LINK.
>
>I've removed the space in CVS tip, we'll see where that takes us.
>
>
>
>

Seems to have done the trick on Windows. I think this stuff is now fixed
- thanks for your help.

cheers

andrew

Patch applied. Thanks. If we made plpython worse, we will hear about
it soon enough. The freebsd-specific changes seem safe, considering
they came from the FreeBSD port maintainers themselves.

---------------------------------------------------------------------------

Jim C. Nasby wrote:
> On Wed, Jul 13, 2005 at 01:24:17PM -0400, Andrew Dunstan wrote:
> > >In any case, I've cleared the conflict and I'm running a build right
> > >now.
>
> octopus is building again, and is back to the behavior I mentioned in
> http://archives.postgresql.org/pgsql-bugs/2005-07/msg00096.php. Is this
> something that should be fixed in code? There are two patches in the
> FreeBSD ports tree for postgresql 8:
>
> decibel(at)flake(dot)2[14:18]/usr/ports/databases/postgresql80-server/files:47>cat patch-plpython-Makefile patch-src-makefiles-Makefile.freebsd
> --- src/pl/plpython/Makefile.orig Fri Nov 19 20:23:01 2004
> +++ src/pl/plpython/Makefile Tue Dec 28 23:32:16 2004
> @@ -9,7 +9,7 @@
> # shared library. Since there is no official way to determine this
> # (at least not in pre-2.3 Python), we see if there is a file that is
> # named like a shared library.
> -ifneq (,$(wildcard $(python_libdir)/libpython*$(DLSUFFIX)*))
> +ifneq (,$(wildcard $(python_libdir)/../../libpython*$(DLSUFFIX)*))
> shared_libpython = yes
> endif
>
> --- src/makefiles/Makefile.freebsd.orig Fri Nov 19 01:41:39 2004
> +++ src/makefiles/Makefile.freebsd Tue Dec 21 02:44:09 2004
> @@ -11,7 +11,7 @@
> ifeq ($(findstring sparc,$(host_cpu)), sparc)
> CFLAGS_SL = -fPIC -DPIC
> else
> -CFLAGS_SL = -fpic -DPIC
> +CFLAGS_SL = -fPIC -DPIC
> endif
>
>
> @@ -29,3 +29,5 @@
> endif
>
> sqlmansect = 7
> +
> +allow_nonpic_in_shlib = yes
>
> The first of these patches makes me think that octopus might actually be
> finding the wrong library, though things are fine on platypus with
> python 2.3 (octopus is running 2.4).
>
> I'm upgrading platypus to 2.4 right now to see what that changes, if
> anything.
> --
> Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
> Give your computer some brain candy! www.distributed.net Team #1828
>
> Windows: "Where do you want to go today?"
> Linux: "Where do you want to go tomorrow?"
> FreeBSD: "Are you guys coming, or what?"
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Patch applied. Thanks. If we made plpython worse, we will hear about
> it soon enough.

You did, and you're hearing about it. Please revert the
plpython/Makefile change; it is certainly wrong for every other
platform. What's more, it's unnecessary given the port makefile
change --- if you have allow_nonpic_in_shlib, it doesn't matter
whether shared_libpython gets set.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Patch applied. Thanks. If we made plpython worse, we will hear about
> > it soon enough.
>
> You did, and you're hearing about it. Please revert the
> plpython/Makefile change; it is certainly wrong for every other
> platform. What's more, it's unnecessary given the port makefile
> change --- if you have allow_nonpic_in_shlib, it doesn't matter
> whether shared_libpython gets set.

OK, backed out.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Unfortunately, it looks like the allow_non_pic_in_shlib setting broke
platypus: http://lnk.nu/pgbuildfarm.org/3l3.pl

If I back that part of the patch out, playtypus works fine.

On Fri, Aug 12, 2005 at 04:57:58PM -0400, Bruce Momjian wrote:
>
> Patch applied. Thanks. If we made plpython worse, we will hear about
> it soon enough. The freebsd-specific changes seem safe, considering
> they came from the FreeBSD port maintainers themselves.
>
> ---------------------------------------------------------------------------
>
>
> Jim C. Nasby wrote:
> > On Wed, Jul 13, 2005 at 01:24:17PM -0400, Andrew Dunstan wrote:
> > > >In any case, I've cleared the conflict and I'm running a build right
> > > >now.
> >
> > octopus is building again, and is back to the behavior I mentioned in
> > http://archives.postgresql.org/pgsql-bugs/2005-07/msg00096.php. Is this
> > something that should be fixed in code? There are two patches in the
> > FreeBSD ports tree for postgresql 8:
> >
> > decibel(at)flake(dot)2[14:18]/usr/ports/databases/postgresql80-server/files:47>cat patch-plpython-Makefile patch-src-makefiles-Makefile.freebsd
> > --- src/pl/plpython/Makefile.orig Fri Nov 19 20:23:01 2004
> > +++ src/pl/plpython/Makefile Tue Dec 28 23:32:16 2004
> > @@ -9,7 +9,7 @@
> > # shared library. Since there is no official way to determine this
> > # (at least not in pre-2.3 Python), we see if there is a file that is
> > # named like a shared library.
> > -ifneq (,$(wildcard $(python_libdir)/libpython*$(DLSUFFIX)*))
> > +ifneq (,$(wildcard $(python_libdir)/../../libpython*$(DLSUFFIX)*))
> > shared_libpython = yes
> > endif
> >
> > --- src/makefiles/Makefile.freebsd.orig Fri Nov 19 01:41:39 2004
> > +++ src/makefiles/Makefile.freebsd Tue Dec 21 02:44:09 2004
> > @@ -11,7 +11,7 @@
> > ifeq ($(findstring sparc,$(host_cpu)), sparc)
> > CFLAGS_SL = -fPIC -DPIC
> > else
> > -CFLAGS_SL = -fpic -DPIC
> > +CFLAGS_SL = -fPIC -DPIC
> > endif
> >
> >
> > @@ -29,3 +29,5 @@
> > endif
> >
> > sqlmansect = 7
> > +
> > +allow_nonpic_in_shlib = yes
> >
> > The first of these patches makes me think that octopus might actually be
> > finding the wrong library, though things are fine on platypus with
> > python 2.3 (octopus is running 2.4).
> >
> > I'm upgrading platypus to 2.4 right now to see what that changes, if
> > anything.
> > --
> > Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
> > Give your computer some brain candy! www.distributed.net Team #1828
> >
> > Windows: "Where do you want to go today?"
> > Linux: "Where do you want to go tomorrow?"
> > FreeBSD: "Are you guys coming, or what?"
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 2: Don't 'kill -9' the postmaster
> >
>
> --
> Bruce Momjian | http://candle.pha.pa.us
> pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
> + If your life is a hard drive, | 13 Roberts Road
> + Christ can be your backup. | Newtown Square, Pennsylvania 19073
>

--
Jim C. Nasby, Sr. Engineering Consultant jnasby(at)pervasive(dot)com
Pervasive Software http://pervasive.com 512-569-9461

"Jim C. Nasby" <jnasby(at)pervasive(dot)com> writes:
> Unfortunately, it looks like the allow_non_pic_in_shlib setting broke
> platypus: http://lnk.nu/pgbuildfarm.org/3l3.pl

> If I back that part of the patch out, playtypus works fine.

So what's different between platypus and the machines where it works?
We might need a version check or something ...

regards, tom lane

Tom Lane wrote:
> "Jim C. Nasby" <jnasby(at)pervasive(dot)com> writes:
>
>>Unfortunately, it looks like the allow_non_pic_in_shlib setting broke
>>platypus: http://lnk.nu/pgbuildfarm.org/3l3.pl
>
>
>>If I back that part of the patch out, playtypus works fine.
>
>
> So what's different between platypus and the machines where it works?
> We might need a version check or something ...

platypus is amd64, not x86. AFAIK, amd64 does not allow non-PIC code to
be mixed with PIC code in the same object, whereas it's just fine for
x86. In NetBSD pkgsrc, we've had to fix a lot of software that makes
this same assumption.

Cheers,

-- Johnny Lam <jlam(at)NetBSD(dot)org>

On Wed, Aug 17, 2005 at 01:22:16PM -0400, Johnny Lam wrote:
> Tom Lane wrote:
> >"Jim C. Nasby" <jnasby(at)pervasive(dot)com> writes:
> >
> >>Unfortunately, it looks like the allow_non_pic_in_shlib setting broke
> >>platypus: http://lnk.nu/pgbuildfarm.org/3l3.pl
> >
> >
> >>If I back that part of the patch out, playtypus works fine.
> >
> >
> >So what's different between platypus and the machines where it works?
> >We might need a version check or something ...
>
> platypus is amd64, not x86. AFAIK, amd64 does not allow non-PIC code to
> be mixed with PIC code in the same object, whereas it's just fine for
> x86. In NetBSD pkgsrc, we've had to fix a lot of software that makes
> this same assumption.

Damn, I'm sorry, I totally mis-interpreted this. Turns out the failures
are due to a perl problem. They appear to be from
http://lnk.nu/developer.postgresql.org/3l8.c.
http://lnk.nu/pgbuildfarm.org/3l9.pl is the log for the latest failure.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby(at)pervasive(dot)com
Pervasive Software http://pervasive.com 512-569-9461

"Jim C. Nasby" <jnasby(at)pervasive(dot)com> writes:
>>> Unfortunately, it looks like the allow_non_pic_in_shlib setting broke
>>> platypus: http://lnk.nu/pgbuildfarm.org/3l3.pl

> Damn, I'm sorry, I totally mis-interpreted this. Turns out the failures
> are due to a perl problem.

Yeah, but the nonpic change caused that.

Can anyone tell me which machine types (host_cpu values) FreeBSD does
support non-PIC code in shlibs for? Is it only x86, and if so exactly
what's the host_cpu setting? We need to throw a conditional into
Makefile.freebsd.

regards, tom lane

Tom Lane wrote:
> "Jim C. Nasby" <jnasby(at)pervasive(dot)com> writes:
>>
>>Damn, I'm sorry, I totally mis-interpreted this. Turns out the failures
>>are due to a perl problem.
>
>
> Yeah, but the nonpic change caused that.
>
> Can anyone tell me which machine types (host_cpu values) FreeBSD does
> support non-PIC code in shlibs for? Is it only x86, and if so exactly
> what's the host_cpu setting? We need to throw a conditional into
> Makefile.freebsd.

Judging from the config.guess script that the PostgreSQL distribution
uses, the host_cpu value should be "x86_64".

I think this mixing of non-relocatable and relocatable code is a problem
that exists for all amd64 machines, regardless of the operating system.
When Googling for the linker error message, I've seen it crop up for
users running NetBSD, FreeBSD and Linux. I think you'll want to add the
same conditional setting of allow_nonpic_in_shlib in
src/makefiles/Makefile.linux based on the host_cpu value.

Cheers,

-- Johnny Lam <jlam(at)NetBSD(dot)org>