Re: BUG #2600: dblink compile with SSL missing libraries

The following bug has been logged online:

Bug reference: 2600
Logged by: Christopher Browne
Email address: cbbrowne(at)ca(dot)afilias(dot)info
PostgreSQL version: CVS HEAD (8.2)
Operating system: AIX 5.3
Description: dblink compile with SSL missing libraries
Details:

If I try to build dblink when PG is configured "--with-openssl", the build
of the contrib module dblink breaks as follows:

pgdba(at)ydb1(dot)int(dot)libertyrms(dot)com:/opt/rg/data_dba/build-farm/HEAD/pgsql.741430/
contrib/dblink $ gmake
../../src/backend/port/aix/mkldexport.sh libdblink.a > libdblink.exp
/opt/prod/gcc-4.1.1/bin/gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Winline -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing
-g -Wl,-bmaxdata:0x80000000 -Wl,-bnoentry -Wl,-H512 -Wl,-bM:SRE -o
libdblink.so libdblink.a -Wl,-bE:libdblink.exp -L../../src/interfaces/libpq
-L../../src/port -L/opt/freeware/lib -lpq -lpthread -lpthreads
-Wl,-bI:../../src/backend/postgres.imp
ld: 0711-317 ERROR: Undefined symbol: .SSL_pending
ld: 0711-317 ERROR: Undefined symbol: .ERR_get_error
ld: 0711-317 ERROR: Undefined symbol: .ERR_reason_error_string
ld: 0711-317 ERROR: Undefined symbol: .SSL_write
ld: 0711-317 ERROR: Undefined symbol: .SSL_get_error
ld: 0711-317 ERROR: Undefined symbol: .SSL_read
ld: 0711-317 ERROR: Undefined symbol: .SSL_shutdown
ld: 0711-317 ERROR: Undefined symbol: .SSL_free
ld: 0711-317 ERROR: Undefined symbol: .X509_free
ld: 0711-317 ERROR: Undefined symbol: .SSL_connect
ld: 0711-317 ERROR: Undefined symbol: .SSL_get_peer_certificate
ld: 0711-317 ERROR: Undefined symbol: .X509_get_subject_name
ld: 0711-317 ERROR: Undefined symbol: .X509_NAME_oneline
ld: 0711-317 ERROR: Undefined symbol: .X509_NAME_get_text_by_NID
ld: 0711-317 ERROR: Undefined symbol: .SSL_new
ld: 0711-317 ERROR: Undefined symbol: .SSL_set_ex_data
ld: 0711-317 ERROR: Undefined symbol: .SSL_set_fd
ld: 0711-317 ERROR: Undefined symbol: .SSL_CTX_free
ld: 0711-317 ERROR: Undefined symbol: .TLSv1_method
ld: 0711-317 ERROR: Undefined symbol: .SSL_CTX_new
ld: 0711-317 ERROR: Undefined symbol: .SSL_CTX_load_verify_locations
ld: 0711-317 ERROR: Undefined symbol: .SSL_CTX_get_cert_store
ld: 0711-317 ERROR: Undefined symbol: .X509_STORE_load_locations
ld: 0711-317 ERROR: Undefined symbol: .SSL_CTX_set_verify
ld: 0711-317 ERROR: Undefined symbol: .SSL_library_init
ld: 0711-317 ERROR: Undefined symbol: .SSL_load_error_strings
ld: 0711-317 ERROR: Undefined symbol: .CRYPTO_set_id_callback
ld: 0711-317 ERROR: Undefined symbol: .CRYPTO_num_locks
ld: 0711-317 ERROR: Undefined symbol: .CRYPTO_set_locking_callback
ld: 0711-317 ERROR: Undefined symbol: .X509_STORE_set_flags
ld: 0711-317 ERROR: Undefined symbol: .SSL_get_ex_data
ld: 0711-317 ERROR: Undefined symbol: .PEM_read_X509
ld: 0711-317 ERROR: Undefined symbol: .PEM_read_PrivateKey
ld: 0711-317 ERROR: Undefined symbol: .X509_check_private_key
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more
information.
collect2: ld returned 8 exit status
gmake: *** [libdblink.so] Error 1

If I add, to the GCC command line, requests for libssl and libcrypto...
-lssl -lcrypto

e.g. - command line:

pgdba(at)ydb1(dot)int(dot)libertyrms(dot)com:/opt/rg/data_dba/build-farm/HEAD/pgsql.741430/
contrib/dblink $ /opt/prod/gcc-4.1.1/bin/gcc -O2 -Wall -Wmissing-prototypes
-Wpointer-arith -Winline -Wdeclaration-after-statement -Wendif-labels
-fno-strict-aliasing -g -Wl,-bmaxdata:0x80000000 -Wl,-bnoentry -Wl,-H512
-Wl,-bM:SRE -o libdblink.so libdblink.a -Wl,-bE:libdblink.exp
-L../../src/interfaces/libpq -L../../src/port -L/opt/freeware/lib -lpq
-lpthread -lpthreads -lssl -lcrypto -Wl,-bI:../../src/backend/postgres.imp

This builds fine without further complaint.

Christopher Browne wrote:
> The following bug has been logged online:

>
> If I try to build dblink when PG is configured "--with-openssl", the build
> of the contrib module dblink breaks as follows:

> If I add, to the GCC command line, requests for libssl and libcrypto...
> -lssl -lcrypto
>
> e.g. - command line:
>
> pgdba(at)ydb1(dot)int(dot)libertyrms(dot)com:/opt/rg/data_dba/build-farm/HEAD/pgsql.741430/
> contrib/dblink $ /opt/prod/gcc-4.1.1/bin/gcc -O2 -Wall -Wmissing-prototypes
> -Wpointer-arith -Winline -Wdeclaration-after-statement -Wendif-labels
> -fno-strict-aliasing -g -Wl,-bmaxdata:0x80000000 -Wl,-bnoentry -Wl,-H512
> -Wl,-bM:SRE -o libdblink.so libdblink.a -Wl,-bE:libdblink.exp
> -L../../src/interfaces/libpq -L../../src/port -L/opt/freeware/lib -lpq
> -lpthread -lpthreads -lssl -lcrypto -Wl,-bI:../../src/backend/postgres.imp
>
> This builds fine without further complaint.

Interesting. I build using "--with-openssl" all the time and have never
had a problem. Can anyone comment on the appropriate Makefile changes
for this?

Thanks,

Joe

Joe Conway wrote:
> Christopher Browne wrote:
>> The following bug has been logged online:
>
>>
>> If I try to build dblink when PG is configured "--with-openssl", the
>> build
>> of the contrib module dblink breaks as follows:
>
>> If I add, to the GCC command line, requests for libssl and libcrypto...
>> -lssl -lcrypto
>>
>> e.g. - command line:
>> pgdba(at)ydb1(dot)int(dot)libertyrms(dot)com:/opt/rg/data_dba/build-farm/HEAD/pgsql.741430/
>>
>> contrib/dblink $ /opt/prod/gcc-4.1.1/bin/gcc -O2 -Wall
>> -Wmissing-prototypes
>> -Wpointer-arith -Winline -Wdeclaration-after-statement -Wendif-labels
>> -fno-strict-aliasing -g -Wl,-bmaxdata:0x80000000 -Wl,-bnoentry -Wl,-H512
>> -Wl,-bM:SRE -o libdblink.so libdblink.a -Wl,-bE:libdblink.exp
>> -L../../src/interfaces/libpq -L../../src/port -L/opt/freeware/lib -lpq
>> -lpthread -lpthreads -lssl -lcrypto
>> -Wl,-bI:../../src/backend/postgres.imp
>>
>> This builds fine without further complaint.
>
> Interesting. I build using "--with-openssl" all the time and have never
> had a problem. Can anyone comment on the appropriate Makefile changes
> for this?

hmm that actually seems to be a rather AIX-centric issue since we have a
ton of buildfarm boxes building with --with-openssl ...

Stefan

mail(at)joeconway(dot)com (Joe Conway) writes:
> Christopher Browne wrote:
>> The following bug has been logged online:
>
>> If I try to build dblink when PG is configured "--with-openssl", the
>> build
>> of the contrib module dblink breaks as follows:
>
>> If I add, to the GCC command line, requests for libssl and libcrypto...
>> -lssl -lcrypto
>> e.g. - command line:
>> pgdba(at)ydb1(dot)int(dot)libertyrms(dot)com:/opt/rg/data_dba/build-farm/HEAD/pgsql.741430/
>> contrib/dblink $ /opt/prod/gcc-4.1.1/bin/gcc -O2 -Wall -Wmissing-prototypes
>> -Wpointer-arith -Winline -Wdeclaration-after-statement -Wendif-labels
>> -fno-strict-aliasing -g -Wl,-bmaxdata:0x80000000 -Wl,-bnoentry -Wl,-H512
>> -Wl,-bM:SRE -o libdblink.so libdblink.a -Wl,-bE:libdblink.exp
>> -L../../src/interfaces/libpq -L../../src/port -L/opt/freeware/lib -lpq
>> -lpthread -lpthreads -lssl -lcrypto -Wl,-bI:../../src/backend/postgres.imp
>> This builds fine without further complaint.
>
> Interesting. I build using "--with-openssl" all the time and have
> never had a problem. Can anyone comment on the appropriate Makefile
> changes for this?

I do notice that on Linux, the command line doesn't include requests
for either library.

I also seem to recall, in past discussions about "library matters,"
that AIX is more sticky about requiring that libraries be named
expressly. Maybe we add something like...

ifeq($(findstring aix, $(host_os)), aix)
LIBS += -lssl -lcrypt
endif

(Though that doesn't seem to do the trick...)
--
let name="cbbrowne" and tld="ntlug.org" in String.concat "@" [name;tld];;
http://www.ntlug.org/~cbbrowne/emacs.html
I'm sure glad we're having this "How many FTP transfers can dance on
the head of a chargeback packet" conversation now, because when
chargebacks happen, it will surely be too expensive to read these
amazing conversations.

Am Mittwoch, 30. August 2006 22:57 schrieb Chris Browne:
> I also seem to recall, in past discussions about "library matters,"
> that AIX is more sticky about requiring that libraries be named
> expressly.

ecpglib has

SHLIB_LINK = -L../pgtypeslib -lpgtypes $(libpq) \
$(filter -lintl -lssl -lcrypto -lkrb5 -lcrypt -lm, $(LIBS))
$(PTHREAD_LIBS)

ifeq ($(PORTNAME), win32)
# Link to shfolder.dll instead of shell32.dll
SHLIB_LINK += -lshfolder
endif

Presumably the same would be necessary everywhere else libpq is used.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Am Mittwoch, 30. August 2006 22:57 schrieb Chris Browne:
>> I also seem to recall, in past discussions about "library matters,"
>> that AIX is more sticky about requiring that libraries be named
>> expressly.

> ecpglib has

> SHLIB_LINK = -L../pgtypeslib -lpgtypes $(libpq) \
> $(filter -lintl -lssl -lcrypto -lkrb5 -lcrypt -lm, $(LIBS))
> $(PTHREAD_LIBS)

> ifeq ($(PORTNAME), win32)
> # Link to shfolder.dll instead of shell32.dll
> SHLIB_LINK += -lshfolder
> endif

> Presumably the same would be necessary everywhere else libpq is used.

Surely it'd be better to put that into the $(libpq) macro instead.
I see that Makefile.global already has an AIX-specific hack to deal with
PTHREAD_LIBS that way, maybe we need to extend it?

Also, the list of potential library dependencies in libpq's own Makefile
is quite a bit longer than this list ... perhaps ecpg's list is too
short?

regards, tom lane

Where are we on this?

---------------------------------------------------------------------------

Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > Am Mittwoch, 30. August 2006 22:57 schrieb Chris Browne:
> >> I also seem to recall, in past discussions about "library matters,"
> >> that AIX is more sticky about requiring that libraries be named
> >> expressly.
>
> > ecpglib has
>
> > SHLIB_LINK = -L../pgtypeslib -lpgtypes $(libpq) \
> > $(filter -lintl -lssl -lcrypto -lkrb5 -lcrypt -lm, $(LIBS))
> > $(PTHREAD_LIBS)
>
> > ifeq ($(PORTNAME), win32)
> > # Link to shfolder.dll instead of shell32.dll
> > SHLIB_LINK += -lshfolder
> > endif
>
> > Presumably the same would be necessary everywhere else libpq is used.
>
> Surely it'd be better to put that into the $(libpq) macro instead.
> I see that Makefile.global already has an AIX-specific hack to deal with
> PTHREAD_LIBS that way, maybe we need to extend it?
>
> Also, the list of potential library dependencies in libpq's own Makefile
> is quite a bit longer than this list ... perhaps ecpg's list is too
> short?
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

A long time ago, in a galaxy far, far away, bruce(at)momjian(dot)us (Bruce Momjian) wrote:
> Where are we on this?

Nothing submitted so far. If someone magically does something about
it, that's super. I'm on vacation 'til Tuesday; if it's not dealt
with, I'll see if I can find something.
--
(format nil "~S(at)~S" "cbbrowne" "cbbrowne.com")
http://linuxdatabases.info/info/lsf.html
Pay no attention to the PDP-11 behind the front panel.
-- PGS, in reference to OZ

peter_e(at)gmx(dot)net (Peter Eisentraut) writes:
> Am Mittwoch, 30. August 2006 22:57 schrieb Chris Browne:
>> I also seem to recall, in past discussions about "library matters,"
>> that AIX is more sticky about requiring that libraries be named
>> expressly.
>
> ecpglib has
>
> SHLIB_LINK = -L../pgtypeslib -lpgtypes $(libpq) \
> $(filter -lintl -lssl -lcrypto -lkrb5 -lcrypt -lm, $(LIBS))
> $(PTHREAD_LIBS)
>
> ifeq ($(PORTNAME), win32)
> # Link to shfolder.dll instead of shell32.dll
> SHLIB_LINK += -lshfolder
> endif
>
> Presumably the same would be necessary everywhere else libpq is used.

I replaced:
SHLIB_LINK = $(libpq)

with
SHLIB_LINK = $(libpq) $(LIBS)

which allowed the compile to get through this.

If I add that very same line:
SHLIB_LINK = $(libpq) $(LIBS)

to contrib/sslinfo/Makefile, it now survives the compile, as well as
successfully running through, for contrib, "make install" and "make
installcheck".
--
"cbbrowne","@","cbbrowne.com"
http://linuxfinances.info/info/unix.html
Do you know where your towel is?

The change Tom made to contrib/sshinfo/Makefile to support Darwin,
adding in $(LIBS), fixed my problem with that contrib module on AIX.

I still need the following, on AIX:

===================================================================
RCS file: /projects/cvsroot/pgsql/contrib/dblink/Makefile,v
retrieving revision 1.11
diff -u -r1.11 Makefile
--- Makefile 27 Feb 2006 12:54:38 -0000 1.11
+++ Makefile 6 Sep 2006 19:48:23 -0000
@@ -3,7 +3,7 @@
MODULE_big = dblink
PG_CPPFLAGS = -I$(libpq_srcdir)
OBJS = dblink.o
-SHLIB_LINK = $(libpq)
+SHLIB_LINK = $(libpq) $(LIBS)

DATA_built = dblink.sql
DATA = uninstall_dblink.sql
--
(reverse (concatenate 'string "moc.enworbbc" "@" "enworbbc"))
http://linuxdatabases.info/info/nonrdbms.html
"Fashion is a form of ugliness so intolerable that we have to alter it
every six months." -- Oscar Wilde

Chris Browne <cbbrowne(at)acm(dot)org> writes:
> I still need the following, on AIX:

> -SHLIB_LINK = $(libpq)
> +SHLIB_LINK = $(libpq) $(LIBS)

No you don't --- see recent warthog complaint. We have to filter LIBS
down to just the minimum.

regards, tom lane

tgl(at)sss(dot)pgh(dot)pa(dot)us (Tom Lane) writes:
> Chris Browne <cbbrowne(at)acm(dot)org> writes:
>> I still need the following, on AIX:
>
>> -SHLIB_LINK = $(libpq)
>> +SHLIB_LINK = $(libpq) $(LIBS)
>
> No you don't --- see recent warthog complaint. We have to filter LIBS
> down to just the minimum.

I'm at a loss, then.

- If LIBS is being filtered to the minimum, then shouldn't it be
appropriate to add it in here?

- There isn't any variable other than LIBS that *does* get bound to
include -lssl and -lcrypto

- Do we need to add an additional LIBSSL, spattered widely through
makefiles, which sometimes gets linked in?

- Or do we need some custom DBLINKLIBS, defined in configure, that is
only used for dblink?
--
(reverse (concatenate 'string "moc.enworbbc" "@" "enworbbc"))
http://www3.sympatico.ca/cbbrowne/languages.html
QT adds to a Linux distribution a level of licencing complexity that
nullifies one of the major virtues of Linux: no licencing complexity.
-- <jedi(at)dementia(dot)mishnet>

Chris Browne <cbbrowne(at)acm(dot)org> writes:
> tgl(at)sss(dot)pgh(dot)pa(dot)us (Tom Lane) writes:
>> No you don't --- see recent warthog complaint. We have to filter LIBS
>> down to just the minimum.

> I'm at a loss, then.

> - If LIBS is being filtered to the minimum, then shouldn't it be
> appropriate to add it in here?

No, LIBS isn't filtered at all. See my recent commit to sslinfo's
Makefile --- I blew it just like this, you should learn from my mistake.

regards, tom lane

tgl(at)sss(dot)pgh(dot)pa(dot)us (Tom Lane) writes:
> Chris Browne <cbbrowne(at)acm(dot)org> writes:
>> tgl(at)sss(dot)pgh(dot)pa(dot)us (Tom Lane) writes:
>>> No you don't --- see recent warthog complaint. We have to filter LIBS
>>> down to just the minimum.
>
>> I'm at a loss, then.
>
>> - If LIBS is being filtered to the minimum, then shouldn't it be
>> appropriate to add it in here?
>
> No, LIBS isn't filtered at all. See my recent commit to sslinfo's
> Makefile --- I blew it just like this, you should learn from my mistake.

OK, the very same change as your recent change to
contrib/sslinfo/Makefile works out fine for contrib/dblink/Makefile.
That allows a buildfarm run to go through perfectly.

I suspect that both Makefiles also need to "filter in" -lgettext or
something similar; see bug #2608, which shows off much the same
problem surrounding NLS support. (I'm happy to see that someone's
running xlC on AIX 5.3, by the way... I should be getting a copy Real
Soon Now, but it may not be soon enough to be helpful :-( )
--
output = reverse("ofni.secnanifxunil" "@" "enworbbc")
http://linuxdatabases.info/info/multiplexor.html
"If God meant us to be vegetarians why'd He make cows out of meat?"
-- seen on a bumper sticker