Re: error after installing on CentOS 5.8 with source

Hi

I am trying to build and install psycopg2-2.4.5 source distribution on
CentOS 5.8 VPS.
Installation ends without errors, but on Python interpreter, import
psycopg2 returns:

Python 2.7.3 (default, Jul 25 2012, 19:59:36)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/python2.7/lib/python2.7/site-packages/psycopg2/__init__.py",
line 67, in <module>
from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError:
/opt/python2.7/lib/python2.7/site-packages/psycopg2/_psycopg.so: undefined
symbol: lo_truncate

I know It can be related to the issue posted in the FAQ
http://initd.org/psycopg/docs/faq.html#problems-compiling-and-deploying-psycopg2

"...You can avoid the problem by using the same version of the
*pg_config*at install time and the libpq at runtime. ..."

But I cannot find the command/configuration needed to control the pg_config
used at runtime, so I can set the pg_config to the newest version both for
installation and runtime.

python setup.py build is returning this warnings:

psycopg/lobject_int.c: In function ‘lobject_write’:
psycopg/lobject_int.c:343: aviso: el paso del argumento 3 de ‘lo_write’
descarta los calificadores del tipo del destino del puntero
(something like "warning: passing third argument to lo_write discards
target type classifiers of the pointer")

psycopg/lobject_int.c: In function ‘lobject_truncate’:
psycopg/lobject_int.c:478: aviso: declaración implícita de la función
‘lo_truncate’
("warning: implicit declaration of function lo_truncate")

I've installed Python 2.7.3 compiled from source
Also installed PostgreSQL 9.1.4 via yum

This is what ldd returns

[root(at)pycom ~] # ldd
/opt/python2.7/lib/python2.7/site-packages/psycopg2/_psycopg.so | grep libpq
libpq.so.4 => /usr/lib/libpq.so.4 (0x00716000)

Note: I have configured setup.cfg so it reads the updated pg_config file
(pg 9.1) before installation.

There is an older pg_config version stored in /usr/bin with this output:

#########################################################################

BINDIR = /usr/bin
DOCDIR = /usr/share/doc/pgsql
INCLUDEDIR = /usr/include
PKGINCLUDEDIR = /usr/include/pgsql
INCLUDEDIR-SERVER = /usr/include/pgsql/server
LIBDIR = /usr/lib
PKGLIBDIR = /usr/lib/pgsql
LOCALEDIR = /usr/share/locale
MANDIR = /usr/share/man
SHAREDIR = /usr/share/pgsql
SYSCONFDIR = /etc/sysconfig/pgsql
PGXS = /usr/lib/pgsql/pgxs/src/makefiles/pgxs.mk
CONFIGURE = '--build=i386-redhat-linux-gnu' '--host=i386-redhat-linux-gnu'
'--target=i386-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr'
'--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin'
'--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include'
'--libdir=/usr/lib' '--libexecdir=/usr/libexec' '--localstatedir=/var'
'--sharedstatedir=/usr/com' '--mandir=/usr/share/man'
'--infodir=/usr/share/info' '--disable-rpath' '--with-perl' '--with-tcl'
'--with-tclconfig=/usr/lib' '--with-python' '--with-openssl' '--with-pam'
'--with-krb5' '--enable-nls' '--enable-thread-safety'
'--sysconfdir=/etc/sysconfig/pgsql' '--datadir=/usr/share/pgsql'
'--with-docdir=/usr/share/doc' 'CFLAGS=-O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic
-fasynchronous-unwind-tables' 'build_alias=i386-redhat-linux-gnu'
'host_alias=i386-redhat-linux-gnu' 'target_alias=i386-redhat-linux-gnu'
CC = gcc
CPPFLAGS = -D_GNU_SOURCE
CFLAGS = -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic
-fasynchronous-unwind-tables -Wall -Wmissing-prototypes -Wpointer-arith
-Winline -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing
-fwrapv
CFLAGS_SL = -fpic
LDFLAGS =
LDFLAGS_SL =
LIBS = -lpgport -lpam -lssl -lcrypto -lkrb5 -lz -lreadline -ltermcap
-lcrypt -lresolv -lnsl -ldl -lm -lbsd
VERSION = PostgreSQL 8.1.23

#########################################################################

The new pg_config output:
#########################################################################

BINDIR = /usr/pgsql-9.1/bin
DOCDIR = /usr/share/doc/pgsql
HTMLDIR = /usr/share/doc/pgsql
INCLUDEDIR = /usr/pgsql-9.1/include
PKGINCLUDEDIR = /usr/pgsql-9.1/include
INCLUDEDIR-SERVER = /usr/pgsql-9.1/include/server
LIBDIR = /usr/pgsql-9.1/lib
PKGLIBDIR = /usr/pgsql-9.1/lib
LOCALEDIR = /usr/pgsql-9.1/share/locale
MANDIR = /usr/pgsql-9.1/share/man
SHAREDIR = /usr/pgsql-9.1/share
SYSCONFDIR = /etc/sysconfig/pgsql
PGXS = /usr/pgsql-9.1/lib/pgxs/src/makefiles/pgxs.mk
CONFIGURE = '--disable-rpath' '--prefix=/usr/pgsql-9.1'
'--includedir=/usr/pgsql-9.1/include' '--mandir=/usr/pgsql-9.1/share/man'
'--datadir=/usr/pgsql-9.1/share' '--with-perl' '--with-python' '--with-tcl'
'--with-tclconfig=/usr/lib' '--with-openssl' '--with-pam' '--with-krb5'
'--with-gssapi' '--with-includes=/usr/include' '--with-libraries=/usr/lib'
'--enable-nls' '--with-ossp-uuid' '--with-libxml' '--with-libxslt'
'--with-ldap' '--with-system-tzdata=/usr/share/zoneinfo'
'--sysconfdir=/etc/sysconfig/pgsql' '--docdir=/usr/share/doc' 'CFLAGS=-O2
-g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic
-fasynchronous-unwind-tables -I/usr/include/et' 'CPPFLAGS=
-I/usr/include/et'
CC = gcc
CPPFLAGS = -I/usr/include/et -D_GNU_SOURCE -I/usr/include/libxml2
-I/usr/include
CFLAGS = -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic
-fasynchronous-unwind-tables -I/usr/include/et -Wall -Wmissing-prototypes
-Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
-Wformat-security -fno-strict-aliasing -fwrapv
CFLAGS_SL = -fpic
LDFLAGS = -L/usr/lib -L/usr/lib
LDFLAGS_EX =
LDFLAGS_SL =
LIBS = -lpgport -lxslt -lxml2 -lpam -lssl -lcrypto -lgssapi_krb5 -lz
-lreadline -ltermcap -lcrypt -ldl -lm
VERSION = PostgreSQL 9.1.4

#########################################################################

Thanks

On Thu, Jul 26, 2012 at 11:21 PM, Alan Etkin <spametki(at)gmail(dot)com> wrote:
> Hi
>
> I am trying to build and install psycopg2-2.4.5 source distribution on
> CentOS 5.8 VPS.
> Installation ends without errors, but on Python interpreter, import psycopg2
> returns:
>
> [...]
> ImportError:
> /opt/python2.7/lib/python2.7/site-packages/psycopg2/_psycopg.so: undefined
> symbol: lo_truncate
>
> I know It can be related to the issue posted in the FAQ
> http://initd.org/psycopg/docs/faq.html#problems-compiling-and-deploying-psycopg2
>
> "...You can avoid the problem by using the same version of the pg_config at
> install time and the libpq at runtime. ..."

Yep, that's the case.

> But I cannot find the command/configuration needed to control the pg_config
> used at runtime, so I can set the pg_config to the newest version both for
> installation and runtime.

You cannot choose explicitly the lib to use at runtime: you should
manipulate the LD_LIBRARY_PATH to include the newest dir... which is a
pain in the neck. You better compile using the older pg_config.

> Note: I have configured setup.cfg so it reads the updated pg_config file (pg
> 9.1) before installation.

Yes, this is the problem. You compile with library and headers
offering lo_truncate but don't make it available at runtime.

> This is what ldd returns
>
> [root(at)pycom ~] # ldd
> /opt/python2.7/lib/python2.7/site-packages/psycopg2/_psycopg.so | grep libpq
> libpq.so.4 => /usr/lib/libpq.so.4 (0x00716000)
>

Yes, this is the 8.1 version.

> There is an older pg_config version stored in /usr/bin with this output:
> LIBDIR = /usr/lib
> VERSION = PostgreSQL 8.1.23

> The new pg_config output:
> LIBDIR = /usr/pgsql-9.1/lib
> VERSION = PostgreSQL 9.1.4

You would live a much healthier life if you could get rid of Postgres
8.1, which is unsupported by the way, and install 9.1 in a standard
location, such as /usr/local/

If you cannot do this, either you compile psycopg against PG 8.1 by
using the pg_config in /usr/bin, or you compile for PG 9.1 but set the
LD_LIBRARY_PATH everywhere you need psycopg2 (you may also try hacking
psycopg's __init__.py file setting os.environ['LD_LIBRARY_PATH'] =
'/usr/pgsql-9.1/lib' before the first "from _psycopg import...")

Note that installing 9.1 in /usr/local could be enough to have the
newer libpq found automatically instead of the older, but I'm not
sure. You can try creating a symlink to the newer libpq.so into
/usr/local/lib and use ldd to check it gets used. This would have a
global effect though: can't guarantee there will be no side effects
from the libraries mixup.

-- Daniele

Thanks to all the detailed tips you've posted for the issue I managed to do
import psycopg2 in the CentOS 5.8 VPS:

-Removed all pg installations in the system (luckily there were no dbs to
backup)
-Built/Installed last PostgreSQL dist. at standard location under /usr/local
-Re-built/installed psycopg2 source dist. (set proper pg_config parameter
in setup.cfg)
-I had to do this too, because psycopg2 still complained about libqp.so.5
location (maybe because of not setting lib dirs in setup.cfg):
# cp /usr/local/<pg lib> /usr/lib

After those steps Python2.7.3 plus psycopg2 runs ok. I think the last step
might have something to do with the fact that the python interpreter
libraries are installed under /opt/... which I think is a distribution
non-standard location setup

Note: perhaps this needs further tests for db connection or other issues

Thank you very much

On Fri, Jul 27, 2012 at 3:39 PM, Alan Etkin <spametki(at)gmail(dot)com> wrote:
> Thanks to all the detailed tips you've posted for the issue I managed to do
> import psycopg2 in the CentOS 5.8 VPS:

Cool

> -Removed all pg installations in the system (luckily there were no dbs to
> backup)
> -Built/Installed last PostgreSQL dist. at standard location under /usr/local
> -Re-built/installed psycopg2 source dist. (set proper pg_config parameter in
> setup.cfg)
> -I had to do this too, because psycopg2 still complained about libqp.so.5
> location (maybe because of not setting lib dirs in setup.cfg):

No: lib dirs in setup.cfg are unused and have been dropped recently.
Only the results from pg_config are used now.

> # cp /usr/local/<pg lib> /usr/lib

I wouldn't have expected this to be requested. But what was the source
dir exactly? Didn't you have /usr/local/lib/libpq.so?

> After those steps Python2.7.3 plus psycopg2 runs ok. I think the last step
> might have something to do with the fact that the python interpreter
> libraries are installed under /opt/... which I think is a distribution
> non-standard location setup

This sounds strange to me: the only problem is usually to find the
libpq. When psycopg.so is loaded, the python-related libs have already
been loaded.

> Note: perhaps this needs further tests for db connection or other issues

You can try running the test suite with "make check", after setting
the PSYCOPG_TESTDB_* env variables and maybe creating a scratch db.
See tests/testconfig.py to check what are the vars used and their
default.

Cheers,

-- Daniele

>
> dir exactly? Didn't you have /usr/local/lib/libpq.so?
>
>
libpq files are stored under /usr/lib (although there is a /usr/local/lib
path)

You can try running the test suite with "make check", after setting

> the PSYCOPG_TESTDB_* env variables and maybe creating a scratch db.
> See tests/testconfig.py to check what are the vars used and their
> default.
>
>
I couldn't run the test until I added the environment PYTHONPATH entry to
the default bash configuration file at etc/profile. with the
...site-packages folder because the python interpreter raised ImportError.

This is surely something misconfigured when installing the python
interpreter in CentOS.

After testing with a new database the result is:

----------------------------------------------------------------------
Ran 382 tests in 131.328s

OK (skipped=45)

Note: The make check test stays idle at test_async_cancel if
PSYCOPG2_TESTDB_PASSWORD is not provided, as the pg database owner is
configured to login for db connection

Thanks again

> But what was the source dir exactly?

New libpq files for pg 9.1.4 are installed (by default, I suppose) in
/usr/local/pgsql/lib

Hi,

On Thu, 2012-07-26 at 19:21 -0300, Alan Etkin wrote:

> I am trying to build and install psycopg2-2.4.5 source distribution on
> CentOS 5.8 VPS.

Why are you trying to build? PostgreSQL RPM repository has both
PostgreSQL RPMs and psycopg RPMs for RHEL 4,5,6:

For example,

http://yum.postgresql.org/9.1/redhat/rhel-5-x86_64/repoview/python-psycopg2.html

Regards,
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org Twitter: http://twitter.com/devrimgunduz

>
> Why are you trying to build? PostgreSQL RPM repository has both
> PostgreSQL RPMs and psycopg RPMs for RHEL 4,5,6:
>
> For example,
>
>
> http://yum.postgresql.org/9.1/redhat/rhel-5-x86_64/repoview/python-psycopg2.html
>
> Regards,
> --
> Devrim GÜNDÜZ
>

First I installed pg with those rpms, but I wasn't aware of the updated rpm
with psycopg2, just searched for installation within the psycopg2 project
page. It's good to know that reinstalling would be much simpler.

Thanks