Re: STL problem in stored procedures

Hi All,

I have downloaded an extension for PostgreSql.
It contains stored procedures written in C.
I compiled the neccessary files into an so file
and I tried to use the functions, but it gives
an error message when it has to load the
shared object file.
The error message says, it cannot locate a symbol.
I think this symbol is part of the C++ STL.

This is the error message:

psql:/usr/share/postgresql/contrib/proximity.sql:5: ERROR: could not load
library "/usr/lib/postgresql/proximity.so": /usr/lib/postgresql/proximity.so:
undefined symbol: _ZNKSs17find_first_not_ofERKSsj

What should I do to be able to use this extension?
Anyone has any experience with this extension?

Thanks,

Zoli

--
Zoltan Dudas
Software Engineer
SEI Hungary
Office +36-52-889-532 Ext: 2032
Fax +36-52-889-599
Email zoltan(dot)dudas(at)sei(dot)hu
URL www.sei-it.com

Zoltán Dudás schrob:

> I have downloaded an extension for PostgreSql.
> It contains stored procedures written in C.
> I compiled the neccessary files into an so file
> and I tried to use the functions, but it gives
> an error message when it has to load the
> shared object file.
> The error message says, it cannot locate a symbol.
> I think this symbol is part of the C++ STL.

Yes, that's a member of the basic_string template and the string
class.

> This is the error message:
>
> psql:/usr/share/postgresql/contrib/proximity.sql:5: ERROR: could not load
> library "/usr/lib/postgresql/proximity.so": /usr/lib/postgresql/proximity.so:
> undefined symbol: _ZNKSs17find_first_not_ofERKSsj
>
> What should I do to be able to use this extension?

It looks like your .so isn't linked to libstdc++. How exactly did you
compile and link the .so?

regards,
Andreas
--

On Tuesday 25 October 2005 12.32, Andreas Seltenreich wrote:
> > This is the error message:
> >
> > psql:/usr/share/postgresql/contrib/proximity.sql:5: ERROR: could not
> > load library "/usr/lib/postgresql/proximity.so":
> > /usr/lib/postgresql/proximity.so: undefined symbol:
> > _ZNKSs17find_first_not_ofERKSsj
> >
> > What should I do to be able to use this extension?
>
> It looks like your .so isn't linked to libstdc++. How exactly did you
> compile and link the .so?

I use a very simple makefile which is from the PostgreSql documentation.

Makefile:

OBJS = pg_prox.o proximity.o ../common/pg_utils.o
MODULE_big = proximity
DATA_built = proximity.sql
DOCS = README.proximity

PGXS := $(shell pg_config --pgxs)
include $(PGXS)

- end of makefile -

Regards,

Zoli

--
Zoltan Dudas
Software Engineer
SEI Hungary
Office +36-52-889-532 Ext: 2032
Fax +36-52-889-599
Email zoltan(dot)dudas(at)sei(dot)hu
URL www.sei-it.com

Hi,

> > It looks like your .so isn't linked to libstdc++. How exactly did you
> > compile and link the .so?

I attached output of the "make -n".

Regards,

Zoli

--
Zoltan Dudas
Software Engineer
SEI Hungary
Office +36-52-889-532 Ext: 2032
Fax +36-52-889-599
Email zoltan(dot)dudas(at)sei(dot)hu
URL www.sei-it.com

On Tue, Oct 25, 2005 at 02:11:31PM +0200, Zoltán Dudás wrote:
> Hi,
>
> > > It looks like your .so isn't linked to libstdc++. How exactly did you
> > > compile and link the .so?
>
> I attached output of the "make -n".

I notice other contrib modules include lines like:

SHLIB_LINK = $(libpq)
SHLIB_LINK += $(filter -lm, $(LIBS))
SHLIB_LINK = -lxml2 -lxslt

Perhaps you need a:

SHLIB_LICK = -lstl

or whatever the lib is called.
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

Hi,

On Tuesday 25 October 2005 14.34, Martijn van Oosterhout wrote:
> I notice other contrib modules include lines like:
>
> SHLIB_LINK = $(libpq)
> SHLIB_LINK += $(filter -lm, $(LIBS))
> SHLIB_LINK = -lxml2 -lxslt
>
> Perhaps you need a:
>
> SHLIB_LICK = -lstl
>
> or whatever the lib is called.

Thanks, it was the solution:

SHLIB_LINK = $(libpq)
SHLIB_LINK += $(filter -lm, $(LIBS))
SHLIB_LINK = -lstdc++

Regards,

Zoli

--
Zoltan Dudas
Software Engineer
SEI Hungary
Office +36-52-889-532 Ext: 2032
Fax +36-52-889-599
Email zoltan(dot)dudas(at)sei(dot)hu
URL www.sei-it.com

Martijn van Oosterhout <kleptog(at)svana(dot)org> writes:
> Perhaps you need a:
> SHLIB_LICK = -lstl
> or whatever the lib is called.

I think he needs to rewrite in C :-(. The backend is not C++ and I
fear it's unlikely that libc++ will play nicely as a dynamic add-on.

regards, tom lane

Tom Lane wrote:

>I think he needs to rewrite in C :-(. The backend is not C++ and I
>fear it's unlikely that libc++ will play nicely as a dynamic add-on.
>
>
Grrr. I also have a C++ library that I'd like to use from PostgreSQL on
Mac OS X.

I have succeeded in calling a tiny toy C++ shared library from PG, but I
don't know enough to understand what problems might be associated with
using more complicated C++ code.

It would be great if some C++/C guru could make a thorough analysis of
C++ integration issues.

Thanks,
Kevin Murphy

Kevin Murphy <murphy(at)genome(dot)chop(dot)edu> writes:
> Tom Lane wrote:
>> I think he needs to rewrite in C :-(. The backend is not C++ and I
>> fear it's unlikely that libc++ will play nicely as a dynamic add-on.

> It would be great if some C++/C guru could make a thorough analysis of
> C++ integration issues.

The main thing that scares me is lack of integration of C error handling
(setjmp/longjmp) with C++ error handling (catch/throw). In a
self-contained module you could perhaps avoid this by not using
catch/throw, but I don't think you get to ignore the issue if you are
using STL modules.

It *might* work to put a generic "catch/report via elog" handler around
each one of your entry-point functions. Haven't tried it.

This all assumes that libc++ can cooperate with, rather than try to
replace, libc in the first place. That would depend a lot on what
platform you are on --- I think it might work OK with modern glibc,
but my last experience with C++ was back when it wasn't so.

regards, tom lane

On Tue, Oct 25, 2005 at 10:24:53AM -0400, Kevin Murphy wrote:
> I have succeeded in calling a tiny toy C++ shared library from PG, but I
> don't know enough to understand what problems might be associated with
> using more complicated C++ code.
>
> It would be great if some C++/C guru could make a thorough analysis of
> C++ integration issues.

Lots of people use PostGIS, which can be configured to use GEOS,
which is a C++ library. You might find useful information in the
PostGIS list archives or by asking on one of their lists.

http://postgis.refractions.net/

--
Michael Fuhr

Tom Lane schrob:

> Kevin Murphy <murphy(at)genome(dot)chop(dot)edu> writes:
>> Tom Lane wrote:
>>> I think he needs to rewrite in C :-(. The backend is not C++ and I
>>> fear it's unlikely that libc++ will play nicely as a dynamic add-on.
>
>> It would be great if some C++/C guru could make a thorough analysis of
>> C++ integration issues.
>
> The main thing that scares me is lack of integration of C error handling
> (setjmp/longjmp) with C++ error handling (catch/throw). In a
> self-contained module you could perhaps avoid this by not using
> catch/throw, but I don't think you get to ignore the issue if you are
> using STL modules.
>
> It *might* work to put a generic "catch/report via elog" handler around
> each one of your entry-point functions. Haven't tried it.

Hmm, this setup worked quite stable here for some smaller educational
projects. The snippet I used to wrap the C++ was:

--8<---------------cut here---------------start------------->8---
#include <stdexcept>

extern "C" {
#include <postgres.h>
#include <fmgr.h>
}

extern "C" {
PG_FUNCTION_INFO_V1(cpptest);
Datum cpptest(PG_FUNCTION_ARGS);
}

Datum
cpptest(PG_FUNCTION_ARGS)
{
try {

throw std::runtime_error("Your code here.");

}
catch (std::exception &e) {
elog(ERROR, "caught C++ exception: %s", e.what());
}
catch (...) {
elog(ERROR, "caught non-standard C++ exception.");
}
}
--8<---------------cut here---------------end--------------->8---

I was even able to spare the exception code and the extern "C"s in
each and every function by hacking together a language handler loading
functors (classes overloading the operator "()") using the class
loader documented in this paper:
<http://www.s11n.net/papers/classloading_cpp.html> (this one is
exceptional in that it contains no demangling code for symbol lookup
and doesn't need any non-portable libraries).

> This all assumes that libc++ can cooperate with, rather than try to
> replace, libc in the first place. That would depend a lot on what
> platform you are on --- I think it might work OK with modern glibc,
> but my last experience with C++ was back when it wasn't so.

The most annoying non-cooperation for me was the STL's ignorance of
Pg's MemoryContexts. I managed to tame it with a wrapper around palloc
with an STL allocator interface:

--8<---------------cut here---------------start------------->8---
// $Id: pg_alloc.hh,v 1.2 2005/10/19 09:59:13 andreas Exp $
//
// This implements an allocator template with STL interface as a
// wrapper around PostgreSQL's MemoryContexts.
//
// It enables STL containers to use appropriate MemoryContexts instead
// of C++'s "free store".

#include <stdexcept>

extern "C" {
#include <postgres.h>
}

template <class T, MemoryContext *M = &CurrentMemoryContext>
class pg_allocator
{
public:
typedef T value_type;
typedef size_t size_type;
typedef ptrdiff_t difference_type;

typedef T* pointer;
typedef const T* const_pointer;

typedef T& reference;
typedef const T& const_reference;

pointer address(reference r) { return &r; }
const_pointer address(const_reference r) { return &r; }

pg_allocator() throw() { };
template <class U> pg_allocator(const pg_allocator<U, M>&) throw() { };
~pg_allocator() throw() { };

pointer allocate(size_type n, const T *hint = 0) throw(std::bad_alloc);
void deallocate(pointer p, size_type n);

void construct(pointer p, const T& val) { new(p) T(val); }
void destroy(pointer p) { p->~T(); }

size_type max_size() const throw();

template <class U>
struct rebind {typedef pg_allocator<U, M> other; };
};

template <class T, MemoryContext *M>
T* pg_allocator<T, M>::allocate(size_type n, const T * hint) throw(std::bad_alloc)
{
void* ptr = 0;

ptr = MemoryContextAlloc(*M, (n * sizeof(T)));

if (ptr)
return static_cast<T*>(ptr);

throw std::bad_alloc();
}

template <class T, MemoryContext *M>
void pg_allocator<T, M>::deallocate(T* ptr, size_t n)
{
pfree(static_cast<void *>(ptr));
}

template <class T, MemoryContext *M>
bool operator==(const pg_allocator<T, M>& a, const pg_allocator<T, M>& b) {
return true;
}

template <class T, MemoryContext *M>
bool operator!=(const pg_allocator<T, M>& a, const pg_allocator<T, M>& b) {
return false;
}
--8<---------------cut here---------------end--------------->8---

E.g., to allocate a list of strings on the CurrentMemoryContext:
list<string, pg_allocator<string>> foo;
You can supply a custom context with the second template argument if
you need, e.g., a session-wide cache of something.

At this level, I find C++ quite usable as a compromise between C and
the comparatively slow procedural languages. Of course, the interface
to PostgreSQL is still all PODs (plain old datatypes), but C++'s C
compatibility should be seen as a strength here.

regards,
Andreas
--

Andreas Seltenreich <andreas+pg(at)gate450(dot)dyndns(dot)org> writes:
> Tom Lane schrob:
>> It *might* work to put a generic "catch/report via elog" handler around
>> each one of your entry-point functions. Haven't tried it.

> Hmm, this setup worked quite stable here for some smaller educational
> projects. The snippet I used to wrap the C++ was:
> [ snip ]
> [ some later mention of calling back into the backend for, eg, palloc ]

I was with you until that last bit. Have you covered the case where
palloc or another called-back backend routine throws an elog? AFAICS
the only clean way to do that is to PG_TRY around every such call,
throw the error as a C++ throw, catch it again at the function exit
level, re-throw as a PG longjmp :-(. Aside from the sheer tedium and
error-proneness, there's the certainty of losing quite a lot of semantic
detail in the error reports. And on top of that, what if the error was
one that the backend can't recover from except by a transaction abort?
If some level of the C++ code thinks it can catch and recover from the
error, you've left things in a pretty bad state. (palloc failure
doesn't have this risk, but a lot of other error conditions do.)

regards, tom lane

Tom Lane schrob:

> Andreas Seltenreich <andreas+pg(at)gate450(dot)dyndns(dot)org> writes:
>> Tom Lane schrob:
>>> It *might* work to put a generic "catch/report via elog" handler around
>>> each one of your entry-point functions. Haven't tried it.
>
>> Hmm, this setup worked quite stable here for some smaller educational
>> projects. The snippet I used to wrap the C++ was:
>> [ snip ]
>> [ some later mention of calling back into the backend for, eg, palloc ]
>
> I was with you until that last bit. Have you covered the case where
> palloc or another called-back backend routine throws an elog? AFAICS

Not at all. It wasn't necessary in my case, since my classes were
mostly mathematical abstractions, and the only resource held was
memory via the allocator template, which got garbage collected anyway
on a "c-exception". But I see that supporting the other way round is a
must-have as soon as you hog other kinds of resources with C++.

> the only clean way to do that is to PG_TRY around every such call,
> throw the error as a C++ throw, catch it again at the function exit
> level, re-throw as a PG longjmp :-(. Aside from the sheer tedium and

How about letting preprocessor magic wrap calls to the backend with a
PG_TRY and an appropriate throw() if __cplusplus is defined for a
start?

> error-proneness, there's the certainty of losing quite a lot of semantic
> detail in the error reports.

Durr, as long as one inherits the exception class from std::exception
for a proper e.what() interface I don't see a way how semantics could
be lost (using automated throwing with the preprocessor magic
mentioned above that is).

> And on top of that, what if the error was one that the backend can't
> recover from except by a transaction abort? If some level of the C++
> code thinks it can catch and recover from the error, you've left
> things in a pretty bad state.

Hmm, but if you replace "C++" with "C" and "catch" with "PG_TRY" in
that sentence, wouldn't that be business as usual?

Thanks,
Andreas

Andreas Seltenreich <seltenreich(at)gmx(dot)de> writes:
> Tom Lane schrob:
>> And on top of that, what if the error was one that the backend can't
>> recover from except by a transaction abort? If some level of the C++
>> code thinks it can catch and recover from the error, you've left
>> things in a pretty bad state.

> Hmm, but if you replace "C++" with "C" and "catch" with "PG_TRY" in
> that sentence, wouldn't that be business as usual?

Sure, but one hopes that someone writing PG_TRY is clued-in enough to
consider this issue. The thing that is worrying me here is that generic
STL code written by someone who never heard of Postgres is going to try
to make decisions about whether it can recover from an error thrown by
some part of the backend.

regards, tom lane