Re: building and linking C user defined functions

Hi all,

As I learned that a native win32 version of PG would be available soon, I began
to look seriously at it (win32 support is necessary in our business). We began
with a 7.xx standard version under Linux. After a few days of learing and
documenting, we were able to create our own types, add user defined functions
written in C and returning sets etc. Worked very well and nicely. Fine (and
thanks to the Postgresql team!).

Now I tried to port this stuff to the win32 native version, building a dll that
can be dynamically loaded by PG at run-time.

Our steps:

1. Download the self extracting binaries and install them. No major problems
encountered.
2. Download the cvs 8.0beta3 snapshoot and build the native libpq.lib, libpq.dll
with nmake etc. Seemed to work well.
3. Try to link our code for building a dll. Here the problem begins:

-----------------------------------------------------------
link /nologo /dll /libpath:"c:\Program Files\mlfi\lib" \
/libpath:"C:\postgresql\postgresql-snapshot\src\interfaces\libpq\Relea
se" \
/libpath:"c:\program files\Microsoft Visual Studio\VC98\Lib" \
/out:libudfs_wrapper.dll \
libpq.lib udfs.obj udfs_wrapper.obj postgresql/pg_wrapper.obj \
libcamlrun.lib
pg_wrapper.obj : error LNK2001: unresolved external symbol _MemoryContextAlloc
pg_wrapper.obj : error LNK2001: unresolved external symbol __imp__CurrentMemoryC
ontext
pg_wrapper.obj : error LNK2001: unresolved external symbol _Float8GetDatum
pg_wrapper.obj : error LNK2001: unresolved external symbol _pg_detoast_datum
pg_wrapper.obj : error LNK2001: unresolved external symbol _end_MultiFuncCall
pg_wrapper.obj : error LNK2001: unresolved external symbol _per_MultiFuncCall
pg_wrapper.obj : error LNK2001: unresolved external symbol _MemoryContextSwitchT
o
pg_wrapper.obj : error LNK2001: unresolved external symbol _init_MultiFuncCall
libudfs_wrapper.dll : fatal error LNK1120: 8 unresolved externals
make: *** [libudfs_wrapper.dll] Error 96

-----------------------------------------------------------

If I understand well, libpq is a library for building PG clients, but doesn’t
contain "enough" functions to build "udf dlls".

So my question:

What is (or will be) the "official" way to achieve this goal for the win32 version ?

Did anybody achieve to build such a dll with native VC++ only (probably by
achieving to compile the transitive closure of what is needed in backend/utils) ?

Or is one supposed to build such a dll through the "cygwin way" ? This would
mean that one is taking a mingw or cygwin compiled PG (that should contain all
needed object files and library files as .a and .o files), compile and link the
own C udfs against these files for producing a dll ? Would such a dll be
compatible with the native win32 binary version ?

Jean-Marc Eber

I think I can answer this. The symbols it is complaining about are
backend symbols. They would never be in libpq.

As far as backend functions, because Visual C doesn't compile the
backend code, I don't think it can be used for backend functions. My
guess is that you are going to have to use the MinGW build environment
for such cases.

Now, on your specific case below, I _think_ the problem is that a DLL
requires some resolution of symbols that would normally appear in the
backend, while a Unix shared library can have those symbols totally
unresolved until dynamic link time. Again, that is just a guess.
Perhaps there is some link switch which will give you that behavior on
Visual C.

On your last point, when creating a shared library on Unix, you don't
normally use any of the backend object files. What happens is that the
backend dynamically links in the object file and resolves all symbols at
runtime. Since you have done this on Unix already you should already
know that though.

One trick would be to look at the shared objects created by MinGW like
libplpgsql.so and see how those are created and try to duplicate the
process in Visual C. MinGW uses gcc and I wonder if it is creating some
special file format that it can dynamically link but Visual C can't.

The short answer is that I think you are going to need to use MinGW to
do what you want at this point. You will find MinGW much closer to
Win32 native that Cygwin because it doesn't have any compatibility layer
in the runtime files.

---------------------------------------------------------------------------

Jean-Marc EBER wrote:
> Hi all,
>
> As I learned that a native win32 version of PG would be available soon, I began
> to look seriously at it (win32 support is necessary in our business). We began
> with a 7.xx standard version under Linux. After a few days of learing and
> documenting, we were able to create our own types, add user defined functions
> written in C and returning sets etc. Worked very well and nicely. Fine (and
> thanks to the Postgresql team!).
>
> Now I tried to port this stuff to the win32 native version, building a dll that
> can be dynamically loaded by PG at run-time.
>
> Our steps:
>
> 1. Download the self extracting binaries and install them. No major problems
> encountered.
> 2. Download the cvs 8.0beta3 snapshoot and build the native libpq.lib, libpq.dll
> with nmake etc. Seemed to work well.
> 3. Try to link our code for building a dll. Here the problem begins:
>
> -----------------------------------------------------------
> link /nologo /dll /libpath:"c:\Program Files\mlfi\lib" \
> /libpath:"C:\postgresql\postgresql-snapshot\src\interfaces\libpq\Relea
> se" \
> /libpath:"c:\program files\Microsoft Visual Studio\VC98\Lib" \
> /out:libudfs_wrapper.dll \
> libpq.lib udfs.obj udfs_wrapper.obj postgresql/pg_wrapper.obj \
> libcamlrun.lib
> pg_wrapper.obj : error LNK2001: unresolved external symbol _MemoryContextAlloc
> pg_wrapper.obj : error LNK2001: unresolved external symbol __imp__CurrentMemoryC
> ontext
> pg_wrapper.obj : error LNK2001: unresolved external symbol _Float8GetDatum
> pg_wrapper.obj : error LNK2001: unresolved external symbol _pg_detoast_datum
> pg_wrapper.obj : error LNK2001: unresolved external symbol _end_MultiFuncCall
> pg_wrapper.obj : error LNK2001: unresolved external symbol _per_MultiFuncCall
> pg_wrapper.obj : error LNK2001: unresolved external symbol _MemoryContextSwitchT
> o
> pg_wrapper.obj : error LNK2001: unresolved external symbol _init_MultiFuncCall
> libudfs_wrapper.dll : fatal error LNK1120: 8 unresolved externals
> make: *** [libudfs_wrapper.dll] Error 96
>
> -----------------------------------------------------------
>
> If I understand well, libpq is a library for building PG clients, but doesn?t
> contain "enough" functions to build "udf dlls".
>
> So my question:
>
> What is (or will be) the "official" way to achieve this goal for the win32 version ?
>
> Did anybody achieve to build such a dll with native VC++ only (probably by
> achieving to compile the transitive closure of what is needed in backend/utils) ?
>
> Or is one supposed to build such a dll through the "cygwin way" ? This would
> mean that one is taking a mingw or cygwin compiled PG (that should contain all
> needed object files and library files as .a and .o files), compile and link the
> own C udfs against these files for producing a dll ? Would such a dll be
> compatible with the native win32 binary version ?
>
> Jean-Marc Eber
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce,

Thanks for this quick answer.
I will have to look more precisely at the minGW build env/stuff.

jm

Bruce Momjian wrote:
> I think I can answer this. The symbols it is complaining about are
> backend symbols. They would never be in libpq.
>
> As far as backend functions, because Visual C doesn't compile the
> backend code, I don't think it can be used for backend functions. My
> guess is that you are going to have to use the MinGW build environment
> for such cases.
>
> Now, on your specific case below, I _think_ the problem is that a DLL
> requires some resolution of symbols that would normally appear in the
> backend, while a Unix shared library can have those symbols totally
> unresolved until dynamic link time. Again, that is just a guess.
> Perhaps there is some link switch which will give you that behavior on
> Visual C.
>
> On your last point, when creating a shared library on Unix, you don't
> normally use any of the backend object files. What happens is that the
> backend dynamically links in the object file and resolves all symbols at
> runtime. Since you have done this on Unix already you should already
> know that though.
>
> One trick would be to look at the shared objects created by MinGW like
> libplpgsql.so and see how those are created and try to duplicate the
> process in Visual C. MinGW uses gcc and I wonder if it is creating some
> special file format that it can dynamically link but Visual C can't.
>
> The short answer is that I think you are going to need to use MinGW to
> do what you want at this point. You will find MinGW much closer to
> Win32 native that Cygwin because it doesn't have any compatibility layer
> in the runtime files.
>
> ---------------------------------------------------------------------------
>
> Jean-Marc EBER wrote:
>
>>Hi all,
>>
>>As I learned that a native win32 version of PG would be available soon, I began
>>to look seriously at it (win32 support is necessary in our business). We began
>>with a 7.xx standard version under Linux. After a few days of learing and
>>documenting, we were able to create our own types, add user defined functions
>>written in C and returning sets etc. Worked very well and nicely. Fine (and
>>thanks to the Postgresql team!).
>>
>>Now I tried to port this stuff to the win32 native version, building a dll that
>>can be dynamically loaded by PG at run-time.
>>
>>Our steps:
>>
>>1. Download the self extracting binaries and install them. No major problems
>>encountered.
>>2. Download the cvs 8.0beta3 snapshoot and build the native libpq.lib, libpq.dll
>>with nmake etc. Seemed to work well.
>>3. Try to link our code for building a dll. Here the problem begins:
>>
>>-----------------------------------------------------------
>>link /nologo /dll /libpath:"c:\Program Files\mlfi\lib" \
>> /libpath:"C:\postgresql\postgresql-snapshot\src\interfaces\libpq\Relea
>>se" \
>> /libpath:"c:\program files\Microsoft Visual Studio\VC98\Lib" \
>> /out:libudfs_wrapper.dll \
>> libpq.lib udfs.obj udfs_wrapper.obj postgresql/pg_wrapper.obj \
>> libcamlrun.lib
>>pg_wrapper.obj : error LNK2001: unresolved external symbol _MemoryContextAlloc
>>pg_wrapper.obj : error LNK2001: unresolved external symbol __imp__CurrentMemoryC
>>ontext
>>pg_wrapper.obj : error LNK2001: unresolved external symbol _Float8GetDatum
>>pg_wrapper.obj : error LNK2001: unresolved external symbol _pg_detoast_datum
>>pg_wrapper.obj : error LNK2001: unresolved external symbol _end_MultiFuncCall
>>pg_wrapper.obj : error LNK2001: unresolved external symbol _per_MultiFuncCall
>>pg_wrapper.obj : error LNK2001: unresolved external symbol _MemoryContextSwitchT
>>o
>>pg_wrapper.obj : error LNK2001: unresolved external symbol _init_MultiFuncCall
>>libudfs_wrapper.dll : fatal error LNK1120: 8 unresolved externals
>>make: *** [libudfs_wrapper.dll] Error 96
>>
>>-----------------------------------------------------------
>>
>>If I understand well, libpq is a library for building PG clients, but doesn?t
>>contain "enough" functions to build "udf dlls".
>>
>>So my question:
>>
>>What is (or will be) the "official" way to achieve this goal for the win32 version ?
>>
>>Did anybody achieve to build such a dll with native VC++ only (probably by
>>achieving to compile the transitive closure of what is needed in backend/utils) ?
>>
>>Or is one supposed to build such a dll through the "cygwin way" ? This would
>>mean that one is taking a mingw or cygwin compiled PG (that should contain all
>>needed object files and library files as .a and .o files), compile and link the
>>own C udfs against these files for producing a dll ? Would such a dll be
>>compatible with the native win32 binary version ?
>>
>>Jean-Marc Eber
>>
>>
>>
>>---------------------------(end of broadcast)---------------------------
>>TIP 4: Don't 'kill -9' the postmaster
>>
>
>