Visual Studio 2005, C-language function - avoiding hacks?

I have PostgreSQL 8.4 installed on Windows XP, and am using Visual Studio
2005 to write a C-Language function. I have the most basic hello-world type
example (just the 'add_one' function from
http://www.postgresql.org/docs/8.4/interactive/xfunc-c.html) in a DLL, set
to compile to C code rather than C++, and I can do CREATE FUNCTION on it
then use it successfully. However, I had to do some questionable hacks to
get it to build, so I'm guessing I've done something the wrong way. I've not
found anything categorical in the archives here, so would very much
appreciate anyone who can tell me if there's a key step I'm missing that
would make the hacks unnecessary. (For instance, I've come across a
requirement in previous versions to replace pg_config.h with
pg_config.h.win32, but that doesn't seem to apply to current versions -
maybe something else does.) I'm concerned the hacks might turn round and
bite me otherwise once I'm implementing something non-trivial.

Having added C:\Program Files\PostgreSQL\8.4\include\server to the include
directories to pick up postgres.h and fmgr.h, the compiler complained about
various missing include files, starting with 'libintl.h'. Having read the
post at http://archives.postgresql.org/pgsql-general/2009-03/msg00332.php I
created an empty libint.h in an include dir, along with a bunch of other
empty dummy files that were expected: netdb.h, pwd.h, netinet/in.h and
arpa/inet.h.

The code then compiles ok, but gives 'inconsistent dll linkage' on the line
with PG_FUNCTION_INFO_V1 and the one with PG_MODULE_MAGIC.

So I'd like to ask:

* Is there some symbol I should be defining, or something, to avoid all
those dummy header files and make the corresponding Visual Studio headers
get picked up?

* Is there something I should be doing to avoid the 'inconsistent dll
linkage' warnings?

(The C-Language functions are going to end up using some in-proc
Windows-only components, so I'm hoping that building them using Visual
Studio will be the least painful way to get at those.)

Thanks in advance for any help

Kevin.

Kevin Flanagan wrote:

> the compiler
> complained about various missing include files, starting with
> ‘libintl.h’. Having read the post at
> http://archives.postgresql.org/pgsql-general/2009-03/msg00332.php I
> created an empty libint.h in an include dir

NFI why Pg for win32 doesn't bundle a copy of the libintl it was built
against. I should poke the EDB guys about it, actually.

> along with a bunch of other
> empty dummy files that were expected: netdb.h, pwd.h, netinet/in.h and
> arpa/inet.h.

Those I wouldn't expect to be included if you're building for win32.

Are you sure you're building with the win32 configuration?

> The code then compiles ok, but gives ‘inconsistent dll linkage’ on the
> line with PG_FUNCTION_INFO_V1 and the one with PG_MODULE_MAGIC.

This would suggest that the macros that insert appropriate
__declspec(dllimport) and __declspec(dllexport) attributes aren't being
triggered - so again, it makes me wonder if Pg knows it's building for
win32.

--
Craig Ringer

On Fri, Mar 5, 2010 at 4:02 AM, Craig Ringer
<craig(at)postnewspapers(dot)com(dot)au> wrote:
> Kevin Flanagan wrote:
>
>> the compiler
>> complained about various missing include files, starting with
>> ‘libintl.h’. Having read the post at
>> http://archives.postgresql.org/pgsql-general/2009-03/msg00332.php I
>> created an empty libint.h in an include dir
>
> NFI why Pg for win32 doesn't bundle a copy of the libintl it was built
> against. I should poke the EDB guys about it, actually.

We do include the library. We don't include the headers or source for
third party code though - that would be considered part of the build
environment, just the same as the Windows SDK.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
PG East Conference: http://www.enterprisedb.com/community/nav-pg-east-2010.do

Dave Page wrote:
> On Fri, Mar 5, 2010 at 4:02 AM, Craig Ringer
> <craig(at)postnewspapers(dot)com(dot)au> wrote:
>> Kevin Flanagan wrote:
>>
>>> the compiler
>>> complained about various missing include files, starting with
>>> ‘libintl.h’. Having read the post at
>>> http://archives.postgresql.org/pgsql-general/2009-03/msg00332.php I
>>> created an empty libint.h in an include dir
>> NFI why Pg for win32 doesn't bundle a copy of the libintl it was built
>> against. I should poke the EDB guys about it, actually.
>
> We do include the library. We don't include the headers or source for
> third party code though - that would be considered part of the build
> environment, just the same as the Windows SDK.

The installer includes the headers for PostgreSQL its self (ie a
"postgresql sdk"). For them to be useful for much you need libintl too.
As on windows there's no "system" libintl, the user has to try to figure
out what libintl Pg was built against and somehow obtain appropriate
headers, including any config headers. There are many different win32
builds of libintl out there and many (most) of them won't match what Pg
was built against.

That seems unnecessarily painful. It seems pretty harmless to ship the
gettext headers, as a separate download if not in the main installer
package.

How do _you_ go about building server extensions for Pg? Where do you
get the headers for gettext etc?

IMO if the gettext headers aren't in the main installer pkg then the Pg
headers shouldn't be either, and a separate "sdk" installer should be
provided with them both. Ditto any other headers required.

I'm increasingly thinking the win32 package _should_ be split into
server binary and separate headers+pdb+sources packages, with the sdk
package including gettext headers and sources too. It'd be a LOT easier
to develop with Pg on win32 this way.

--
Craig Ringer

On Fri, Mar 5, 2010 at 9:05 AM, Craig Ringer
<craig(at)postnewspapers(dot)com(dot)au> wrote:

> How do _you_ go about building server extensions for Pg? Where do you
> get the headers for gettext etc?

Same place I get the binaries - gnuwin32 mostly.

> I'm increasingly thinking the win32 package _should_ be split into
> server binary and separate headers+pdb+sources packages, with the sdk
> package including gettext headers and sources too. It'd be a LOT easier
> to develop with Pg on win32 this way.

How does breaking it up into multiple packages make it easier?

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
PG East Conference: http://www.enterprisedb.com/community/nav-pg-east-2010.do

Dave Page wrote:
> On Fri, Mar 5, 2010 at 9:05 AM, Craig Ringer
> <craig(at)postnewspapers(dot)com(dot)au> wrote:
>
>> How do _you_ go about building server extensions for Pg? Where do you
>> get the headers for gettext etc?
>
> Same place I get the binaries - gnuwin32 mostly.
>
>> I'm increasingly thinking the win32 package _should_ be split into
>> server binary and separate headers+pdb+sources packages, with the sdk
>> package including gettext headers and sources too. It'd be a LOT easier
>> to develop with Pg on win32 this way.
>
> How does breaking it up into multiple packages make it easier?

What I was trying to say was "if you don't want to include gettext in
the main download, perhaps splitting all the dev files into a separate
package would permit you to add gettext and the rest".

I don't much like the fact that presently users have to go hunting for
the libraries, with not even a pointer included in the sources about
where they should look to find headers matching the shipped libraries,
and what version they need.

Why _not_ distribute gettext headers, though? Sources I can understand
for size reasons, but the headers are small and fuss free, and you need
the _right_ _versions_ to build against the Pg backend.

--
Craig Ringer

On Fri, Mar 5, 2010 at 9:31 AM, Craig Ringer
<craig(at)postnewspapers(dot)com(dot)au> wrote:

> Why _not_ distribute gettext headers, though? Sources I can understand
> for size reasons, but the headers are small and fuss free, and you need
> the _right_ _versions_ to build against the Pg backend.

No reason, other than I didn't realise they were needed to build extension.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
PG East Conference: http://www.enterprisedb.com/community/nav-pg-east-2010.do

Dave Page wrote:
> On Fri, Mar 5, 2010 at 9:31 AM, Craig Ringer
> <craig(at)postnewspapers(dot)com(dot)au> wrote:
>
>> Why _not_ distribute gettext headers, though? Sources I can understand
>> for size reasons, but the headers are small and fuss free, and you need
>> the _right_ _versions_ to build against the Pg backend.
>
> No reason, other than I didn't realise they were needed to build extension.
>

Ah, fair enough. I read:

> We do include the library. We don't include the headers or source for
> third party code though - that would be considered part of the build
> environment, just the same as the Windows SDK.

as "we don't want to distribute third-party headers even if required by
Pg's own headers" and thus thought you *did* know but by policy didn't
want to distribute them.

--
Craig Ringer

Ok, re "building with the win32 configuration" ... that sounds like just the thing I should know about. All I've done is downloaded and installed the 1-click installer for Windows from http://www.enterprisedb.com/products/pgdownload.do#windows ... so while I'm sure it knows it's running on Win32, is there some other configuration change I should make for dev purposes to indicate that it's "the win32 configuration"? Or does "building with the win32 configuration" refer to those who are building the server from source, or something?

Thanks

Kevin

-----Original Message-----
From: Craig Ringer [mailto:craig(at)postnewspapers(dot)com(dot)au]
Sent: 05 March 2010 04:02
To: Kevin Flanagan
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [HACKERS] Visual Studio 2005, C-language function - avoiding hacks?

Kevin Flanagan wrote:

> the compiler
> complained about various missing include files, starting with
> ‘libintl.h’. Having read the post at
> http://archives.postgresql.org/pgsql-general/2009-03/msg00332.php I
> created an empty libint.h in an include dir

NFI why Pg for win32 doesn't bundle a copy of the libintl it was built
against. I should poke the EDB guys about it, actually.

> along with a bunch of other
> empty dummy files that were expected: netdb.h, pwd.h, netinet/in.h and
> arpa/inet.h.

Those I wouldn't expect to be included if you're building for win32.

Are you sure you're building with the win32 configuration?

> The code then compiles ok, but gives ‘inconsistent dll linkage’ on the
> line with PG_FUNCTION_INFO_V1 and the one with PG_MODULE_MAGIC.

This would suggest that the macros that insert appropriate
__declspec(dllimport) and __declspec(dllexport) attributes aren't being
triggered - so again, it makes me wonder if Pg knows it's building for
win32.

--
Craig Ringer

Kevin Flanagan wrote:
> Ok, re "building with the win32 configuration" ... that sounds like just the thing I should know about. All I've done is downloaded and installed the 1-click installer for Windows from http://www.enterprisedb.com/products/pgdownload.do#windows ... so while I'm sure it knows it's running on Win32, is there some other configuration change I should make for dev purposes to indicate that it's "the win32 configuration"? Or does "building with the win32 configuration" refer to those who are building the server from source, or something?

I wasn't too specific because it's been a while since I did any coding
against Pg on win32, and I couldn't remember exactly how it selected the
right code to use for a given platform - whether it was a macro that
must be defined, or what.

Having had a look at the sources: It's done by header search path. You
need to make sure that include/port/win32_msvc is on the header search
path as well as the main include/ directory.

I *think* port/win32 is for the MinGW win32 port and thus shouldn't be
included in the search path for msvc builds, but I'm not 100% sure of
that and a quick look doesn't reveal any documentation on the matter.

--
Craig Ringer

On Fri, Mar 5, 2010 at 9:50 AM, Craig Ringer
<craig(at)postnewspapers(dot)com(dot)au> wrote:
> Dave Page wrote:
>> On Fri, Mar 5, 2010 at 9:31 AM, Craig Ringer
>> <craig(at)postnewspapers(dot)com(dot)au> wrote:
>>
>>> Why _not_ distribute gettext headers, though? Sources I can understand
>>> for size reasons, but the headers are small and fuss free, and you need
>>> the _right_ _versions_ to build against the Pg backend.
>>
>> No reason, other than I didn't realise they were needed to build extension.
>>
>
> Ah, fair enough. I read:
>
>> We do include the library. We don't include the headers or source for
>> third party code though - that would be considered part of the build
>> environment, just the same as the Windows SDK.
>
> as "we don't want to distribute third-party headers even if required by
> Pg's own headers" and thus thought you *did* know but by policy didn't
> want to distribute them.

I didn't know in this case, but was making a general statement about
how I felt the policy should be.

Plus I was feeling a little grumpy in my pre-coffee state. Sorry :-p

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
PG East Conference: http://www.enterprisedb.com/community/nav-pg-east-2010.do

Ok, that got me on the right track, thanks. I think the key points for this build scenario are these:

1. you have to define the symbol BUILDING_DLL in your code before including postgres.h (as that then means PGDLLIMPORT gets defined right in pg_config_os.h). That makes the 'inconsistent dll linkage' warnings go away.
2. you have to have include\server\port\win32 in the include dirs list as well as include\server (as that provides a bunch of otherwise-missing headers such as netdb.h)

However, that still leaves one missing include file - libintl.h, which c.h tries to include because ENABLE_NLS is defined (and that seems to get defined as 1 in pg_config.h whether you like it or not). And in fact, it seems Rostic Sheykhet posted at http://www.postgresql.org/docs/8.2/interactive/xfunc-c.html with the same problem (now that I know how to do it, I know what to Google for to, er, find out how to do it :) ). You can get round it by commenting out the include or creating a dummy libintl.h.

Just posting the results here in case they're relevant for anything.

Kevin.

-----Original Message-----
From: Craig Ringer [mailto:craig(at)postnewspapers(dot)com(dot)au]
Sent: 05 March 2010 10:05
To: Kevin Flanagan
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [HACKERS] Visual Studio 2005, C-language function - avoiding hacks?

Kevin Flanagan wrote:
> Ok, re "building with the win32 configuration" ... that sounds like just the thing I should know about. All I've done is downloaded and installed the 1-click installer for Windows from http://www.enterprisedb.com/products/pgdownload.do#windows ... so while I'm sure it knows it's running on Win32, is there some other configuration change I should make for dev purposes to indicate that it's "the win32 configuration"? Or does "building with the win32 configuration" refer to those who are building the server from source, or something?

I wasn't too specific because it's been a while since I did any coding
against Pg on win32, and I couldn't remember exactly how it selected the
right code to use for a given platform - whether it was a macro that
must be defined, or what.

Having had a look at the sources: It's done by header search path. You
need to make sure that include/port/win32_msvc is on the header search
path as well as the main include/ directory.

I *think* port/win32 is for the MinGW win32 port and thus shouldn't be
included in the search path for msvc builds, but I'm not 100% sure of
that and a quick look doesn't reveal any documentation on the matter.

--
Craig Ringer

"Kevin Flanagan" <kevin-f(at)linkprior(dot)com> wrote:

> 1. you have to define the symbol BUILDING_DLL in your code before
> including postgres.h

No, BUILDING_DLL does not work. We use PGDLLIMPORT both exported global
variables and PG_MODULE_MAGIC/PG_FUNCTION_INFO_V1 for now, but actually
we should always use __declspec (dllexport) for the latter cases.
They are exported from *user dlls*, not the postgres' one.

I'd like to propose to define PGALWAYSEXPORT macro:
#ifdef WIN32
#define PGALWAYSEXPORT __declspec (dllexport)
#endif
and modify PG_MODULE_MAGIC and PG_FUNCTION_INFO_V1 to use it
instead of PGDLLEXPORT.

Regards,
---
Takahiro Itagaki
NTT Open Source Software Center

Takahiro Itagaki <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp> writes:
> I'd like to propose to define PGALWAYSEXPORT macro:
> #ifdef WIN32
> #define PGALWAYSEXPORT __declspec (dllexport)
> #endif
> and modify PG_MODULE_MAGIC and PG_FUNCTION_INFO_V1 to use it
> instead of PGDLLEXPORT.

This seems like change for the sake of change. The existing mechanism
works (as demonstrated by the fact that the contrib modules work on
Windows). I don't think we should invent a new mechanism that won't be
backwards-compatible.

regards, tom lane

Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Takahiro Itagaki <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp> writes:
> > I'd like to propose to define PGALWAYSEXPORT macro:
> > #ifdef WIN32
> > #define PGALWAYSEXPORT __declspec (dllexport)
> > #endif
> > and modify PG_MODULE_MAGIC and PG_FUNCTION_INFO_V1 to use it
> > instead of PGDLLEXPORT.
>
> This seems like change for the sake of change. The existing mechanism
> works (as demonstrated by the fact that the contrib modules work on
> Windows).

I wonder why the contrib modules can be compiled correctly because:
1. PG_MODULE_MAGIC requires dllexport.
2. Other exported variables from postgres requires dllimport.
3. Exported functions from the contrib DLLs require dllexport,
but they don't have any PGDLLEXPORT tags in their functions.

Did we use non-standard tools except msvc in the build frameword
for core code? And what should I do for an external project?

Regards,
---
Takahiro Itagaki
NTT Open Source Software Center

2010/3/8 Takahiro Itagaki <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>:
>
> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
>> Takahiro Itagaki <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp> writes:
>> > I'd like to propose to define PGALWAYSEXPORT macro:
>> >     #ifdef WIN32
>> >     #define PGALWAYSEXPORT  __declspec (dllexport)
>> >     #endif
>> > and modify PG_MODULE_MAGIC and PG_FUNCTION_INFO_V1 to use it
>> > instead of PGDLLEXPORT.
>>
>> This seems like change for the sake of change.  The existing mechanism
>> works (as demonstrated by the fact that the contrib modules work on
>> Windows).
>
> I wonder why the contrib modules can be compiled correctly because:
>    1. PG_MODULE_MAGIC requires dllexport.
>    2. Other exported variables from postgres requires dllimport.
>    3. Exported functions from the contrib DLLs require dllexport,
>       but they don't have any PGDLLEXPORT tags in their functions.
>
> Did we use non-standard tools except msvc in the build frameword
> for core code? And what should I do for an external project?

Yes, we use mingw.

In this particular case, it may be the non-standard behavior that
mingw exports *all* symbols in a DLL. We have some scripts in the MSVC
build system that does this - it auto-generates a .DEF file that lists
all symbols inthe file, and makes sure those are all exported.

In fact, this even requires us to remove warnings created by modern
versions of Visual Studio, since you're not supposed to use both
dllexport and DEF files for the same symbol, but we do.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus(at)hagander(dot)net> wrote:

> >> The existing mechanism
> >> works (as demonstrated by the fact that the contrib modules work on
> >> Windows).
> >
> > Did we use non-standard tools except msvc in the build framework
> > for core code? And what should I do for an external project?
>
> Yes, we use mingw.

Hmmm, it means the existing mechanism can only work on limited environments.
Should we make the code to be more portable for standard developers?

Third party developer might not build the postgres server,
but they would want to build their extension modules without mingw.

Regards,
---
Takahiro Itagaki
NTT Open Source Software Center

2010/3/9 Takahiro Itagaki <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>:
>
> Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>
>> >> The existing mechanism
>> >> works (as demonstrated by the fact that the contrib modules work on
>> >> Windows).
>> >
>> > Did we use non-standard tools except msvc in the build framework
>> > for core code? And what should I do for an external project?
>>
>> Yes, we use mingw.
>
> Hmmm, it means the existing mechanism can only work on limited environments.
> Should we make the code to be more portable for standard developers?

That would definitely be good.

> Third party developer might not build the postgres server,
> but they would want to build their extension modules without mingw.

Well, the tool for generating the full export of symbols is in the
build tree, so perhaps we jus tneed to ship that.

But usually you *don't* want that because, well, it's non-standard.
And exporting all symbols makes the binary *much* larger (iirc,
postgres.exe grows about 40%).

It would perhaps be better to encourage (and document) how people
create their own .DEF files to export the specific symbols they need.
Because AFAIK, that will work if you do that - or do you know of
issues with that method?

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/