Hacking PostgreSQL to work in Mac OS X 10.3 (Panther 7B85)

All,

After toying with a few builds of the up coming OS X 10.3 (Panther)
release, I've written a doc on how to get postgresql to build using
Panther build 7B85 and Xcode build 7B85 (which are rumored to be the
gold master and the builds that will be shipping later this month).

Two things needed to be hacked to get postgresql to build from
darwinports, and I'm assuming the same fixes would apply to a build
from a .tar.gz. Essentially, a header file installed by Apple's Xcode
devel tools needs to be edited, and the template for darwin needs a
line removed. This fix is also known to work with OS X 10.3 build 7B85
and Xcode 7B68. Whilst I'm not in a position to analyse the root cause
of why these hacks are needed, this doc is simply a quick and dirty
guide to hacking it together.

Scenario
========

Fresh install of OS X 10.3 7B85
Fresh install of Xcode 7B85

howrare:~ jwilson$ sudo port install postgresql

btodb Compiler Error
====================

Error: Target com.apple.build returned: shell command "cd
"/Library/DarwinPorts/dports/databases/postgresql/work/postgresql
-7.3.2" && CPPFLAGS=-I/opt/local/include CFLAGS=-I/opt/local/include
LDFLAGS=-L/opt/local/lib gnumake all" returned error 2
Command output: gnumake[2]: Nothing to be done for `all'.
gnumake -C backend all
gnumake -C ../../src/port all
gnumake[3]: Nothing to be done for `all'.
prereqdir=`cd parser/ >/dev/null && pwd` && \ cd
../../src/include/parser/ && rm -f parse.h && \ ln -s
"$prereqdir/parse.h" .
gnumake -C utils fmgroids.h
CPP='gcc -traditional-cpp -E' AWK='awk' /bin/sh Gen_fmgrtab.sh
../../../src/include/catalog/pg_proc.h
cd ../../src/include/utils/ && rm -f fmgroids.h && \
ln -s ../../../src/backend/utils/fmgroids.h .
gnumake -C access all
gnumake -C common SUBSYS.o
gcc -traditional-cpp -I/opt/local/include -Wall -Wmissing-prototypes
-Wmissing-declarations -I../../../../src/include -I/opt/local/include
-c -o heaptuple.o heaptuple.c
gcc -traditional-cpp -I/opt/local/include -Wall -Wmissing-prototypes
-Wmissing-declarations -I../../../../src/include -I/opt/local/include
-c -o indextuple.o indextuple.c
gcc -traditional-cpp -I/opt/local/include -Wall -Wmissing-prototypes
-Wmissing-declarations -I../../../../src/include -I/opt/local/include
-c -o indexvalid.o indexvalid.c
gcc -traditional-cpp -I/opt/local/include -Wall -Wmissing-prototypes
-Wmissing-declarations -I../../../../src/include -I/opt/local/include
-c -o printtup.o printtup.c
In file included from /usr/include/machine/param.h:30,
from /usr/include/sys/socket.h:67,
from ../../../../src/include/libpq/pqcomm.h:28,
from ../../../../src/include/libpq/libpq-be.h:24,
from ../../../../src/include/libpq/libpq.h:21,
from printtup.c:20:
/usr/include/ppc/param.h:98: macro "btodb" requires 2 arguments, but
only 1 given
/usr/include/ppc/param.h:100: macro "dbtob" requires 2 arguments, but
only 1 given
gnumake[4]: *** [printtup.o] Error 1gnumake[3]: *** [common-recursive]
Error 2
gnumake[2]: *** [access-recursive] Error 2
gnumake[1]: *** [all] Error 2
gnumake: *** [all] Error 2

Workaround
----------

howrare:~ jwilson$ sudo cp /usr/include/ppc/param.h
/usr/include/ppc/param.h_pgfix
howrare:~ jwilson$ sudo vim /usr/include/ppc/param.h
* Go to line 90 (Type '90gg')

Note this section:

/* bytes to pages */
#define btoc(x) (((unsigned)(x)+(PGOFSET))>>PGSHIFT)
#ifdef __APPLE__
#define btodb(bytes, devBlockSize) \
((unsigned)(bytes) / devBlockSize)
#define dbtob(db, devBlockSize) \
((unsigned)(db) * devBlockSize)
#else
#define btodb(bytes) /* calculates (bytes /
DEV_BSIZE) */ \
((unsigned)(bytes) >> DEV_BSHIFT)
#define dbtob(db) /* calculates (db * DEV_BSIZE)
*/ \
((unsigned)(db) << DEV_BSHIFT)
#endif

It seems that __APPLE__ is defined somehow (perhaps it should be) when
postgres is compiling. Hence, the btodb macros are expecting two args
and getting only one. The quick and dirty fix is to remove the ifdef
__APPLE__ portion of the code and leave it such that the only macros
defined are the single arg ones.

Hence, delete these lines:

#ifdef __APPLE__
#define btodb(bytes, devBlockSize) \
((unsigned)(bytes) / devBlockSize)
#define dbtob(db, devBlockSize) \
((unsigned)(db) * devBlockSize)
#else

And then delete the "#endif" line at the end of that section of code
(Just above the comment block for the next section which starts with
"Map a ``block device block``"

As such, after deleting the lines the code section at line 90 and
beyond should look like this:

/* bytes to pages */
#define btoc(x) (((unsigned)(x)+(PGOFSET))>>PGSHIFT)
#define btodb(bytes) /* calculates (bytes /
DEV_BSIZE) */ \
((unsigned)(bytes) >> DEV_BSHIFT)
#define dbtob(db) /* calculates (db * DEV_BSIZE)
*/ \
((unsigned)(db) << DEV_BSHIFT)

Now save the file (:w! to override the read-only) and try the build of
postgresql again. This time it will fail with a precompiler error.

Precompiler Errors
==================

Error: Target com.apple.build returned: shell command "cd
"/Library/DarwinPorts/dports/databases/postgresql/work/postgresql
-7.3.2" && CPPFLAGS=-I/opt/local/include CFLAGS=-I/opt/local/include
LDFLAGS=-L/opt/local/lib gnumake all" returned error 2
Command output: gcc -traditional-cpp -I/opt/local/include -Wall
-Wmissing-prototypes -Wmissing-declarations -I. -I../../../src/include
-I/opt/local/include -c -o bootparse.o bootparse.c
bootparse.y: In function `Int_yyparse':
bootparse.y:273: error: syntax error at '##' token
bootparse.y:273: error: `T_' undeclared (first use in this function)
bootparse.y:273: error: (Each undeclared identifier is reported only
once
bootparse.y:273: error: for each function it appears in.)
bootparse.y:273: error: parse error before "IndexElem"
In file included from bootparse.y:339:
lex.Int_yy.c: At top level:
lex.Int_yy.c:1122: warning: `Int_yy_get_next_buffer' was used with no
prototype before its definition
lex.Int_yy.c:1254: warning: `Int_yy_get_previous_state' was used with
no prototype before its definition
lex.Int_yy.c:1292: warning: `Int_yy_try_NUL_trans' was used with no
prototype before its definition
lex.Int_yy.c:1440: warning: no previous prototype for `Int_yyrestart'
lex.Int_yy.c:1455: warning: no previous prototype for
`Int_yy_switch_to_buffer'
lex.Int_yy.c:1486: warning: no previous prototype for
`Int_yy_load_buffer_state'
lex.Int_yy.c:1498: warning: no previous prototype for
`Int_yy_create_buffer'
lex.Int_yy.c:1529: warning: no previous prototype for
`Int_yy_delete_buffer'
lex.Int_yy.c:1555: warning: no previous prototype for
`Int_yy_init_buffer'
lex.Int_yy.c:1582: warning: no previous prototype for
`Int_yy_flush_buffer'
lex.Int_yy.c:1613: warning: no previous prototype for
`Int_yy_scan_buffer'
lex.Int_yy.c:1651: warning: no previous prototype for
`Int_yy_scan_string'
lex.Int_yy.c:1668: warning: no previous prototype for
`Int_yy_scan_bytes'
lex.Int_yy.c:1762: warning: `Int_yy_fatal_error' was used with no
prototype before its definition
lex.Int_yy.c:1826: warning: `Int_yy_flex_alloc' was used with no
prototype before its definition
lex.Int_yy.c:1836: warning: `Int_yy_flex_realloc' was used with no
prototype before its definition
lex.Int_yy.c:1854: warning: `Int_yy_flex_free' was used with no
prototype before its definition
gnumake[3]: *** [bootparse.o] Error 1
gnumake[2]: *** [bootstrap-recursive] Error 2
gnumake[1]: *** [all] Error 2
gnumake: *** [all] Error 2

Solution
--------

The gcc in 10.3's Xcode behaves differently (perhaps properly) with the
-traditional-cpp flag. I.e it doesn't allow use of the '##' token not
available in traditional c/c++. The solution was to edit the template
file for darwin in src/template and just removed the 'CC="$CC
-traditional-cpp"' definition all together.

sudo vim
{DarwinPortsDir}/dports/databases/postgresql/work/postgresql-7.3.2/src/
template/darwin

Remove the following line:

CC="$CC -traditional-cpp"

Of course, the
{DarwinPortsDir}/dports/databases/postgresql/work/
.darwinports.postgresql.state file then needs to be edited to cause a
reconfigure of the source with the new template.

sudo vim
{DarwinPortsDir}/dports/databases/postgresql/work/
.darwinports.postgresql.state

Remove the following lines:

target: com.apple.configure

Try the build of postgresql again, DO NOT clean it, just try the
install again.

-------------

Best of luck (was re: works for me.)

-------------------
James Wilson
CCIE #6662

LithiumCorp Pty Ltd

mobile: 0422 22 3742
email: jwilson(at)lithiumcorp(dot)com

James, we've spent some time lately with Mac OS X related changes to
the code base. Please try this again with a fresh CVS copy of
PostgreSQL and let us know if there are any further changes you think
we'd need to work properly with Panther.

Thanks,
mk

On 11.10.2003, at 04:32, James Wilson wrote:
> After toying with a few builds of the up coming OS X 10.3 (Panther)
> release, I've written a doc on how to get postgresql to build using
> Panther build 7B85 and Xcode build 7B85 (which are rumored to be the
> gold master and the builds that will be shipping later this month).

Hi Marko,

I've done a checkout from CVS and performed a build under OS X Panther
7B85 with the non-hacked header files. The problem with the param.h
header file seems to be fixed, however the build is failing with
undefined symbols in libs:

make -C ecpglib all
gcc -no-cpp-precomp -g -O2 -Wall -Wmissing-prototypes
-Wmissing-declarations -bundle execute.o typename.o descriptor.o
data.o error.o prepare.o memory.o connect.o misc.o
-L../../../../src/port -L/opt/local/lib -L../pgtypeslib -lpgtypes
-L../../../../src/interfaces/libpq -lpq -lm -o libecpg.so.4.0
ld: Undefined symbols:
_PQfreemem
_PQresultErrorField
_PQsetNoticeReceiver
_last_path_separator

I noticed it's not linking with the "-undefined suppress" flag which
I've found it required when building 'bundles' for darwin.

Cheers,

On 12/10/2003, at 3:28 AM, Marko Karppinen wrote:

> James, we've spent some time lately with Mac OS X related changes to
> the code base. Please try this again with a fresh CVS copy of
> PostgreSQL and let us know if there are any further changes you think
> we'd need to work properly with Panther.
>
> Thanks,
> mk
>
> On 11.10.2003, at 04:32, James Wilson wrote:
>> After toying with a few builds of the up coming OS X 10.3 (Panther)
>> release, I've written a doc on how to get postgresql to build using
>> Panther build 7B85 and Xcode build 7B85 (which are rumored to be the
>> gold master and the builds that will be shipping later this month).
>
>
>
-------------------
James Wilson
CCIE #6662

LithiumCorp Pty Ltd

mobile: 0422 22 3742
email: jwilson(at)lithiumcorp(dot)com

On 12.10.2003, at 03:52, James Wilson wrote:
> I've done a checkout from CVS and performed a build under OS X Panther
> 7B85 with the non-hacked header files. The problem with the param.h
> header file seems to be fixed, however the build is failing with
> undefined symbols in libs:

I believe you have some old files lingering on your tree:

[marko:~/src] mka% rm -rf pgsql
[marko:~/src] mka% cvs -z3 -d
:pserver:anoncvs(at)anoncvs(dot)postgresql(dot)org:/projects/cvsroot co -P pgsql
[marko:~/src] mka% cd pgsql
[marko:~/src/pgsql] mka% autoconf
[marko:~/src/pgsql] mka% ./configure
. . .
[marko:~/src/pgsql] mka% make
. . .
All of PostgreSQL successfully made. Ready to install.
[marko:~/src/pgsql] mka%

Either that, or your usage of mismatching OS X and dev tools versions
is causing you problems.

mk

Is there something special about function current_user?
In v.7.3.4 it only works when I do "select current_user;"
and breaks when called with the empty arg. list:

syncdb=# select current_user();
ERROR: parser: parse error at or near "(" at character 20

This behavior is just the opposite to that of some other functions
that take no arguments (e.g. version, timeofday)

Mike.

Michael Brusser writes:

> Is there something special about function current_user?

Yes.

> In v.7.3.4 it only works when I do "select current_user;"
> and breaks when called with the empty arg. list:

That is the special thing. It's bizarre, but that's how SQL wants it.
I've wanted to extend it so that you can call it both ways, but didn't get
enough support for it.

--
Peter Eisentraut peter_e(at)gmx(dot)net

James Wilson <jwilson(at)lithiumcorp(dot)com> writes:
> make -C ecpglib all
> gcc -no-cpp-precomp -g -O2 -Wall -Wmissing-prototypes
> -Wmissing-declarations -bundle execute.o typename.o descriptor.o
> data.o error.o prepare.o memory.o connect.o misc.o
> -L../../../../src/port -L/opt/local/lib -L../pgtypeslib -lpgtypes
> -L../../../../src/interfaces/libpq -lpq -lm -o libecpg.so.4.0
> ld: Undefined symbols:
> _PQfreemem
> _PQresultErrorField
> _PQsetNoticeReceiver
> _last_path_separator

I'll bet you have an old libpq in /opt/local/lib. Why is that getting
included in the link anyway?

regards, tom lane

Nicely spotted, yeah that's what is causing these issue. The reason
-L/opt/local/lib is in there is due to it being ./configured with
--with-libraries=/opt/local/lib and --with-includes=/opt/local/include.
Those configure options are necessary to ensure the configure found the
darwinport-installed readline et.al. which are in /opt/local/lib and
/opt/local/include

Removing the old postgres libs would resolve this build issue, but
makes it a bit painful for users upgrading from an existing
installation.

On 15/10/2003, at 4:03 AM, Tom Lane wrote:

> James Wilson <jwilson(at)lithiumcorp(dot)com> writes:
>> make -C ecpglib all
>> gcc -no-cpp-precomp -g -O2 -Wall -Wmissing-prototypes
>> -Wmissing-declarations -bundle execute.o typename.o descriptor.o
>> data.o error.o prepare.o memory.o connect.o misc.o
>> -L../../../../src/port -L/opt/local/lib -L../pgtypeslib -lpgtypes
>> -L../../../../src/interfaces/libpq -lpq -lm -o libecpg.so.4.0
>> ld: Undefined symbols:
>> _PQfreemem
>> _PQresultErrorField
>> _PQsetNoticeReceiver
>> _last_path_separator
>
> I'll bet you have an old libpq in /opt/local/lib. Why is that getting
> included in the link anyway?
>
> regards, tom lane
>
>
-------------------
James Wilson
CCIE #6662

LithiumCorp Pty Ltd

mobile: 0422 22 3742
email: jwilson(at)lithiumcorp(dot)com

After spending a few hours of trying to get Postgresql7.3.4 to build
from source (tar.gz) on a Panther (release, not beta) system, I find
that I am still having the same errors described in the original post
to this thread. It is a clean install as of yesterday, with readline
installed via fink.

# ./configure --with-includes=/sw/include/ --with-libraries=/sw/lib
--with-perl

then:

#make

yields this error at then end:

In file included from /usr/include/machine/param.h:30,
from /usr/include/sys/socket.h:67,
from ../../../../src/include/libpq/pqcomm.h:28,
from ../../../../src/include/libpq/libpq-be.h:24,
from ../../../../src/include/libpq/libpq.h:21,
from printtup.c:20:
/usr/include/ppc/param.h:98: macro "btodb" requires 2 arguments, but
only 1 given
/usr/include/ppc/param.h:100: macro "dbtob" requires 2 arguments, but
only 1 given
make[4]: *** [printtup.o] Error 1
make[3]: *** [common-recursive] Error 2
make[2]: *** [access-recursive] Error 2
make[1]: *** [all] Error 2
make: *** [all] Error 2

I have installed the XCode tools that came with Panther and ran
Software update to get everything up to date. Is there another
(better?) way to fix this than altering the header files as James
inidcated in his original post? there is no /opt, so I don't think it
is a lingering library there.

Kirk Gleason

jwilson(at)lithiumcorp(dot)com (James Wilson) wrote in message news:<6CF21BDE-FEAA-11D7-A7CA-000A957C9590(at)lithiumcorp(dot)com>...
> Nicely spotted, yeah that's what is causing these issue. The reason
> -L/opt/local/lib is in there is due to it being ./configured with
> --with-libraries=/opt/local/lib and --with-includes=/opt/local/include.
> Those configure options are necessary to ensure the configure found the
> darwinport-installed readline et.al. which are in /opt/local/lib and
> /opt/local/include
>
> Removing the old postgres libs would resolve this build issue, but
> makes it a bit painful for users upgrading from an existing
> installation.
>
> On 15/10/2003, at 4:03 AM, Tom Lane wrote:
>
> > James Wilson <jwilson(at)lithiumcorp(dot)com> writes:
> >> make -C ecpglib all
> >> gcc -no-cpp-precomp -g -O2 -Wall -Wmissing-prototypes
> >> -Wmissing-declarations -bundle execute.o typename.o descriptor.o
> >> data.o error.o prepare.o memory.o connect.o misc.o
> >> -L../../../../src/port -L/opt/local/lib -L../pgtypeslib -lpgtypes
> >> -L../../../../src/interfaces/libpq -lpq -lm -o libecpg.so.4.0
> >> ld: Undefined symbols:
> >> _PQfreemem
> >> _PQresultErrorField
> >> _PQsetNoticeReceiver
> >> _last_path_separator
> >
> > I'll bet you have an old libpq in /opt/local/lib. Why is that getting
> > included in the link anyway?
> >
> > regards, tom lane
> >
> >
> -------------------
> James Wilson
> CCIE #6662
>
> LithiumCorp Pty Ltd
>
> mobile: 0422 22 3742
> email: jwilson(at)lithiumcorp(dot)com
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: don't forget to increase your free space map settings

kgleason(at)finelight(dot)com writes:
> After spending a few hours of trying to get Postgresql7.3.4 to build
> from source (tar.gz) on a Panther (release, not beta) system,

Try 7.4RC1 instead. Apple made some incompatible changes in their
compiler in Panther.

regards, tom lane

Tom Lane wrote:
> kgleason(at)finelight(dot)com writes:
>
>>After spending a few hours of trying to get Postgresql7.3.4 to build
>>from source (tar.gz) on a Panther (release, not beta) system,
>
>
> Try 7.4RC1 instead. Apple made some incompatible changes in their
> compiler in Panther.

I was going to recommend the same thing. I compiled a 7.4 beta out of
the box without a hitch, so I'd assume the RC would be fine as well.

--

Jeff Hoffmann
PropertyKey.com

Just build RC1 today on Panther, no problems.

On Nov 4, 2003, at 5:06 PM, Jeff Hoffmann wrote:

> Tom Lane wrote:
>> kgleason(at)finelight(dot)com writes:
>>> After spending a few hours of trying to get Postgresql7.3.4 to build
>>> from source (tar.gz) on a Panther (release, not beta) system,
>> Try 7.4RC1 instead. Apple made some incompatible changes in their
>> compiler in Panther.
>
> I was going to recommend the same thing. I compiled a 7.4 beta out of
> the box without a hitch, so I'd assume the RC would be fine as well.
>
> --
>
> Jeff Hoffmann
> PropertyKey.com
>
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 8: explain analyze is your friend
>
--------------------

Andrew Rawnsley
President
The Ravensfield Digital Resource Group, Ltd.
(740) 587-0114
www.ravensfield.com

Is this a bug we should fix for 7.3.5 when it eventually comes out?

On Tue, 4 Nov 2003, Andrew Rawnsley wrote:

>
> Just build RC1 today on Panther, no problems.
>
>
> On Nov 4, 2003, at 5:06 PM, Jeff Hoffmann wrote:
>
> > Tom Lane wrote:
> >> kgleason(at)finelight(dot)com writes:
> >>> After spending a few hours of trying to get Postgresql7.3.4 to build
> >>> from source (tar.gz) on a Panther (release, not beta) system,
> >> Try 7.4RC1 instead. Apple made some incompatible changes in their
> >> compiler in Panther.
> >
> > I was going to recommend the same thing. I compiled a 7.4 beta out of
> > the box without a hitch, so I'd assume the RC would be fine as well.
> >
> > --
> >
> > Jeff Hoffmann
> > PropertyKey.com
> >
> >
> > ---------------------------(end of
> > broadcast)---------------------------
> > TIP 8: explain analyze is your friend
> >
> --------------------
>
> Andrew Rawnsley
> President
> The Ravensfield Digital Resource Group, Ltd.
> (740) 587-0114
> www.ravensfield.com
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faqs/FAQ.html
>
>