Re: "Fat" binaries for OS X (was Re: [GENERAL] Postgres Library natively available for Mac OSX Intel?)

Hello!

I would like to know if somebody already has a Mac OSX Intel 10.4.5
pg-Library (for C, C++, Objective C) or knows how to compile it?

Or even better, a fat-library (powerpc, i386) would be even better :-)

Thanks for any information,
regards
Philipp Ott

# philipp(dot)ott(at)avalon(dot)at / 2006-03-31 10:05:06 +0200:
> I would like to know if somebody already has a Mac OSX Intel 10.4.5
> pg-Library (for C, C++, Objective C) or knows how to compile it?

What problems did you have building libpq?

Note: I'm not an OSX user.

--
How many Vietnam vets does it take to screw in a light bulb?
You don't know, man. You don't KNOW.
Cause you weren't THERE. http://bash.org/?255991

Hello!

Am 07.04.2006 um 13:50 schrieb User Roman:

> # philipp(dot)ott(at)avalon(dot)at / 2006-03-31 10:05:06 +0200:
>> I would like to know if somebody already has a Mac OSX Intel 10.4.5
>> pg-Library (for C, C++, Objective C) or knows how to compile it?
>
> What problems did you have building libpq?
>
> Note: I'm not an OSX user.

I just wanted to know - I would like to have universal binaries of
libpg and psql to deploy.

Currently 8.1.3 compiles and runs just fine on OSX 10.4.6 + XCode
2.2.1, but generates binaries just for the current host architecture.
Now when I add -arch i386 -arch ppc to CFLAGS and LDFLAGS for
configure, then it compiles everything just fine, however at linking
stage I get various problems for missing architecture files. Every
generated .o file in the src build tree is actually an universal
binary now, like for example

./src/timezone/SUBSYS.o:
Mach header
magic cputype cpusubtype filetype ncmds sizeofcmds flags
0xfeedface 7 3 1 4 852 0x00002000
Fat headers
fat_magic 0xcafebabe
nfat_arch 2
architecture 0
cputype 7
cpusubtype 3
offset 64
size 48100
align 2^5 (32)
architecture 1
cputype 18
cpusubtype 0
offset 48192
size 69332
align 2^5 (32)
./src/timezone/zic.o:
Mach header
magic cputype cpusubtype filetype ncmds sizeofcmds flags
0xfeedface 7 3 1 3 772 0x00002000

So when make comes to the first linking this happens:

gcc -no-cpp-precomp -arch i386 -arch ppc -Wall -Wmissing-prototypes -
Wpointer-arith -Winline -Wdeclaration-after-statement -Wendif-labels -
fno-strict-aliasing -L../../src/port -arch i386 -arch ppc access/
SUBSYS.o bootstrap/SUBSYS.o catalog/SUBSYS.o parser/SUBSYS.o commands/
SUBSYS.o executor/SUBSYS.o lib/SUBSYS.o libpq/SUBSYS.o main/SUBSYS.o
nodes/SUBSYS.o optimizer/SUBSYS.o port/SUBSYS.o postmaster/SUBSYS.o
regex/SUBSYS.o rewrite/SUBSYS.o storage/SUBSYS.o tcop/SUBSYS.o utils/
SUBSYS.o ../../src/timezone/SUBSYS.o ../../src/port/
libpgport_srv.a -lz -lreadline -lresolv -ldl -lm -o postgres
/usr/bin/ld: for architecture ppc
/usr/bin/ld: warning access/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning bootstrap/SUBSYS.o cputype (7, architecture
i386) does not match cputype (18) for specified -arch flag: ppc (file
not loaded)
/usr/bin/ld: warning catalog/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning parser/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning commands/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning executor/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning lib/SUBSYS.o cputype (7, architecture i386) does
not match cputype (18) for specified -arch flag: ppc (file not loaded)
/usr/bin/ld: warning libpq/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning main/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning nodes/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning optimizer/SUBSYS.o cputype (7, architecture
i386) does not match cputype (18) for specified -arch flag: ppc (file
not loaded)
/usr/bin/ld: warning port/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning postmaster/SUBSYS.o cputype (7, architecture
i386) does not match cputype (18) for specified -arch flag: ppc (file
not loaded)
/usr/bin/ld: warning regex/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning rewrite/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning storage/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning tcop/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning utils/SUBSYS.o cputype (7, architecture i386)
does not match cputype (18) for specified -arch flag: ppc (file not
loaded)
/usr/bin/ld: warning ../../src/timezone/SUBSYS.o cputype (7,
architecture i386) does not match cputype (18) for specified -arch
flag: ppc (file not loaded)
/usr/bin/ld: Undefined symbols:
_main
collect2: ld returned 1 exit status
lipo: can't open input file: /var/tmp//cc0yANLg.out (No such file or
directory)
make[2]: *** [postgres] Error 1
make[1]: *** [all] Error 2
make: *** [all] Error 2

However all the mentioned .o files in this linking list above are
both architecture 7 and 18.

Did someone else encounter similar problems and could fix this and
let me know please?

I used

configure CFLAGS="-arch i386 -arch ppc" LDFLAGS="-arch i386 -arch ppc"

for the initial configure run.

Thank you
Philipp

# philipp(dot)ott(at)avalon(dot)at / 2006-04-08 14:04:28 +0200:
> Am 07.04.2006 um 13:50 schrieb User Roman:
> ># philipp(dot)ott(at)avalon(dot)at / 2006-03-31 10:05:06 +0200:
> >>I would like to know if somebody already has a Mac OSX Intel 10.4.5
> >>pg-Library (for C, C++, Objective C) or knows how to compile it?
> >
> > What problems did you have building libpq?
> >
> > Note: I'm not an OSX user.
>
> I just wanted to know - I would like to have universal binaries of
> libpg and psql to deploy.
>
> Currently 8.1.3 compiles and runs just fine on OSX 10.4.6 + XCode
> 2.2.1, but generates binaries just for the current host architecture.
> Now when I add -arch i386 -arch ppc to CFLAGS and LDFLAGS for
> configure, then it compiles everything just fine, however at linking
> stage I get various problems for missing architecture files. Every
> generated .o file in the src build tree is actually an universal
> binary now, like for example

sorry, this is way over my head.

--
How many Vietnam vets does it take to screw in a light bulb?
You don't know, man. You don't KNOW.
Cause you weren't THERE. http://bash.org/?255991

Philipp Ott <philipp(dot)ott(at)avalon(dot)at> writes:
> Currently 8.1.3 compiles and runs just fine on OSX 10.4.6 + XCode
> 2.2.1, but generates binaries just for the current host architecture.
> Now when I add -arch i386 -arch ppc to CFLAGS and LDFLAGS for
> configure, then it compiles everything just fine, however at linking
> stage I get various problems for missing architecture files.

I looked into this and found that the problem is our habit of using
"ld -r" to aggregate multiple .o files into a single SUBSYS.o file
that's still relocatable. The Darwin version of ld is pretty brain-dead
when it comes to "fat" files containing code for more than one
architecture --- the man page says

UNIVERSAL FILE SUPPORT
The link editor accepts ``universal'' (multiple-architecture) input
files, but always creates a ``thin'' (single-architecture), standard
Mach-O output file. The architecture is specified using the -arch
arch_type option. If this option is not used, ld(1) attempts to deter-
mine the output architecture by examining the first object file encoun-
tered on the command line. If it is a ``thin'' file, its architecture
determines that of the output file. If the first input file is a
``universal'' file, the ``best'' architecture for the host is used.
(See the explanation of the -arch option, below.)

The compiler driver cc(1) handles creating universal executables by
calling ld(1) multiple times and using lipo(1) to create a ``univer-
sal'' file from the results of the ld(1) executions.

So what you're seeing is that one of the arches has been dropped from
the SUBSYS.o files.

Unfortunately, there doesn't seem to be any way to use cc/gcc to emulate
"ld -r", in the sense of just combining multiple fat .o files into one
fat .o file. At least I couldn't see one after perusing the man page
for a bit. I also found out that lipo(1) is not by itself smart enough
to do this.

So it looks like you'd have to write a small shell script to do what the
above snippet describes cc as doing. Not out of the question by any
means, but still a PITA. Any Apple experts around who know a better
answer? Is Apple likely to improve this situation in the near future?

BTW, our configure script is not real flexible about adjusting the
command used to produce the SUBSYS.o files ... if you want anything
except "$(LD) -r -o SUBSYS.o *.o", you have to edit Makefile.global
after configuring. But without having a solution that actually works
for multi-arch Darwin, I'm not seeing the point of improving that yet.

regards, tom lane

Hi Tom!

Am 09.04.2006 um 04:34 schrieb Tom Lane:

> Philipp Ott <philipp(dot)ott(at)avalon(dot)at> writes:
>> Currently 8.1.3 compiles and runs just fine on OSX 10.4.6 + XCode
>> 2.2.1, but generates binaries just for the current host architecture.
>> Now when I add -arch i386 -arch ppc to CFLAGS and LDFLAGS for
>> configure, then it compiles everything just fine, however at linking
>> stage I get various problems for missing architecture files.
>
> [...]
> So what you're seeing is that one of the arches has been dropped from
> the SUBSYS.o files.
>
> Unfortunately, there doesn't seem to be any way to use cc/gcc to
> emulate
> "ld -r", in the sense of just combining multiple fat .o files into one
> fat .o file. At least I couldn't see one after perusing the man page
> for a bit. I also found out that lipo(1) is not by itself smart
> enough
> to do this.
>
> So it looks like you'd have to write a small shell script to do
> what the
> above snippet describes cc as doing. Not out of the question by any
> means, but still a PITA. Any Apple experts around who know a better
> answer? Is Apple likely to improve this situation in the near future?
> [...]

Thank you for the suggestion. I followed your suggestion and came up
with a ldfat script and a change to src/Makefile.global after running
configure. I put my ldfat script into the top_builddir and reference
it in the src/Makefile.global.

So from the download of postgresql-8.1.3.tar.gz I made the following
steps:

tar xzf postgresql-8.1.3.tar.gz
cd postgres-8.1.3
./configure CFLAGS="-arch i386 -arch ppc"

vi src/Makefile.global
....
LD = ${top_builddir}/ldfat
...

And in the postgres top directory I put the following script called
ldfat.

#!/bin/bash

OFILES=""
RELOPT=""
OUTPUT=""
OTHERS=""

while [ "$#" != "0" ];
do
case "$1" in
-r) RELOPT="-r";;
-o) OUTPUT=`basename -s .o "$2"`; shift;;
*.o) OFILES="$OFILES $1";;
*) OTHERS="$OTHERS $1";;
esac
shift
done

if [ "$RELOPT" == "-r" ];
then
echo ldfat $RELOPT -o $OUTPUT $OFILES $OTHERS
`/usr/bin/ld -r -arch i386 -o ${OUTPUT}_i386.o $OFILES $OTHERS`
`/usr/bin/ld -r -arch ppc -o ${OUTPUT}_ppc.o $OFILES $OTHERS`
`lipo -create -output ${OUTPUT}.o ${OUTPUT}_i386.o ${OUTPUT}_ppc.o`
else
echo ld -o $OUTPUT $OFILES $OTHERS
`/usr/bin/ld -o $OUTPUT $OFILES $OTHERS`
fi
exit $?

Now make and sudo make install run fine and in default config put a
working psql (and the rest I think will work too) into /usr/local/
pgsql on my imac. At the moment I just have this Intel iMac around
but I will try this with a PowerPC Mac later and let you know, if the
generated binaries and libraries work on an out-of-the-box OSX.

I vote for an --enable-osxuniversal option to ./configure which adds
"-arch ppc -arch i386" to the CFLAGS and replaces "LD" with a script
solution for the time being, unless an Apple expert has a better
solution at hand. If Apple makes changes to ld later this --enable-
osxuniversal would handle this transparently. However my configure
knowledge is next to non-existant :-) Also I only made a cursory
glance over all Makefiles ${LD} invoctions for calling patterns and
atm it works, however it would need some checking for available
options and errors.

Regards,
Philipp Ott

On Sat, 08 Apr 2006 14:04:28 +0200, Philipp Ott wrote:

> (..snippetysnip..)
> I just wanted to know - I would like to have universal binaries of libpg
> and psql to deploy.
>
>
> Currently 8.1.3 compiles and runs just fine on OSX 10.4.6 + XCode 2.2.1,
> but generates binaries just for the current host architecture. Now when I
> add -arch i386 -arch ppc to CFLAGS and LDFLAGS for configure, then it
> compiles everything just fine, however at linking stage I get various
> problems for missing architecture files. Every generated .o file in the

Are you sure you have all libs/frameworks installed as fat versions? On
NeXTSTEP you could select the architectures to install; unselected ones
were lipo'd out. I don't have OSX but the messages look like you only have
x86 libs to link against.

-h