Re: Snow Leopard bison/flex build problem

Upgraded to Snow Leopard Saturday, and am having problems building now.

The build logs are here
http://buildfarm.postgresql.org/cgi-bin/show_status.pl?member=polecat

And the failing part is here:
make -C preproc all
make -C ../../../../src/port all
make[5]: Nothing to be done for `all'.
"/usr/bin/perl" ./parse.pl . < ../../../backend/parser/gram.y >
preproc.y
/usr/bin/bison -d -o preproc.c preproc.y
/usr/bin/flex -o'pgc.c' pgc.l
/opt/local/bin/ccache gcc -no-cpp-precomp -I/opt/local/include -Wall -
Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -
Wendif-labels -fno-strict-aliasing -fwrapv -g -DECPG_COMPILE -I../
include -I../../../../src/interfaces/ecpg/include -I. -I. -
DMAJOR_VERSION=4 -DMINOR_VERSION=6 -DPATCHLEVEL=0 -I../../../../src/
include -I/usr/include/libxml2 -c -o preproc.o preproc.c
In file included from preproc.y:12065:
pgc.c:161: error: conflicting types for 'yyleng'
extern.h:43: error: previous declaration of 'yyleng' was here
pgc.c:288: error: conflicting types for 'yyleng'
extern.h:43: error: previous declaration of 'yyleng' was here
make[4]: *** [preproc.o] Error 1
make[3]: *** [all] Error 2
make[2]: *** [all] Error 2
make[1]: *** [all] Error 2
make: *** [all] Error 2

% perl --version

This is perl, v5.10.0 built for darwin-thread-multi-2level
(with 2 registered patches, see perl -V for more detail)

Copyright 1987-2007, Larry Wall

Perl may be copied only under the terms of either the Artistic License
or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to
the
Internet, point your browser at http://www.perl.org/, the Perl Home
Page.

% bison --version
bison (GNU Bison) 2.3
Written by Robert Corbett and Richard Stallman.

Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There
is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.

% flex --version
flex 2.5.35

Any thoughts here?

Thanks,
Rob

Robert Creager wrote:
>
> Upgraded to Snow Leopard Saturday, and am having problems building now.
>
> The build logs are here
> http://buildfarm.postgresql.org/cgi-bin/show_status.pl?member=polecat
>
> And the failing part is here:
> make -C preproc all
> make -C ../../../../src/port all
> make[5]: Nothing to be done for `all'.
> "/usr/bin/perl" ./parse.pl . < ../../../backend/parser/gram.y > preproc.y
> /usr/bin/bison -d -o preproc.c preproc.y
> /usr/bin/flex -o'pgc.c' pgc.l
> /opt/local/bin/ccache gcc -no-cpp-precomp -I/opt/local/include -Wall
> -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement
> -Wendif-labels -fno-strict-aliasing -fwrapv -g -DECPG_COMPILE
> -I../include -I../../../../src/interfaces/ecpg/include -I. -I.
> -DMAJOR_VERSION=4 -DMINOR_VERSION=6 -DPATCHLEVEL=0
> -I../../../../src/include -I/usr/include/libxml2 -c -o preproc.o
> preproc.c
> In file included from preproc.y:12065:
> pgc.c:161: error: conflicting types for 'yyleng'
> extern.h:43: error: previous declaration of 'yyleng' was here
> pgc.c:288: error: conflicting types for 'yyleng'
> extern.h:43: error: previous declaration of 'yyleng' was here
> make[4]: *** [preproc.o] Error 1
> make[3]: *** [all] Error 2
> make[2]: *** [all] Error 2
> make[1]: *** [all] Error 2
> make: *** [all] Error 2
>

Please try building by hand and examine the files to find out what the
conflict is between these declarations.

This is really a -hackers question.

cheers

andrew

On Sep 7, 2009, at 11:29 AM, Andrew Dunstan wrote:

>
>
> Robert Creager wrote:
>>
>> Upgraded to Snow Leopard Saturday, and am having problems building
>> now.
>>
>> The build logs are here
>> http://buildfarm.postgresql.org/cgi-bin/show_status.pl?member=polecat
>>
>> And the failing part is here:
>> make -C preproc all
>> make -C ../../../../src/port all
>> make[5]: Nothing to be done for `all'.
>> "/usr/bin/perl" ./parse.pl . < ../../../backend/parser/gram.y >
>> preproc.y
>> /usr/bin/bison -d -o preproc.c preproc.y
>> /usr/bin/flex -o'pgc.c' pgc.l
>> /opt/local/bin/ccache gcc -no-cpp-precomp -I/opt/local/include -
>> Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-
>> statement -Wendif-labels -fno-strict-aliasing -fwrapv -g -
>> DECPG_COMPILE -I../include -I../../../../src/interfaces/ecpg/
>> include -I. -I. -DMAJOR_VERSION=4 -DMINOR_VERSION=6 -DPATCHLEVEL=0 -
>> I../../../../src/include -I/usr/include/libxml2 -c -o preproc.o
>> preproc.c
>> In file included from preproc.y:12065:
>> pgc.c:161: error: conflicting types for 'yyleng'
>> extern.h:43: error: previous declaration of 'yyleng' was here
>> pgc.c:288: error: conflicting types for 'yyleng'
>> extern.h:43: error: previous declaration of 'yyleng' was here
>> make[4]: *** [preproc.o] Error 1
>> make[3]: *** [all] Error 2
>> make[2]: *** [all] Error 2
>> make[1]: *** [all] Error 2
>> make: *** [all] Error 2
>>
>
> Please try building by hand and examine the files to find out what
> the conflict is between these declarations.
>
> This is really a -hackers question.

pgc.c - 161:

extern yy_size_t yyleng;

extern.h - 43 :

extern int yylineno,
yyleng;

Thanks,
Rob

Robert Creager <robert(at)logicalchaos(dot)org> writes:
> On Sep 7, 2009, at 11:29 AM, Andrew Dunstan wrote:
>> Please try building by hand and examine the files to find out what
>> the conflict is between these declarations.

> pgc.c - 161:

> extern yy_size_t yyleng;

Bizarre --- my copy of flex 2.5.35 definitely puts out "int", and
so does everybody else's. Did Apple take it on their own head to
change this?

regards, tom lane

On Sep 7, 2009, at 1:18 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Robert Creager <robert(at)logicalchaos(dot)org> writes:
>> On Sep 7, 2009, at 11:29 AM, Andrew Dunstan wrote:
>>> Please try building by hand and examine the files to find out what
>>> the conflict is between these declarations.
>
>> pgc.c - 161:
>
>> extern yy_size_t yyleng;
>
> Bizarre --- my copy of flex 2.5.35 definitely puts out "int", and
> so does everybody else's. Did Apple take it on their own head to
> change this?
>
> regards, tom lane

I'll try the macports version, and see what happens.

Cheers,
Rob

Sent from my iPhone

>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

On 9/7/09, Robert Creager <robert(at)logicalchaos(dot)org> wrote:
>
> On Sep 7, 2009, at 1:18 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
>> Robert Creager <robert(at)logicalchaos(dot)org> writes:
>>> On Sep 7, 2009, at 11:29 AM, Andrew Dunstan wrote:
>>>> Please try building by hand and examine the files to find out what
>>>> the conflict is between these declarations.
>>
>>> pgc.c - 161:
>>
>>> extern yy_size_t yyleng;
>>
>> Bizarre --- my copy of flex 2.5.35 definitely puts out "int", and
>> so does everybody else's. Did Apple take it on their own head to
>> change this?
>>
>> regards, tom lane
>
> I'll try the macports version, and see what happens.
>

FYI, I've been building from source on Snow Leopard without any problems.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

On Sep 7, 2009, at 1:18 PM, Tom Lane wrote:

> Robert Creager <robert(at)logicalchaos(dot)org> writes:
>> On Sep 7, 2009, at 11:29 AM, Andrew Dunstan wrote:
>>> Please try building by hand and examine the files to find out what
>>> the conflict is between these declarations.
>
>> pgc.c - 161:
>
>> extern yy_size_t yyleng;
>
> Bizarre --- my copy of flex 2.5.35 definitely puts out "int", and
> so does everybody else's. Did Apple take it on their own head to
> change this?

Apparently so - the /opt version is from macports. It works.

% /opt/local/bin/flex --version
flex 2.5.35

% /usr/bin/flex --version
flex 2.5.35

Next problems, with HEAD and 8_4, 8_3, 8_2, are here - all with the
same error:

http://buildfarm.postgresql.org/cgi-bin/show_status.pl?member=polecat

...
============== running regression test queries ==============
test tablespace ... FAILED
...
================== pgsql.41144/src/test/regress/regression.diffs
===================
*** /usr/local/src/build-farm-3.2/builds/HEAD/pgsql.41144/src/test/
regress/expected/tablespace.out Mon Sep 7 14:03:30 2009
--- /usr/local/src/build-farm-3.2/builds/HEAD/pgsql.41144/src/test/
regress/results/tablespace.out Mon Sep 7 14:03:43 2009
***************
*** 73,75 ****
--- 73,76 ----
drop cascades to table testschema.atable
-- Should succeed
DROP TABLESPACE testspace;
+ ERROR: could not read directory "pg_tblspc/16388": Invalid argument

Sigh,
Rob

On Sep 7, 2009, at 2:17 PM, Dave Page wrote:

>
> FYI, I've been building from source on Snow Leopard without any
> problems.

If your building from the official tarball, bison/flex are not used.
I'm building from CVS, where bison/flex are used.

Cheers,
rob

On 9/7/09, Robert Creager <robert(at)logicalchaos(dot)org> wrote:
>
> On Sep 7, 2009, at 2:17 PM, Dave Page wrote:
>
>>
>> FYI, I've been building from source on Snow Leopard without any
>> problems.
>
> If your building from the official tarball, bison/flex are not used.
> I'm building from CVS, where bison/flex are used.

Building from CVS, but on closer examination I do have Macports
versions of bison/flex ahead of Apple's in the path.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

Robert Creager <robert(at)logicalchaos(dot)org> writes:
> On Sep 7, 2009, at 1:18 PM, Tom Lane wrote:
>> Robert Creager <robert(at)logicalchaos(dot)org> writes:
>>> extern yy_size_t yyleng;
>>
>> Bizarre --- my copy of flex 2.5.35 definitely puts out "int", and
>> so does everybody else's. Did Apple take it on their own head to
>> change this?

> Apparently so - the /opt version is from macports. It works.

[ Well, I knew I was going to be buying a copy of Snow Leopard sometime
soon. One trip to the Apple store later ... ]

Yeah, I've confirmed this. It appears that Apple has absorbed this
as-yet-unreleased upstream patch into their "2.5.35" version:
http://flex.cvs.sourceforge.net/viewvc/flex/flex/flex.skl?r1=2.212&r2=2.213

This is probably not the brightest thing the flex developers have ever
done, as there is now absolutely no way to predict the type of yyleng
externally to the generated scan.c file. They might as well not export
it at all.

However, I think we can work around it. AFAICS, the only reason ecpg
has an extern for yyleng is because preproc.y's make_name() uses it,
and there doesn't seem to be any compelling reason why that function
shouldn't just do "mm_strdup(yytext)" instead. I don't see any other
places where we are referring to yyleng outside of a scanner file.
(Alternatively, make_name() could be moved into pgc.l, but I doubt
it's worth the work to avoid one extra strlen calculation.)

Presumably, versions of flex containing this change will start to show
up at other places besides Apple sometime soon. So I'm thinking we
need to back-patch the fix to whatever branches we think are likely
to get compiled from CVS on newer platforms. Any feelings about that?
Should I just hit everything back to 7.4 to be safe?

regards, tom lane

On Sep 7, 2009, at 2:41 PM, Dave Page <dpage(at)pgadmin(dot)org> wrote:

> On 9/7/09, Robert Creager <robert(at)logicalchaos(dot)org> wrote:
>>
>> On Sep 7, 2009, at 2:17 PM, Dave Page wrote:
>>
>>>
>>> FYI, I've been building from source on Snow Leopard without any
>>> problems.
>>
>> If your building from the official tarball, bison/flex are not used.
>> I'm building from CVS, where bison/flex are used.
>
> Building from CVS, but on closer examination I do have Macports
> versions of bison/flex ahead of Apple's in the path.

Does your <make config> work? Are you using the 32 or 64 bit kernel?
>

Cheers,
Rob

Sent from my iPhone

On Sep 7, 2009, at 4:08 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Any feelings about that?
> Should I just hit everything back to 7.4 to be safe?

I've noticed on 7.4, Mac gets a spinlock compile error (see polecat
logs on buildfarm). Should I give up on the mac for 7.4?

Cheers,
Rob

Sent from my iPhone

Robert Creager <robert(at)logicalchaos(dot)org> writes:
> Next problems, with HEAD and 8_4, 8_3, 8_2, are here - all with the
> same error:

> -- Should succeed
> DROP TABLESPACE testspace;
> + ERROR: could not read directory "pg_tblspc/16388": Invalid argument

Yeah, this is a known Snow Leopard bug --- see last month's report
from Jan Otto. I think our plan is to wait and see if Apple fixes
it in 10.6.1 before we consider whether it's worth installing a
kluge workaround.

regards, tom lane

Robert Creager <robert(at)logicalchaos(dot)org> writes:
> I've noticed on 7.4, Mac gets a spinlock compile error (see polecat
> logs on buildfarm). Should I give up on the mac for 7.4?

7.4 thinks that Darwin only runs on PPC. We are not going to fix it.

regards, tom lane

On Sep 7, 2009, at 3:12 PM, Robert Creager wrote:

>> Building from CVS, but on closer examination I do have Macports
>> versions of bison/flex ahead of Apple's in the path.
>
> Does your <make config> work? Are you using the 32 or 64 bit kernel?

Also, anyone tried compiling with LLVM and Clang?

http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/9

And are there places where PostgreSQL can take advantage of GCD?

http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/12

Cheers,

David

I wrote:
> Yeah, I've confirmed this. It appears that Apple has absorbed this
> as-yet-unreleased upstream patch into their "2.5.35" version:
> http://flex.cvs.sourceforge.net/viewvc/flex/flex/flex.skl?r1=2.212&r2=2.213
> ...
> However, I think we can work around it. AFAICS, the only reason ecpg
> has an extern for yyleng is because preproc.y's make_name() uses it,
> and there doesn't seem to be any compelling reason why that function
> shouldn't just do "mm_strdup(yytext)" instead.

I've committed patches for this --- it should be safe to go back to
using OS X's native flex on your buildfarm machine.

regards, tom lane

try without the ccache? perhaps it is pulling in some stuff you don't expect.

On Mon, Sep 7, 2009 at 12:35 PM, Robert Creager<robert(at)logicalchaos(dot)org> wrote:
> Upgraded to Snow Leopard Saturday, and am having problems building now.