News on Clang

A couple of months back, Greg Stark reported mixed results on getting
Clang/LLVM to build Postgres. It could be done, but there were some
bugs, including a bug that caused certain grammar TUs to take way too
long to compile. 2 bug reports were filed. He said that the long
compile time problem was made a lot better by using SVN-tip as it
existed at the time - it brought compile time down from over a minute
to just 15 seconds for preproc.c, which was of particular concern
then.

Last night, I had a go at getting Postgres to build using my own build
of Clang SVN-tip (2.9). I was successful, but I too found certain
grammar TUs to be very slow to compile. Total times were 3m23s for
Clang, to GCC 4.6's 2m1s. I made a (perhaps duplicate) bug report,
which had a preprocessed gram.c as a testcase. Here was the compile
time that I saw for the file:

[peter(at)peter postgresql]$ time /home/peter/build/Release/bin/clang -O2
-Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wformat-security
-fno-strict-aliasing -fwrapv -Wno-error -I. -I. -I
/home/peter/postgresql/src/include -D_GNU_SOURCE -c -o gram.o
/home/peter/postgresql/src/backend/parser/gram.c
In file included from gram.y:12939:
scan.c:16246:23: warning: unused variable 'yyg' [-Wunused-variable]
struct yyguts_t * yyg = (struct yyguts_t*)yyscanner; /* This var ...
^
1 warning generated.

real 1m5.780s
user 1m4.915s
sys 0m0.086s

The compile time is astronomically high, considering it takes about 2
seconds to compile gram.c on the same machine using GCC 4.6 with the
same flags.

This problem is reportedly a front-end issue. Observe what happens
when I omit the -Wall flag:

[peter(at)peter postgresql]$ time /home/peter/build/Release/bin/clang -O2
-Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement
-Wendif-labels -Wformat-security -fno-strict-aliasing -fwrapv
-Wno-error -I. -I. -I /home/peter/postgresql/src/include -D_GNU_SOURCE
-c -o gram.o /home/peter/postgresql/src/backend/parser/gram.c

real 0m2.153s
user 0m2.073s
sys 0m0.065s
[peter(at)peter postgresql]$

This is very close to the time for GCC. The problem has been further
isolated to the flag -Wuninitialized.

I hacked our configure file to omit -Wall in $CFLAGS . -Wall is just a
flag to have GCC/Clang "show all reasonable warnings" - it doesn't
affect binaries. I then measured the total build time for Postgres
using Clang SVN-tip. Here's what "time make" outputs:

* SNIP *
real 2m7.102s
user 1m49.015s
sys 0m10.635s

I'm very encouraged by this - Clang is snapping at the heels of GCC
here. I'd really like to see Clang as a better supported compiler,
because the whole LLVM infrastructure seems to have a lot to offer -
potentially improved compile times, better warnings/errors, a good
static analysis tool, a nice debugger, and a nice modular architecture
for integration with third party components. It is still a bit
immature, but it has the momentum.

At a large presentation that I and other PG community members were
present at during FOSDEM, Postgres was specifically cited as an
example of a medium-sized C program that had considerably improved
compile times on Clang. While I was obviously unable to reproduce the
very impressive compile-time numbers claimed (at -O0), I still think
that Clang has a lot of promise. Here are the slides from that
presentation:

http://www.scribd.com/doc/48921683/LLVM-Clang-Advancing-Compiler-Technology

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

On Fri, Jun 24, 2011 at 1:02 PM, Peter Geoghegan <peter(at)2ndquadrant(dot)com> wrote:
> [research]

Nice stuff. Thanks for the update.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2011-06-25 00:02, Peter Geoghegan wrote:

> At a large presentation that I and other PG community members were
> present at during FOSDEM, Postgres was specifically cited as an
> example of a medium-sized C program that had considerably improved
> compile times on Clang. While I was obviously unable to reproduce the
> very impressive compile-time numbers claimed (at -O0), I still think
> that Clang has a lot of promise. Here are the slides from that
> presentation:
>
> http://www.scribd.com/doc/48921683/LLVM-Clang-Advancing-Compiler-Technology

I notice that the slide about compilation speed on postgres compares
only front-end speeds between gcc and clang, not the speeds for
optimization and code generation. That may explain why the difference
is more pronounced on the slide than it is for a real build.

By the way, I was amazed to see such a young compiler build libpqxx with
no other problems than a few justified warnings or errors that gcc
hadn't issued. And that's C++, which is a lot harder than C! The
output was also far more helpful than gcc's. IIRC I found clang just
slightly faster than gcc on a full configure/build/test.

Jeroen

On fre, 2011-06-24 at 18:02 +0100, Peter Geoghegan wrote:
> I'm very encouraged by this - Clang is snapping at the heels of GCC
> here. I'd really like to see Clang as a better supported compiler,

We have a build farm member for Clang:
http://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=smew&br=HEAD

That doesn't capture the compile time issues that you described, but
several other issues that caused the builds to fail altogether have been
worked out recently, and I'd consider clang 2.9+ to be fully supported
as of PG 9.1.

On 25 June 2011 00:27, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On fre, 2011-06-24 at 18:02 +0100, Peter Geoghegan wrote:
>> I'm very encouraged by this - Clang is snapping at the heels of GCC
>> here. I'd really like to see Clang as a better supported compiler,
>
> We have a build farm member for Clang:
> http://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=smew&br=HEAD
>
> That doesn't capture the compile time issues that you described, but
> several other issues that caused the builds to fail altogether have been
> worked out recently, and I'd consider clang 2.9+ to be fully supported
> as of PG 9.1.

I did see the D_GNU_SOURCE issue that you described a while back with
2.8. My bleeding edge Fedora 15 system's yum repository only has 2.8,
for some reason.

I'm glad that you feel we're ready to officially support Clang -
should this be in the 9.1 release notes?

Anyway, since the problem has been narrowed to a specific compiler
flag, and we have a good test case, I'm optimistic that the bug can be
fixed quickly.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services

On lör, 2011-06-25 at 01:02 +0100, Peter Geoghegan wrote:
> I'm glad that you feel we're ready to officially support Clang -
> should this be in the 9.1 release notes?

Done

On 8 July 2011 15:15, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On lör, 2011-06-25 at 01:02 +0100, Peter Geoghegan wrote:
>> I'm glad that you feel we're ready to officially support Clang -
>> should this be in the 9.1 release notes?
>
> Done

Reportedly, LLVM r134697 speeds things up considerably for these
problem TUs. I intend to blog on the subject soon enough.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services