build times

I have been investigating some build performance issues, and trying to
narrow down causes of slowness, and observed an odd effect, which was
suggested by a huge time difference between building from git and
building from a tarball.

If I do

make -C src/port all

and then wait 10 seconds or so and do

make -j 3

or even just plain

make

the build finishes much much faster (like 1m vs 5m) than if I had not
run the first command.

I have seen this on a Fedora VM (VirtualBox, W7 host, Athlon II X2) and
a ScientificLinux 6 machine (dual quad xeon E5620).

The setup is a vpath build configured thus:

../pgsql/configure --enable-cassert --enable-debug
--enable-integer-datetimes --with-perl --with-python --with-tcl
--with-krb5 --with-includes=/usr/include/et --with-openssl
--with-ldap --with-libxml --with-libxslt

Can anyone with a bit more make-fu than I have suggest why this should
be so? Can we tweak the make files so hacks like this aren't required to
get a fast build? Can anyone replicate this odd result?

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> I have been investigating some build performance issues, and trying to
> narrow down causes of slowness, and observed an odd effect, which was
> suggested by a huge time difference between building from git and
> building from a tarball.

> If I do
> make -C src/port all
> and then wait 10 seconds or so and do
> make -j 3
> or even just plain
> make

> the build finishes much much faster (like 1m vs 5m) than if I had not
> run the first command.

Can't reproduce that here. What I do notice on a Fedora 14 machine is
that ccache seems to be enabled by default, ie you get caching even when
you just say "gcc", and that makes a huge difference in build times.
I see 70 seconds after "rm -rf ~/.ccache", versus 4 seconds with it
fully populated. Building src/port first saves nothing in either
starting state.

I wonder whether your experiments got affected by something similar.

regards, tom lane

On 10/02/2011 05:25 PM, Tom Lane wrote:
> Andrew Dunstan<andrew(at)dunslane(dot)net> writes:
>> I have been investigating some build performance issues, and trying to
>> narrow down causes of slowness, and observed an odd effect, which was
>> suggested by a huge time difference between building from git and
>> building from a tarball.
>> If I do
>> make -C src/port all
>> and then wait 10 seconds or so and do
>> make -j 3
>> or even just plain
>> make
>> the build finishes much much faster (like 1m vs 5m) than if I had not
>> run the first command.
> Can't reproduce that here. What I do notice on a Fedora 14 machine is
> that ccache seems to be enabled by default, ie you get caching even when
> you just say "gcc", and that makes a huge difference in build times.
> I see 70 seconds after "rm -rf ~/.ccache", versus 4 seconds with it
> fully populated. Building src/port first saves nothing in either
> starting state.
>
> I wonder whether your experiments got affected by something similar.
>
>

Yes, possibly, although SL6 (which is an RHEL clone) doesn't have ccache
by default. I'm not sure what happened there, as now I can't reproduce
it either.

Sorry for the noise.

cheers

andrew