Re: Something fishy about the current Makefiles

Whilst fooling around with GIN, I have repeatedly observed that doing
"make" in src/backend/access/gin, followed by "make install-bin" in
src/backend, fails to rebuild the postgres executable --- it just
installs the existing one. A second execution of "make install-bin"
does notice that postgres is out of date and rebuilds it. This
procedure for rebuilding after changing one or two .c files has always
worked for me before. I can't avoid the suspicion that the recent
changes to make things more parallel-friendly broke something.

regards, tom lane

On Thu, Jan 6, 2011 at 11:57 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Whilst fooling around with GIN, I have repeatedly observed that doing
> "make" in src/backend/access/gin, followed by "make install-bin" in
> src/backend, fails to rebuild the postgres executable --- it just
> installs the existing one.  A second execution of "make install-bin"
> does notice that postgres is out of date and rebuilds it.  This
> procedure for rebuilding after changing one or two .c files has always
> worked for me before.  I can't avoid the suspicion that the recent
> changes to make things more parallel-friendly broke something.

I've noticed something like this as well, but haven't been able to
figure out exactly what is going wrong.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> On Thu, Jan 6, 2011 at 11:57 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Whilst fooling around with GIN, I have repeatedly observed that doing
>> "make" in src/backend/access/gin, followed by "make install-bin" in
>> src/backend, fails to rebuild the postgres executable --- it just
>> installs the existing one. A second execution of "make install-bin"
>> does notice that postgres is out of date and rebuilds it. This
>> procedure for rebuilding after changing one or two .c files has always
>> worked for me before. I can't avoid the suspicion that the recent
>> changes to make things more parallel-friendly broke something.

> I've noticed something like this as well, but haven't been able to
> figure out exactly what is going wrong.

I'm not entirely sure either, but it looks like the first upper "make"
updates the objfiles.txt file in src/backend/access and then the second
one realizes it has to rebuild postgres. Something about multi-level
dependencies isn't quite right.

regards, tom lane

On Thu, 2011-01-06 at 13:53 -0500, Tom Lane wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> > On Thu, Jan 6, 2011 at 11:57 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >> Whilst fooling around with GIN, I have repeatedly observed that doing
> >> "make" in src/backend/access/gin, followed by "make install-bin" in
> >> src/backend, fails to rebuild the postgres executable --- it just
> >> installs the existing one. A second execution of "make install-bin"
> >> does notice that postgres is out of date and rebuilds it. This
> >> procedure for rebuilding after changing one or two .c files has always
> >> worked for me before. I can't avoid the suspicion that the recent
> >> changes to make things more parallel-friendly broke something.
>
> > I've noticed something like this as well, but haven't been able to
> > figure out exactly what is going wrong.
>
> I'm not entirely sure either, but it looks like the first upper "make"
> updates the objfiles.txt file in src/backend/access and then the second
> one realizes it has to rebuild postgres. Something about multi-level
> dependencies isn't quite right.

I had a weirdness with the dependency files a few days ago. The error
was a missing delimiter : at the end of one of the files. Stopped
everything cold until I added that character, then re-configured and not
a hint of the earlier problem.

Sounds like dependency is a common issue.

--
Simon Riggs http://www.2ndQuadrant.com/books/
PostgreSQL Development, 24x7 Support, Training and Services

On tor, 2011-01-06 at 11:57 -0500, Tom Lane wrote:
> Whilst fooling around with GIN, I have repeatedly observed that doing
> "make" in src/backend/access/gin, followed by "make install-bin" in
> src/backend, fails to rebuild the postgres executable --- it just
> installs the existing one. A second execution of "make install-bin"
> does notice that postgres is out of date and rebuilds it. This
> procedure for rebuilding after changing one or two .c files has always
> worked for me before. I can't avoid the suspicion that the recent
> changes to make things more parallel-friendly broke something.

This fixes it, but it's beyond me why.

diff --git i/src/backend/common.mk w/src/backend/common.mk
index 4e0a5da..5d599db 100644
--- i/src/backend/common.mk
+++ w/src/backend/common.mk
@@ -34,7 +34,7 @@ objfiles.txt: Makefile $(SUBDIROBJS) $(OBJS)
expand_subsys = $(foreach file,$(1),$(if $(filter %/objfiles.txt,$(file)),$(patsubst ../../src/backend/%,%,$(addprefix $(top_builddir)/,$(shell cat $(file)))),$(file)))

# Parallel make trickery
-$(SUBDIROBJS): | $(SUBDIRS:%=%-recursive) ;
+$(SUBDIROBJS): $(SUBDIRS:%=%-recursive) ;

.PHONY: $(SUBDIRS:%=%-recursive)
$(SUBDIRS:%=%-recursive):

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> On tor, 2011-01-06 at 11:57 -0500, Tom Lane wrote:
>> Whilst fooling around with GIN, I have repeatedly observed that doing
>> "make" in src/backend/access/gin, followed by "make install-bin" in
>> src/backend, fails to rebuild the postgres executable --- it just
>> installs the existing one. A second execution of "make install-bin"
>> does notice that postgres is out of date and rebuilds it. This
>> procedure for rebuilding after changing one or two .c files has always
>> worked for me before. I can't avoid the suspicion that the recent
>> changes to make things more parallel-friendly broke something.

> This fixes it, but it's beyond me why.

Further experimentation suggests that *only* subdirectories of access/
are vulnerable to the issue. I wonder whether this syntax:

> -$(SUBDIROBJS): | $(SUBDIRS:%=%-recursive) ;

is weird with respect to the first SUBDIRS value. Or maybe it's a plain
old bug in gmake.

regards, tom lane

On Thu, 2011-01-06 at 12:27 -0500, Robert Haas wrote:
> I've noticed something like this as well, but haven't been able to
> figure out exactly what is going wrong.

I've been having build problems ever since:

http://archives.postgresql.org/message-id/1291256879.25389.6.camel@jdavis-ux.asterdata.local

It's gotten much worse recently, where changing a c file seems to
require multiple top-level "make install" commands to take effect (as
Tom describes). I don't think it's limited to the access/ subdirectory,
though.

Regards,
Jeff Davis

Excerpts from Tom Lane's message of jue ene 06 13:57:11 -0300 2011:
> Whilst fooling around with GIN, I have repeatedly observed that doing
> "make" in src/backend/access/gin, followed by "make install-bin" in
> src/backend, fails to rebuild the postgres executable --- it just
> installs the existing one. A second execution of "make install-bin"
> does notice that postgres is out of date and rebuilds it. This
> procedure for rebuilding after changing one or two .c files has always
> worked for me before. I can't avoid the suspicion that the recent
> changes to make things more parallel-friendly broke something.

FWIW this explains why I was having such a hard time testing the FOR KEY
LOCK patch. Please commit whatever fix you have.

--
Álvaro Herrera <alvherre(at)commandprompt(dot)com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support