Re: pgxs problem...

I'm trying to build something (pljava, dont laugh) for IBM AIX (don't
laugh). I have my own build of postgresql 9.1.6 on AIX, compiled with
IBM XLC, it works quite nicely, but this build is bombing..

.......
/opt/pgsql91/lib/pgxs/src/makefiles/../../src/backend/port/aix/mkldexport.sh
libpljava.a >libpljava.exp
/bin/sh:
/opt/pgsql91/lib/pgxs/src/makefiles/../../src/backend/port/aix/mkldexport.sh:
not found.
...

now, I have a lib/pgxs/src/makefiles, but there's no src/backend

Do I need to use a special ./configure option in postgres to build
this? or run some kind of special make ?

--
john r pierce N 37, W 122
santa cruz ca mid-left coast

John R Pierce <pierce(at)hogranch(dot)com> writes:
> I'm trying to build something (pljava, dont laugh) for IBM AIX (don't
> laugh). I have my own build of postgresql 9.1.6 on AIX, compiled with
> IBM XLC, it works quite nicely, but this build is bombing..

> /opt/pgsql91/lib/pgxs/src/makefiles/../../src/backend/port/aix/mkldexport.sh
> libpljava.a >libpljava.exp
> /bin/sh:
> /opt/pgsql91/lib/pgxs/src/makefiles/../../src/backend/port/aix/mkldexport.sh:
> not found.

Hm. There is a src/backend/port/aix/mkldexport.sh in the source tree,
and I see that Makefile.shlib relies on it, but I don't see any evidence
that we make any effort to install it. For the moment you could just
copy it over to the pgxs tree ... but if we wanted to support AIX a bit
better it seems like we ought to do that automatically.

regards, tom lane

On 10/09/12 11:49 AM, John R Pierce wrote:
>
> and, muh-oh... $PG_PREFIX/bin/postgres is the binary executable, so
> I *can't* put a copy of postgres.imp there, hah. and a bit of poking
> around the pljava makefiles, I'm not figuring out how its cooking that
> path name up?!?

ok, its coming from the pgxs side.

$ find /opt/pgsql91 -name Makefile\* | xargs grep 'postgres.imp'
/opt/pgsql91/lib/pgxs/src/Makefile.port:POSTGRES_IMP= postgres.imp
$ find /opt/pgsql91 -name Makefile\* | xargs grep 'POSTGRES_IMP'
/opt/pgsql91/lib/pgxs/src/Makefile.port:POSTGRES_IMP= postgres.imp
/opt/pgsql91/lib/pgxs/src/Makefile.port:BE_DLLLIBS=
-Wl,-bI:$(bindir)/postgres/$(POSTGRES_IMP)
/opt/pgsql91/lib/pgxs/src/Makefile.port:BE_DLLLIBS=
-Wl,-bI:$(top_builddir)/src/backend/$(POSTGRES_IMP)

$(bindir)/postgres/* can't possibly be right, since $(bindir)/postgres
is the runtime binary for the server ?!?

--
john r pierce N 37, W 122
santa cruz ca mid-left coast

John R Pierce <pierce(at)hogranch(dot)com> writes:
> mm, k. so I manually copied that file over, and now get...
> ....
> /usr/vacpp/bin/xlc -q64 -I/home/postgres/src/include -O2 -qarch=pwr5
> -qtune=balanced -qnoansialias -o pljava.so libpljava.a
> -Wl,-bE:libpljava.exp -L/opt/pgsql91/lib -L/home/postgres/src/lib
> -Wl,-blibpath:'/opt/pgsql91/lib:/opt/pgsql91/lib:/home/postgres/src/lib:/usr/lib:/lib'
> -Wl,-bnoentry -Wl,-H512 -Wl,-bM:SRE
> -Wl,-bI:/opt/pgsql91/bin/postgres/postgres.imp -L.
> -L"/usr/java5_64/jre/bin/j9vm" -ljvm
> ld: 0706-003 Cannot find or read import file:
> /opt/pgsql91/bin/postgres/postgres.imp
> ld:accessx(): A parameter must be a directory.

> and, muh-oh... $PG_PREFIX/bin/postgres is the binary executable, so I
> *can't* put a copy of postgres.imp there, hah. and a bit of poking
> around the pljava makefiles, I'm not figuring out how its cooking that
> path name up?!?

Looks like it's coming from src/makefiles/Makefile.aix:

ifdef PGXS
BE_DLLLIBS= -Wl,-bI:$(bindir)/postgres/$(POSTGRES_IMP)
else
BE_DLLLIBS= -Wl,-bI:$(top_builddir)/src/backend/$(POSTGRES_IMP)
endif

I think the first case is just wrong (evidently never been tested
before). A look in src/backend/Makefile shows that the POSTGRES_IMP
file is actually installed into $(pkglibdir), so that's probably what
you want there instead of $(bindir)/postgres.

regards, tom lane

On 10/09/12 11:49 AM, John R Pierce wrote:
> $ find /opt/pgsql91 -name postgres.imp
> /opt/pgsql91/lib/postgres.imp

ah, and so I find the $PG_PREFIX/lib/pgxs/src/Makefile.port has that bad
reference to $(bindir)/postgres/postgres.imp and hack it like...

POSTGRES_IMP= postgres.imp

ifdef PGXS
BE_DLLLIBS= -Wl,-bI:$(libdir)/$(POSTGRES_IMP)
else
BE_DLLLIBS= -Wl,-bI:$(top_builddir)/src/backend/$(POSTGRES_IMP)
endif

....

and still no good. but this could be tossing the ball back to the
pljava side of things, except it does appear to be pgxs stuff still.

gmake[1]: Entering directory
`/home/postgres/src/pljava-1.4.3/build/objs'
/home/postgres/src/pljava-1.4.3/src/C/pljava/Makefile:232: warning:
overriding commands for target `pljava.so'
/opt/pgsql91/lib/pgxs/src/makefiles/../../src/Makefile.shlib:387:
warning: ignoring old commands for target `pljava.so'
/usr/bin/ar crs libpljava.a Backend.o Exception.o ExecutionPlan.o
Function.o HashMap.o Invocation.o Iterator.o JNICalls.o PgObject.o
PgSavepoint.o SPI.o SQLInputFromChunk.o SQLInputFromTuple.o
SQLOutputToChunk.o SQLOutputToTuple.o Session.o SubXactListener.o
XactListener.o backports.o type/AclId.o type/Any.o type/Array.o
type/BigDecimal.o type/Boolean.o type/Byte.o type/Coerce.o
type/Composite.o type/Date.o type/Double.o type/ErrorData.o
type/Float.o type/HeapTupleHeader.o type/Integer.o
type/JavaWrapper.o type/LargeObject.o type/Long.o type/Oid.o
type/Portal.o type/Relation.o type/Short.o type/String.o type/Time.o
type/Timestamp.o type/TriggerData.o type/Tuple.o type/TupleDesc.o
type/TupleTable.o type/Type.o type/UDT.o type/Void.o type/byte_array.o
touch libpljava.a
/opt/pgsql91/lib/pgxs/src/makefiles/../../src/backend/port/aix/mkldexport.sh
libpljava.a >libpljava.exp
/usr/vacpp/bin/xlc -q64 -I/home/postgres/src/include -O2 -qarch=pwr5
-qtune=balanced -qnoansialias -o pljava.so libpljava.a
-Wl,-bE:libpljava.exp -L/opt/pgsql91/lib -L/home/postgres/src/lib
-Wl,-blibpath:'/opt/pgsql91/lib:/opt/pgsql91/lib:/home/postgres/src/lib:/usr/lib:/lib'
-Wl,-bnoentry -Wl,-H512 -Wl,-bM:SRE
-Wl,-bI:/opt/pgsql91/lib/postgres.imp -L.
-L"/usr/java5_64/jre/bin/j9vm" -ljvm
ld: 0711-317 ERROR: Undefined symbol: .floor
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more
information.
gmake[1]: *** [libpljava.a] Error 8
gmake[1]: *** Deleting file `libpljava.a'
gmake[1]: Leaving directory `/home/postgres/src/pljava-1.4.3/build/objs'
gmake: *** [c_all] Error 2

<sound="bashing head into desk.mp3"/>

--
john r pierce N 37, W 122
santa cruz ca mid-left coast

On 10/09/12 12:11 PM, Tom Lane wrote:
> Looks like it's coming from src/makefiles/Makefile.aix:
>
> ifdef PGXS
> BE_DLLLIBS= -Wl,-bI:$(bindir)/postgres/$(POSTGRES_IMP)
> else
> BE_DLLLIBS= -Wl,-bI:$(top_builddir)/src/backend/$(POSTGRES_IMP)
> endif
>
> I think the first case is just wrong (evidently never been tested
> before). A look in src/backend/Makefile shows that the POSTGRES_IMP
> file is actually installed into $(pkglibdir), so that's probably what
> you want there instead of $(bindir)/postgres.

do I need to submit bug reports on this stuff, or is this discussion
good enough to get it into the "fix me" queue for eventual release?

btw, $pkglibdir didn't fare any better than $libdir, I'm stuck at...

$ ./makeit c_all
gmake[1]: Entering directory `/home/postgres/src/pljava-1.4.3/build/objs'
/home/postgres/src/pljava-1.4.3/src/C/pljava/Makefile:232: warning:
overriding commands for target `pljava.so'
/opt/pgsql91/lib/pgxs/src/makefiles/../../src/Makefile.shlib:387:
warning: ignoring old commands for target `pljava.so'
/usr/bin/ar crs libpljava.a Backend.o Exception.o ExecutionPlan.o
Function.o HashMap.o Invocation.o Iterator.o JNICalls.o PgObject.o
PgSavepoint.o SPI.o SQLInputFromChunk.o SQLInputFromTuple.o
SQLOutputToChunk.o SQLOutputToTuple.o Session.o SubXactListener.o
XactListener.o backports.o type/AclId.o type/Any.o type/Array.o
type/BigDecimal.o type/Boolean.o type/Byte.o type/Coerce.o
type/Composite.o type/Date.o type/Double.o type/ErrorData.o type/Float.o
type/HeapTupleHeader.o type/Integer.o type/JavaWrapper.o
type/LargeObject.o type/Long.o type/Oid.o type/Portal.o type/Relation.o
type/Short.o type/String.o type/Time.o type/Timestamp.o
type/TriggerData.o type/Tuple.o type/TupleDesc.o type/TupleTable.o
type/Type.o type/UDT.o type/Void.o type/byte_array.o
touch libpljava.a
/opt/pgsql91/lib/pgxs/src/makefiles/../../src/backend/port/aix/mkldexport.sh
libpljava.a >libpljava.exp
/usr/vacpp/bin/xlc -q64 -I/home/postgres/src/include -O2 -qarch=pwr5
-qtune=balanced -qnoansialias -o pljava.so libpljava.a
-Wl,-bE:libpljava.exp -L/opt/pgsql91/lib -L/home/postgres/src/lib
-Wl,-blibpath:'/opt/pgsql91/lib:/opt/pgsql91/lib:/home/postgres/src/lib:/usr/lib:/lib'
-Wl,-bnoentry -Wl,-H512 -Wl,-bM:SRE
-Wl,-bI:/opt/pgsql91/lib/postgres.imp -L. -L"/usr/java5_64/jre/bin/j9vm"
-ljvm
ld: 0711-317 ERROR: Undefined symbol: .floor
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more
information.
gmake[1]: *** [libpljava.a] Error 8
gmake[1]: *** Deleting file `libpljava.a'
gmake[1]: Leaving directory `/home/postgres/src/pljava-1.4.3/build/objs'
gmake: *** [c_all] Error 2

(makeit sets up the path and other environment stuff for the pljava
gmake... c_all tells pljava's make to just build the C side of things
and not the java side, which has its OWN problems far out of scope of
this list)

--
john r pierce N 37, W 122
santa cruz ca mid-left coast

John R Pierce <pierce(at)hogranch(dot)com> writes:
> btw, $pkglibdir didn't fare any better than $libdir, I'm stuck at...

> ld: 0711-317 ERROR: Undefined symbol: .floor

Well, you're making progress anyway. That one probably needs -lm
added to the command. Hard to tell if the fact that it's not there
already is our fault or pljava's. On my machine, if I go into say
contrib/cube and do "make", I see -lm in the link command --- do you?

regards, tom lane

On 10/09/12 1:45 PM, Tom Lane wrote:
> Well, you're making progress anyway. That one probably needs -lm
> added to the command. Hard to tell if the fact that it's not there
> already is our fault or pljava's. On my machine, if I go into say
> contrib/cube and do "make", I see -lm in the link command --- do you?

I do.

...
/usr/vacpp/bin/xlc -q64 -I/home/postgres/src/include -O2 -qarch=pwr5
-qtune=balanced -qnoansialias -o cube.so libcube.a
-Wl,-bE:libcube.exp -L../../src/port -L/home/postgres/src/lib
-Wl,-blibpath:'/opt/pgsql91/lib:/home/postgres/src/lib:/usr/lib:/lib' -Wl,-bnoentry
-Wl,-H512 -Wl,-bM:SRE -lm -Wl,-bI:../../src/backend/postgres.imp
...

Further, it appears the link command pljava is using for the AIX case is
given in its makefile as...

$(COMPILER) $(LDFLAGS_NO_L) $(LDFLAGS_SL) -o $(plugin) $<
-Wl,-bE:$(NAME)$(EXPSUFF) $(SHLIB_LINK)

I can't find anywhere LDFLAGS_NO_L is defined. however, SHLIB_LINK is
defined to concatenate PLJAVA_LDFLAGS, so I set that to -lm and POOF,
its built. scary!

--
john r pierce N 37, W 122
santa cruz ca mid-left coast

John R Pierce <pierce(at)hogranch(dot)com> writes:
> Further, it appears the link command pljava is using for the AIX case is
> given in its makefile as...

> $(COMPILER) $(LDFLAGS_NO_L) $(LDFLAGS_SL) -o $(plugin) $<
> -Wl,-bE:$(NAME)$(EXPSUFF) $(SHLIB_LINK)

> I can't find anywhere LDFLAGS_NO_L is defined. however, SHLIB_LINK is
> defined to concatenate PLJAVA_LDFLAGS, so I set that to -lm and POOF,
> its built. scary!

Well, the reason I mentioned contrib/cube is that it's known to need
libm. I see in its makefile

SHLIB_LINK += $(filter -lm, $(LIBS))

so apparently that's the de rigueur way to add libm when you need it.
I'd suggest pestering the pljava people to do likewise. They might be
getting away without this on more-forgiving platforms, but that doesn't
make it good practice to omit.

So if I've not lost track, the scorecard is:

1. We need to install mkldexport.sh when on AIX, so that pgxs builds can
use it.

2. Makefile.aix has the wrong idea about where to find postgres.imp when
in pgxs mode.

3. pljava needs -lm and isn't explicitly asking for it.

I will see about fixing the first two, but the third is on pljava to
fix.

(These aren't new bugs BTW --- it looks to me like this has been wrong
since the pgxs code was created, in 8.1. I guess we don't have many
AIX users :-()

regards, tom lane

I wrote:
> So if I've not lost track, the scorecard is:

> 1. We need to install mkldexport.sh when on AIX, so that pgxs builds can
> use it.

> 2. Makefile.aix has the wrong idea about where to find postgres.imp when
> in pgxs mode.

> 3. pljava needs -lm and isn't explicitly asking for it.

> I will see about fixing the first two, but the third is on pljava to
> fix.

I've committed a patch for the first two things:
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=bd0ef304f8a306522983f3b4b06274fdc45beed8
... but not having an AIX machine, I can't actually test it. Would
you verify it works?

regards, tom lane

On 10/09/12 6:09 PM, Tom Lane wrote:
> I wrote:
>> >So if I've not lost track, the scorecard is:
>> >1. We need to install mkldexport.sh when on AIX, so that pgxs builds can
>> >use it.
>> >2. Makefile.aix has the wrong idea about where to find postgres.imp when
>> >in pgxs mode.
>> >3. pljava needs -lm and isn't explicitly asking for it.
>> >I will see about fixing the first two, but the third is on pljava to
>> >fix.
> I've committed a patch for the first two things:
> http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=bd0ef304f8a306522983f3b4b06274fdc45beed8
> ... but not having an AIX machine, I can't actually test it. Would
> you verify it works?

hmm, i've not worked with git much. whats the best way I can verify
this? should I just run the patches against a 9.1.6 tarball or should
I attempt a full checkout from git?

> (These aren't new bugs BTW --- it looks to me like this has been wrong
> since the pgxs code was created, in 8.1. I guess we don't have many
> AIX users :-()

at least not AIX users using the PGXS infrastructure, we've been running
PG on AIX for a couple years now in house.

--
john r pierce N 37, W 122
santa cruz ca mid-left coast

John R Pierce <pierce(at)hogranch(dot)com> writes:
> On 10/09/12 6:09 PM, Tom Lane wrote:
>> I've committed a patch for the first two things:
>> http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=bd0ef304f8a306522983f3b4b06274fdc45beed8
>> ... but not having an AIX machine, I can't actually test it. Would
>> you verify it works?

> hmm, i've not worked with git much. whats the best way I can verify
> this? should I just run the patches against a 9.1.6 tarball or should
> I attempt a full checkout from git?

It's enough to apply the patch to a 9.1.6 tarball. If you go to that
page and click the "patch" button, you'll get a plain-text patch you
can feed to "patch -p1".

regards, tom lane