Re: [HACKERS] Patch to fix plpython on OS X

Attached is a plpython_error_1.out file that will fix cuckoo.
Unfortunately, I couldn't figure out how to submit it as an actual
patch, so it's just a file attachment. It should go in
src/pl/plpython/expected.
--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

"Jim C. Nasby" <decibel(at)decibel(dot)org> writes:
> Attached is a plpython_error_1.out file that will fix cuckoo.

What is the reason for the difference in the error message spelling
in the first place? Is this a Python version thing (and if so,
which version is newer --- that should have pride of place as
plpython_error.out I should think)? Or is there some other reason
that we need to understand more closely instead of just slapping on
a band-aid?

regards, tom lane

On Tue, Jul 19, 2005 at 10:03:39AM -0400, Tom Lane wrote:
> "Jim C. Nasby" <decibel(at)decibel(dot)org> writes:
> > Attached is a plpython_error_1.out file that will fix cuckoo.
>
> What is the reason for the difference in the error message spelling
> in the first place? Is this a Python version thing (and if so,
> which version is newer --- that should have pride of place as
> plpython_error.out I should think)? Or is there some other reason
> that we need to understand more closely instead of just slapping on
> a band-aid?

I don't think it's a version issue; cuckoo is at 2.4, platypus used to
be at 2.3 but I upgraded it to 2.4 to see if that was the issue, but
platypus kept working.
--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

"Jim C. Nasby" <decibel(at)decibel(dot)org> writes:
> On Tue, Jul 19, 2005 at 10:03:39AM -0400, Tom Lane wrote:
>> "Jim C. Nasby" <decibel(at)decibel(dot)org> writes:
>>> Attached is a plpython_error_1.out file that will fix cuckoo.
>>
>> What is the reason for the difference in the error message spelling
>> in the first place? Is this a Python version thing (and if so,
>> which version is newer --- that should have pride of place as
>> plpython_error.out I should think)? Or is there some other reason
>> that we need to understand more closely instead of just slapping on
>> a band-aid?

> I don't think it's a version issue; cuckoo is at 2.4, platypus used to
> be at 2.3 but I upgraded it to 2.4 to see if that was the issue, but
> platypus kept working.

Hmm ... if it's *not* a version thing then I really do want to know
what's causing it. Anyone have an idea why this machine is saying
'\u80' where everyone else's python says u'\x80' ?

*** ./expected/plpython_error.out Mon Jul 18 22:06:49 2005
--- ./results/plpython_error.out Mon Jul 18 23:53:30 2005
***************
*** 24,38 ****
--
SELECT unicode_return_error();
ERROR: plpython: function "unicode_return_error" could not create return value
! DETAIL: exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\x80' in position 0: ordinal not in range(128)
INSERT INTO unicode_test (testvalue) VALUES ('test');
ERROR: plpython: function "unicode_trigger_error" could not modify tuple
! DETAIL: exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\x80' in position 0: ordinal not in range(128)
SELECT unicode_plan_error1();
WARNING: plpython: in function unicode_plan_error1:
DETAIL: plpy.Error: Unknown error in PLy_spi_execute_plan
ERROR: plpython: function "unicode_plan_error1" could not execute plan
! DETAIL: exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\x80' in position 0: ordinal not in range(128)
SELECT unicode_plan_error2();
ERROR: plpython: function "unicode_plan_error2" could not execute plan
! DETAIL: exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\x80' in position 0: ordinal not in range(128)
--- 24,38 ----
--
SELECT unicode_return_error();
ERROR: plpython: function "unicode_return_error" could not create return value
! DETAIL: exceptions.UnicodeEncodeError: 'ascii' codec can't encode character '\u80' in position 0: ordinal not in range(128)
INSERT INTO unicode_test (testvalue) VALUES ('test');
ERROR: plpython: function "unicode_trigger_error" could not modify tuple
! DETAIL: exceptions.UnicodeEncodeError: 'ascii' codec can't encode character '\u80' in position 0: ordinal not in range(128)
SELECT unicode_plan_error1();
WARNING: plpython: in function unicode_plan_error1:
DETAIL: plpy.Error: Unknown error in PLy_spi_execute_plan
ERROR: plpython: function "unicode_plan_error1" could not execute plan
! DETAIL: exceptions.UnicodeEncodeError: 'ascii' codec can't encode character '\u80' in position 0: ordinal not in range(128)
SELECT unicode_plan_error2();
ERROR: plpython: function "unicode_plan_error2" could not execute plan
! DETAIL: exceptions.UnicodeEncodeError: 'ascii' codec can't encode character '\u80' in position 0: ordinal not in range(128)

regards, tom lane

On Tue, Jul 19, 2005 at 02:48:52PM -0400, Tom Lane wrote:
> "Jim C. Nasby" <decibel(at)decibel(dot)org> writes:
> > I don't think it's a version issue; cuckoo is at 2.4, platypus used to
> > be at 2.3 but I upgraded it to 2.4 to see if that was the issue, but
> > platypus kept working.
>
> Hmm ... if it's *not* a version thing then I really do want to know
> what's causing it. Anyone have an idea why this machine is saying
> '\u80' where everyone else's python says u'\x80' ?

Is it possible that plpython.so is linked against an old version
of libpython? I see that the error message changed a few years ago:

http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Python/exceptions.c?r1=1.44&r2=1.45
http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Python/exceptions.c?r1=1.45&r2=1.46

As I recall, Python must be configured with --enable-shared or you
don't get a shared version of libpython, so if you installed a new
Python but not a new version of libpython.*.so, then plpython.so
might be linked against an old version.

Does this machine have ldd or the equivalent? If so, can you compare
"ldd /path/to/python" and "ldd /path/to/plpython.so"?

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

On Tue, Jul 19, 2005 at 01:54:00PM -0600, Michael Fuhr wrote:
> On Tue, Jul 19, 2005 at 02:48:52PM -0400, Tom Lane wrote:
> > "Jim C. Nasby" <decibel(at)decibel(dot)org> writes:
> > > I don't think it's a version issue; cuckoo is at 2.4, platypus used to
> > > be at 2.3 but I upgraded it to 2.4 to see if that was the issue, but
> > > platypus kept working.
> >
> > Hmm ... if it's *not* a version thing then I really do want to know
> > what's causing it. Anyone have an idea why this machine is saying
> > '\u80' where everyone else's python says u'\x80' ?
>
> Is it possible that plpython.so is linked against an old version
> of libpython? I see that the error message changed a few years ago:
>
> http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Python/exceptions.c?r1=1.44&r2=1.45
> http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Python/exceptions.c?r1=1.45&r2=1.46
>
> As I recall, Python must be configured with --enable-shared or you
> don't get a shared version of libpython, so if you installed a new
> Python but not a new version of libpython.*.so, then plpython.so
> might be linked against an old version.
>
> Does this machine have ldd or the equivalent? If so, can you compare
> "ldd /path/to/python" and "ldd /path/to/plpython.so"?

Oddly, no, it doesn't seem to have ldd. And the buildfarm script seems
to clean everything up even in the pgsqlkeep directories; or at least I
couldn't find a plpython.so laying around.
--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

On Tue, Jul 19, 2005 at 03:11:35PM -0500, Jim C. Nasby wrote:
> On Tue, Jul 19, 2005 at 01:54:00PM -0600, Michael Fuhr wrote:
> > Does this machine have ldd or the equivalent? If so, can you compare
> > "ldd /path/to/python" and "ldd /path/to/plpython.so"?
>
> Oddly, no, it doesn't seem to have ldd. And the buildfarm script seems
> to clean everything up even in the pgsqlkeep directories; or at least I
> couldn't find a plpython.so laying around.

[googles]

"otool -L" appears to be the Darwin equivalent of ldd. If you can
manage to find a plpython.so then it would be interesting to see
which libpython it's linked against.

Can you search the system for all files named libpython* and post
what you find?

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

Jim C. Nasby wrote:

>And the buildfarm script seems
>to clean everything up even in the pgsqlkeep directories; or at least I
>couldn't find a plpython.so laying around.
>
>

Nothing should be removed. If you are using the experimental code I
recently gave you all bets are off, but under normal circumstances if
you run with --keepall then your plpython.so should be there.

cheers

andrew

On Tue, Jul 19, 2005 at 02:42:07PM -0600, Michael Fuhr wrote:
> On Tue, Jul 19, 2005 at 03:11:35PM -0500, Jim C. Nasby wrote:
> > On Tue, Jul 19, 2005 at 01:54:00PM -0600, Michael Fuhr wrote:
> > > Does this machine have ldd or the equivalent? If so, can you compare
> > > "ldd /path/to/python" and "ldd /path/to/plpython.so"?
> >
> > Oddly, no, it doesn't seem to have ldd. And the buildfarm script seems
> > to clean everything up even in the pgsqlkeep directories; or at least I
> > couldn't find a plpython.so laying around.
>
> [googles]
>
> "otool -L" appears to be the Darwin equivalent of ldd. If you can
> manage to find a plpython.so then it would be interesting to see
> which libpython it's linked against.

I'm going to run a build with the non-experimental version of run_build.pl and
see if I'll have some files left then.

> Can you search the system for all files named libpython* and post
> what you find?

buildfar(at)phonebook(dot)1[16:42]~:11%locate libpython
/Applications/NeoOfficeJ.app/Contents/MacOS/libpython.dylib
/Applications/NeoOfficeJ.app/Contents/MacOS/libpython2.2.dylib
/Applications/NeoOfficeJ.app/Contents/MacOS/libpython2.dylib
/Applications/OpenOffice.org1.1.2/program/libpython.dylib
/Applications/OpenOffice.org1.1.2/program/libpython2.2.dylib
/Applications/OpenOffice.org1.1.2/program/libpython2.dylib
/opt/local/lib/libpython2.4.dylib
/opt/local/lib/python2.4/config/libpython2.4.a
/opt/local/var/db/dports/software/python24/2.4_3/opt/local/lib/libpython2.4.dylib
/opt/local/var/db/dports/software/python24/2.4_3/opt/local/lib/python2.4/config/libpython2.4.a
buildfar(at)phonebook(dot)1[16:42]~:12%

--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

On Tue, Jul 19, 2005 at 04:51:03PM -0500, Jim C. Nasby wrote:
> > Can you search the system for all files named libpython* and post
> > what you find?
>
> buildfar(at)phonebook(dot)1[16:42]~:11%locate libpython
> /Applications/NeoOfficeJ.app/Contents/MacOS/libpython.dylib
> /Applications/NeoOfficeJ.app/Contents/MacOS/libpython2.2.dylib
> /Applications/NeoOfficeJ.app/Contents/MacOS/libpython2.dylib
> /Applications/OpenOffice.org1.1.2/program/libpython.dylib
> /Applications/OpenOffice.org1.1.2/program/libpython2.2.dylib
> /Applications/OpenOffice.org1.1.2/program/libpython2.dylib
> /opt/local/lib/libpython2.4.dylib
> /opt/local/lib/python2.4/config/libpython2.4.a
> /opt/local/var/db/dports/software/python24/2.4_3/opt/local/lib/libpython2.4.dylib
> /opt/local/var/db/dports/software/python24/2.4_3/opt/local/lib/python2.4/config/libpython2.4.a
> buildfar(at)phonebook(dot)1[16:42]~:12%

buildfar(at)phonebook(dot)1[18:00]~/buildfarm/HEAD/pgsqlkeep.1121809875/src/pl/plpython:41%otool -L libplpython.0.0.so
libplpython.0.0.so:
/System/Library/Frameworks/Python.framework/Versions/2.3/Python (compatibility version 2.3.0, current version 2.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 71.1.3)
buildfar(at)phonebook(dot)1[18:00]~/buildfarm/HEAD/pgsqlkeep.1121809875/src/pl/plpython:42%

buildfar(at)phonebook(dot)1[18:01]~/buildfarm/HEAD/lastrun-logs:12%grep -i python configure.log make.log
configure.log:checking whether to build Python modules... yes
configure.log:checking for python... /opt/local/bin/python
configure.log:checking for Python distutils module... yes
configure.log:checking Python configuration directory... /opt/local/lib/python2.4/config
configure.log:checking how to link an embedded Python application... -L/opt/local/lib/python2.4/config -lpython2.4 -ldl
make.log:ccache gcc -no-cpp-precomp -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels -fno-strict-aliasing -g -DFRONTEND -I../../../src/interfaces/libpq -DVAL_CONFIGURE="\"'--with-includes=/opt/local/include' '--with-libraries=/opt/local/lib' '--enable-cassert' '--enable-debug' '--enable-integer-datetimes' '--with-perl' '--with-python' '--with-tcl' '--with-openssl' '--enable-depend' '--enable-nls' '--prefix=/Users/buildfarm/buildfarm/HEAD/inst' '--with-pgport=5678' 'CC=ccache gcc'\"" -I../../../src/include -I/opt/local/include -c -o pg_config.o pg_config.c -MMD
make.log:ccache gcc -no-cpp-precomp -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels -fno-strict-aliasing -g -I. -I/opt/local/include/python2.4 -I../../../src/include -I/opt/local/include -c -o plpython.o plpython.c -MMD
make.log:In file included from /opt/local/include/python2.4/Python.h:77,
make.log: from plpython.c:37:
make.log:/opt/local/include/python2.4/objimpl.h:255: warning: use of `long double' type; its size may change in a future release
make.log:/opt/local/include/python2.4/objimpl.h:255: warning: (Long double usage is reported only once for each file.
make.log:/opt/local/include/python2.4/objimpl.h:255: warning: To disable this warning, use -Wno-long-double.)
make.log:ar crs libplpython.a `lorder plpython.o | tsort`
make.log:ranlib libplpython.a
make.log:ccache gcc -no-cpp-precomp -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels -fno-strict-aliasing -g -bundle plpython.o -L../../../src/port -L/opt/local/lib -bundle_loader ../../../src/backend/postgres -framework Python -o libplpython.0.0.so
make.log:rm -f libplpython.0.so
make.log:ln -s libplpython.0.0.so libplpython.0.so
make.log:rm -f libplpython.so
make.log:ln -s libplpython.0.0.so libplpython.so
buildfar(at)phonebook(dot)1[18:02]~/buildfarm/HEAD/lastrun-logs:13%

Neither /opt/local/lib/libpython2.4.dylib or
/opt/local/lib/python2.4/config/libpython2.4.a reference the /System/Library
python, so I have no idea how it's finding it's way to that...

buildfar(at)phonebook(dot)1[18:03]~/buildfarm/HEAD/lastrun-logs:13%otool -L /opt/local/lib/libpython2.4.dylib
/opt/local/lib/libpython2.4.dylib:
/opt/local/lib/libpython2.4.dylib (compatibility version 2.4.0, current version 2.4.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 71.1.1)
buildfar(at)phonebook(dot)1[18:03]~/buildfarm/HEAD/lastrun-logs:14%otool -L /opt/local/lib/python2.4/config/libpython2.4.a
Archive : /opt/local/lib/python2.4/config/libpython2.4.a
/opt/local/lib/python2.4/config/libpython2.4.a(getbuildinfo.o):
/opt/local/lib/python2.4/config/libpython2.4.a(acceler.o):
/opt/local/lib/python2.4/config/libpython2.4.a(grammar1.o):
/opt/local/lib/python2.4/config/libpython2.4.a(listnode.o):
/opt/local/lib/python2.4/config/libpython2.4.a(node.o):
/opt/local/lib/python2.4/config/libpython2.4.a(parser.o):
/opt/local/lib/python2.4/config/libpython2.4.a(parsetok.o):
/opt/local/lib/python2.4/config/libpython2.4.a(bitset.o):
/opt/local/lib/python2.4/config/libpython2.4.a(metagrammar.o):
/opt/local/lib/python2.4/config/libpython2.4.a(firstsets.o):
/opt/local/lib/python2.4/config/libpython2.4.a(grammar.o):
/opt/local/lib/python2.4/config/libpython2.4.a(pgen.o):
/opt/local/lib/python2.4/config/libpython2.4.a(myreadline.o):
/opt/local/lib/python2.4/config/libpython2.4.a(tokenizer.o):
/opt/local/lib/python2.4/config/libpython2.4.a(abstract.o):
/opt/local/lib/python2.4/config/libpython2.4.a(boolobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(bufferobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(cellobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(classobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(cobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(complexobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(descrobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(enumobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(genobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(fileobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(floatobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(frameobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(funcobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(intobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(iterobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(listobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(longobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(dictobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(methodobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(moduleobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(object.o):
/opt/local/lib/python2.4/config/libpython2.4.a(obmalloc.o):
/opt/local/lib/python2.4/config/libpython2.4.a(rangeobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(setobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(sliceobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(stringobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(structseq.o):
/opt/local/lib/python2.4/config/libpython2.4.a(tupleobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(typeobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(weakrefobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(unicodeobject.o):
/opt/local/lib/python2.4/config/libpython2.4.a(unicodectype.o):
/opt/local/lib/python2.4/config/libpython2.4.a(bltinmodule.o):
/opt/local/lib/python2.4/config/libpython2.4.a(exceptions.o):
/opt/local/lib/python2.4/config/libpython2.4.a(ceval.o):
/opt/local/lib/python2.4/config/libpython2.4.a(compile.o):
/opt/local/lib/python2.4/config/libpython2.4.a(codecs.o):
/opt/local/lib/python2.4/config/libpython2.4.a(errors.o):
/opt/local/lib/python2.4/config/libpython2.4.a(frozen.o):
/opt/local/lib/python2.4/config/libpython2.4.a(frozenmain.o):
/opt/local/lib/python2.4/config/libpython2.4.a(future.o):
/opt/local/lib/python2.4/config/libpython2.4.a(getargs.o):
/opt/local/lib/python2.4/config/libpython2.4.a(getcompiler.o):
/opt/local/lib/python2.4/config/libpython2.4.a(getcopyright.o):
/opt/local/lib/python2.4/config/libpython2.4.a(getmtime.o):
/opt/local/lib/python2.4/config/libpython2.4.a(getplatform.o):
/opt/local/lib/python2.4/config/libpython2.4.a(getversion.o):
/opt/local/lib/python2.4/config/libpython2.4.a(graminit.o):
/opt/local/lib/python2.4/config/libpython2.4.a(import.o):
/opt/local/lib/python2.4/config/libpython2.4.a(importdl.o):
/opt/local/lib/python2.4/config/libpython2.4.a(marshal.o):
/opt/local/lib/python2.4/config/libpython2.4.a(modsupport.o):
/opt/local/lib/python2.4/config/libpython2.4.a(mystrtoul.o):
/opt/local/lib/python2.4/config/libpython2.4.a(mysnprintf.o):
/opt/local/lib/python2.4/config/libpython2.4.a(pyfpe.o):
/opt/local/lib/python2.4/config/libpython2.4.a(pystate.o):
/opt/local/lib/python2.4/config/libpython2.4.a(pythonrun.o):
/opt/local/lib/python2.4/config/libpython2.4.a(structmember.o):
/opt/local/lib/python2.4/config/libpython2.4.a(symtable.o):
/opt/local/lib/python2.4/config/libpython2.4.a(sysmodule.o):
/opt/local/lib/python2.4/config/libpython2.4.a(traceback.o):
/opt/local/lib/python2.4/config/libpython2.4.a(getopt.o):
/opt/local/lib/python2.4/config/libpython2.4.a(pystrtod.o):
/opt/local/lib/python2.4/config/libpython2.4.a(dynload_next.o):
/opt/local/lib/python2.4/config/libpython2.4.a(mactoolboxglue.o):
/opt/local/lib/python2.4/config/libpython2.4.a(thread.o):
/opt/local/lib/python2.4/config/libpython2.4.a(config.o):
/opt/local/lib/python2.4/config/libpython2.4.a(getpath.o):
/opt/local/lib/python2.4/config/libpython2.4.a(main.o):
/opt/local/lib/python2.4/config/libpython2.4.a(gcmodule.o):
/opt/local/lib/python2.4/config/libpython2.4.a(threadmodule.o):
/opt/local/lib/python2.4/config/libpython2.4.a(signalmodule.o):
/opt/local/lib/python2.4/config/libpython2.4.a(posixmodule.o):
/opt/local/lib/python2.4/config/libpython2.4.a(errnomodule.o):
/opt/local/lib/python2.4/config/libpython2.4.a(_sre.o):
/opt/local/lib/python2.4/config/libpython2.4.a(_codecsmodule.o):
/opt/local/lib/python2.4/config/libpython2.4.a(zipimport.o):
/opt/local/lib/python2.4/config/libpython2.4.a(symtablemodule.o):
/opt/local/lib/python2.4/config/libpython2.4.a(xxsubtype.o):
buildfar(at)phonebook(dot)1[18:04]~/buildfarm/HEAD/lastrun-logs:15%

--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

On Tue, Jul 19, 2005 at 06:06:00PM -0500, Jim C. Nasby wrote:
> buildfar(at)phonebook(dot)1[18:00]~/buildfarm/HEAD/pgsqlkeep.1121809875/src/pl/plpython:41%otool -L libplpython.0.0.so
> libplpython.0.0.so:
> /System/Library/Frameworks/Python.framework/Versions/2.3/Python (compatibility version 2.3.0, current version 2.3.0)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 71.1.3)

If that first object has something to do with Python 2.3 then we
might have found the culprit. But how'd you get that?

> configure.log:checking Python configuration directory... /opt/local/lib/python2.4/config
> configure.log:checking how to link an embedded Python application... -L/opt/local/lib/python2.4/config -lpython2.4 -ldl

The above looks reasonable.

> make.log:ccache gcc -no-cpp-precomp -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels -fno-strict-aliasing -g -bundle plpython.o -L../../../src/port -L/opt/local/lib -bundle_loader ../../../src/backend/postgres -framework Python -o libplpython.0.0.so

Hmmm...what's that "-framework Python" business? Looks mighty
suspicious in light of the otool output.

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

On Tue, Jul 19, 2005 at 05:47:37PM -0600, Michael Fuhr wrote:
> On Tue, Jul 19, 2005 at 06:06:00PM -0500, Jim C. Nasby wrote:
> > buildfar(at)phonebook(dot)1[18:00]~/buildfarm/HEAD/pgsqlkeep.1121809875/src/pl/plpython:41%otool -L libplpython.0.0.so
> > libplpython.0.0.so:
> > /System/Library/Frameworks/Python.framework/Versions/2.3/Python (compatibility version 2.3.0, current version 2.3.0)
> > /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 71.1.3)
>
> If that first object has something to do with Python 2.3 then we
> might have found the culprit. But how'd you get that?

No idea at all. Is there some way to find out from how it was built?

> > configure.log:checking Python configuration directory... /opt/local/lib/python2.4/config
> > configure.log:checking how to link an embedded Python application... -L/opt/local/lib/python2.4/config -lpython2.4 -ldl
>
> The above looks reasonable.
>
> > make.log:ccache gcc -no-cpp-precomp -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels -fno-strict-aliasing -g -bundle plpython.o -L../../../src/port -L/opt/local/lib -bundle_loader ../../../src/backend/postgres -framework Python -o libplpython.0.0.so
>
> Hmmm...what's that "-framework Python" business? Looks mighty
> suspicious in light of the otool output.

Again, no idea. IANAC (I am not a coder) :)
--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

Tom Lane wrote:

>"Jim C. Nasby" <decibel(at)decibel(dot)org> writes:
>
>
>>On Tue, Jul 19, 2005 at 10:03:39AM -0400, Tom Lane wrote:
>>
>>
>>>"Jim C. Nasby" <decibel(at)decibel(dot)org> writes:
>>>
>>>
>>>>Attached is a plpython_error_1.out file that will fix cuckoo.
>>>>
>>>>
>>>What is the reason for the difference in the error message spelling
>>>in the first place? Is this a Python version thing (and if so,
>>>which version is newer --- that should have pride of place as
>>>plpython_error.out I should think)? Or is there some other reason
>>>that we need to understand more closely instead of just slapping on
>>>a band-aid?
>>>
>>>
>
>
>
>>I don't think it's a version issue; cuckoo is at 2.4, platypus used to
>>be at 2.3 but I upgraded it to 2.4 to see if that was the issue, but
>>platypus kept working.
>>
>>
>
>Hmm ... if it's *not* a version thing then I really do want to know
>what's causing it. Anyone have an idea why this machine is saying
>'\u80' where everyone else's python says u'\x80' ?
>
>
>
>
>

Another OSX box on buildfarm, wallaroo, is exhibiting the same
behaviour, albeit currently masked by interval regression failures.

cheers

andrew

On Sat, Jul 23, 2005 at 07:58:21PM -0400, Andrew Dunstan wrote:
> Tom Lane wrote:
> >"Jim C. Nasby" <decibel(at)decibel(dot)org> writes:
> >>I don't think it's a version issue; cuckoo is at 2.4, platypus used to
> >>be at 2.3 but I upgraded it to 2.4 to see if that was the issue, but
> >>platypus kept working.
> >
> >Hmm ... if it's *not* a version thing then I really do want to know
> >what's causing it. Anyone have an idea why this machine is saying
> >'\u80' where everyone else's python says u'\x80' ?
>
> Another OSX box on buildfarm, wallaroo, is exhibiting the same
> behaviour, albeit currently masked by interval regression failures.

I suspect this is indeed a Python version issue:

http://archives.postgresql.org/pgsql-hackers/2005-07/msg00669.php
http://archives.postgresql.org/pgsql-hackers/2005-07/msg00684.php

It looks like the Macs have some kind of Python framework that
PL/Python is linking against even if a newer version of Python has
been installed. Unfortunately I don't have a Mac I could use to
do any deeper investigating.

The regression tests that are failing are from the patch I submitted
about a month ago to fix a core dump in PL/Python:

http://archives.postgresql.org/pgsql-patches/2005-06/msg00519.php

The tests exercise the error checking that the patch added, doing
things that previously caused a segmentation fault but that now
raise an exception. Should those tests remain in place? If so,
should we rewrite them to avoid the version-specific Python messages
(possibly by wrapping them in a PL/pgSQL function that traps the
errors), or should we just leave the tests alone now that we think
we understand what's happening?

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

Michael Fuhr <mike(at)fuhr(dot)org> writes:
>> Tom Lane wrote:
>>> Hmm ... if it's *not* a version thing then I really do want to know
>>> what's causing it. Anyone have an idea why this machine is saying
>>> '\u80' where everyone else's python says u'\x80' ?

> The regression tests that are failing are from the patch I submitted
> about a month ago to fix a core dump in PL/Python:

> http://archives.postgresql.org/pgsql-patches/2005-06/msg00519.php

> The tests exercise the error checking that the patch added, doing
> things that previously caused a segmentation fault but that now
> raise an exception. Should those tests remain in place? If so,
> should we rewrite them to avoid the version-specific Python messages
> (possibly by wrapping them in a PL/pgSQL function that traps the
> errors), or should we just leave the tests alone now that we think
> we understand what's happening?

Well, if it is just a Python version issue then all we need do is add
a variant expected-output file to match. I was just expressing a
desire to know that for sure before we wallpaper over the symptom...

regards, tom lane

On Sat, Jul 23, 2005 at 10:38:59PM -0400, Tom Lane wrote:
> Well, if it is just a Python version issue then all we need do is add
> a variant expected-output file to match. I was just expressing a
> desire to know that for sure before we wallpaper over the symptom...

I just built Python 2.3 and it does indeed format the error differently
than later versions (the format appears to have changed in 2.3.1):

% python2.3
Python 2.3 (#1, Jul 24 2005, 06:18:30)
[GCC 3.4.2] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> str(u'\x80')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character '\u80' in position 0: ordinal not in range(128)

% python2.4
Python 2.4.1 (#1, Apr 6 2005, 09:52:02)
[GCC 3.4.2] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> str(u'\x80')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character u'\x80' in position 0: ordinal not in range(128)

One could check the version of Python that PL/Python is using with
the following function (assuming that Python isn't so broken that
it would use the core of one version but find modules from another):

CREATE FUNCTION pyversion() RETURNS text AS $$
import sys
return sys.version
$$ LANGUAGE plpythonu;

I've attached two new files that should go in the plpython directory:

resultmap
expected/plpython_error_py23.out

A problem with this patch is that it assumes a version of Python
based on the OS, which might clean up the current buildfarm but
that isn't really correct. Is there a better way to handle this?

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

Michael Fuhr said:

> I just built Python 2.3 and it does indeed format the error differently
> than later versions (the format appears to have changed in 2.3.1):
>
[snip]
> I've attached two new files that should go in the plpython directory:
>
> resultmap
> expected/plpython_error_py23.out
>
> A problem with this patch is that it assumes a version of Python
> based on the OS, which might clean up the current buildfarm but
> that isn't really correct. Is there a better way to handle this?

This is completely unnecessary - pg_regress has an alternative result
mechanism that doesn't rely on a resultmap file. Just name your alternative
result file foo_n.out instead of foo.out, for some n in [0-9]. In this case,
call it, say, plpython_error_1.out. Job done, and no OS dependence.

cheers

andrew

Michael Fuhr <mike(at)fuhr(dot)org> writes:
> A problem with this patch is that it assumes a version of Python
> based on the OS, which might clean up the current buildfarm but
> that isn't really correct. Is there a better way to handle this?

Yes --- just let pg_regress deal with it as if it were a locale
problem. I've committed it that way.

regards, tom lane

On Sun, Jul 24, 2005 at 08:40:42AM -0500, Andrew Dunstan wrote:
> This is completely unnecessary - pg_regress has an alternative result
> mechanism that doesn't rely on a resultmap file. Just name your alternative
> result file foo_n.out instead of foo.out, for some n in [0-9]. In this case,
> call it, say, plpython_error_1.out. Job done, and no OS dependence.

Thanks -- I overlooked that in src/test/regress/README.

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

Michael Fuhr wrote:

>On Sun, Jul 24, 2005 at 08:40:42AM -0500, Andrew Dunstan wrote:
>
>
>>This is completely unnecessary - pg_regress has an alternative result
>>mechanism that doesn't rely on a resultmap file. Just name your alternative
>>result file foo_n.out instead of foo.out, for some n in [0-9]. In this case,
>>call it, say, plpython_error_1.out. Job done, and no OS dependence.
>>
>>
>
>Thanks -- I overlooked that in src/test/regress/README.
>
>
>

We should probably generalise that section of the README a bit. People
might skip over it thinking "this isn't a locale difference".

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Michael Fuhr wrote:
>> Thanks -- I overlooked that in src/test/regress/README.

> We should probably generalise that section of the README a bit. People
> might skip over it thinking "this isn't a locale difference".

I'm wondering why we still have a README there at all --- it's entirely
superseded by the SGML documentation.

http://developer.postgresql.org/docs/postgres/regress-evaluation.html

regards, tom lane

On Sun, Jul 24, 2005 at 10:55:08AM -0400, Tom Lane wrote:
> Michael Fuhr <mike(at)fuhr(dot)org> writes:
> > A problem with this patch is that it assumes a version of Python
> > based on the OS, which might clean up the current buildfarm but
> > that isn't really correct. Is there a better way to handle this?
>
> Yes --- just let pg_regress deal with it as if it were a locale
> problem. I've committed it that way.
>
> regards, tom lane
>
FYI, cuckoo went green with this build:
http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=cuckoo&dt=2005-07-25%2008:05:02
--
Jim C. Nasby, Database Consultant decibel(at)decibel(dot)org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

Am Sonntag, 24. Juli 2005 17:53 schrieb Tom Lane:
> I'm wondering why we still have a README there at all --- it's entirely
> superseded by the SGML documentation.
>
> http://developer.postgresql.org/docs/postgres/regress-evaluation.html

I think we kept it there so people can read it during the installation.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Am Sonntag, 24. Juli 2005 17:53 schrieb Tom Lane:
>> I'm wondering why we still have a README there at all --- it's entirely
>> superseded by the SGML documentation.
>>
>> http://developer.postgresql.org/docs/postgres/regress-evaluation.html

> I think we kept it there so people can read it during the installation.

Yeah. I desisted from deleting it after I noticed that there are
provisions for re-generating it over in the doc/src/sgml Makefile.
However, I'm now wondering why it's not handled exactly like INSTALL
--- ie, don't keep it in CVS, but auto-generate it during tarball build.
The current manual procedure definitely isn't keeping it up to date.

regards, tom lane

Am Freitag, 29. Juli 2005 16:34 schrieb Tom Lane:
> Yeah. I desisted from deleting it after I noticed that there are
> provisions for re-generating it over in the doc/src/sgml Makefile.
> However, I'm now wondering why it's not handled exactly like INSTALL
> --- ie, don't keep it in CVS, but auto-generate it during tarball build.
> The current manual procedure definitely isn't keeping it up to date.

I don't see where the INSTALL file is generated either and put in place
either. I suppose this is somewhere in the release building scripts. It
should probably be done in the makefiles.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/