pltcl - cannot create 'normal' interpreter - Tcl_CreateSlave() fails - A solution

A follow-up of:
http://archives.postgresql.org/pgsql-cygwin/2003-11/msg00074.php
http://archives.postgresql.org/pgsql-cygwin/2003-01/msg00131.php
http://sources.redhat.com/ml/insight/2003-q1/msg00049.php
http://archives.postgresql.org/pgsql-interfaces/2003-04/msg00164.php

Here is a description of how I succeeded to run a
PL/Tcl script on Postgresql/Cygwin.

1. First, some words about what appends, and the
"failure stack":
When you try to run a PL/Tcl code, the operation fails
with the message:
"ERROR: pltcl: internal error - cannot create
'normal' interpreter"
This is due to the failure of:
-> pl/tcl/pltcl.c - pltcl_init_all()
-> tcl8.4.1/generic/tclInterp.c - Tcl_CreateSlave()
-> tcl8.4.1/win/tclWinInit.c - Tcl_Init()
-> Tcl_Eval(interp, initScript)
-> tcl8.4.1/generic/tclInitScript.h - error $msg -
Can't find a usable init.tcl ...

With some debugging trace, one can see that the
directory list to search for a init.tcl is empty.
Conclusion:
It is mandatory to feed the initialization step with
some directory paths.
Let's see how.

2. tclInitScript.h mentions 4 ways to specify the
directories.
- $tcl_library: this variable should be set after
Tcl_CreateInterp() and before Tcl_Init(),
so in tclInterp.c. As I don't want to touch the tcl
package, forget it.
- $env(TCL_LIBRARY): this way seems to solve the
problem, but another one appears (see 6. below).
- $tclDefaultLibrary: is set for the unix version
(tcl8.4.1/unix/tclUnixInit.c),
but not for the win version
(tcl8.4.1/win/tclWinInit.c). Forget it.
- $tcl_libPath: is set with the result of
TclGetLibraryPath().
Fine, but who is in charge to call
TclSetLibraryPath()?

Some investigation leads me to the following tree:
TclSetLibraryPath()
<- win/tclWinInit.c - TclpInitLibraryPath()
<- generic/tclEncoding.c - TclFindEncodings()
<- generic/tclEncoding.c - Tcl_FindExecutable(argv[0])

3. TclpInitLibraryPath() is a very interesting place,
indeed.
This is where the directory path is initialized.
Let's browse the categories:

a. installLib[2], "library", developLib:
What may be fine with tclsh.exe
(X:/cygwin/bin/tclsh.exe) is not with pgtclsh.exe or
postgres.exe
(X:/cygwin/usr/local/pgsql/bin/ path).
These options compose the following useless
directories:
X:/cygwin/usr/local/pgsql/share/tcl8.4
X:/cygwin/usr/local/pgsql/usr/share/tcl8.4
X:/cygwin/usr/local/share/tcl8.4
X:/cygwin/usr/local/pgsql/library
X:/cygwin/usr/local/library
X:/cygwin/usr/local/../tcl8.4.1/library
X:/cygwin/usr/../tcl8.4.1/library

b. Tcl_GetDefaultEncodingDir(): is empty; useless

c. AppendDllPath(): would be the perfect solution, but
is not yet adapted for cygwin standard
(see
http://cygwin.com/ml/cygwin/2004-01/msg00108.html)

d. AppendEnvironment(): have to be THE solution,
because there is no one else.
So TCL_LIBRARY have to be defined as an environment
variable.

4. For pgtclsh.exe or if you run postgres.exe
manually, you can do:
export TCL_LIBRARY=$(cygpath -w /usr/share/tcl8.4)

[Jason, you were on the right way with it]

5. But if you run postmaster as a windows Service, you
should define TCL_LIBRARY
as a system environment variable. Something like
"X:\cygwin\usr\share\tcl8.4".
Stop/Start postmaster is not enough: don't forget (as
I did) to reboot the machine,
as Windows loads the environment variables which are
available to services when
the machine is booted.

6. Add a call to Tcl_FindExecutable() in pltcl.c -
pltcl_init_all(),
just before the call to Tcl_CreateInterp().

It should be called with argv[0] as argument, but I
don't know how to get this piece of information for
pltcl.dll. Anyway, for Windows, it is unused (see
win/tclWinFile.c - TclpFindExecutable()). Just a value
!= NULL is OK, so I used an empty string.

[Jan, I'm only interested in cygwin platform. I
suggest you investigate deeper to make this
"workaround" cleaner, in order to be also compatible
with pure Unix platforms.]

Without this call, and with only the setting of the
environment variable, Tcl_CreateSlave() seems to fail
somewhere else, with this message:

--------------------------
server closed the connection unexpectedly
This probably means the server terminated
abnormally
before or while processing the request.
The connection to the server was lost. Attempting
reset: Succeeded.
--------------------------
I didn't spend time on this issue, just added the
call.

7. For the pltcl.dll to be built, I had to:

a. Permute the order of tcl.h and postgres.h.
See
http://archives.postgresql.org/pgsql-bugs/2003-12/msg00184.php

[Peter, Tom, I know you don't consider it as a
definitive fix, but meanwhile it worked for me.]

b. Add $(BE_DLLLIBS) to SHLIB_LINK in Makefile.
See
http://archives.postgresql.org/pgsql-bugs/2003-12/msg00183.php
Is fixed in version 7.4

This was suggested by Jason in his never-submitted
patch, see:

http://archives.postgresql.org/pgsql-cygwin/2003-01/msg00080.php

http://archives.postgresql.org/pgsql-cygwin/2003-11/msg00074.php

Just for completeness, here are the output of my build
and my patch:
--------------------------
Samson(at)pc2020 /opt/postgresql-7.3.5/src/pl/tcl
$ make
gcc -O2 -Wall -Wmissing-prototypes
-Wmissing-declarations -I../../../src/include -c -o
pltcl.o pltcl.c
In file included from
../../../src/include/pg_config.h:673,
from ../../../src/include/c.h:53,
from
../../../src/include/postgres.h:48,
from pltcl.c:43:
../../../src/include/pg_config_os.h:26:1: warning:
"DLLIMPORT" redefined
In file included from pltcl.c:41:
/usr/include/tcl.h:201:1: warning: this is the
location of the previous definition
pltcl.c: In function `pltcl_init_interp':
pltcl.c:270: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:272: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:274: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:276: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:279: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:281: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:283: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:285: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c: In function `pltcl_trigger_handler':
pltcl.c:827: warning: passing arg 4 of `Tcl_SplitList'
from incompatible pointer type
pltcl.c: In function `pltcl_SPI_prepare':
pltcl.c:1754: warning: passing arg 4 of
`Tcl_SplitList' from incompatible pointer type
pltcl.c: In function `pltcl_SPI_execp':
pltcl.c:2063: warning: passing arg 4 of
`Tcl_SplitList' from incompatible pointer type
dlltool --export-all --output-def pltcl.def pltcl.o
dllwrap -o pltcl.dll --dllname pltcl.dll --def
pltcl.def pltcl.o ../../../src/utils/dllinit.o
-L/usr/local/lib -L../../../src/backend -lpostgres
-L/usr/lib -ltcl84
dlltool --dllname pltcl.dll --def pltcl.def
--output-lib libpltcl.a
make -C modules all
make[1]: Entering directory
`/opt/postgresql-7.3.5/src/pl/tcl/modules'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory
`/opt/postgresql-7.3.5/src/pl/tcl/modules'
--------------------------

$ diff -u pltcl.orig pltcl.c
--- pltcl.orig 2003-10-30 03:00:44.000000000 +0100
+++ pltcl.c 2004-01-06 13:54:03.937500000 +0100
@@ -35,10 +35,13 @@
*

**********************************************************************/

-#include "postgres.h"
-
+//psa: change order of tcl.h and postgres.h
+//
http://archives.postgresql.org/pgsql-cygwin/2003-01/msg00080.php
+//
http://archives.postgresql.org/pgsql-cygwin/2003-11/msg00074.php
#include <tcl.h>

+#include "postgres.h"
+
#include <unistd.h>
#include <fcntl.h>
#include <setjmp.h>
@@ -207,6 +210,9 @@
* Create the dummy hold interpreter to
prevent close of
* stdout and stderr on DeleteInterp

************************************************************/
+//psa: argument should be argv[0]. But anyway, it is
unused on Windows: just have to be != NULL.
+Tcl_FindExecutable("");
+
if ((pltcl_hold_interp = Tcl_CreateInterp())
== NULL)
{
elog(ERROR, "pltcl: internal error -
cannot create 'hold' "
@@ -219,6 +225,8 @@
if ((pltcl_norm_interp =
Tcl_CreateSlave(pltcl_hold_interp,
"norm", 0)) == NULL)
{
+//psa: tell me more about the fail
+elog(LOG,Tcl_GetStringResult(pltcl_hold_interp));
elog(ERROR,
"pltcl: internal error - cannot
create 'normal' interpreter");
}
@@ -227,6 +235,8 @@
if ((pltcl_safe_interp =
Tcl_CreateSlave(pltcl_hold_interp,
"safe", 1)) == NULL)
{
+//psa: tell me more about the fail
+elog(LOG,Tcl_GetStringResult(pltcl_hold_interp));
elog(ERROR,
"pltcl: internal error -
cannot create 'safe' interpreter");
}

__________________________________
Do you Yahoo!?
Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes
http://hotjobs.sweepstakes.yahoo.com/signingbonus

Patrick,

On Wed, Jan 07, 2004 at 05:21:38AM -0800, Patrick Samson wrote:
> Here is a description of how I succeeded to run a PL/Tcl script on
> Postgresql/Cygwin.

Thanks for finding a solution *and* your hard work! Would you be
willing to go the extra mile and submit patches to Insight (see below)
and PostgreSQL to finally put this one to bed?

> c. AppendDllPath(): would be the perfect solution, but is not yet
> adapted for cygwin standard (see
> http://cygwin.com/ml/cygwin/2004-01/msg00108.html)

If this is the best way to fix the problem, then I strongly encourage
you to go this route. What Chris was trying to tell you in his terse
response is the following:

If you have problems with this version of tcltk PLEASE SEND BUG
REPORTS TO THE INSIGHT MAILING LIST at "insight at sources dot
redhat dot com". Then the insight maintainers can help rectify
these issues. They are familiar with cygwin but, for obvious
reasons, should not be forced to read the cygwin mailing list to
find tcltk/insight problems.

Hence, you may get a better response if you send your Cygwin Tcl patch
to <insight at sources dot redhat dot com>.

> d. AppendEnvironment(): have to be THE solution, because there is no
> one else.

See above...

> 6. Add a call to Tcl_FindExecutable() in pltcl.c - pltcl_init_all(),
> just before the call to Tcl_CreateInterp().

Will this be obviated if Insight accepts your patch?

> 7. For the pltcl.dll to be built, I had to:
>
> a. Permute the order of tcl.h and postgres.h. See
> http://archives.postgresql.org/pgsql-bugs/2003-12/msg00184.php
>
> [Peter, Tom, I know you don't consider it as a definitive fix, but
> meanwhile it worked for me.]

The above is an ugly issue -- unfortunately, I don't know of a good
solution. :,(

Thanks,
Jason

--
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D 8784 1AFD E4CC ECF4 8EF6

--- Jason Tishler <jason(at)tishler(dot)net> wrote:
> Patrick,
>
> On Wed, Jan 07, 2004 at 05:21:38AM -0800, Patrick
> Samson wrote:
> > Here is a description of how I succeeded to run a
> > PL/Tcl script on
> > Postgresql/Cygwin.
>
> Thanks for finding a solution *and* your hard work!
> Would you be
> willing to go the extra mile and submit patches to
> Insight (see below)
> and PostgreSQL to finally put this one to bed?

Done.
http://sources.redhat.com/ml/insight/2004-q1/msg00010.html
http://sources.redhat.com/ml/insight/2004-q1/msg00009.html
patch submitted to pgsql-patches (stalled)

> > 6. Add a call to Tcl_FindExecutable() in pltcl.c -
> > pltcl_init_all(),
> > just before the call to Tcl_CreateInterp().
>
> Will this be obviated if Insight accepts your patch?

I don't think so. The tcl patch is a way to allow
init.tcl to be found even if TCL_LIBRARY is not
defined. But as I wrote, without this call and only
the variable defined, init.tcl seems to be found but
Tcl_CreateSlave() fails further with another error
message.

__________________________________
Do you Yahoo!?
Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes
http://hotjobs.sweepstakes.yahoo.com/signingbonus