pltcl - cannot create 'normal' interpreter - Tcl_CreateSlave() fails - A solution

A follow-up of:
http://archives.postgresql.org/pgsql-cygwin/2003-11/msg00074.php
http://archives.postgresql.org/pgsql-cygwin/2003-01/msg00131.php
http://sources.redhat.com/ml/insight/2003-q1/msg00049.php
http://archives.postgresql.org/pgsql-interfaces/2003-04/msg00164.php

Here is a description of how I succeeded to run a
PL/Tcl script on Postgresql/Cygwin.

1. First, some words about what appends, and the
"failure stack":
When you try to run a PL/Tcl code, the operation fails
with the message:
"ERROR: pltcl: internal error - cannot create
'normal' interpreter"
This is due to the failure of:
-> pl/tcl/pltcl.c - pltcl_init_all()
-> tcl8.4.1/generic/tclInterp.c - Tcl_CreateSlave()
-> tcl8.4.1/win/tclWinInit.c - Tcl_Init()
-> Tcl_Eval(interp, initScript)
-> tcl8.4.1/generic/tclInitScript.h - error $msg -
Can't find a usable init.tcl ...

With some debugging trace, one can see that the
directory list to search for a init.tcl is empty.
Conclusion:
It is mandatory to feed the initialization step with
some directory paths.
Let's see how.

2. tclInitScript.h mentions 4 ways to specify the
directories.
- $tcl_library: this variable should be set after
Tcl_CreateInterp() and before Tcl_Init(),
so in tclInterp.c. As I don't want to touch the tcl
package, forget it.
- $env(TCL_LIBRARY): this way seems to solve the
problem, but another one appears (see 6. below).
- $tclDefaultLibrary: is set for the unix version
(tcl8.4.1/unix/tclUnixInit.c),
but not for the win version
(tcl8.4.1/win/tclWinInit.c). Forget it.
- $tcl_libPath: is set with the result of
TclGetLibraryPath().
Fine, but who is in charge to call
TclSetLibraryPath()?

Some investigation leads me to the following tree:
TclSetLibraryPath()
<- win/tclWinInit.c - TclpInitLibraryPath()
<- generic/tclEncoding.c - TclFindEncodings()
<- generic/tclEncoding.c - Tcl_FindExecutable(argv[0])

3. TclpInitLibraryPath() is a very interesting place,
indeed.
This is where the directory path is initialized.
Let's browse the categories:

a. installLib[2], "library", developLib:
What may be fine with tclsh.exe
(X:/cygwin/bin/tclsh.exe) is not with pgtclsh.exe or
postgres.exe
(X:/cygwin/usr/local/pgsql/bin/ path).
These options compose the following useless
directories:
X:/cygwin/usr/local/pgsql/share/tcl8.4
X:/cygwin/usr/local/pgsql/usr/share/tcl8.4
X:/cygwin/usr/local/share/tcl8.4
X:/cygwin/usr/local/pgsql/library
X:/cygwin/usr/local/library
X:/cygwin/usr/local/../tcl8.4.1/library
X:/cygwin/usr/../tcl8.4.1/library

b. Tcl_GetDefaultEncodingDir(): is empty; useless

c. AppendDllPath(): would be the perfect solution, but
is not yet adapted for cygwin standard
(see
http://cygwin.com/ml/cygwin/2004-01/msg00108.html)

d. AppendEnvironment(): have to be THE solution,
because there is no one else.
So TCL_LIBRARY have to be defined as an environment
variable.

4. For pgtclsh.exe or if you run postgres.exe
manually, you can do:
export TCL_LIBRARY=$(cygpath -w /usr/share/tcl8.4)

[Jason, you were on the right way with it]

5. But if you run postmaster as a windows Service, you
should define TCL_LIBRARY
as a system environment variable. Something like
"X:\cygwin\usr\share\tcl8.4".
Stop/Start postmaster is not enough: don't forget (as
I did) to reboot the machine,
as Windows loads the environment variables which are
available to services when
the machine is booted.

6. Add a call to Tcl_FindExecutable() in pltcl.c -
pltcl_init_all(),
just before the call to Tcl_CreateInterp().

It should be called with argv[0] as argument, but I
don't know how to get this piece of information for
pltcl.dll. Anyway, for Windows, it is unused (see
win/tclWinFile.c - TclpFindExecutable()). Just a value
!= NULL is OK, so I used an empty string.

[Jan, I'm only interested in cygwin platform. I
suggest you investigate deeper to make this
"workaround" cleaner, in order to be also compatible
with pure Unix platforms.]

Without this call, and with only the setting of the
environment variable, Tcl_CreateSlave() seems to fail
somewhere else, with this message:

--------------------------
server closed the connection unexpectedly
This probably means the server terminated
abnormally
before or while processing the request.
The connection to the server was lost. Attempting
reset: Succeeded.
--------------------------
I didn't spend time on this issue, just added the
call.

7. For the pltcl.dll to be built, I had to:

a. Permute the order of tcl.h and postgres.h.
See
http://archives.postgresql.org/pgsql-bugs/2003-12/msg00184.php

[Peter, Tom, I know you don't consider it as a
definitive fix, but meanwhile it worked for me.]

b. Add $(BE_DLLLIBS) to SHLIB_LINK in Makefile.
See
http://archives.postgresql.org/pgsql-bugs/2003-12/msg00183.php
Is fixed in version 7.4

This was suggested by Jason in his never-submitted
patch, see:

http://archives.postgresql.org/pgsql-cygwin/2003-01/msg00080.php

http://archives.postgresql.org/pgsql-cygwin/2003-11/msg00074.php

Just for completeness, here are the output of my build
and my patch:
--------------------------
Samson(at)pc2020 /opt/postgresql-7.3.5/src/pl/tcl
$ make
gcc -O2 -Wall -Wmissing-prototypes
-Wmissing-declarations -I../../../src/include -c -o
pltcl.o pltcl.c
In file included from
../../../src/include/pg_config.h:673,
from ../../../src/include/c.h:53,
from
../../../src/include/postgres.h:48,
from pltcl.c:43:
../../../src/include/pg_config_os.h:26:1: warning:
"DLLIMPORT" redefined
In file included from pltcl.c:41:
/usr/include/tcl.h:201:1: warning: this is the
location of the previous definition
pltcl.c: In function `pltcl_init_interp':
pltcl.c:270: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:272: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:274: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:276: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:279: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:281: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:283: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c:285: warning: passing arg 3 of
`Tcl_CreateCommand' from incompatible pointer type
pltcl.c: In function `pltcl_trigger_handler':
pltcl.c:827: warning: passing arg 4 of `Tcl_SplitList'
from incompatible pointer type
pltcl.c: In function `pltcl_SPI_prepare':
pltcl.c:1754: warning: passing arg 4 of
`Tcl_SplitList' from incompatible pointer type
pltcl.c: In function `pltcl_SPI_execp':
pltcl.c:2063: warning: passing arg 4 of
`Tcl_SplitList' from incompatible pointer type
dlltool --export-all --output-def pltcl.def pltcl.o
dllwrap -o pltcl.dll --dllname pltcl.dll --def
pltcl.def pltcl.o ../../../src/utils/dllinit.o
-L/usr/local/lib -L../../../src/backend -lpostgres
-L/usr/lib -ltcl84
dlltool --dllname pltcl.dll --def pltcl.def
--output-lib libpltcl.a
make -C modules all
make[1]: Entering directory
`/opt/postgresql-7.3.5/src/pl/tcl/modules'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory
`/opt/postgresql-7.3.5/src/pl/tcl/modules'
--------------------------

$ diff -u pltcl.orig pltcl.c
--- pltcl.orig 2003-10-30 03:00:44.000000000 +0100
+++ pltcl.c 2004-01-06 13:54:03.937500000 +0100
@@ -35,10 +35,13 @@
*

**********************************************************************/

-#include "postgres.h"
-
+//psa: change order of tcl.h and postgres.h
+//
http://archives.postgresql.org/pgsql-cygwin/2003-01/msg00080.php
+//
http://archives.postgresql.org/pgsql-cygwin/2003-11/msg00074.php
#include <tcl.h>

+#include "postgres.h"
+
#include <unistd.h>
#include <fcntl.h>
#include <setjmp.h>
@@ -207,6 +210,9 @@
* Create the dummy hold interpreter to
prevent close of
* stdout and stderr on DeleteInterp

************************************************************/
+//psa: argument should be argv[0]. But anyway, it is
unused on Windows: just have to be != NULL.
+Tcl_FindExecutable("");
+
if ((pltcl_hold_interp = Tcl_CreateInterp())
== NULL)
{
elog(ERROR, "pltcl: internal error -
cannot create 'hold' "
@@ -219,6 +225,8 @@
if ((pltcl_norm_interp =
Tcl_CreateSlave(pltcl_hold_interp,
"norm", 0)) == NULL)
{
+//psa: tell me more about the fail
+elog(LOG,Tcl_GetStringResult(pltcl_hold_interp));
elog(ERROR,
"pltcl: internal error - cannot
create 'normal' interpreter");
}
@@ -227,6 +235,8 @@
if ((pltcl_safe_interp =
Tcl_CreateSlave(pltcl_hold_interp,
"safe", 1)) == NULL)
{
+//psa: tell me more about the fail
+elog(LOG,Tcl_GetStringResult(pltcl_hold_interp));
elog(ERROR,
"pltcl: internal error -
cannot create 'safe' interpreter");
}

__________________________________
Do you Yahoo!?
Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes
http://hotjobs.sweepstakes.yahoo.com/signingbonus