Re: [JDBC] Conversion between UNICODE and LATIN1 is not supported

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all!
I get the following error when trying to connect to a pg-7.3 database on
Solaris8 with the 7.3 JDBC-driver.
Here is my output on the Solaris box:
www=> select version();
version
- -----------------------------------------------------------------------
PostgreSQL 7.3 on sparc-sun-solaris2.8, compiled by GCC gcc (GCC) 3.2
(1 row)

www=> set client_encoding = 'UNICODE';
ERROR: Conversion between UNICODE and LATIN1 is not supported
www=>

Here is the same commands on my Linux box:
andreak=# select version();
version
- --------------------------------------------------------------------------------------------------
PostgreSQL 7.3 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.2 (Mandrake
Linux 9.0 3.2-1mdk)
(1 row)

andreak=# set client_encoding = 'UNICODE';
SET
andreak=#

=================

Does anybody know why this doesn't work on Solaris8? They are both compiled
with "./configure --with-pam" and multibyte support is enabled as default in
7.3. I can allways recompile the JDBC-drivers with "set client_encoding =
'LATIN1'; show autocommit" in AbstractJdbc1Connection.java, but that seems
hacky.

- --
Andreas Joseph Krogh <andreak(at)officenet(dot)no>
The difference between insanity and genius is measured by success
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE+AGxvUopImDh2gfQRAoLbAJ9y9ZOZNS513zahIkPMDP5tJ1zSGACgj6Xo
lxVLkYwE/I/sxRRRNvRSZFo=
=FsMP
-----END PGP SIGNATURE-----

Andreas,

This would be better addressed to the hackers list, and I am forwarding
it on.

Dave
On Wed, 2002-12-18 at 07:39, Andreas Joseph Krogh wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi all!
> I get the following error when trying to connect to a pg-7.3 database on
> Solaris8 with the 7.3 JDBC-driver.
> Here is my output on the Solaris box:
> www=> select version();
> version
> - -----------------------------------------------------------------------
> PostgreSQL 7.3 on sparc-sun-solaris2.8, compiled by GCC gcc (GCC) 3.2
> (1 row)
>
> www=> set client_encoding = 'UNICODE';
> ERROR: Conversion between UNICODE and LATIN1 is not supported
> www=>
>
>
> Here is the same commands on my Linux box:
> andreak=# select version();
> version
> - --------------------------------------------------------------------------------------------------
> PostgreSQL 7.3 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.2 (Mandrake
> Linux 9.0 3.2-1mdk)
> (1 row)
>
> andreak=# set client_encoding = 'UNICODE';
> SET
> andreak=#
>
> =================
>
> Does anybody know why this doesn't work on Solaris8? They are both compiled
> with "./configure --with-pam" and multibyte support is enabled as default in
> 7.3. I can allways recompile the JDBC-drivers with "set client_encoding =
> 'LATIN1'; show autocommit" in AbstractJdbc1Connection.java, but that seems
> hacky.
>
> - --
> Andreas Joseph Krogh <andreak(at)officenet(dot)no>
> The difference between insanity and genius is measured by success
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.0.7 (GNU/Linux)
>
> iD8DBQE+AGxvUopImDh2gfQRAoLbAJ9y9ZOZNS513zahIkPMDP5tJ1zSGACgj6Xo
> lxVLkYwE/I/sxRRRNvRSZFo=
> =FsMP
> -----END PGP SIGNATURE-----
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo(at)postgresql(dot)org
--
Dave Cramer <dave(at)fastcrypt(dot)com>
Cramer Consulting

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thursday 19 December 2002 08:37, you wrote:
> Andreas Joseph Krogh wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Hi all!
> > I get the following error when trying to connect to a pg-7.3 database on
> > Solaris8 with the 7.3 JDBC-driver.
> > Here is my output on the Solaris box:
> > www=> select version();
> > version
> > - -----------------------------------------------------------------------
> > PostgreSQL 7.3 on sparc-sun-solaris2.8, compiled by GCC gcc (GCC) 3.2
> > (1 row)
> >
> > www=> set client_encoding = 'UNICODE';
> > ERROR: Conversion between UNICODE and LATIN1 is not supported
>
> This is not JDBC interface, it is PSQL command line utility!
>
> The message most likely is due to your database being LATIN1 encoded and I
> must agree with PostgreSQL - Unicode cannot be converted to Latin1, not all
> of it.
>
> So, check the database encoding: "psql -l".

I'm sorry if I was unclear, I initially got the error using the JDBC-driver
which issues the commands:
java.sql.ResultSet acRset =
ExecSQL("set client_encoding = 'UNICODE'; show autocommit");

//set encoding to be unicode
encoding = Encoding.getEncoding("UNICODE", null);

if (!acRset.next())
{
throw new PSQLException("postgresql.con.failed", "failed getting
autocommit status");
}

in AbstractJdbc1Connection.openConnection()

I had to recompile the JDBC-drivers with the method changed to
"set client_encoding = 'LATIN1'", as my database is as you guessed created
with "createdb -E LATIN1 dbname"

Then I realised I could reproduse it from the psql cmd-line, and posted the
error I got from there. Sorry for the confusion.

But my question still stands, why does this work on Linux and not Solaris8?
The docs say PostgreSQL can convert to and from LATIN1<->UNICODE as long as
multibyte support is enabled, which it is default in 7.3.

- --
Andreas Joseph Krogh <andreak(at)officenet(dot)no>
The difference between insanity and genius is measured by success
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE+AXljUopImDh2gfQRAlnYAJkBD+9kBPH8ME0doqOPctHApgHLowCeKkxK
5m5GngiT8tW7gUAPPTRV9JI=
=Hosd
-----END PGP SIGNATURE-----

Hello,

Do you have other databases that work with UNICODE? You can check the
encoding of all your databases with 'psql -l'. You can check if you can
create any unicode databases with 'createdb -E UNICODE mytestdatabase'.

I didn't use postgresql on solaris yet, so the following is just a
guess: maybe you need to install additional solaris packages. For
example to get tomcat/java working with utf-8/unicode/.. instead of
plain ascii, we need at least the following on our systems:

SUNWloc System Localization
SUNWlocx System Localization (64-bit)
SUNWeuluf UTF-8 L10N For Language Environment User Files
SUNWeulux UTF-8 L10N For Language Environment User Files (64-bit)

Kind regards,
Dries Verachtert

On Wed, 2002-12-18 at 13:39, Andreas Joseph Krogh wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi all!
> I get the following error when trying to connect to a pg-7.3 database on
> Solaris8 with the 7.3 JDBC-driver.
> Here is my output on the Solaris box:
> www=> select version();
> version
> - -----------------------------------------------------------------------
> PostgreSQL 7.3 on sparc-sun-solaris2.8, compiled by GCC gcc (GCC) 3.2
> (1 row)
>
> www=> set client_encoding = 'UNICODE';
> ERROR: Conversion between UNICODE and LATIN1 is not supported
> www=>
>
>
> Here is the same commands on my Linux box:
> andreak=# select version();
> version
> - --------------------------------------------------------------------------------------------------
> PostgreSQL 7.3 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.2 (Mandrake
> Linux 9.0 3.2-1mdk)
> (1 row)
>
> andreak=# set client_encoding = 'UNICODE';
> SET
> andreak=#
>
> =================
>
> Does anybody know why this doesn't work on Solaris8? They are both compiled
> with "./configure --with-pam" and multibyte support is enabled as default in
> 7.3. I can allways recompile the JDBC-drivers with "set client_encoding =
> 'LATIN1'; show autocommit" in AbstractJdbc1Connection.java, but that seems
> hacky.
>
> - --
> Andreas Joseph Krogh <andreak(at)officenet(dot)no>
> The difference between insanity and genius is measured by success
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.0.7 (GNU/Linux)
>
> iD8DBQE+AGxvUopImDh2gfQRAoLbAJ9y9ZOZNS513zahIkPMDP5tJ1zSGACgj6Xo
> lxVLkYwE/I/sxRRRNvRSZFo=
> =FsMP
> -----END PGP SIGNATURE-----
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo(at)postgresql(dot)org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thursday 19 December 2002 16:09, you wrote:
> Hello,
>
> Do you have other databases that work with UNICODE? You can check the
> encoding of all your databases with 'psql -l'. You can check if you can
> create any unicode databases with 'createdb -E UNICODE mytestdatabase'.
>
> I didn't use postgresql on solaris yet, so the following is just a
> guess: maybe you need to install additional solaris packages. For
> example to get tomcat/java working with utf-8/unicode/.. instead of
> plain ascii, we need at least the following on our systems:
>
> SUNWloc System Localization
> SUNWlocx System Localization (64-bit)
> SUNWeuluf UTF-8 L10N For Language Environment User Files
> SUNWeulux UTF-8 L10N For Language Environment User Files (64-bit)

No, all our databases are LATIN1, it's the JDBC driver which want to "set
client_encoding = 'UNICODE'". I havn't tried to connect to a UNICODE (-E
UNICODE) database with the JDBC driver.

I now created a test database <andreak> with -E UNICODE
Here is some output:

[andreak(at)www2] ~$ uname -a
SunOS www2.java.no 5.8 Generic_108528-16 sun4u sparc SUNW,Ultra-250
[andreak(at)www2] ~$ psql -l
List of databases
Name | Owner | Encoding
- -----------+----------+-----------
andreak | andreak | UNICODE
template0 | postgres | SQL_ASCII
template1 | postgres | SQL_ASCII
www | onp | LATIN1
(4 rows)

[andreak(at)www2] ~$ psql andreak
Welcome to psql 7.3, the PostgreSQL interactive terminal.

Type: \copyright for distribution terms
\h for help with SQL commands
\? for help on internal slash commands
\g or terminate with semicolon to execute query
\q to quit

andreak=# select version();
version
- -----------------------------------------------------------------------
PostgreSQL 7.3 on sparc-sun-solaris2.8, compiled by GCC gcc (GCC) 3.2
(1 row)

andreak=# show client_encoding ;
client_encoding
- -----------------
UNICODE
(1 row)

andreak=# set client_encoding = 'LATIN1';
ERROR: Conversion between LATIN1 and UNICODE is not supported
andreak=# set client_encoding = 'UNICODE';
SET
andreak=# \q
[andreak(at)www2] ~$ psql www
Welcome to psql 7.3, the PostgreSQL interactive terminal.

Type: \copyright for distribution terms
\h for help with SQL commands
\? for help on internal slash commands
\g or terminate with semicolon to execute query
\q to quit

www=# set client_encoding = 'LATIN1';
SET
www=# set client_encoding = 'UNICODE';
ERROR: Conversion between UNICODE and LATIN1 is not supported
www=#

This works fine on Linux, I havn't tried to patch Solaris with l10n-stuff yet,
but it would be nice to get some feedback and understand why this is
happening and why I don't get a more descriptive error. For some reason none
of the conversions work at all, except, of course, conversion between similar
encodings like UNICODE<->UNICODE.

- --
Andreas Joseph Krogh <andreak(at)officenet(dot)no>
There will always be someone who agrees with you
but is, inexplicably, a moron.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE+AeQ4UopImDh2gfQRAuRvAKCbk4pyp7CbXiA4NGq8ntKpZ2IriwCeKcaC
/pPGUgPzEuzDmIdX3shR8Ps=
=EYP3
-----END PGP SIGNATURE-----