Lists: | pgsql-generalpgsql-hackers |
---|
From: | Joseph Shraibman <jks(at)selectacast(dot)net> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | How do I change the server encoding? |
Date: | 2003-02-25 00:13:21 |
Message-ID: | b3eceu$atu$1@news.hub.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-general pgsql-hackers |
I have a server that has LATIN1 encoding. I want to convert it to run UTF encoding. How
do I do that? Simply changing the encoding in a dump file does not work.
From: | Antti Haapala <antti(dot)haapala(at)iki(dot)fi> |
---|---|
To: | Joseph Shraibman <jks(at)selectacast(dot)net> |
Cc: | pgsql-general(at)postgresql(dot)org |
Subject: | Re: How do I change the server encoding? |
Date: | 2003-02-25 07:55:09 |
Message-ID: | Pine.GSO.4.44.0302250949390.18775-100000@paju.oulu.fi |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-general pgsql-hackers |
On Mon, 24 Feb 2003, Joseph Shraibman wrote:
> I have a server that has LATIN1 encoding. I want to convert it to run UTF encoding. How
> do I do that? Simply changing the encoding in a dump file does not work.
So have you done both of these:
- dropped and recreated your db with encoding 'utf-8'
- converted your dumps to utf-8 or
added set client_encoding to 'latin1' in the dumps
--
Antti Haapala
From: | Philippe Kiener <philippe(dot)kiener(at)eivd(dot)ch> |
---|---|
To: | Antti Haapala <antti(dot)haapala(at)iki(dot)fi> |
Cc: | <pgsql-general(at)postgresql(dot)org> |
Subject: | Re: How do I change the server encoding? |
Date: | 2003-02-25 15:13:06 |
Message-ID: | BA814692.54AC%philippe.kiener@eivd.ch |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-general pgsql-hackers |
Hello
I have the same question that Joseph Shraibman.
I have dump the db, created a new db with utf-8 encoding
My database should be transform from SQL_ASCII to utf-8
I have added that line to my dumps:
SET CLIENT_ENCODING TO 'SQL_ASCII';
Now when I load the dump into my db, I get that error on tables with text:
psql:tcom-database.sql:7111: ERROR: copy: line 1, Invalid UNICODE character
sequence found (0xe96500)
psql:tcom-database.sql:7111: lost synchronization with server, resetting
connection
psql:tcom-database.sql:7409: ERROR: copy: line 1, Invalid UNICODE character
sequence found (0xe97265)
psql:tcom-database.sql:7409: lost synchronization with server, resetting
connection
psql:tcom-database.sql:7456: ERROR: copy: line 3, Invalid UNICODE character
sequence found (0xe90007)
psql:tcom-database.sql:7456: lost synchronization with server, resetting
connection
psql:tcom-database.sql:7468: ERROR: copy: line 6, Invalid UNICODE character
sequence found (0xe97300)
Any ideas?
Thanks for your help.
Philippe Kiener
Le 25.2.2003 8:55, "Antti Haapala" <antti(dot)haapala(at)iki(dot)fi> wrote:
>
> On Mon, 24 Feb 2003, Joseph Shraibman wrote:
>
>> I have a server that has LATIN1 encoding. I want to convert it to run UTF
>> encoding. How
>> do I do that? Simply changing the encoding in a dump file does not work.
>
> So have you done both of these:
> - dropped and recreated your db with encoding 'utf-8'
> - converted your dumps to utf-8 or
> added set client_encoding to 'latin1' in the dumps
From: | Peter Eisentraut <peter_e(at)gmx(dot)net> |
---|---|
To: | Philippe Kiener <philippe(dot)kiener(at)eivd(dot)ch> |
Cc: | Antti Haapala <antti(dot)haapala(at)iki(dot)fi>, <pgsql-general(at)postgresql(dot)org> |
Subject: | Re: How do I change the server encoding? |
Date: | 2003-02-25 22:27:46 |
Message-ID: | Pine.LNX.4.44.0302251915090.2581-100000@peter.localdomain |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-general pgsql-hackers |
Philippe Kiener writes:
> My database should be transform from SQL_ASCII to utf-8
>
> I have added that line to my dumps:
>
> SET CLIENT_ENCODING TO 'SQL_ASCII';
>
> Now when I load the dump into my db, I get that error on tables with text:
>
> psql:tcom-database.sql:7111: ERROR: copy: line 1, Invalid UNICODE character
> sequence found (0xe96500)
The client encoding SQL_ASCII means that the data will be passed through
unchanged. Try setting it to LATIN1.
--
Peter Eisentraut peter_e(at)gmx(dot)net
From: | Joseph Shraibman <jks(at)selectacast(dot)net> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | Re: How do I change the server encoding? |
Date: | 2003-02-26 01:16:47 |
Message-ID: | b3h4hs$197v$1@news.hub.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-general pgsql-hackers |
Peter Eisentraut wrote:
> Philippe Kiener writes:
>
>
>>My database should be transform from SQL_ASCII to utf-8
>>
>>I have added that line to my dumps:
>>
>>SET CLIENT_ENCODING TO 'SQL_ASCII';
>>
>>Now when I load the dump into my db, I get that error on tables with text:
>>
>>psql:tcom-database.sql:7111: ERROR: copy: line 1, Invalid UNICODE character
>>sequence found (0xe96500)
>
>
> The client encoding SQL_ASCII means that the data will be passed through
> unchanged. Try setting it to LATIN1.
>
I tried with latin1 and it didn't work.
From: | Joseph Shraibman <jks(at)selectacast(dot)net> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | Re: How do I change the server encoding? |
Date: | 2003-02-26 03:13:18 |
Message-ID: | b3hbc9$1rje$1@news.hub.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-general pgsql-hackers |
Joseph Shraibman wrote:
After further experimenting I think the problem is in psql. When I try
update mytable set firstname = 'Oné' where ukey = 12911;
It works with a latin1 database, but when I try it on a unicode database:
utfowl=# update mytable set firstname = 'Oné' where ukey = 12911;
utfowl'#
It thinks there is an open quote or something. This is even if I set the client encoding
to be latin1. Of course dumps are read with the copy command but maybe it is the same
problem.
From: | Antti Haapala <antti(dot)haapala(at)iki(dot)fi> |
---|---|
To: | Joseph Shraibman <jks(at)selectacast(dot)net> |
Cc: | pgsql-general(at)postgresql(dot)org |
Subject: | Re: How do I change the server encoding? |
Date: | 2003-02-26 08:56:21 |
Message-ID: | Pine.GSO.4.44.0302261048340.2358-100000@paju.oulu.fi |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-general pgsql-hackers |
On Tue, 25 Feb 2003, Joseph Shraibman wrote:
> Peter Eisentraut wrote:
> > Philippe Kiener writes:
> >>
> >>My database should be transform from SQL_ASCII to utf-8
> >>
> >>I have added that line to my dumps:
> >>
> >>SET CLIENT_ENCODING TO 'SQL_ASCII';
> >>
> >>Now when I load the dump into my db, I get that error on tables with text:
> >>
> >>psql:tcom-database.sql:7111: ERROR: copy: line 1, Invalid UNICODE character
> >>sequence found (0xe96500)
> >
> >
> > The client encoding SQL_ASCII means that the data will be passed through
> > unchanged. Try setting it to LATIN1.
> >
> I tried with latin1 and it didn't work.
Hmm... still caused errors? I think that because newer dumps have those
\connects, you need to add explicit char set settings after all of those.
The better way would be converting the whole dump with iconv, though.
Iconv comes by default with many unixen. For example command
iconv -f iso-8859-1 -t utf-8 < text_dump > text_dump_converted
will convert your dump from latin1 to utf-8.
--
Antti Haapala
From: | Joseph Shraibman <jks(at)selectacast(dot)net> |
---|---|
To: | Joseph Shraibman <jks(at)selectacast(dot)net> |
Subject: | Re: How do I change the server encoding? SOLVED |
Date: | 2003-02-26 22:11:18 |
Message-ID: | 3E5D3B86.3000105@selectacast.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-general pgsql-hackers |
Joseph Shraibman wrote:
> Joseph Shraibman wrote:
> After further experimenting I think the problem is in psql. When I try
> update mytable set firstname = 'Oné' where ukey = 12911;
>
> It works with a latin1 database, but when I try it on a unicode database:
>
> utfowl=# update mytable set firstname = 'Oné' where ukey = 12911;
> utfowl'#
>
> It thinks there is an open quote or something. This is even if I set
> the client encoding to be latin1. Of course dumps are read with the
> copy command but maybe it is the same problem.
>
I solved the problem. "set client_encoding = 'latin1';" does not work, but "\encoding
latin1" does. I suggest that pg_dump put a "\encoding <encoding>" after every \connect
in the dump. I would do this myself but I can't figure out where that is done in the dump
program.
I did modify pg_dump.c so the encoding used during the dump can be specified on the
command line, but since that isn't what solved the problem I'm not sure there is a point
to having it. Is anyone interested?