Re: [HACKERS] UTF8 or Unicode

-----Original Message-----
From: pgsql-hackers-owner(at)postgresql(dot)org on behalf of Oliver Jowett
Sent: Fri 2/18/2005 11:27 AM
To: Karel Zak
Cc: List pgsql-hackers
Subject: Re: [HACKERS] UTF8 or Unicode

Karel Zak wrote:

>> Yes, I think we should fix it and remove UNICODE and WIN encoding names
>> from PG code.
>
> The JDBC driver asks for a UNICODE client encoding before it knows the
> server version it is talking to. How do you avoid breaking this?

So does pgAdmin.

Regards, Dave

Dave Page wrote:
> Karel Zak wrote:
>
> >> Yes, I think we should fix it and remove UNICODE and WIN encoding names
> >> from PG code.
> >
> > The JDBC driver asks for a UNICODE client encoding before it knows the
> > server version it is talking to. How do you avoid breaking this?
>
> So does pgAdmin.

I think we just need to _favor_ UTF8. The question is where are we
favoring Unicode rather than UTF8?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> I think we just need to _favor_ UTF8.

I agree.

> The question is where are we
> favoring Unicode rather than UTF8?

It's the canonical name of the encoding, both in the code and the docs.

regression=# create database e encoding 'utf-8';
CREATE DATABASE
regression=# \l
List of databases
Name | Owner | Encoding
------------+----------+-----------
e | postgres | UNICODE
regression | postgres | SQL_ASCII
template0 | postgres | SQL_ASCII
template1 | postgres | SQL_ASCII
(5 rows)

As soon as we decide whether the canonical name is "UTF8" or "UTF-8"
;-) we can fix it.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > I think we just need to _favor_ UTF8.
>
> I agree.
>
> > The question is where are we
> > favoring Unicode rather than UTF8?
>
> It's the canonical name of the encoding, both in the code and the docs.
>
> regression=# create database e encoding 'utf-8';
> CREATE DATABASE
> regression=# \l
> List of databases
> Name | Owner | Encoding
> ------------+----------+-----------
> e | postgres | UNICODE
> regression | postgres | SQL_ASCII
> template0 | postgres | SQL_ASCII
> template1 | postgres | SQL_ASCII
> (5 rows)
>
> As soon as we decide whether the canonical name is "UTF8" or "UTF-8"
> ;-) we can fix it.

I checked and it looks like "UTF-8" is the correct usage:

http://www.unicode.org/glossary/

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

I do not object the changing UNICODE->UTF-8, but all these discussions
sound a little bit funny to me.

If you want to blame UNICODE, you should blame LATIN1 etc. as
well. LATIN1(ISO-8859-1) is actually a character set name, not an
encoding name. ISO-8859-1 can be encoded in 8-bit single byte
stream. But it can be encoded in 7-bit too. So when we refer to
LATIN1(ISO-8859-1), it's not clear if it's encoded in 7/8-bit.
--
Tatsuo Ishii

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
Subject: Re: [HACKERS] UTF8 or Unicode
Date: Mon, 21 Feb 2005 22:08:25 -0500 (EST)
Message-ID: <200502220308(dot)j1M38PV03238(at)candle(dot)pha(dot)pa(dot)us>

> Tom Lane wrote:
> > Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > > I think we just need to _favor_ UTF8.
> >
> > I agree.
> >
> > > The question is where are we
> > > favoring Unicode rather than UTF8?
> >
> > It's the canonical name of the encoding, both in the code and the docs.
> >
> > regression=# create database e encoding 'utf-8';
> > CREATE DATABASE
> > regression=# \l
> > List of databases
> > Name | Owner | Encoding
> > ------------+----------+-----------
> > e | postgres | UNICODE
> > regression | postgres | SQL_ASCII
> > template0 | postgres | SQL_ASCII
> > template1 | postgres | SQL_ASCII
> > (5 rows)
> >
> > As soon as we decide whether the canonical name is "UTF8" or "UTF-8"
> > ;-) we can fix it.
>
> I checked and it looks like "UTF-8" is the correct usage:
>
> http://www.unicode.org/glossary/
>
> --
> Bruce Momjian | http://candle.pha.pa.us
> pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
> + If your life is a hard drive, | 13 Roberts Road
> + Christ can be your backup. | Newtown Square, Pennsylvania 19073
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
> http://archives.postgresql.org
>

Tatsuo Ishii wrote:
> I do not object the changing UNICODE->UTF-8, but all these discussions
> sound a little bit funny to me.
>
> If you want to blame UNICODE, you should blame LATIN1 etc. as
> well. LATIN1(ISO-8859-1) is actually a character set name, not an
> encoding name. ISO-8859-1 can be encoded in 8-bit single byte
> stream. But it can be encoded in 7-bit too. So when we refer to
> LATIN1(ISO-8859-1), it's not clear if it's encoded in 7/8-bit.

Wow, Tatsuo has a point here. Looking at encnames.c, I see:

"UNICODE", PG_UTF8

but also:

"WIN", PG_WIN1251
"LATIN1", PG_LATIN1

and I see conversions for those:

"iso88591", PG_LATIN1
"win", PG_WIN1251

so I see what he is saying. We are not consistent in favoring the
official names vs. the common names.

I will work on a patch that people can review and test.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> We are not consistent in favoring the
> official names vs. the common names.

The problem is rather that there are too many standards and conventions
to choose from.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Thu, 2005-02-24 at 23:51 -0500, Bruce Momjian wrote:
> Tatsuo Ishii wrote:
> > I do not object the changing UNICODE->UTF-8, but all these discussions
> > sound a little bit funny to me.
> >
> > If you want to blame UNICODE, you should blame LATIN1 etc. as
> > well. LATIN1(ISO-8859-1) is actually a character set name, not an
> > encoding name. ISO-8859-1 can be encoded in 8-bit single byte
> > stream. But it can be encoded in 7-bit too. So when we refer to
> > LATIN1(ISO-8859-1), it's not clear if it's encoded in 7/8-bit.
>
> Wow, Tatsuo has a point here. Looking at encnames.c, I see:
>
> "UNICODE", PG_UTF8
>
> but also:
>
> "WIN", PG_WIN1251
> "LATIN1", PG_LATIN1

> so I see what he is saying. We are not consistent in favoring the
> official names vs. the common names.

Yes. I said already. For example "WIN" is extremely bad alias. It all is
heritage from old versions.

> I will work on a patch that people can review and test.

Thanks.

Karel

--
Karel Zak <zakkr(at)zf(dot)jcu(dot)cz>

Am Freitag, 25. Februar 2005 05:51 schrieb Bruce Momjian:
> so I see what he is saying. We are not consistent in favoring the
> official names vs. the common names.
>
> I will work on a patch that people can review and test.

I think this is what we should do:

UNICODE => UTF8
ALT => WIN866
WIN => WIN1251
TCVN => WIN1258

That should clear it up.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Am Freitag, 25. Februar 2005 05:51 schrieb Bruce Momjian:
> > so I see what he is saying. We are not consistent in favoring the
> > official names vs. the common names.
> >
> > I will work on a patch that people can review and test.
>
> I think this is what we should do:
>
> UNICODE => UTF8
> ALT => WIN866
> WIN => WIN1251
> TCVN => WIN1258

OK, but what about latin1?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Peter Eisentraut wrote:
>> I think this is what we should do:
>>
>> UNICODE => UTF8
>> ALT => WIN866
>> WIN => WIN1251
>> TCVN => WIN1258

> OK, but what about latin1?

I think LATIN1 is fine as-is. It's a reasonably popular name for the
character set, and despite Tatsuo's complaint, it's not going to confuse
anyone in practice --- the 7-bit version of that standard has no traction.
The reason UNICODE is a bad name for UTF8 is exactly that there are
multiple physical encodings of Unicode that are in common use.

regards, tom lane

Am Freitag, 25. Februar 2005 16:26 schrieb Bruce Momjian:
> OK, but what about latin1?

The following character set names are specified in the SQL standard and
therefore somewhat non-negotiable:

SQL_CHARACTER
GRAPHIC_IRV
LATIN1
ISO8BIT
UTF16
UTF8
UCS2
SQL_TEXT
SQL_IDENTIFIER

So we have to use LATIN1, even though it creates an inconsistency. We
discussed this a while ago during the last great renaming, I think.

Btw., I think ISO8BIT is the correct name for what we call SQL_ASCII, but I
haven't analyzed that in detail, yet.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Am Freitag, 25. Februar 2005 16:26 schrieb Bruce Momjian:
> > OK, but what about latin1?
>
> The following character set names are specified in the SQL standard and
> therefore somewhat non-negotiable:
>
> SQL_CHARACTER
> GRAPHIC_IRV
> LATIN1
> ISO8BIT
> UTF16
> UTF8
> UCS2
> SQL_TEXT
> SQL_IDENTIFIER
>
> So we have to use LATIN1, even though it creates an inconsistency. We
> discussed this a while ago during the last great renaming, I think.
>

Oh, UTF8 and not UTF-8? I thought UTF-8 was the standard name, but if
ANSI uses UTF8 we will have to use that.

> Btw., I think ISO8BIT is the correct name for what we call SQL_ASCII, but I
> haven't analyzed that in detail, yet.

OK, please let us know.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Peter Eisentraut wrote:
> Am Freitag, 25. Februar 2005 05:51 schrieb Bruce Momjian:
> > so I see what he is saying. We are not consistent in favoring the
> > official names vs. the common names.
> >
> > I will work on a patch that people can review and test.
>
> I think this is what we should do:
>
> UNICODE => UTF8
> ALT => WIN866
> WIN => WIN1251
> TCVN => WIN1258
>
> That should clear it up.

OK, here is a patch that makes those changes.

The only uncertainty I have is with the the use of the TCVN conversion
routine names, e.g.:

SELECT CONVERT('foo' USING tcvn_to_utf_8);

I assume this is the same as:

SELECT CONVERT('foo', 'WIN1258', 'UTF8');
and
SELECT CONVERT('foo', 'TCVN', 'UTF8'); -- alias usage

So, why would people use the routine name? Both forms are documented.
The first one with USING does not accept aliases, while the others do.

I think this should be renamed to win1258_to_utf_8. However, this would
be an incompatibility. We should mention it in the release notes.

Other than that the other conversion files were already named fine, e.g.
ascii_to_utf_8 (no UNICODE), however it is utf_8 and not utf8. I am
unsure how to handle these.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

The routines that do the conversion could have alternate names specified
in the conversion_create.sql. There is not reason that I can see why you
cannot have two function names pointing to the same routine.

like

CREATE OR REPLACE FUNCTION ascii_to_mic (INTEGER, INTEGER, CSTRING,
CSTRING, INTEGER) RETURNS VOID AS '$libdir/ascmic', 'ascii_to_mic'
LANGUAGE 'c' STRICT;

and

CREATE OR REPLACE FUNCTION ascii_to_whatever (INTEGER, INTEGER, CSTRING,
CSTRING, INTEGER) RETURNS VOID AS '$libdir/ascmic', 'ascii_to_mic'
LANGUAGE 'c' STRICT;

I just tried with those two and it works OK

That way you do not break compatibility with existing SQL scripts and/or
programs using the existing function names.

Lorne

In <200502262050(dot)j1QKoNi10358(at)candle(dot)pha(dot)pa(dot)us>, on 02/26/05
at 03:50 PM, Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> said:

>Peter Eisentraut wrote:
>> Am Freitag, 25. Februar 2005 05:51 schrieb Bruce Momjian:
>> > so I see what he is saying. We are not consistent in favoring the
>> > official names vs. the common names.
>> >
>> > I will work on a patch that people can review and test.
>>
>> I think this is what we should do:
>>
>> UNICODE => UTF8
>> ALT => WIN866
>> WIN => WIN1251
>> TCVN => WIN1258
>>
>> That should clear it up.

>OK, here is a patch that makes those changes.

>The only uncertainty I have is with the the use of the TCVN conversion
>routine names, e.g.:

> SELECT CONVERT('foo' USING tcvn_to_utf_8);

>I assume this is the same as:

> SELECT CONVERT('foo', 'WIN1258', 'UTF8');
>and
> SELECT CONVERT('foo', 'TCVN', 'UTF8'); -- alias usage

>So, why would people use the routine name? Both forms are documented.
>The first one with USING does not accept aliases, while the others do.

>I think this should be renamed to win1258_to_utf_8. However, this would
>be an incompatibility. We should mention it in the release notes.

>Other than that the other conversion files were already named fine, e.g.
>ascii_to_utf_8 (no UNICODE), however it is utf_8 and not utf8. I am
>unsure how to handle these.

--
-----------------------------------------------------------
lsunley(at)mb(dot)sympatico(dot)ca
-----------------------------------------------------------

Further to my earlier e-mail, there would have to be two lines added to
conversion_create.sql for each alternate function name

Like:

CREATE OR REPLACE FUNCTION ascii_to_whatever (INTEGER, INTEGER, CSTRING,
CSTRING, INTEGER) RETURNS VOID AS '$libdir/ascmic', 'ascii_to_mic'
LANGUAGE 'c' STRICT;

CREATE CONVERSION pg_catalog.ascii_to_whatever FOR 'SQL_ASCII' TO
'MULE_INTERNAL' FROM ascii_to_whatever;

Lorne

In <200502262050(dot)j1QKoNi10358(at)candle(dot)pha(dot)pa(dot)us>, on 02/26/05
at 03:50 PM, Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> said:

>Peter Eisentraut wrote:
>> Am Freitag, 25. Februar 2005 05:51 schrieb Bruce Momjian:
>> > so I see what he is saying. We are not consistent in favoring the
>> > official names vs. the common names.
>> >
>> > I will work on a patch that people can review and test.
>>
>> I think this is what we should do:
>>
>> UNICODE => UTF8
>> ALT => WIN866
>> WIN => WIN1251
>> TCVN => WIN1258
>>
>> That should clear it up.

>OK, here is a patch that makes those changes.

>The only uncertainty I have is with the the use of the TCVN conversion
>routine names, e.g.:

> SELECT CONVERT('foo' USING tcvn_to_utf_8);

>I assume this is the same as:

> SELECT CONVERT('foo', 'WIN1258', 'UTF8');
>and
> SELECT CONVERT('foo', 'TCVN', 'UTF8'); -- alias usage

>So, why would people use the routine name? Both forms are documented.
>The first one with USING does not accept aliases, while the others do.

>I think this should be renamed to win1258_to_utf_8. However, this would
>be an incompatibility. We should mention it in the release notes.

>Other than that the other conversion files were already named fine, e.g.
>ascii_to_utf_8 (no UNICODE), however it is utf_8 and not utf8. I am
>unsure how to handle these.

--
-----------------------------------------------------------
lsunley(at)mb(dot)sympatico(dot)ca
-----------------------------------------------------------

In <200502270055(dot)j1R0te122973(at)candle(dot)pha(dot)pa(dot)us>, on 02/26/05
at 07:55 PM, Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> said:

>lsunley(at)mb(dot)sympatico(dot)ca wrote:
>> Further to my earlier e-mail, there would have to be two lines added to
>> conversion_create.sql for each alternate function name
>>
>> Like:
>>
>> CREATE OR REPLACE FUNCTION ascii_to_whatever (INTEGER, INTEGER, CSTRING,
>> CSTRING, INTEGER) RETURNS VOID AS '$libdir/ascmic', 'ascii_to_mic'
>> LANGUAGE 'c' STRICT;
>>
>> CREATE CONVERSION pg_catalog.ascii_to_whatever FOR 'SQL_ASCII' TO
>> 'MULE_INTERNAL' FROM ascii_to_whatever;

>I would like to avoid bloating the system with aliases. I think it is OK
>to have it happen with specific names but doing it for all the
>combinations is too much, and these duplicates are in the system tables,
>not in the C code. It just doesn't seem worth it to me.

Good point, perhaps having a script generated that can be executed by the
DBA to insert any deprecated conversion routine names would be the best
route to take.

That way they would only be there if needed by the particular
installation.

--
-----------------------------------------------------------
lsunley(at)mb(dot)sympatico(dot)ca
-----------------------------------------------------------

lsunley(at)mb(dot)sympatico(dot)ca wrote:
> Further to my earlier e-mail, there would have to be two lines added to
> conversion_create.sql for each alternate function name
>
> Like:
>
> CREATE OR REPLACE FUNCTION ascii_to_whatever (INTEGER, INTEGER, CSTRING,
> CSTRING, INTEGER) RETURNS VOID AS '$libdir/ascmic', 'ascii_to_mic'
> LANGUAGE 'c' STRICT;
>
> CREATE CONVERSION pg_catalog.ascii_to_whatever FOR 'SQL_ASCII' TO
> 'MULE_INTERNAL' FROM ascii_to_whatever;

I would like to avoid bloating the system with aliases. I think it is
OK to have it happen with specific names but doing it for all the
combinations is too much, and these duplicates are in the system tables,
not in the C code. It just doesn't seem worth it to me.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

lsunley(at)mb(dot)sympatico(dot)ca wrote:
> In <200502270055(dot)j1R0te122973(at)candle(dot)pha(dot)pa(dot)us>, on 02/26/05
> at 07:55 PM, Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> said:
>
> >lsunley(at)mb(dot)sympatico(dot)ca wrote:
> >> Further to my earlier e-mail, there would have to be two lines added to
> >> conversion_create.sql for each alternate function name
> >>
> >> Like:
> >>
> >> CREATE OR REPLACE FUNCTION ascii_to_whatever (INTEGER, INTEGER, CSTRING,
> >> CSTRING, INTEGER) RETURNS VOID AS '$libdir/ascmic', 'ascii_to_mic'
> >> LANGUAGE 'c' STRICT;
> >>
> >> CREATE CONVERSION pg_catalog.ascii_to_whatever FOR 'SQL_ASCII' TO
> >> 'MULE_INTERNAL' FROM ascii_to_whatever;
>
> >I would like to avoid bloating the system with aliases. I think it is OK
> >to have it happen with specific names but doing it for all the
> >combinations is too much, and these duplicates are in the system tables,
> >not in the C code. It just doesn't seem worth it to me.
>
> Good point, perhaps having a script generated that can be executed by the
> DBA to insert any deprecated conversion routine names would be the best
> route to take.

I am unsure why anyone would use the awkward names when you can more
easily supply the actual encoding names, and all the aliases work there.

Also, consider this is going to be a major release change so they are
going to dump/reload anyway. I see I need to update
conversion_create.sql too. I was previously only looking in C files.
Let me do a bigger search.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Here is an updated version that handles all cases. It does rename the
routine names so the primary encoding name is used for the routine
names. This will be documented in the release notes if anyone actually
uses those names in their code.

This patch requires renaming of the utf8_and_tcvn directory so it will
not apply cleanly.

I left the routines named utf_8 alone because the code splits encoding
names at breaks, like this iso_8859_7_to_utf_8. I assume that is OK.

---------------------------------------------------------------------------

Bruce Momjian wrote:
> Peter Eisentraut wrote:
> > Am Freitag, 25. Februar 2005 05:51 schrieb Bruce Momjian:
> > > so I see what he is saying. We are not consistent in favoring the
> > > official names vs. the common names.
> > >
> > > I will work on a patch that people can review and test.
> >
> > I think this is what we should do:
> >
> > UNICODE => UTF8
> > ALT => WIN866
> > WIN => WIN1251
> > TCVN => WIN1258
> >
> > That should clear it up.
>
> OK, here is a patch that makes those changes.
>
> The only uncertainty I have is with the the use of the TCVN conversion
> routine names, e.g.:
>
> SELECT CONVERT('foo' USING tcvn_to_utf_8);
>
> I assume this is the same as:
>
> SELECT CONVERT('foo', 'WIN1258', 'UTF8');
> and
> SELECT CONVERT('foo', 'TCVN', 'UTF8'); -- alias usage
>
> So, why would people use the routine name? Both forms are documented.
> The first one with USING does not accept aliases, while the others do.
>
> I think this should be renamed to win1258_to_utf_8. However, this would
> be an incompatibility. We should mention it in the release notes.
>
> Other than that the other conversion files were already named fine, e.g.
> ascii_to_utf_8 (no UNICODE), however it is utf_8 and not utf8. I am
> unsure how to handle these.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

In <200502270409(dot)j1R49hc08394(at)candle(dot)pha(dot)pa(dot)us>, on 02/26/05
at 11:09 PM, Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> said:

>Here is an updated version that handles all cases. It does rename the
>routine names so the primary encoding name is used for the routine names.
>This will be documented in the release notes if anyone actually uses
>those names in their code.

>This patch requires renaming of the utf8_and_tcvn directory so it will
>not apply cleanly.

>I left the routines named utf_8 alone because the code splits encoding
>names at breaks, like this iso_8859_7_to_utf_8. I assume that is OK.

You cannot patch conversion_create.sql

You have to patch the makefile that generates it.

The CONVERSIONS variable has the names etc and the

@set $(CONVERSIONS) etc code generates the conversion_create.sql file

--
-----------------------------------------------------------
lsunley(at)mb(dot)sympatico(dot)ca
-----------------------------------------------------------

Ignore previous e-mail...

Missed the patch to the makefile

Sorry about that

--
-----------------------------------------------------------
lsunley(at)mb(dot)sympatico(dot)ca
-----------------------------------------------------------

В Сбт, 26/02/2005 в 15:50 -0500, Bruce Momjian пишет:

> Other than that the other conversion files were already named fine, e.g.
> ascii_to_utf_8 (no UNICODE), however it is utf_8 and not utf8. I am
> unsure how to handle these.

General remark about the spelling of this encoding:

The correct encoding name is "UTF-8".

"The official name and spelling of this encoding is UTF-8, where UTF
stands for UCS Transformation Format. Please do not write UTF-8 in any
documentation text in other ways (such as utf8 or UTF_8), unless of
course you refer to a variable name and not the encoding itself."

from

http://www.cl.cam.ac.uk/~mgk25/unicode.html#utf-8

Thanks

--
Markus Bertheau ☭ <twanger(at)bluetwanger(dot)de>

Markus Bertheau ? wrote:
> ? ???, 26/02/2005 ? 15:50 -0500, Bruce Momjian ?????:
>
> > Other than that the other conversion files were already named fine, e.g.
> > ascii_to_utf_8 (no UNICODE), however it is utf_8 and not utf8. I am
> > unsure how to handle these.
>
> General remark about the spelling of this encoding:
>
> The correct encoding name is "UTF-8".
>
> "The official name and spelling of this encoding is UTF-8, where UTF
> stands for UCS Transformation Format. Please do not write UTF-8 in any
> documentation text in other ways (such as utf8 or UTF_8), unless of
> course you refer to a variable name and not the encoding itself."
>
> from
>
> http://www.cl.cam.ac.uk/~mgk25/unicode.html#utf-8

True, but Peter says the ANSI standard calls it UTF8 so that's what I
used.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
>> The correct encoding name is "UTF-8".

> True, but Peter says the ANSI standard calls it UTF8 so that's what I
> used.

What SQL99 actually says is

- UTF8 specifies the name of a character repertoire that consists
of every character represented by The Unicode Standard Version
2.0 and by ISO/IEC 10646 UTF-8, where each character is encoded
using the UTF-8 encoding, occupying from 1 (one) through 6
octets.

That is, "UTF8" is an identifier chosen to refer to an encoding which
they know perfectly well is really called UTF-8. We should probably
follow the same convention of using UTF8 in code identifiers and UTF-8
in documentation. In particular, UTF_8 with an underscore is sanctioned
by nobody and should be avoided.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> >> The correct encoding name is "UTF-8".
>
> > True, but Peter says the ANSI standard calls it UTF8 so that's what I
> > used.
>
> What SQL99 actually says is
>
> - UTF8 specifies the name of a character repertoire that consists
> of every character represented by The Unicode Standard Version
> 2.0 and by ISO/IEC 10646 UTF-8, where each character is encoded
> using the UTF-8 encoding, occupying from 1 (one) through 6
> octets.
>
> That is, "UTF8" is an identifier chosen to refer to an encoding which
> they know perfectly well is really called UTF-8. We should probably
> follow the same convention of using UTF8 in code identifiers and UTF-8
> in documentation. In particular, UTF_8 with an underscore is sanctioned
> by nobody and should be avoided.

OK, new version with utf_8 changed to utf8. The docs mention UTF-8 when
talking about the encoding, but UTF8 when listing the code to use.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073