pgsql: Add support for Windows codepages 1253, 1254, 1255, and 1257 and

From: petere(at)postgresql(dot)org (Peter Eisentraut)
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: Add support for Windows codepages 1253, 1254, 1255, and 1257 and
Date: 2006-02-18 16:15:23
Message-ID: 20060218161523.7B0A59DC84A@postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Log Message:
-----------
Add support for Windows codepages 1253, 1254, 1255, and 1257 and clean
up a bunch of the support utilities.

In src/backend/utils/mb/Unicode remove nearly duplicate copies of the
UCS_to_XXX perl script and replace with one version to handle all generic
files. Update the Makefile so that it knows about all the map files.
This produces a slight difference in some of the map files, using a
uniform naming convention and not mapping the null character.

In src/backend/utils/mb/conversion_procs create a master utf8<->win
codepage function like the ISO 8859 versions instead of having a separate
handler for each conversion.

There is an externally visible change in the name of the win1258 to utf8
conversion. According to the documentation notes, it was named
incorrectly and this changes it to a standard name.

Running the Unicode mapping perl scripts has shown some additional mapping
changes in koi8r and iso8859-7.

Modified Files:
--------------
pgsql/doc/src/sgml:
charset.sgml (r2.75 -> r2.76)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/charset.sgml.diff?r1=2.75&r2=2.76)
func.sgml (r1.306 -> r1.307)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/doc/src/sgml/func.sgml.diff?r1=1.306&r2=1.307)
pgsql/src/backend/utils/mb:
README (r1.4 -> r1.5)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/README.diff?r1=1.4&r2=1.5)
encnames.c (r1.28 -> r1.29)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/encnames.c.diff?r1=1.28&r2=1.29)
wchar.c (r1.53 -> r1.54)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/wchar.c.diff?r1=1.53&r2=1.54)
pgsql/src/backend/utils/mb/Unicode:
Makefile (r1.9 -> r1.10)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/Makefile.diff?r1=1.9&r2=1.10)
iso8859_7_to_utf8.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/iso8859_7_to_utf8.map.diff?r1=1.1&r2=1.2)
koi8r_to_utf8.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/koi8r_to_utf8.map.diff?r1=1.1&r2=1.2)
utf8_to_iso8859_7.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/utf8_to_iso8859_7.map.diff?r1=1.1&r2=1.2)
utf8_to_koi8r.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/utf8_to_koi8r.map.diff?r1=1.1&r2=1.2)
utf8_to_win1250.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/utf8_to_win1250.map.diff?r1=1.1&r2=1.2)
utf8_to_win1251.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/utf8_to_win1251.map.diff?r1=1.1&r2=1.2)
utf8_to_win1252.map (r1.3 -> r1.4)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/utf8_to_win1252.map.diff?r1=1.3&r2=1.4)
utf8_to_win1258.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/utf8_to_win1258.map.diff?r1=1.1&r2=1.2)
utf8_to_win866.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/utf8_to_win866.map.diff?r1=1.1&r2=1.2)
utf8_to_win874.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/utf8_to_win874.map.diff?r1=1.1&r2=1.2)
win1250_to_utf8.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/win1250_to_utf8.map.diff?r1=1.1&r2=1.2)
win1251_to_utf8.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/win1251_to_utf8.map.diff?r1=1.1&r2=1.2)
win1252_to_utf8.map (r1.3 -> r1.4)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/win1252_to_utf8.map.diff?r1=1.3&r2=1.4)
win1258_to_utf8.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/win1258_to_utf8.map.diff?r1=1.1&r2=1.2)
win874_to_utf8.map (r1.1 -> r1.2)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/win874_to_utf8.map.diff?r1=1.1&r2=1.2)
pgsql/src/backend/utils/mb/conversion_procs:
Makefile (r1.15 -> r1.16)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/Makefile.diff?r1=1.15&r2=1.16)
pgsql/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic:
utf8_and_cyrillic.c (r1.12 -> r1.13)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c.diff?r1=1.12&r2=1.13)
pgsql/src/bin/initdb:
initdb.c (r1.109 -> r1.110)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/bin/initdb/initdb.c.diff?r1=1.109&r2=1.110)
pgsql/src/include/mb:
pg_wchar.h (r1.66 -> r1.67)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/include/mb/pg_wchar.h.diff?r1=1.66&r2=1.67)
pgsql/src/test/regress/expected:
conversion.out (r1.11 -> r1.12)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/test/regress/expected/conversion.out.diff?r1=1.11&r2=1.12)
pgsql/src/test/regress/sql:
conversion.sql (r1.8 -> r1.9)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/test/regress/sql/conversion.sql.diff?r1=1.8&r2=1.9)

Added Files:
-----------
pgsql/src/backend/utils/mb/Unicode:
UCS_to_most.pl (r1.1)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/UCS_to_most.pl?rev=1.1&content-type=text/x-cvsweb-markup)
utf8_to_win1253.map (r1.1)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/utf8_to_win1253.map?rev=1.1&content-type=text/x-cvsweb-markup)
utf8_to_win1254.map (r1.1)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/utf8_to_win1254.map?rev=1.1&content-type=text/x-cvsweb-markup)
utf8_to_win1255.map (r1.1)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/utf8_to_win1255.map?rev=1.1&content-type=text/x-cvsweb-markup)
utf8_to_win1257.map (r1.1)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/utf8_to_win1257.map?rev=1.1&content-type=text/x-cvsweb-markup)
win1253_to_utf8.map (r1.1)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/win1253_to_utf8.map?rev=1.1&content-type=text/x-cvsweb-markup)
win1254_to_utf8.map (r1.1)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/win1254_to_utf8.map?rev=1.1&content-type=text/x-cvsweb-markup)
win1255_to_utf8.map (r1.1)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/win1255_to_utf8.map?rev=1.1&content-type=text/x-cvsweb-markup)
win1257_to_utf8.map (r1.1)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/win1257_to_utf8.map?rev=1.1&content-type=text/x-cvsweb-markup)
pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win:
Makefile (r1.1)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win/Makefile?rev=1.1&content-type=text/x-cvsweb-markup)
utf8_and_win.c (r1.1)
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c?rev=1.1&content-type=text/x-cvsweb-markup)

Removed Files:
-------------
pgsql/src/backend/utils/mb/Unicode:
UCS_to_8859.pl
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/UCS_to_8859.pl)
UCS_to_BIG5.pl
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/UCS_to_BIG5.pl)
UCS_to_GBK.pl
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/UCS_to_GBK.pl)
UCS_to_JOHAB.pl
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/UCS_to_JOHAB.pl)
UCS_to_UHC.pl
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/UCS_to_UHC.pl)
UCS_to_WIN874.pl
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/UCS_to_WIN874.pl)
UCS_to_WINX.pl
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/UCS_to_WINX.pl)
UCS_to_cyrillic.pl
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/Unicode/UCS_to_cyrillic.pl)
pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1250:
Makefile
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1250/Makefile)
utf8_and_win1250.c
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1250/utf8_and_win1250.c)
pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1252:
Makefile
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1252/Makefile)
utf8_and_win1252.c
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1252/utf8_and_win1252.c)
pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1256:
Makefile
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1256/Makefile)
utf8_and_win1256.c
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1256/utf8_and_win1256.c)
pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1258:
Makefile
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1258/Makefile)
utf8_and_win1258.c
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1258/utf8_and_win1258.c)
pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win874:
Makefile
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win874/Makefile)
utf8_and_win874.c
(http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win874/utf8_and_win874.c)

Browse pgsql-committers by date

  From Date Subject
Next Message Neil Conway 2006-02-18 20:48:51 pgsql: Patch from Marko Kreen: pgcrypto crypt()/md5 and hmac() leak
Previous Message Peter Eisentraut 2006-02-18 16:13:17 pgsql: New Directory