Re: UTF16 surrogate pairs in UTF8 encoding

* Tom Lane:

> I just noticed that we are now advertising the ability to insert UTF16
> surrogate pairs in strings and identifiers (see section 4.1.2.2 in
> current docs, in particular). Is this really wise? I thought that
> surrogate pairs were specifically prohibited in UTF8 strings, because
> of the security hazards implicit in having more than one way to
> represent the same code point.

There is relatively little risk because surrogate pairs cannot encode
characters in the BMP, and presumably, most of the critical characters
are located there.

However, if this is converted to regular UTF-8, I really question the
sense of this. Usually, people want CESU-8 to preserve ordering
between languages such as C# and Java and their database, and
conversion destroys this property.

--
Florian Weimer <fweimer(at)bfk(dot)de>
BFK edv-consulting GmbH http://www.bfk.de/
Kriegsstraße 100 tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99