Bytea misconceptions

From: Peter Eisentraut <peter_e(at)gmx(dot)net>
To: PostgreSQL Development <pgsql-hackers(at)postgresql(dot)org>
Subject: Bytea misconceptions
Date: 2003-02-19 11:45:11
Message-ID: Pine.LNX.4.44.0302191235290.1714-100000@peter.localdomain
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

The bytea type seems to be liable to character set conversions to the
effect that it falsifies the stored data.

Example: Create a cluster with non-C CTYPE, create a LATIN1 database,
create a table with a bytea column, and store something with non-ASCII
characters in it. Then change the client encoding (to UNICODE, say) and
read the data. I stored 'ätsch bätsch' and got 'Àtsch bÀtsch', which is
not a suitable result for bytea data.

The bytea output function uses isprint() to determine which characters not
to escape, which fails to give at least the documented results in most
locales. In general, the only safe solution would be to escape *all* byte
values on output. Then the client can reconstruct the byte sequence based
on the character entities in the delivered string and does not have to
rely on the character codes staying the same during the conversion.
(Alternatively, we do not pass bytea values through the character set
conversion, but that might be unfeasible for other reasons.)

--
Peter Eisentraut peter_e(at)gmx(dot)net

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2003-02-19 11:48:37 Re: Hard problem with concurrency
Previous Message Dave Cramer 2003-02-19 11:35:51 request for sql3 compliance for the update command